Download entire public S3 bucket without credentials

Published: Wednesday, Oct 9, 2013 Last modified: Monday, Apr 8, 2024

Say for example you have a S3 bucket named s3://mr2011/

If you have made your bucket public (Hint: s3cmd -P) it should reachable upon, which may again redirect you to

Now you should see an XML listing like:

<ListBucketResult xmlns="">

If not, perhaps you are missing the aws s3api put-bucket-acl --bucket --acl public-read listing permission.

You’re not limited to 1k objects. Attach ?max-keys=2147483647 to get more. If <IsTruncated> is still True, one strategy is to break results down by specifying first letter of path by attaching ?prefix=a, ?prefix=b…. etc.

Fetch first give objects on the day 2015-10-12 providing that’s how you store your objects (HINT: prefix with YYYY-MM-DD is a good idea)

curl -s '' | xml sel -N w="" -T -t -m "//w:Key" -o "${s3url%/}/" -v . -n

You can download all the resources with a bit of xmlstarlet magic like so:

Alternatively to xmlstarlet, xml2 is a nice way of working xml.