Downloading an entire S3 bucket?

by

AWS CLI

See the “AWS CLI Command Reference” for more information.

AWS recently released their Command Line Tools, which work much like boto and can be installed using 

sudo easy_install awscli

or 

sudo pip install awscli

Once installed, you can then simply run:

aws s3 sync s3://<source_bucket> <local_destination>

For example:

aws s3 sync s3://mybucket .

will download all the objects in mybucket to the current directory.

And will output:

download: s3://mybucket/test.txt to test.txt
download: s3://mybucket/test2.txt to test2.txt

This will download all of your files using a one-way sync. It will not delete any existing files in your current directory unless you specify --delete, and it won’t change or delete any files on S3.

You can also do S3 bucket to S3 bucket, or local to S3 bucket sync.

Check out the documentation and other examples.

Whereas the above example is how to download a full bucket, you can also download a folder recursively by performing

aws s3 cp s3://BUCKETNAME/PATH/TO/FOLDER LocalFolderName --recursive

This will instruct the CLI to download all files and folder keys recursively within the PATH/TO/FOLDER directory within the BUCKETNAME bucket.

More Related Contents:

  1. How list Amazon S3 bucket contents by modified date?
  2. The AWS Access Key Id does not exist in our records
  3. SSL CERTIFICATE_VERIFY_FAILED in aws cli
  4. How do I delete a versioned bucket in AWS S3 using the CLI?
  5. How do I set Content-Type when uploading to S3 with AWS CLI?
  6. Getting 403 forbidden from s3 when attempting to download a file
  7. AWS CLI S3 A client error (403) occurred when calling the HeadObject operation: Forbidden
  8. S3 – Access-Control-Allow-Origin Header
  9. CloudFront + S3 Website: “The specified key does not exist” when an implicit index document should be displayed
  10. Amazon AWS Route 53 Hosted Zone does not work
  11. How long should I wait after applying an AWS IAM policy before it is valid?
  12. How to make all Objects in AWS S3 bucket public by default?
  13. AWS EFS vs EBS vs S3 (differences & when to use?) [closed]
  14. Multiple Cloudfront Origins with Behavior Path Redirection
  15. Nginx proxy Amazon S3 resources
  16. Append data to an S3 object
  17. Serving a multitude of static sites from a wildcard domain in AWS
  18. Supporting HTTPS URL redirection with a single CloudFront distribution
  19. Deploying react-redux app in AWS S3
  20. How to change permission recursively to folder with AWS s3 or AWS s3api
  21. How to use MFA with AWS CLI?
  22. Difference between upload() and putObject() for uploading a file to S3?
  23. How do you search an amazon s3 bucket?
  24. AWS sts assume role in one command
  25. Adding AWS Lambda with VPC configuration causes timeout when accessing S3
  26. How to change content type of Amazon S3 Objects
  27. AWS Glue output file name
  28. Amazon S3 bucket returning 403 Forbidden
  29. Get last modified object from S3 using AWS CLI
  30. Initial setup of terraform backend using terraform

Leave a Comment