Syncing a lot of data from a directory to Amazon S3 Cloud

In an attempt to upload a directory 40+ Gigabytes worth of files on t0 Amazons S3.

Typically I use the AWS Console, but I ran into issues, I believe my browser was timing out.  I tried a couple downloading and installing third party applications, the only success I got from that was possibly inheriting additional malware.

I did manage to find a solution, I tend to pass on the command line tools, but I was desperate, turns out in this instance command line is actually sooo much easier and stable, This is basically all I had to do.

Download AWS CLI  and Install it

Open your terminal and run

aws configure

youll need to provide it your AWS ID, AWS KEY, Region, and a format (I choose JSON). Then all i had to do was run the following command to start syncing

aws s3 sync C://Your_Local_Directory_To_SYNC s3://your_S3_bucket

if you want to see more options simply run. You can sync bi-directionally or from s3 bucket to bucket by interchanging the last two parameters accordingly

aws s3 help

You are now free to do whatever it is you do.

Note: If you step away from your computer for too long or your computer goes inactive, this will discontinue your transfer. If that happens rerun the command to continue.

Leave a Comment

You must be logged in to post a comment.