Syncing a lot of data from a directory to Amazon S3 Cloud
In an attempt to upload a directory 40+ Gigabytes worth of files on t0 Amazons S3.
Typically I use the AWS Console, but I ran into issues, I believe my browser was timing out. I tried a couple downloading and installing third party applications, the only success I got from that was possibly inheriting additional malware.
I did manage to find a solution, I tend to pass on the command line tools, but I was desperate, turns out in this instance command line is actually sooo much easier and stable, This is basically all I had to do.
Download AWS CLI and Install it
Open your terminal and run
youll need to provide it your AWS ID, AWS KEY, Region, and a format (I choose JSON). Then all i had to do was run the following command to start syncing
aws s3 sync C://Your_Local_Directory_To_SYNC s3://your_S3_bucket
if you want to see more options simply run. You can sync bi-directionally or from s3 bucket to bucket by interchanging the last two parameters accordingly
aws s3 help
You are now free to do whatever it is you do.
Note: If you step away from your computer for too long or your computer goes inactive, this will discontinue your transfer. If that happens rerun the command to continue.
Leave a Comment
You must be logged in to post a comment.