{"id":844,"date":"2018-01-12T09:34:02","date_gmt":"2018-01-12T17:34:02","guid":{"rendered":"https:\/\/www.nicknguyen.com\/?p=844"},"modified":"2020-02-04T22:31:19","modified_gmt":"2020-02-05T06:31:19","slug":"syncing-a-lot-of-data-from-a-directory-to-amazon-s3-cloud","status":"publish","type":"post","link":"https:\/\/nicknguyen.com\/old\/syncing-a-lot-of-data-from-a-directory-to-amazon-s3-cloud\/","title":{"rendered":"Syncing a lot of data from a directory to Amazon S3 Cloud"},"content":{"rendered":"<p>In an attempt to upload a directory 40+ Gigabytes worth of files on t0 Amazons S3.<\/p>\n<p>Typically I use the AWS Console, but I ran into issues, I believe my browser was timing out.\u00a0 I tried a couple downloading and installing third party applications, the only success I got from that was possibly inheriting additional malware.<\/p>\n<p>I did manage to find a solution, I tend to pass on the command line tools, but I was desperate, turns out in this instance command line is actually sooo much easier and stable, This is basically all I had to do.<\/p>\n<p><a href=\"https:\/\/aws.amazon.com\/cli\/\">Download AWS CLI\u00a0 and Install it<\/a><\/p>\n<p>Open your terminal and run<\/p>\n<blockquote><p>aws configure<\/p><\/blockquote>\n<p>youll need to provide it your AWS ID, AWS KEY, Region, and a format (I choose JSON). Then all i had to do was run the following command to start syncing<\/p>\n<blockquote><p>aws\u00a0s3 sync C:\/\/Your_Local_Directory_To_SYNC s3:\/\/your_S3_bucket<\/p><\/blockquote>\n<p>if you want to see more options simply run. You can sync bi-directionally or from s3 bucket to bucket by interchanging the last two parameters accordingly<\/p>\n<blockquote><p>aws s3 help<\/p><\/blockquote>\n<p>You are now free to do whatever it is you do.<\/p>\n<p>Note: If you step away from your computer for too long or your computer goes inactive, this will discontinue your transfer. If that happens rerun the command to continue.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In an attempt to upload a directory 40+ Gigabytes worth of files on t0 Amazons S3. Typically I use the AWS Console, but I ran into issues, I believe my browser was timing out.\u00a0 I tried a couple downloading and installing third party applications, the only success I got from that was possibly inheriting additional&hellip;<\/p>\n","protected":false},"author":1,"featured_media":949,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_crdt_document":"","_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"Syncing a lot of data from a directory on your computer to a Amazon S3","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1,151,162],"tags":[168,93,169],"class_list":["post-844","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-all","category-how-to","category-technology","tag-aws","tag-cloud","tag-s3"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/media.nicknguyen.com\/wp-content\/uploads\/2018\/01\/21011218\/467771121-crop-copy-56a6a4815f9b58b7d0e41c09.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/nicknguyen.com\/old\/wp-json\/wp\/v2\/posts\/844","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nicknguyen.com\/old\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nicknguyen.com\/old\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nicknguyen.com\/old\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nicknguyen.com\/old\/wp-json\/wp\/v2\/comments?post=844"}],"version-history":[{"count":2,"href":"https:\/\/nicknguyen.com\/old\/wp-json\/wp\/v2\/posts\/844\/revisions"}],"predecessor-version":[{"id":846,"href":"https:\/\/nicknguyen.com\/old\/wp-json\/wp\/v2\/posts\/844\/revisions\/846"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nicknguyen.com\/old\/wp-json\/wp\/v2\/media\/949"}],"wp:attachment":[{"href":"https:\/\/nicknguyen.com\/old\/wp-json\/wp\/v2\/media?parent=844"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nicknguyen.com\/old\/wp-json\/wp\/v2\/categories?post=844"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nicknguyen.com\/old\/wp-json\/wp\/v2\/tags?post=844"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}