Monday, December 30, 2013

S3 Bucket downloading and uploading with Java

Due to the growth of Mosaime, I recently faced the necessity of transferring a bucket from one AWS account to another. There are several tools on the Internet that could help me do that, such us S3 Browser or s3cmd, but I had some problems when starting to work with them.

I realized that this task was very simple and repetitive. Its only inconvenient has to do with the amount of time spent processing when the bucket is very big. Thankfully, the buckets of Mosaime only store a few hundred objects (right now), so I didn't have to worry about that.

So I developed a very simple program in Java called Java S3 All or Nothing. It allows to:
  1. Download an entire bucket to a folder
  2. Upload all the contents of a folder to a bucket
  3. Delete a bucket an all its contents
It's very simple because the amount of files that it has to deal with isn't very big. The code is published in GitHub so everyone can learn from it or use it. Also, it would be lovely to see another developer making it better, maybe by implementing simultaneous downloads using threads or making it a more packaged piece of software.





No comments:

Post a Comment