I recently needed to share some open source files via BitTorrent and wanted to host them on my Amazon S3 account. For those of you familiar with S3, here is the short-answer: add ?torrent to the end of the URL of a public-shared file to get the *.torrent file, so the link would be http://s3.amazonaws.com/your_bucket_name/your_file_name?torrent
If you’re new to Amazon S3, read on for more info and detailed instructions.
Why would you want to distribute a file using BitTorrent?
I wanted to do this because, when sharing a file via a torrent, you use less of your Amazon S3 bandwidth and lower your monthly expense. To be more accurate, according the S3 documentation:
The difference is that if a lot of clients are requesting the same object simultaneously via BitTorrent, then the amount of data S3 must serve to satisfy those clients will be lower than with client/server delivery.
I knew it was possible to host a torrent on S3, but wasn’t quite sure how to do it. After quite a bit of searching and reading, I found that it was actually pretty easy to do.
Here are the details on how to copy a file to your Amazon S3 account, make it public, create the torrent seed / tracker, and have Amazon S3 share both the .torrent file and seed the download.
Preliminary Steps for new Amazon S3 Users:
- You must first have an Amazon S3 account
- Secondly, you need a way to create/manage Amazon S3 buckets, so you’ll need to install a client on your PC.
- I recommend using Bucket Explorer, which is a full-featured and easy-to-use client that runs on Windows and Linux; (a Mac version is in private beta and should be available Oct 2007). A free, less-featured alternative is the S3 Organizer add-on for the Mozilla Firefox web browser.
- Install your application of choice and either:
- Open Bucket Explorer -or-
- Open Firefox and go to Tools menu -> S3 organizer, and click the Manage Accounts button
- Enter your AmazonAWS Access Key and Secret Key
- These are available by going to http://aws.amazon.com, mousing-over the “Your Web Services Account” in the upper right-hand corner, and selecting “AWS Access Identifiers“
How to Create and Seed a Torrent Download on Amazon S3:
- Create a new “bucket” (a.k.a. folder) by clicking the “create folder” icon in the right-hand panel of S3 Organizer. Note: you must use a unique bucket name; you won’t be able to create bucket if the name is already being used by someone else.
- Upload the file which will be shared into the desired bucket using S3 Organizer
- Make the bucket public read by right-clicking on the bucket name and selecting “Edit ACL”. Select “Read” access for “Everyone”. Note: you may also have to repeat the read access step for each file that is uploaded to the bucket.
- Now that the file is uploaded and shared, it can be downloaded via standard http from the following location: http://s3.amazonaws.com/your_bucket_name/your_file_name .
- Note 1: You can also find the URL by right-clicking the file in S3 organizer and selecting Copy URL to Clipboard.
- Note 2: Unfortunately, the file will always be available via http download, which can be faster than BitTorrent download, but will use more of your S3 bandwidth.)
- Now you want to create the torrent file itself. Just enter the following URL: http://s3.amazonaws.com/your_bucket_name/your_file_name?torrent and the torrent file will be created and you will be able to download it the *.torrent file to your computer.
- You can share this downloaded *.torrent file with others via e-mail, webpage download, etc.
According to this article, you can use Amazon S3 as the torrent tracker only by following the steps above, opening the torrent file on your local PC, start seeding from your local PC, and then remove the original file. S3 will continue to be the (ultra reliable) tracker for the file but you don’t have to use S3 bandwidth to share the file. To save bandwidth usage and prevent someone from going directly to the http download of the file instead of using the torrent download, it’s probably best to download the small *.torrent file and share that with others instead publishing the URL to the torrent file on the Amazon S3 server.
Assuming you have the file being shared stored on your local computer, you can download the *.torrent and start seeding from your local PC without re-downloading (the shared file) from S3, further reducing your S3 bandwidth.That’s it, my start-to-finish guide on how to take a file and share/seed/tracker it via BitTorrent and Amazon S3. If I missed something or if something isn’t clear, let me know in the comments and I’ll fix it.