A Cloud Based FX Library – Possible?

I’ve been doing a bit of data wrangling lately – I just added another 8 TB of storage to my sound library (which actually means 4TB as I maintain two copies of it), so my sound library now has a total capacity of 8TB (4Tb existing and 4TB new) but I’ve been investigating the idea of keeping a copy in the cloud ie online, and I thought I’d share with you my number crunching & theorising to see what input & experiences you have on the subject. So I’m going to work out what it would currently cost to have 8TB of data archived and securely accessible online.

data centre
photo credit

Before anyone pipes up and asks what my upload speed is and how many years/decades it will take to get 8TB of sounds uploaded, my immediate answer is AWS Import/Export! So yes, for this cloud based scenario I am investigating using Amazon S3 storage which has the option of shipping physical drives thus enabling local data transfer speeds.


But this isn’t an ad for Amazon S3, it is just an example and as with doing any due diligence I’ll check out other similar options (feel free to suggest them?) but ever since I started researching this I started noticing companies & large sites that also use Amazon S3, although you’d never know it -they don’t advertise the fact, I just noticed when I went to download something, the link suddenly switched to Amazon S3, examples such as Vimeo and E-Junkie. So funnily enough if you use these sites you are actually paying to use Amazon S3!

So ok the costing. I’m going to ship 8TB, so I’ll need 8TB worth of USB drives to send, a quick search & price comparison indicates I can get a Seagate 2TB drive in a case for US$150, I’ll need four of them so thats US$600 for the drives.

AWS Import/Export pricing is:
Device Handling: $80.00 per storage device handled and Data Loading Time: $2.49 per data-loading-hour. There is a handy calculator here and so entering the data it estimates:
Device Charges: $320.00 for 4 device(s) plus Data Loading Charges $298.80 – 120 hour(s)
Total Estimated Charges $618.80

So for US$600 of drives and US$618 of data transfer my library is securely in the cloud, now what does it cost to keep it there?

First 1 TB / month of Storage Used: $0.140 per GB
Next 49 TB / month of Storage Used: $0.125 per GB

So for the first 1TB = 1000GB @0.14 = US$140 and for the other 7TB = 7000GB @0.125 = US$875
Youch! Thats a monthly bill of US$1,015 – Me not likee 🙁

And what about access costs?
First 1 GB/month data transfer out is free, Up to 10 TB / month data transfer out $0.150 per GB
You can always do a data export via the same way it go in there, ie a physical drive, with the same costs ie US$618.80 transfer costs…

In terms of actual ease of use, that will be dependent on your internet connection; auditioning 24 bit 96k files is probably not that realistic, but even if you had to run a script across your whole library to generate MP3 audition versions that aspect is workable… And with transfer apps like Transmit not only can you treat your Amazon S3 archive like an ftp site, you can also mount virtual drives on your desktop, so a local copy of SoundMiner could be used to search & audition….

But wow it’s that monthly bill of US$1,015 that put the brakes on this idea – back to cloning drives and storing them at work & at home & at my parents house etc… Sneakernet to the rescue!

6 thoughts on “A Cloud Based FX Library – Possible?

  1. DW

    The speed of access is the killer for those of us in the antipodes. I use S3 via a 3rd party for data backup of my home PC. Even though I have a fast connection, the trans-Pacific speed is not great. It’s not really feasible for anything larger than documents and photos.

  2. Pingback: Tweets that mention Music of Sound » A Cloud Based FX Library – Possible? -- Topsy.com

  3. rene coronado

    I’ve looked into s3 for long term archival and come up with the same conclusion. Monthly expense far too high to be feasible.

    FWIW there’s a cheaper way to get data onto sata drives though:
    get a USB dock and buy the drives unenclosed. I have this one: http://www.newegg.com/Product/Product.aspx?Item=N82E16817193075&Tpk=sata%20usb%20dock

    works like a champ. With one of those in place a 2tb naked segate is $90.

    plugs right in, quiet as can be and immediate plug and play access.

  4. Tim Walston

    ** Great topic! **
    You mention USB a few times – isn’t FireWire 800 faster than USB 2.0? Is there a reason why you prefer USB drives to firewire?

    What are you using now for your 8TB of storage? Have you looked into some sort of small raid system? There are many products of this type out there:

    I like the idea of the drive dock and bare drives, but I’m longing for more long term security as well.

    1. tim Post author

      Amazon S3 only accept either USB or SATA drives…

      I use this for library storage at the moment; Stardom SohoTank 2

      which are a 2 drive bay case with FW400, FW800 and USB (I use FW800)
      So each case can put 2 x 2TB

      My original library is on 2 x 2TB GTech drives

      I don’t want RAID – i don’t need the performance (its only for auditioning, I always transfer to internal work drives on my Mac) and I prefer to manage redundancy myself, so if a drive fails I instantly replace it and clone from another….

  5. MarD

    If the purpose basically is, that you want to have online access to your Library, you could always set up a small server/nas. Also, you could use your existing hard drives and backing up would not change either. Depending on the internet connection that could work just fine.

Leave a Reply

Your email address will not be published.