So firstly congrats to all the recordists who have uploaded their material – what a truly BRILLIANT collection it is rapidly becoming. The total currently sits at 70GB and tonight I have sent an email to everyone who committed to the project but hasn’t uploaded as yet to clarify whether their material is coming soon or not.
In the meantime I have been checking files, metadata etc and I have two topics for discussion here:
1. How best to organise the library. While most people access their libraries through searches using SoundMiner etc I believe it is still valuable to organise the folder structure logically. So I’d appreciate your thoughts on the matter – as a contributor or a potential user of the library.
Currently the library remains in the form it was uploaded
ie THE DOORS/Recordist Name/Folder for each door
I imagine re-organising them by type, so for example
THE DOORS/APARTMENT/Folder for each Apartment door
THE DOORS/HOUSE/Folder for each House door
THE DOORS/KITCHEN/Folder for each Kitchen door
THE DOORS/OFFICE/Folder for each Office door
Is that how you imagine it being presented?
Each individual door folder is still tagged with the recordists name….
Possible issues: I seem to remember there being a limit to the number of folders you can have in one directory before the OS becomes bogged down – is that true?
2. The final library will be approx 100GB which means it is beyond the realistic size of a download. Currently the largest Flash drives are 35GB. I investigated BluRay disks but I am not sure everyone has access to BluRay readers, plus the blank disks are still reasonably costly. Which leads me to believe the only way to deliver this library is via a small USB hard drive. I’d appreciate your thoughts on this – are there any options I havent thought of? And what are a good reliable model of small USB drives?
[...] This post was mentioned on Twitter by Tim Prebble, Gustavo Carvalho. Gustavo Carvalho said: THE DOORS – update & discussion: So firstly congrats to all the recordists who have uploaded their material – what… http://bit.ly/bJMDS8 [...]
Regarding the library´s structure I think the method you described is the best option (classifying per building type)!
As for how to deliver the final library…hmmm…thats getting BIG!
Bluray disks I would avoid as indeed, it is not yet accessible everywhere. Flash drives would be ideal but 100 GB is too big for that. So I would think small USB drives are the best solution but they will considerably increase the cost of the library… Tough one I must say! But quite proud to think we have put together 100GB worth of doors!
I noticed that other FX Library companies ship FX as:
download > DVD > Flash disk > Hard drive
depending on the size of the library….
And it was inevitable – after THE DOGS I am VERY keen to crowdsource an Ambience Library, and that would be an even larger library as every ambience would be 3 minutes minimum!
ambiances! now THATS a library I can sink my teeth into.
Ambiences! Yes! One can’t have enough of those. Question is, if those have to be recorded in 96 kHz or if 48 is enough, since one would probably use ambiences rather as they are. There won’t be much processing except EQing in most cases, right?
Not so much. Ambiences are really good to slow down, so the extra high-frequency info is helpful. One of my favourite tricks is to take a standard “birds” ambiences and play it at half speed — Bingo-instant “spooky forest” atmos.
re USB hard drives:
1. One brand to stay well away from, especially w/ 2.5″ drives – Western Digital. Another problematic brand is LaCie. (personal experience w/ both)
2. Enclosures will hold you hostage with regards to speed and power consumption. With 2.5″ drives, Firewire is not necessarily better than USB.
3. I’ve had reliable performance with (internal) drives from two brands: Toshiba GA* series, and Fujitsu (also owned by Toshiba from what I see). Toshiba market portable drives in their own enclosures relatively cheap, at the cost of often looking funny.
A 100GB library is still small enough to be copied to a larger storage medium though so you could regard the drive as mostly transport?
If you’re looking for a cheap way to distribute the lib to participants, have you considered maybe utilising p2p technology for a limited time? 10 parts 10GB each might work?
1.
Folders by building type would make sense to me. I think the recordist’s name should be stripped out of the folders and file names, but it will still be in the metadata in case you want to know.
2.
Regardless of how you get it to all of us, how are you going to deliver the sound library to people who purchase it going forward? I think in that case, including a hard drive would be the best option.
For all of us recordists, while I really do want to get the entire library, I’d rather do it in a downloaded fashion, whether it’s 5 gigs a week for however many weeks it takes, or whatever.
George has a good point about getting space on a fileshare site. It’s not a bad way to go, but it does have to be broken up into whatever max file size. In this case you wouldn’t be hit for bandwidth, and all the files are securely stored as long as you need them to be available.
You could always convert them all to mp3s and then it will be nothing to download….. heh.
Woah! 100 gigs of doors? I hope I didn’t leave too much room tone in my files!
)
I’d be happier to receive multiple SDHC or CF cards than a USB disk in the mail. Might be cheaper to ship also.
Yes I agree & much prefer the idea of shipping devices with no moving parts!
Using rar it can reduce the total file size by up to a third…
so 100GB could become 65GB which might fit on two CF cards…
And to think: I still remember when a 1GB drive seemed big!
No moving parts was the main motivation behind the thought. But also the usefulness of another CF card in the field too. You get the sound effects delivered, and then a bonus CF to go gather more sound effects. Double points!
I think the file structure you described sounds like the best way. I personally have no problem leaving the recordists initials in the folder or file name.
As for delivery method, I’m with Colin and George. I’d prefer to download, even if it takes a while to download all the content.
But if shipping is the best method I can see this becoming quite a large time investment for you Tim and expensive with shipping to other countries to get the library to all the contributors. Just throwing this out there, but what if you got people to volunteer in countries with a large number of contributors. So if you have 30 recordists from the US, then ship or download to one point person in the States and have them be the point person for delivery. Just a thought.
Thats a great idea! Even if it meant I couriered ten drives to different countries and the distribution occurred from there
I don’t know if it makes a difference, but would this work for downloading as well? Have mirror sites for the US/ Europe/ Asia?
If you do go with this shipping method, I’d be happy to be a point of contact for the US.
Hi Tim,
First of all good to hear that the material is so good!
I will sent you an email in 5 minutes about my (almost finished) contribution.
Distribution:
I work at The Institute for Sound and Vision in The Netherlands and we have a superfast fiber network/uplink. Maybe i can arrange something for this project so there can be an upload for European contributors.
Furthermore, the Institute has a trade concept: they get/store the media (be it tape or discs) and in exchange they have the right to make it available to schools and industry. Not commercially just for subsidized parties (such as universities or national television).
Maybe it could be interesting to have some kind of trade between “Hiss and Roar Ltd.” and “The Institute” ? Let me know.
Here is a link:
http://instituut.beeldengeluid.nl/index.aspx?ChapterID=8532
Greetings!
Arnoud
thats a great concept – will investigate
thanks Arnoud!
ey Arnoud
All good?
Anton
Re delivery
+1 for download.
Bittorrent? I.e. crowdsourcing.
torrents def. the way to go. It is possible to have a secure torrent right?
I’m quite sure, it’s possible to create secure/private torrents. The whole point of torrents is to distribute large amounts of data, fast and efficiently between many people.That’s exactly what we want isn’t it?
On a side note:
The national broadcaster here in Norway (NRK) has started to offer certain tv-series and other programmes, as an experiment, free to download as torrents. One of them is for a 7 1/2 hour (!) long documentary. It is available under a Creative-Commons licence and is offered as 246 GB (!) Pro Res file. Check it out here: http://goo.gl/Gum5
I’ll vote for download as well, though I’m certainly not married to it if its logistically too difficult.
The way the music libraries deal with it is they ship us a thumb drive and a SASE so we can send the drive back after we dump the files. That could also be an option.
love the map BTW!
+1 for download. Mostly because physical shipping is logistically more complicated and expensive and secondly because I wouldn’t mind to download the 100 GB broken down to packages of 5 to 10 GB. It only lasts longer but it is possible. So…
I vote download! Shipping cost is to much for a low paid student ;D
I presume you have broadband access with unlimited data useage?
In New Zealand all the crappy ISPS (and I mean ALL the ISPs) have data caps in place, and the largest domestic broadband fixed price data cap I can find is 20GB per month… Thankfully broadband at my work studio is unmetered…
A relevant question then: How long would it take you to download 100GB?
I have a download rate of 2 megabytes per second if the server on the other side allows that, no data-limit. So downloading would be okay, it would take a couple of hours but with an ftp-download that can be resumed in case of failure that’s no problem.
Like the idea of passing a drive around as well!
Let me say a huge thank you for this ginormous effort! I can’t wait to listen to it all.
A thought regarding categorizing: What about by type of door? Example,
THE DOORS/WOOD/Folder for each Wood door
THE DOORS/METAL/Folder for each Metal door
THE DOORS/GLASS/Folder for each Glass door
etc. etc.
Could even break down to MODERN, ANTIQUE, etc etc.
True!
This could be helpful for downloading too ie I wont do a multi-part rar archive but would rar each category, so you can decompress each folder once its downloaded…. One aspect of making it available for download is getting the 100GB uploaded from my end (upload is way slower than download) but I will enquire about shipping a hard drive to my hosting company, as that would only take a couple of days….
I have to agree with Jay here, while the OFFICE, HOUSE, KITCHEN tags would be useful metadata, I never use terms like that when searching. I look up “door squeak wood” or “door metal slam.” Something to that effect.
Remember, download costs will be reduced by the fact that a lot of the contributors probably know each other IRL. For example, there were three of us from my work that contributed, so we’ll only need to download it once between us and then share it locally amongst ourselves.
Also, you can get the filesize down by archiving them into zip or rar bundles. I know they’re not optimised for audio, but they do alright when the signal has a lot of low-complexity stuff, such as silence or quiet noise. Just trying it with 7-zip (probably the most advanced algorithm) on my set got it down to between 1/2 and 2/3 of the original filesize.
I am open on the shipping or download options, but I completely understand the download cap and expense on Tim’s end.
Personally, I like the idea of shipping SDHC cards with the zipped files to copy to our own audio drives. Maybe even only one or two sets cards, mailed from one person to the next on the list, all the way around the world. This way, even though we worked completely apart, we can touch/share something that was completed together. Perhaps even include a photo or a autograph or the such from each of us as we ship it to each other to go back to Tim.
But perhaps I am a bit of a romantic trapped in a digital age
Nice Idea
I love this idea. While it will take a bit longer to get, it definitely adds a personal touch. Both for all the time and organization Tim’s put in as well as everyone involved.
Only issue is how will you sort out shipping costs between people – some peoples shipping might end up costing more depending on where the next person on the list is?
Personally, I prefers downloads (even if its multiple packages) but I see the problems with ISP’s etc and , at the same time, the problems with shipping. Either way I’ll be buying at some point in time.
Its probably way OTT but if there you are looking at doing big libraries, we use http://www.accellion.com/products-services at work – server based FTP which we end 100′s of GB of animation/uncompressed HD footage to overseas. Just another option I guess
Speaking as someone who sadly did not contribute (although I would love the chance to contribute on an Ambience version of this experiment) I am eagerly awaiting the opportunity to the purchase this doors library. I think the proposed organization by building type is a good way to go.
In terms of distribution, 100gig makes it a difficult situation. Lots of people in my country (Canada) have download data caps on internet at home, but most business accounts do not. I could be a happy camper starting a new download when I leave the studio each night for a week so it pulls it down while I am sleeping. Seems easy enough.
If you need a Canadian distributor/organizer feel free to let me know.
I vote for download too, if possible.
I see from the map image that someone has sent you something from South Australia? Who’s over here??????
That would be me!
Seems we do miss asian and african doors, but i hope we’ll get some asian and african dogs !
If we go for the shipment solution : anyone from europe can send me a mail with their exact location then i’m willing to do a list of european user, a shipment path and try to find a way to share costs equally.
I am willing to do the same for the US, Canada, and (what looks like it could be on the map) Columbia/Ecuador? Can’t quite tell, hahaha
I also have unrestricted internet access at the studio and at home. So if there is need for some kind of distribution/organization in the UK I am up for helping.
I’m also up for a download if it is feasible for Tim.
Would categorising things by building/material type really be the way to go? I only ask because I know that I submitted doors which were wooden frame and glass panelling. So would it go into the wood category or the glass category?
I am split between downloading and receiving physical storage media. For the doors, I think downloading would be fine, however, if this is a trial run for the proposed Ambience library, which will no doubt be much bigger then physical media makes more sense to me.
Your organizing idea is great, Tim!
I think download is the best option… at least for me, and also easy for you. Another idea: You could split each of the categories of the doors and offer different download options. ie: download the apartment doors only, etc.. whatever each recordist chooses.
You could also offer that option to future HISS and a ROAR clients… the possibility of acquire each type of door separately with an specific price, or get the complete package…
If any of the contributors fall through and your looking for a few more doors let me know. I would love to be a part of this. I’ve got some heavy 150+ pound doors in a quiet studio environment that could sound quite nice.
Also, if you are looking anything specific (stairwell doors, doors at the end of a long corridor) please let me know.
Thanks,
Chris
Hey Tim
Some updates for the delivery method ?
Thanks!
M
I will use a combination of hard drive delivery and download.
1. For those who wish to receive a hard drive, I suggest I buy 5-10 hard drives and ship them to someone in each country/continent. There would be a small cost to be spread between everyone who wants drive delivery, and each person would have to commit to paying the postage to ship the drive to the next person.
2. I will upload the whole library to my server and give everyone download links. The library is now at 80GB, and I plan to announce 31st August as final upload date. So once I finish sorting the library I will rar it into individual archives, so you can download & use each category without downloading the entire library. Just bear in mind it will take approx 2 weeks to upload it all!!
I am in the process of setting up a forum attached to HISSandaROAR.com for discussion and opt in for this & future group libraries. I think a forum would be a better means of ongoing discussion, as posts on this blog slowly disappear into the archive…. Should be live this coming week – will email an invite to all the contributors
… and if you opt for p2p distribution you do not need to upload anywhere, just create a .tor out of it and give it to everyone interested. you don’t even need a tracker, if i remember correct.
1. I cant feed 100GB of data via my home broadband as it has data cap and since I’m the only one with all the data, the whole 100GB has to come from me first right?
2. I cant use torrents at work, as they are not allowed by the ISP
100gb = 4 days x 24 hours at 300k/sec downstream.
raise that to modern day 8mbit basic broadband at 600k/sec = 2 days x 24 hours.
I think your maths is not right…
I did speed test at work using server location Texas (where hostgator is)
Download 870kbps
Upload 202kbps
It said download 250MB = 16 minutes
which means 1GB = 2.5hours
which means 100G = 250 hours = 11 days of continuous download.
If upload speed is a quarter of that, it means 100GB would take 44 days of continuous upload!
I’ll email everyone a login to a private google docs spreadsheet and each person can enter a field to specify how they want to receive the data, plus their address if they choose physical media
[...] http://www.musicofsound.co.nz/blog/the-doors-update-discussion [...]