keep your data safe

After posting those stats on my data generation during a two month Japan trip (106GB of field recordings, 428Gb of photos & video) a couple of people asked me how I manage my data while travelling, so here is my methodology & associated tech. Warning: there are no great revelations or shortcuts – it is all fairly simple, and boring… but it works!

The basic requirement is to keep your data safe – so that means creating clones in two, preferably three separate places, as soon as possible. And, at crucial times, keeping them separate ie physically separate.

A final caveat, no doubt there are better ways to do this, and if at any point you read this & think ‘WTF, why doesn’t he just get a thing, or do this then comment & help us evolve for the next trip!

Backup for me = [laptop] + [2 x 2TB drives] + [bag of cables]

In a nutshell, I use a 2013 macbook pro laptop to clone data asap – it has 2 x usb, 2 x thunderbolt and a built in SD card reader. I carry two LaCie rugged 2TB hard drives, which power off the bus and have both USB3 and FW800 connectors – available in NZ here

backups

My sources of data:

sound:
sound devices 788T – 8040×2 ORTF + 8020×2 spaced omni = 4 x 24bit 96kHz
sony d100 = 2 x 24bit 96kHz

visual
sony a6300 capturing 4k, 1080p/24fps-120fps + JPG+RAW stills
contax T2 shooting TriX to neg x TIFF hi rez scans
fujinon TX2/xPan2 shooting Velvia50/TriX/400Pro to neg x TIFF hi rez scans

Each of these create different amounts of data, at different rates (especially shooting film) but my aim is at the end of every day, or as soon as the data is available, to transfer on to one of the 2TB drives, and then backup to a clone on the other 2Tb drive.

I have a strict folder naming policy of ‘DATE-DEVICE-DESCRIPTION’
For example the top level folder might be called:

201610 JAPAN TRIP Master

and sub folders labelled
20161107 a6300 Kyoto
20161107 788T Kyoto

The date is generated via a TextExpander shortcut, so the format is consistent.
And I don’t reorganise the contents – this backup is strictly chronological.

On the second 2TB hard drive I have a folder labelled
201610 JAPAN TRIP Clone

And I use a simple OSX app called FoldersSyncronizer to backup from the Master to the Clone.

My Sound Devices 788T has a large enough hard drive in it that I also leave the original data on the device, so for all sound recordings I have three separate copies of the data.

For video & photos, after a week or so I have to clean off the SD cards – it amazes me that as per the photo of the hard drives above, in less space than my business card I have 2 x 128GB and 1 64GB SD cards.

If I was only shooting photos JPG+RAW, I could probably cope leaving the data on the SD cards, but when it comes to shooting 4k video or 120fps 1080, it chews through data… So I tended to let the SD cards fill up, and switch to the next SD card, backing up daily but only then cleaning the SD card after doing an extra check that all data had been transferred.

So thats about it. The only other point to make, and it is an important one, is with regards to the physical location of your backups. While I was in Japan I stayed the first week in Tokyo in one apartment, then six weeks in an apartment in Osaka, then 1 week in a different apartment in Tokyo. So including my international flights there was four times when I was relocating with everything I owned and all data with me. Now it is stating he obvious but, at those times my data is at most risk i.e. if both hard drives were in the same bag as my 788T, and that bag got stolen.. then all of my data is gone.

So my final comment is when travelling/changing locations, place your three copies of data in three different bags. For my international flights, two sets of data went via each of my checked baggage suitcases and one set travelled with my as carry on.

Worst case scenario, travel insurance would replace my recorder & cameras. But there is no such possible insurance to replace data that will never exist in that form ever again.

In an ideal world I would also be uploading my data to a server as/when I had time… But this is rarely possible while travelling eg I rented the fastest 4G LTE wifi modem I could find, with no data cap. But all mobile data is assymetrical and is geared for fast down/slow up. The only other way would be to find a friendly ISP and pay to do a rapid data dump upload, but good luck negotiating that with my sub-primary school Japanese language skills 🙂

Oh, one other thing: those 2 x 2TB drives also contain Carbon Copy Clones of OSX from my laptops internal SSD… so if I had a devestating drive crash or had to replace the SSD in my laptop while travelling, I have a CC OSX copy to reinstall back on to a new drive. That laptop wont be doing many backups if its internal drive died!

3 thoughts on “keep your data safe

  1. Guy

    Being a black belt mac & data whizz… i would be finding a fibre connection in Japan, then periodically upload SD cards or larger blocks to your own HAAR repository on Amazon S3. Then on return pull it down to your NAS, (you could run a sync on the NAS while you were overseas) meaning everything in your field recording folder got pulled down to the NAS while you were away. You get back open the NAS and its all there waiting to edit. Plus its home, its on Amazon, its with you on your Mac, and its on your backup which might by now be overkill. Relocating the data to your NAS is more convenient than leaving it to your return so why not break the chore down in blocks while your travelling.

    1. tim Post author

      I agree in theory, but that is a lot more work – for example even try doing that every day in Auckland, for 2 months… How much time do you want to spend travelling to/from wherever the fibre is? And remember the SD cards aren’t the master of all data – I’m capturing to multiple devices every day… I would have to be carrying one of the hard drives with me every day, which is far riskier imho

      When I got home relocating the data to my NAS was very easy – across network, happened while i slept 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *