Best Practices For Storing Photos and Videos
Whether you are a photographer, film maker, blogger, vlogger, or just a casual picture-taker, you probably struggle with how to organize and store your digital files. I see a lot of people in Facebook groups I'm in looking for advice on how others are storing and backing up their content, so I thought I would put together a post on how we have storage system stuff set up.
I shot 45,502 photos in 2015; about 27,599 in 2016; and, since starting the vlog 7 months ago, we have captured over 2.5 TB of 4K video. That is A LOT of frigging pictures and videos. When storing files you not only have to worry about the storage itself, but you also have to think about the redundant back up, because lets face it, hard drives will fail. It's not a matter of IF it's WHEN. Trust me. So here is how we do it; I must warn you that it doesn't look pretty! This also isn't the only way to do these things but it's what we find the most efficient. If you have any other suggestions please feel free to leave them in the comments below. Ok, let's talk storage in general first and then we will chat about how we organize it all.
Storage
We have two big hard drive bays that hold our files. We have one that is 12 TB which holds photo and design files, and we have one that is 15TB strictly dedicated to video and video project files. Each of these hard drive bays holds 4 hard drives and they are set up in a RAID 5 array, which means 1/4 of the total drive capacity is dedicated to parity (data redundancy). So our 12 TB array is really made up of four 4 TB drives (16 TB), with 4 TB "lost" in redundancy. The entire four drive array can suffer one drive failure and still function (albeit slower than normal). We chose RAID 5 because it was a good happy medium between performance (increased speed due to writing to multiple drives simultaneously) and redundancy. One of the towers is setup using hardware RAID built into the tower itself and the other tower is setup as JBOD using SoftRAID to create the RAID 5 array through OSX.
For our Time Machine backup, we have an 8 bay tower set up in RAID 1+0. This means that four mirrored pairs (RAID 1) are created and the data is then striped across those mirrored pairs (RAID 0). This dedicates half of the total space to redundancy (theoretically allowing up to four drive failures as long as they're in different mirrored pairs) and allows for some performance increase as all four pairs are written to simultaneously.
Backups
Having multiple RAID arrays is a really great way to ensure you don't lose all of your files due to a drive failure, but what if your house burns down? Or you get robbed? Or there's a flood? Your RAID set up won't save your ass then. Negative, yes but THE TRUTH! Backing up your files is KEY, and, up until this year, we were doing three back ups, yes THREE! Excessive? I don't think so. So how do we do it?
Firstly, we have everything backing up with time machine into it's own separate RAID array. The nice thing about this is if a file gets corrupt or you save over a file only to find out you want to roll back to another version, you can go back into Time Machine and retrieve the file hopefully before the corruption has happened. Have you ever accidentally resized and saved over a PSD with multiple layers? I have, and Time Machine has saved me.
We do a third back up on a cloud based back up service called Backblaze. If you have a shit ton of files like us, you want to make sure you have a mighty fast internet connection with no capped data transfer limit. A service like Backblaze is excellent since it stores your files on a server outside of your home, so grilled cheesus forbid if there WAS a fire or you DID get robbed, at least your files would still be safe and sound on "the cloud". There are numerous companies that offer cloud based backup, but we ultimately chose Backblaze as it was the only one that we tried that allowed us to maximize the full upload bandwidth of our fiber optic connection. Other services were so slow that our initial backup would have taken something ridiculous like a year.
Hard Drive Organization
Let's take our video drive for example. In the root directory we have two folders; "video dump" and "video projects. Video dump is where we dump all of our cards; this folder is broken down in to folders per camera, within each is the year, and within that there is a folder for each day of the month where videos were shot. So if I know I shot something on the drone in April 2016, I just have to go drone - 2016 - April and search for the date. These folders are also all organized in our Lightroom catalog which I will get to in a minute.
Under the projects folder I have the following categories "Projects" and "YouTube Exports". The latter contains all of my videos exported for YouTube for quick access. Under projects we have things broken down into business, so "The Uncommon Law", "Bold Projects", and "Personal Projects". Within each are folders organizing each client/category of video, then folders within that include the video project files and anything that go along with it, like titles, audio files etc. All the raw footage files stay in the video dump folder.
In App Organization
A lot of people ask how we organize our photos and videos, and the answer is Adobe Lightroom. We have been using Lightroom for YEARS and it's the best program I have ever used in my life. Seriously. It's basically replaced photoshop for me. It's amazing for editing photos, but that's not what I want to talk about. I want to get into keeping your shit organized in the Lightroom Library. Every time we dump a card, we dump through Lightroom. Whether it's video or photo files, we choose the correct directory the files go in and dump away. It's a great place to have an overview of your footage, and also an amazing way to star and filter out images as well as organize them into collections for easy access. I have heard of a few other apps that are similar, like Photo Mechanic, I haven't tried it yet and not sure if it works with video files, but apparently it's a workflow game changer.
So that is how we handle our files and back ups. Let us know in the comments below, on Facebook, or on Twitter if you have any questions about our process! We would also love to know what works for you!
Here are some links to the hardrive systems we use:
Disclosure: By purchasing through our affiliate links, you are directly supporting our blog. So thank you!