Backup Strategy - please critique

Greetings! Serious photography enthusiast here whose current storage solution is fading away with all the digital images I have been (and will be) creating. As such, I am adding more storage, while at the same time, implementing a new backup strategy. My current setup is very simple. One user (myself), running a 2015 edition MacBook Pro (1 TB drive). Most of my 2015 images are there. I also have a 1 TB external HDD where all of my 2014 (and earlier images reside). This drive is also partitioned for Time Machine backups. Finally, I have an Amazon S3 account where I also backup (via Jungle Disk) to act as my "offsite" backup. That is it. Here is what I am thinking about ... please let me know your thoughts:

LaCie3TB d2 Thunderbolt 2 External Hard Drive:
This would be my main image storage drive. I would plan to get most, if not all, of my images off my MacBook Pro, and onto this drive

LaCie6TB d2 Thunderbolt 2 External Hard Drive
This would be my main backup storage drive. I would use Apple’s Time Machine app to backup my photo storage LaCie3TB drive above, plus the drive inside my computer, plus my other 1 TB HDD that I would bring on the road with my when I go on the road for some travel photography. Time Machine would keep backing up until it fills the entire 6TB drive, then it deletes the oldest backup - which would be old enough at that point to not care about it. If the main 3TB photo drive should ever get corrupted, all I would have to do is reformat it, enter Time Machine, and place the backed up folders on the drive from this drive. Super simple, I think ...

In lieu of cloud storage, I would put this 6TB drive into a fireproof safe (when I travel) in my closet far away from my office. In the future, as online storage begins to get faster, that would inevitably become another viable option. But, trying to back up 6TB to an online server daily would take a long time to accomplish that large of a data transfer in our current market. But, I could be wrong ...

Thoughts/comments??? Thanks!

Automate your backup.
You seemed to say that you have ONE copy of some images. Danger.

I'd dump S3 (too expensive). Get a 2-bay NAS like Synology or QNAP. Let Apple Time Machine backup to the NAS. These two NASes can do that. And Synology has their own Time Machine for all comptuers, called Time Backup (with file versioning). I don't use RAID. Let each drive be its own volume.
Backup the NAS to USB3. Keep the USB3 out of sight of burglars.

LaCie - I'd avoid that kick-the-can brand. Definitely.

I'd avoid bleeding edge 6TB drives. Too much in one place, for a home user.

Clould storage is not very practical due to your ISP upload speeds. Mine's typical: 50Mbps down, 5Mbps up. Up is the problem. I do use ADrive (due to their low price) for non-private info and sharing, like family photos.

Thanks stevech. But, when you say "Backup the NAS to USB3" -- what is USB3? I assume your are not talking about the port on the side of my MacBook Pro -- right?

He means by hanging a USB3 hard drive/enclosure off the USB3 port on a NAS.

Are you a professional photographer? IE you derive an income from your images? If so, in the field you need at least a back-up solution there. It could be storing the images on your MacBook as well as on an external HDD. If that doesn't provide enough space to store them with a redundant copy, then you need two external HDDs. ALWAYS back-up BEFORE you erase your memory card. I'd suggest when in the field shooting, if something is staying at one site, take the other with you. It would suck if you are leaving your MacBook and that HDD in your hotel room or something and have your room broken in to and them both stolen. A small HDD easily fits in a day bag/camera bag. If out and about and your gear is stolen/forgotten, then you should have the MacBook/other HDD back in your room.

At home, dump your images to a 2-bay (or 4 or whatever) NAS. Have the NAS set to back-up these images to an external USB3 drive/enclosure on a pretty frequent basis, like hourly. Or do it manually after you make a dump. Verify that the images are on the NAS as WELL as the back-up before you remove them from your travel HDDs or MacBook (or leave them on them as extra back-up copies).

You can always do only some for cloud storage. If a professional photographer, I'd suggest any final files you produce get copied to a separate folder in your storage and only the finals get backed up to cloud storage and not the proofs/extra files. I am not a professional photographer, but typically if I shoot 500 images at an event, I might end up with 70-150 final picks that I'll edit up. It is easy enough to drop those final picks in a separate folder, or have a back-up program back-up only the JPEGs/or edited RAW/DNG/ORF files to cloud storage. Sure, if you lose it all, you'd lose a lot of stuff. However, you'd at least have the images that (at the time) you deemed the most valuable pictures.

It is the equivalent of running out of the burning house with the wedding album and the baby book. A hell of a lot better than losing it all.

If that is still too difficult/expensive, then I suggest getting a safe deposit box at a bank (preferably very close to your house). Then I'd do one of two things. I'd either keep a HDD in that safe deposit box that you go there, get it, take it home and back everything up to it and then take it back to the safe deposit box on some set schedule. Maybe every 6 months, maybe ever 3 months. Maybe monthly.

Or for super redundancy, and frankly to save time. have 2 HDDs. Keep backing up to the one at home and then ever 1-3 months, swing by the bank and swap the most up-to-date backup with the one in the safe deposit box and then bring the old one up-to-date and continuously backup to that one.

Is this expensive? Not really. If your photography is very important to you or especially if you derive an income from it, what does it cost?

If you need maybe 4TB of storage, you are out maybe $500-600 for a 2 disk NAS with a single 4TB drive in it, a USB3 enclosure with a 4TB drive in it (leaving room to add a 2nd drive in the NAS and the enclosure later if you need more space). The spare HDDs might cost you $50-100 for the spare travel HDD (of 1-2TB of capacity). The two HDDs that you are swapping in your safe deposit box might be another $200-250. All up, less than $1000.

Thanks! No, I am not a professional photographer and do not rely on this activity to support me in life. For me, I'm purely an avid weekend hobbyist that shoots a lot -- every weekend! Maybe someday I will go professional -- but not anytime soon. That being said, how would your strategy above potentially change?

I probably wouldn't do the offsite back-up in a safe deposit box, but otherwise I'd do most of the rest.

My current back-up strategy is, on the road...

I load images to whatever portable computer I have with me (tablet or laptop), leaving the images on the SD card. Upon arriving home, I'll load the images to either my server or desktop, depending on what I am doing at the time (either from the SD card or from my tablet/laptop). About a week or so later I'll format the SD card and probably delete off my laptop/tablet.

The reason I wait a week is I have a weekly back-up job that runs on my server that backs up everything to/from my desktop and server. Perfect mirror. So after a week (Saturday nights) it is all backed up and I have at least 2 copies. I try monthly, or a little more frequently, to back-up my desktop to a 5TB external USB3 hard drive that I keep disconnected except when doing back-ups.

At some point I plan to tear down a shed in my backyard and build a 1 car garage and workshop out there. When that happens, I'll load up a hard drive out there, probably hanging off of something like an Intel NUC running back-up server duties.

I want at least 2 copies of all data at all times and preferably 3 copies. I'd prefer to have an offsite, or at least semi-offsite (that shed is 80+ft from my house, so a fire is pretty friggen unlikely to spread and they are at different elevations and I am not anywhere near a flood plain. In fact I am about 120ft above the river about a mile away). If I ever become paranoid enough, my bank is opening a branch about 3 miles from my house, more or less on the way too/from work for me. If they have safe deposit boxes there, I plan to move mine to that branch as it stands (not that I really keep much of anything in it). I might get around to doing my HDD rotation strategy at that point. For $200, it is a lot of peace of mind.

Ignoring the sentimental value of all of my pictures, I have a lot of movies that I have ripped from my collection. If something happened, like my house burning down, and I lost my DVD/BR collection as well as some how trashing all of my on-site HDDs, even if insurance compensated me fully, it still represents something on the order of probably 50-80 hours of actual work on my part to rip my entire collection. Not counting the probably 1000hrs of unattended computer time transcoding.

Certainly worth $200. The risk versus security isn't worth the time it would take me to go to my current branch where I have a safe deposit box (about a 40 minute drive).

OK ... some follow-ups. You state above:
At home, dump your images to a 2-bay (or 4 or whatever) NAS. Have the NAS set to back-up these images to an external USB3 drive/enclosure on a pretty frequent basis, like hourly. Or do it manually after you make a dump. Verify that the images are on the NAS as WELL as the back-up before you remove them from your travel HDDs or MacBook (or leave them on them as extra back-up copies).
1) Are you connecting your laptop/desktop to the NAS via Thunderbolt?
2) When backing up your NAS to an external USB3, could you also use Thunderbolt if the backup drive had Thunderbolt capability (as well as your NAS)?
3) Are you also backing up your laptop/desktop to this USB3 as well as the NAS?
4) If on a Mac, are you using Time Machine on your laptop to control the backups to the USB3? Or, are you using the NAS software (eg. Synology) to do this? How do you backup what is on your Mac laptop/desktop?

Thanks!!!

wmarson said:

Thanks stevech. But, when you say "Backup the NAS to USB3" -- what is USB3? I assume your are not talking about the port on the side of my MacBook Pro -- right?

Click to expand...

Yes, USB3. No, not your MAC's USB3.
The NAS has USB3 ports and you plug in 2-6TB disk drives wihin USB3 enclosures. These are non-Apple and thus relatively inexpensive and widely used.

Apple's freakish Thunderthing can't be used in NASes as Apple keeps it trade secret.
NAS automation backs up what you select as important (e.g., don't backup drive image backups from PCs). Maybe 2+ USB3 drives on the NAS if you need.
Also Synology's Time Backup will copy out all revised files, and has a nice GUI way to browse the stored versions by age.

Well, no, it is Intel's, not Apples. It just isn't used on NAS because it is expensive and the 5Gbps that USB3 provides is more than enough for your typical 2-4 bay HDD enclosure, why go with 20Gbps that Thunderbolt supplies? Especially when it costs probably $10-20 in parts and a $20-40 cable instead of probably a $1-2 USB3 controller and you can supply or have the user buy a $2 cable.

It sounds like you aren't very familiar with networking or computers in general...

Connecting to the NAS is done through network cables, not Thunderbolt.

My home setup I don't have a NAS, I have a server, but functionally it is very similar to a NAS. Server is connected to my core switch, desktop is connected to the same core switch. Laptop/tablet are sometimes connected to the core switch through a wired connection, sometimes I just transfer things wirelessly through one of my access points, that are connected to my core switch.

For a NAS, I'd probably suggest using the software that comes with the NAS to perform back-up jobs. Of course you can also do this manually, especially if you don't leave your laptop on all the time.

I run Windows on my server and I have a scheduled task the runs in the wee hours of Sunday morning (I call it Saturday night, but it is really 12:45am Sunday mornings) that backs up my desktop as well as mirrors my picture folder between the server and desktop as I sometimes drop images right on my server from my tablet/laptop, but I never transfer applications, movies, music, etc. to my server from anything other than my desktop...so my desktop is the authoritative source to get those materials from.

I just manually plug in my external hard drive every few weeks and back-up from my desktop and then disconnect the drive. My desktop has a copy of everything that my server does, so I just connect my USB3 drive to my desktop for the back-up.

In my suggested scenario, your external hard drive would probably be connected to the NAS full time and the NAS would preform a back-up job daily/weekly that would pull a copy of everything from the NAS to drop it on the hard drive. If a bit paranoid like me, I'd only connect the external hard drive every once in awhile for the back-up and then disconnect it (less chance of a power surge taking everything out, as well as less change of getting a virus that destroys everything).

For my Mac's, I use Time Machine with external USB/Firewire drives. I then image those drives weekly and write those images to my NAS.

The NAS is backed up to it's own external USB enclosure.

For this poster, I think a NAS may be just too much to handle.

Could be, yes. But I've f0und that most personal comptuer users don't use discipline and automation for backups. Or they think they have a backup but it doesn't meet the 3-2-1 concept and they get burned.
Just recently a family friend bemoaned loss of a decade of photos. Drive crashed inside Mac. She thought that it was OK as she copied them from camera to Mac. But over time, that Mac disk became the only copy, as camera cards changed, cameras were replaced, etc.

I has a somewhat similar experience with a customer. I had upgraded their computer many times and each time I asked them to at least buy a backup drive to copy the data before I worked on the system. They politely said no each time. The last time I worked on that computer, I had upgraded the OS to Windows 7, had put 4GB of RAM (up from 1GB) and after they saw what an SSD could do to a system, I installed one for them too. The old data was copied from the old drive (which they refused to buy a $5 external enclosure for) and was promptly thrown out (I found out about this afterwards). They used the system for a few weeks and were really enjoying its nimbleness.

Then, it was stolen.

Many years of important documents, images and the only copies of pictures from their grandkids, graduations etc.

Discipline was not their downfall here. It was simply believing that since they never had a failure before, they would save the few dollars and hope they didn't have a failure in the future.

Theft (although it was one of the things I mentioned to them for a reason for a backup or two) was never even on their radar of possible catastrophes with their data.

Since then, they have not only made a 180 degree turn around but make sure others are taking steps to avoid the loss they faced too. One day, this will not just be common knowledge but something everyone acts on too. Automated or not.

stevech said:

Could be, yes. But I've f0und that most personal comptuer users don't use discipline and automation for backups. Or they think they have a backup but it doesn't meet the 3-2-1 concept and they get burned.
Just recently a family friend bemoaned loss of a decade of photos. Drive crashed inside Mac. She thought that it was OK as she copied them from camera to Mac. But over time, that Mac disk became the only copy, as camera cards changed, cameras were replaced, etc.

Click to expand...

I doubt it'll be common. Why? Because the "computer age" has been around for well over 20 years. Heck, the majority of at least American's have probably had a household computer for roughly that long.

Among friends, neighbors, etc. I would guess probably less than 50% of them keep any kind of backups. Among ones who have been bitten by an accidently deleted critical file, theft, crashed drive, fire, etc., I'd guess that the back-up rate is closer to 80%, but it is still no where near 100%, even among the folks I know who have been bitten.

It is work. It is extra expense and all too many believe they are special butterflies and the laws of averages do not apply to them. This will ALWAYS be the case.

For maybe 10 years I've been using a 2 tray RAID 1 enclosure connected to, and shared out by, an older Mac I invariably have lying around (currently a 2009 Mac mini), which means it could be a Time Machine target. I have 3 trays, and 3 matching drives. Once a week, I unmount the RAID 1 volume, remove a tray, take it to safe offsite location, bring the tray from the safe offsite location home, insert it into the tray slot, and let the RAID rebuild the volume from the tray / disk that stayed in the enclosure.

I've considered but then shied away from NASs because I don't trust that the disks are formatted in such a way that the data will be recoverable unless it is in one of the NAS enclosures made by the NAS manufacturer. For that reason I prefer to use a small computer running a "normal" desktop OS and have the shared volume formatted with a well-known vanilla file system format.

The ONLY problem with my system that I've had is that if I were to simply pull out one of the trays, the enclosure insists on behaving as though it has had the cable disconnected. Disk removal or failure should be the same with a RAID 1 - if it is "hot swappable" then don't act like I've unplugged the cable. I suppose the disk caches do need to flush, so perhaps a hardware button to unmount the volume would be best. But I prefer to run this headless.

Most of the consumer NAS boxes use EXT3/4 filesystems and unless you're running RAID, recovering via Windows or OS X is pretty easy, even if it takes a Linux VM or Live CD to do it.

For me I am running NAS4Free with a 3 drives mirrored meaning I can lose two drives without losing data. Once a month I pull one drive, bring it to work, then take last month's drive home and rebuild the ZFS RaidZ2.

This gives me safe off site storage of my cold backup. I am covered for fire, flood, theft, etc. Of course if I drop the drive while in transit my backup is hosed.

If you don't have storage at work perhaps a family member or friend would let you have a drawer.

One humorous note: A friend told me he mailed dvds to a family member for backup. One day he asked for a dvd but was shocked to find out the family was discarding the dvd's!

lo pro joe said:

For maybe 10 years I've been using a 2 tray RAID 1 enclosure connected to, and shared out by, an older Mac I invariably have lying around (currently a 2009 Mac mini), which means it could be a Time Machine target. I have 3 trays, and 3 matching drives. Once a week, I unmount the RAID 1 volume, remove a tray, take it to safe offsite location, bring the tray from the safe offsite location home, insert it into the tray slot, and let the RAID rebuild the volume from the tray / disk that stayed in the enclosure.

Click to expand...

RAID is not a backup solution. RAID-1 provides redundancy so that if a single disk fails you shouldn't need to restore from backup, but you should still have a backup (e.g. to a USB disk) anyway for obvious reasons. What you are doing is not what RAID is designed for and could lead to complete data loss. SATA connectors are designed for a limited number of pulls and if you overuse them you may damage them. If you power down, remove a disk and damage it without realising it through electro-static discharge, and the disk that remains in the device is failing unbeknownst to you and you put the third disk in the device and it is wiped and the failing disk fails in the middle of the resulting rebuild you're in trouble. If you remove a disk while the device is on and expect to use the removed disk as your backup that's even worse and more likely to lead to trouble. Have a look at this post: http://www.snbforums.com/threads/netgear-kicks-loose-six-bay-smb-nases.580/page-3#post-6580

lo pro joe said:

The ONLY problem with my system that I've had is that if I were to simply pull out one of the trays, the enclosure insists on behaving as though it has had the cable disconnected. Disk removal or failure should be the same with a RAID 1 - if it is "hot swappable" then don't act like I've unplugged the cable. I suppose the disk caches do need to flush, so perhaps a hardware button to unmount the volume would be best. But I prefer to run this headless.

Click to expand...

"Hot swappable" is meant for replacing failed components or components you have permanently decided to no longer use (e.g. you want to use a higher capacity disk so you remove the old one, or a PSU fails in a hot-swappable dual-PSU system and you wish to replace the failed PSU). Your system (whatever it is) has no way of knowing that it was you who removed the disk (unless there is a process for you to tell it that you are doing this) so it needs to alert you as to what has happened.

mdgm said:

RAID is not a backup solution. RAID-1 provides redundancy so that if a single disk fails you shouldn't need to restore from backup, but you should still have a backup (e.g. to a USB disk) anyway for obvious reasons. What you are doing is not what RAID is designed for and could lead to complete data loss. SATA connectors are designed for a limited number of pulls and if you overuse them you may damage them. If you power down, remove a disk and damage it without realising it through electro-static discharge, and the disk that remains in the device is failing unbeknownst to you and you put the third disk in the device and it is wiped and the failing disk fails in the middle of the resulting rebuild you're in trouble. If you remove a disk while the device is on and expect to use the removed disk as your backup that's even worse and more likely to lead to trouble. Have a look at this post: http://www.snbforums.com/threads/netgear-kicks-loose-six-bay-smb-nases.580/page-3#post-6580

"Hot swappable" is meant for replacing failed components or components you have permanently decided to no longer use (e.g. you want to use a higher capacity disk so you remove the old one, or a PSU fails in a hot-swappable dual-PSU system and you wish to replace the failed PSU). Your system (whatever it is) has no way of knowing that it was you who removed the disk (unless there is a process for you to tell it that you are doing this) so it needs to alert you as to what has happened.

Click to expand...

Your comments gave me pause. I hadn't considered that the extra wear the rebuilds were imposing on the drives. And I now believe I should be glad that the RAID 1 enclosure will soft-disconnect from its host when a tray is removed. That forced me to manually unmount the RAID-1 volume or shut down the system before removing the tray. (The trays have their own connectors with the backplane in the enclosure, so there should be no direct wear on the SATA connectors themselves.) In the last year or so, I've shifted to using it in JBOD mode, with each disk being a discrete Time Machine target. I'm still not super comfortable using EXT3 / EXT4 for a file system simply because if the NAS dies, it isn't clear to me that I can connect an EXT3 or EXT4 disk on a Mac directly and perform a Time Machine restore. But some NASs seem to have a button or perhaps a schedule with which they back up the stored data to a USB disk (authenticated backup would be better) so perhaps I should take a look at a 2 bay NAS, now that it is 2015.

KenZ71 said:

For me I am running NAS4Free with a 3 drives mirrored meaning I can lose two drives without losing data. Once a month I pull one drive, bring it to work, then take last month's drive home and rebuild the ZFS RaidZ2.

This gives me safe off site storage of my cold backup. I am covered for fire, flood, theft, etc. Of course if I drop the drive while in transit my backup is hosed.

If you don't have storage at work perhaps a family member or friend would let you have a drawer.

One humorous note: A friend told me he mailed dvds to a family member for backup. One day he asked for a dvd but was shocked to find out the family was discarding the dvd's!

Click to expand...

This is quite similar to the old configuration I described earlier except the third mirror disk.

No RAID in my small NAS.