Need help with storage and backup strategy

[imahawki]

First post here but I've been reading through a lot of reviews, etc. Great site!

I am undergoing a project to put all of my DVD library (~700 disks) into iTunes so I can stream them to my Apple TVs and put some on my iPad when I travel for work, etc. I think this is around 1.5TB right now

In addition, I am an "advance amateur" photographer and have tens of thousands of photographs consuming many GBs (thought not nearly as much as the movies). I actually don't have an exact idea, this is somewhere between 60GB and 200GB.

I consider the photographs to be absolutely mission critical. I CAN NOT lose them. I would be absolutely devastated. I consider the movies to be a very close second. I could reproduce the content frankly but there are HUNDREDS of hours of work into ripping and encoding them.

I have two problems.
1) In general, I am running out of storage. I currently have a 3TB Seagate FW800 drive and have used 2TB of it. The last TB will last me a while, but not super long and I want have a plan in place before I'm completely full.

2) I am in even worse shape for backups. My backup drive is only 2TB. So basically between the photos, movies, and my iMac system backup, my drive is full and can't even complete a backup right now.

I signed up for crashplan and started a backup. Its telling me I have nearly 50 days remaining which sounds about right. I could let this keep going or I could pay $125 and seed the backup. *See note 2 below*

If I relied solely on crashplan as my offsite backup, this would ALSO free up my additional 2TB drives for local storage (I actually have 2 because I currently rotate my backups offsite). But it means I don't have full control of my backups which worries me a little, though not overly.

Then ideally if I can make crashplan work for me for a backup, I would get a 4-5 bay NAS. But I'm really worried about this because if I finish the crashplan backup and them get a NAS, is Crashplan going to recognize it as the same data after it changes locations and is no longer mapped the same on my computer (network share vs. DAS) or is it going to make me re-backup 1.6TB AGAIN???

Additional thoughts:
1) I'm mildly concerned if I let the crashplan backup finish online that my ISP will shut me off eventually.
2) Crashplan says they send a 1.5TB drive for your seed, which wouldn't even cover all of my data.
3) As my data grows, if I continue with the crashplan option, I could have 4TB, 5, 6, who knows. Are there going to be significant additional fees from crashplan if I have to do a mail-me-a-hard-drive restore of 6TB???

 

[stevech]

IMO
With TB of data to be backed up, and lots more to be added, with slowwww internet uplink speeds we all have ... I suggest simply using an external eSATA or USB3 drive in combination with some constant backup software that automatically makes the external match the internal. I do that with "SecondCopy" ($30) and have done so for years. It can do the copy operations immediately or at a late-hour time of your choice, and can be told which folders to watch. There are other such programs, and I've tried and rejected most all others. Acronis can do but it's not simple and obvious in this role, rather than a partition backup.

I don't use a RAID1 because if the file system gets corrupted on one, it's corrupted on both. An external drive with a separate file system doesn't have this issue. Same with RAID5. "RAID is not a backup", they say.

Also the external drive can be unplugged and taken off-site for safe keeping. Another drive can replace it.

I just starting using a USB3 drive dock (drop in a drive, pluck it out). Rather than a full-up enclosure. I can just drive-swap two drives now and then, so off-site is kept current and I don't try to do many GB on my 1Mbps uplink cable modem.

 

[imahawki]

IMO
With TB of data to be backed up, and lots more to be added, with slowwww internet uplink speeds we all have ... I suggest simply using an external eSATA or USB3 drive in combination with some constant backup software that automatically makes the external match the internal. I do that with "SecondCopy" ($30) and have done so for years. It can do the copy operations immediately or at a late-hour time of your choice, and can be told which folders to watch. There are other such programs, and I've tried and rejected most all others. Acronis can do but it's not simple and obvious in this role, rather than a partition backup.

I don't use a RAID1 because if the file system gets corrupted on one, it's corrupted on both. An external drive with a separate file system doesn't have this issue. Same with RAID5. "RAID is not a backup", they say.

Also the external drive can be unplugged and taken off-site for safe keeping. Another drive can replace it.

I just starting using a USB3 drive dock (drop in a drive, pluck it out). Rather than a full-up enclosure. I can just drive-swap two drives now and then, so off-site is kept current and I don't try to do many GB on my 1Mbps uplink cable modem.

How do I backup a data set that is larger than the largest external drives. I.e. once I get to 4TB it's going to get very hard to backup. Adding multiple backup drives will add a ton of complexity as I'll have to modify my backup job for destination disk A vs B.

 

[thiggins]

There is no magic solution. It all comes down to cost / security tradeoffs.

I would not backup your ripped DVDs. The DVDs themselves are your backup. Plus the data is not unique.

Second, I'd look at BackBlaze vs. Crashplan if you don't want to worry about excess data charges.

If your photos are "absolutely mission critical", then you already should have seeded the backup with whichever capacity is offered. I'm sure you can pay another $125 and fill a second drive.

Most cloud backup services won't recognize mapped drives. So if you want to use a cloud service with a NAS, you are limited to the services that the NASes support. You can get around this by running a backup from NAS to a shared folder on a Mac or Windows machine that is then backed up by the cloud backup agent.

Some backup programs will automatically span drives. If I remember correctly, Retrospect does. But it stores data in a proprietary format.

 

[imahawki]

There is no magic solution. It all comes down to cost / security tradeoffs.

I would not backup your ripped DVDs. The DVDs themselves are your backup. Plus the data is not unique.

Second, I'd look at BackBlaze vs. Crashplan if you don't want to worry about excess data charges.

If your photos are "absolutely mission critical", then you already should have seeded the backup with whichever capacity is offered. I'm sure you can pay another $125 and fill a second drive.

Most cloud backup services won't recognize mapped drives. So if you want to use a cloud service with a NAS, you are limited to the services that the NASes support. You can get around this by running a backup from NAS to a shared folder on a Mac or Windows machine that is then backed up by the cloud backup agent.

Some backup programs will automatically span drives. If I remember correctly, Retrospect does. But it stores data in a proprietary format.

Why would backblaze solve my excess data charges concern?

Crashplan does offer a work around for backing up a NAS but honestly I have so many concerns around this overall that it might not work.

I saw you responded to my other thread. Bear in mind its all related... its all the same problem I'm trying to solve.

As far as the DVDs, I understand what you mean, its not unique data. But the flip side is I have HUNDREDS of hours in ripping and encoding involved in this. If I lost the data I would probably just give up, making the entire effort a HUGE waste of time and expense.

 

[thiggins]

Why would backblaze solve my excess data charges concern?

Flat fee with no file size or total backup space limit.

As far as the DVDs, I understand what you mean, its not unique data. But the flip side is I have HUNDREDS of hours in ripping and encoding involved in this. If I lost the data I would probably just give up, making the entire effort a HUGE waste of time and expense.

As I said, it's a cost/benefit tradeoff. The decision is up to you.

 

[imahawki]

Crashplan has no limit based on size either. Their only extra charges are if you want to seed or restore via disk because you have a lot of data.

As far as the backup costs, I understand, I'm just looking for options though. I mean, I could have three NASes, one live, one backup onsite, one backup offsite that I rotate. But that's going to be pretty freaking excessive, right. That could get to $6,000 pretty quickly Or, someone presented an option to me today that I could get a ReadyNAS Ultra 6 and setup two pools of three drives and have one be the live data and the other be the backup. Then I'd only need ONE NAS and 9 drives. But, I don't know if the device will actually support that setup. If anyone knows that would be great to know!

 

[thiggins]

You never, never, never want primary store and backup on the same device. That's not backup.

 

[imahawki]

You never, never, never want primary store and backup on the same device. That's not backup.

What if my backup isn't my only backup? (i.e. as I implied, I would have 9 drives for a 6 drive unit).

 

[travisco_nabisco]

What if my backup isn't my only backup? (i.e. as I implied, I would have 9 drives for a 6 drive unit).

If you are going to look at a setup where you have a 6 drive unit and 9 drives to swap out, you might be better off to invest in 2 4-bay NASes and have them in separate areas of the house. Then have one for live data, and the other one mirrored as a backup. It isn't an ideal solution, you are still vulnerable to fire, and perhaps theft if the NASes are easily accessible, but it does provide two distinct copies of the data. If one NAS has the power supply die, you still have everything available.

 

[imahawki]

If you are going to look at a setup where you have a 6 drive unit and 9 drives to swap out, you might be better off to invest in 2 4-bay NASes and have them in separate areas of the house. Then have one for live data, and the other one mirrored as a backup. It isn't an ideal solution, you are still vulnerable to fire, and perhaps theft if the NASes are easily accessible, but it does provide two distinct copies of the data. If one NAS has the power supply die, you still have everything available.

Good call. It might make just as much sense as you say.