Help! NAS for >20TB Storage + Cloud Sync for small business

[VtSoundman]

Hi Folks -

I have a business is not at the stage where I need to spend big money on IT/complex solutions/contracts, but it tetering on edge of disaster if I don't address work flow.

...edit: business that is NOT at the stage where I need to spend lots of money...

Here is the issue: image capture studio that moves tremendous amounts of data around for use by onsite employees (~5) and contractors (~20) around the world. The data sets these folks work on are typically 50-100 gigs - that has to be downloaded to the the local machines, modified/rendered & then re-synced with the server/cloud service.

For local storage, I need ~20TB of real-time space that can by synced with Google Drive services. (I have 12TB now, but will need addition 6TB in the next few months. I also never want to worry about space/expanding). Since the 750-1TB datasets come in quickly via RAID 1 SSD units, and a local Ethernet connection, the NAS needs to be relatively fast to handle the uploads, sync'ing/etc.

What NAS do you recommend for

1. Sync with Google Drive
2. Offer RAID (RAID 10 vs RAID 5/6...open to discussion)
3. Can perform automatic syncing to a 2nd redundant/passive server
4. Can perform 'automated' offline backups using USB/ESATA drives that are stored offsite
5. This is an all MAC office, so the ability to be a time machine server would be great too.
6. Has 2 or more 1GB Link aggregation (prefer 4 ports)

I was considering the 12-15 drive bay units that have expansion capability as spinning storage needs will grow. The QNAP expansion units connect by what appears to be a USB3.0 cable, so that is out - the SYNOLOGY boxes look better - something like the DS1815+...

Does anyone have experience with the SYNOLOGY expansion units? Am I better off buying additional NAS boxes instead?

If I go with DS1515+, there isn't much expansion room or ability to have a hot spare. The DS1815+ allows a hot spare + an SSD cache (I'm assuming this can utilize an SSD cache). I was also considering a DS2416+

For drives, I am considering the HGST 4TB NAS drives. I wanted to stay away from larger drives in the primary NAS to avoid long rebuild times.

I need some help with direction/architecture & components. Budget is not unlimited, but this data is my livelihood and that of ~30 people. IT is not my business, so it needs to be relatively straightforward & robust.

Help! Creativity would be much appreciated!

 

[coxhaus]

Aren’t you going to want hot swap on the drives so the system does not need to come down to replace bad drives? In the Windows world I think a couple of servers linked could perform this function. If you want somebody to call if something goes wrong then you need to buy them from a Vendor like Dell etc. You talk about big Ethernet connections but not how much data flow do you really need to be able to maintain?

I am old server guy and I am interested in what the NAS guys think. I will keep an eye on this thread.

 

[VtSoundman]

Yes...absolutely would need to pull drives to prevent downtime...hence the desire for hot swapping/hot spare. Looked like Synology has a hot spare feature.

Regarding BW : the issue is syncing among the local clients. The ~5 employees onsite arrange the datasets/stage the data for the technicians offsite - and I already have a managed switches to that can handle the Link Aggregation. The offsite folks will download from google drive - my internet upload rate is currently 10mbps peak...pretty slow. The download rate is much faster (avg ~50mbps) and the final product is released from the local office.

I am looking at adding a 2nd internet service and using a multi-wan router to do some load balancing to gain higher upload speeds...really depends on the number of connections google drive sync uses. Since the upload speeds are so poor in the local office, the remote users will download/move content around from Google Drive - make more sense?

 

[thiggins]

QNAP is relatively new to using expansion cabinets. Synology has been doing it for awhile and I think uses Infiniband connections for their higher priced models.

Either Synology or QNAP has the features you want including Time Machine.

Have you thought at all about 10GbE? Check out this series
http://www.smallnetbuilder.com/lanw...ions-of-a-10-gbe-network-newbie-part-1-basics

You might also contact Dennis Wood for input on his experience with QNAP handling video files.

Remember also drive capacity is a 6TB now and still growing. So you might not need an expansion cabinet. Also consider setting up multiple volumes and or multiple NASes. Expansion cabinets are pretty pricey!

 

[VtSoundman]

There will be no manipulation of video files (these are all stills) - so the only duty is file storage and movement. I noticed that non of the benchmarks address RAID 6 speeds - any idea of what sort of hit I might expect with some of the NAS machines?

The 10Gbe is an interesting thought - the clients all Apple products, and the cost for 10Gbe/thuderbolt adapters seem to be cost prohibitive. Also, I am not sure that any ready built NAS could handle the render IOs/thoughtput necessary to keep the CPUs at full throttle.

Here is a quick summary of drive costs (sources of drive costs are not optimized yet...just to provide some directionality). My experience has been generally good with the HGST NAS drives...Any coments would be welcome.

RE: expansion units - wow those things are expensive. Assuming the Google Drive sync software for the NAS would allow selective folder sync, looks I would be better off getting add'l NAS boxes.

Any comments on some of the larger NAS units? Which brands are preferred?
ASUSTOR & QNAP consistently rank higher on your lists. I am not adverse to spending a few more dollars on a higher end unit if it will give me bump in quality.

Anyone have experience using the Google Drive sync App?

 

[thiggins]

I suggested the 10GbE series, not because it is video, but for large file handling. It also has a lot of information about performance.

10GbE can be helpful between the NAS and switch instead of eating up Gigabit ports for aggregation.

Sorry, I have no RAID 6 performance information. RAID 10 is generally regarded as the best for combination of speed and robustness (two drives can fail).

This article has more on QNAP Google Drive sync.

Keep in mind our Charts are based only on performance. Features, reliability, support are not reflected.

QNAP and Synology have been around for awhile. ASUSTOR is relatively new. All have U.S. based support. But I have not tried any of them.

 

[coxhaus]

I am not a NAS guy but did Microsoft servers at some point in my career. Since you are making money and this needs to be up otherwise you don’t make money. I would put together a system with hot standby, failover, etc. What happens with a RAID system when a hard drive fails and you replace the failed hard drive, the system needs to rebuild. I would setup the system with failover so when a hard drive fails you can force the system to the other failover device and replace the failed hard drive and let the system rebuild. Once the system is back transfer back to the primary system as this also works to test your failover. This also provides you redundancy in case of hardware failure.

I hope this gives you some direction.

 

[VtSoundman]

RE: Failover & hot spares...this is exactly where I am headed.

I am not convinced the NAS appliances/companies boxes are ready for this type of use. The base cost for a suitable SMB/enterprise NAS appliance isn't too far from a nicely equipped file server from a reputable company. Since this is the company's first foray into responsible data management, I'm headed down the path of getting a nice server today and purchasing a 2nd fail over/archive/slower server in about 8-10 months.

I am planning on a 3.5 tier system
1. Primary Server - nice high speed machine from a reputable company familiar with digital arts creation - and something that can combat bit rot.

2. Cloud Service - sync to contractors all over the place

2.5 Cloud Service Backup

3. Onsite backup using NAS

 

[sfx2000]

Well, looking at your requirements and environment, the right solution would have been an Xsan installation, but Apple has discontinued that...

Quantum has the StorNext 5 SAN appliances, and they're the migration path for Xsan... it's plug and play with the XSan client SW, and scales very well...

On the Macs, Promise has the SANLink 10Gig Thunderbolt adapters, for about $600 each...

A desktop NAS isn't really going to meet your needs, so spend wisely... Storage Area Network is going to be a much more scalable solution for you...

Building everything around 10G is a very good idea, and will give you room to grow as the business evolves...

 

[coxhaus]

Nothing wrong with a SAN. They tend to be a little more spendy than a server but they do scale higher.