Best NAS for image compositing and editing?

[dddjef]

Hi everyone,
First, thanks for the author of this site, really helpful and well done.

I'm about to buy a NAS for my company. We mainly work for HD films. This films are broken in shots, which are broken in image sequence (24 files of nearly 1Mo per second) So we need good transfer rated. And we often work collaboratively, so I was wondering if having 2 network cards will remove a bottleneck...

What is important for us:
1. Better transfer speed for READING little files (nearly 1 mo)
2. Better Multi users transfer speed
3. 4 To minimum

What would be cool:
1. Installing a linux app managing our renderfarm
2. Easy management interface

What doesn't matter
1. Write speed
2. Noise
3. Raid recovery (I will rely on daily backup for that)

From what I read, I hesitate between Thecus 5200 PRO (dual network card or switch.. I' don't know), the 4100 pro seems pretty fast, and the synology DS 508 interface looks great...
Thanks a lot for your answers!

 

[mgraves]

My day job is in the broadcast business where I see the kind of files you describe routinely. Throughput is paramount.

You might consider one of the newer NAS units that are more than 4 drives. Thecus just started shipping a 7 drive unit with dual NICs. The more drives the greater the total theoretical throughput.

Michael

 

[beisser]

My day job is in the broadcast business where I see the kind of files you describe routinely. Throughput is paramount.

You might consider one of the newer NAS units that are more than 4 drives. Thecus just started shipping a 7 drive unit with dual NICs. The more drives the greater the total theoretical throughput.

Michael

its actually single NIC.

 

[dddjef]

Thanks mgraves, I thought, theorically, when you have more than 4 (good) drives in Raid 0, the single NIC becomes the bottleneck.

beisser: the news tells "There are two PCI-e gigabit Ethernet LAN interfaces supporting 802.3ad link aggregation, load balancing and fail over capability." about the N7700, this is a mistake?

With NAS under Linux, can I install a software to manage my renderfarm? or it's totally close to proprietary softwares?

 

[mgraves]

I'm not certain exactly. An acquaintance got one of the first units shipped in the US and blogged about it here:

http://deancollinsblog.blogspot.com/2008/10/thecus-n7700.html

Nothing specific technically but some decent pics. Then the mfgr has details as well.

Michael

 

[dddjef]

Seems to have a LAN and WAN port, it's OK.

Have you tried both configs? with and without load balancing? bandwith is really twice?

 

[beisser]

Thanks mgraves, I thought, theorically, when you have more than 4 (good) drives in Raid 0, the single NIC becomes the bottleneck.

beisser: the news tells "There are two PCI-e gigabit Ethernet LAN interfaces supporting 802.3ad link aggregation, load balancing and fail over capability." about the N7700, this is a mistake?

With NAS under Linux, can I install a software to manage my renderfarm? or it's totally close to proprietary softwares?

what i was getting at was the n7700. i just had a look at the specs really say it has 2 lan ports. i could swear it said 1 lan port the last time i looked

so its a mistake yes, but on my part

 

[Dennis Wood]

DDD, we've been doing extensive testing on video editing application for NAS as they pertain specifically to throughput, link aggregation, sustained large file reads/writes and editing directly over gigabit using a variety of workstation builds, and two QNAP TS509 units.

Dual load balancing NICS do zero to improve a single channel's throughput...however if you have two or more workstations loading the unit..big difference. If you can post up your NLE software, workstation OS and configuration (RAID) average file size etc., I could give you an answer based on about 300 tests, and two months of research Our comprehensive article on the subject is in progress. You'll want to look over my posts on the QNAP TS509 as they pertain to link aggregation and the update to 4GB RAM. We've been using a 5.3 GB test set that is based on a SONY EX1 SXS (1080 24P HQ) card image with file sizes from 1.8GB to a few K. Also are you using proxy, uncompressed, HDV, DVCproHD, 2k, 4K etc. etc.?

You may want to look at one of these units in RAID0 mode with 5 drives. 85 to 95MB/s sustained large file reads/writes are doable providing your workstations are using RAID0. After performing a RAM upgrade on the unit to 4GB, files under that size will write to the NAS at just over 100MB/s. Consider buying two of them and syncing for redundancy. No joy on the net render concept unless you are prepared to get under the hood and mess with Linux. If this is your goal, just build your own box and use something like Ubuntu. Either way, you'll need an 802.3ad fully capable gigabit switch and a NAS unit like the 509 which supports load balancing over dual NICs. We've tested 2 16 port switches, and a $110 fanless Netgear model (8 port, web managed) too. Feel free to give a call during the day (contact info on the web site) if you'd like to chat.

Cheers,

 

[dddjef]

Thanks for your answer Dennis,
Our footage are mainly images sequences, in example, 500 files of 1 mo per shot. We don't use proxy, but I think I should integrate this concept to the pipeline soon.
By the way, upgrading RAM shouldn't affect my transfers, since we work on tiny files?

I don't understand why my workstation should use RaidO, isn't this a NAS setting?

Thanks for the advice about switch.

 

[Dennis Wood]

D, the workstation's hard drive will the limiting factor after a file exceeds the size of about 50% of the workstation's RAM. If you want to sustain file reads and writes beyond ~60MB/s from the NAS, then you'll need RAID 0 on the workstation to accept the higher data rate. With a busy NAS unit, more RAM means more files can be cached..and these (at least on the TS509) will be fed to your workstations at speeds in excess of 100MB/s. I'm not sure what "mo" refers to but it that means "MB" (megabtye) then 500 files at 1MB each shouldn't be a problem for the NAS units. What you will run into is some network and OS overhead because these are 500 discrete files..not one large file.

Just as an example though, we've tested Premiere Pro CS3 in the timeline with 3 HDV (35Mbit variable bit rate) files which resided on the NAS....and it works quite well. Keep in mind that very little load testing has been published on any NAS units...and for this reason I've posted up results from our NAS units based on multi-workstation loading. I've also been pushing the QNAP folks pretty hard to do some SAMBA tuning to improve this area..and they have.

At some point in the very near future, we hope to have complete solutions for folks just like you

 

[dddjef]

OK Dennis,
I've just ordered a TS509 and five hitachi 1TB. (you should ask for Qnap stock options, lol)
I know it's not done for that, but, I wondered what would happen if I link a workstation to the TS509 with a eSata cable... would it be magic, and the workstation recognize a ultrafast external disk, or would it going to burn... did you try?

 

[Dennis Wood]

No joy there. The esata port is only used to connect external drives. You'll find it very useful for backing up large files to an eSATA drive enclosure. We use the fairly inexpensive 1TB WD "green" drives in $45 eSATA enclosures for this.