Pulling ahead of the RAID 5 Pack: QNAP TS-509 Pro Reviewed

[thiggins]

The QNAP TS-509 Pro sets new performance highs. But at almost $900, ya gotta pay big to play. [article link]

 

[Unregistered]

QNAP state that they don't support Jumbo Frames becuase it is a limitation of the hardware NIC. They found that the trade off is that they get better performance without JF support using this NIC chipset rather than a slower one with JF support.

Also, they have said that they will support cameras in an upcoming firmware release due to customer requests for the feature.

 

[boerner]

Nice review, as always. I would be really curious to see how FreeNAS (both the 0.6 and 0.7 variant) run on it (if at all). Also would love to see how OpenSolaris would work.

 

[Dennis Wood]

Real world testing Intel vs Qnap for video editing.

Tim, first of all thanks for the all the review information and the inspiration for our own little test lab. We're working on building/reviewing an NAS system for video editing and archiving as it would seem that gigabit ethernet is almost there for a cost effective workflow. What we've found is that our results more or less match up with what you're seeing with a few important exceptions.

We've been doing extensive testing over the last week of just about every scenario possible in terms of writing and reading from RAID 0 and RAID 5 arrays not only on the same box, but over gigabit LAN as well between workstations and the NAS. The tests involve a batch file that incorporates the iozone tests, a five GB file set copy, and a ffmpeg encode splitting an MP4 file into two streams and writing them from the source drive/array to the target drive/array. Every scenario is tested in both directions, logged and time stamped at each step so that we could evaluate what was happening.

The Intel NAS we're testing (with 4 x 1TB Seagate ES2 drives) is showing real world transfer speeds of 60MB/s but we are only seeing this in the downstream direction from the NAS to a RAID 0 array (3 drives, Nvidia chipset). Once the results are tabulated we'll post them up. The ffmpeg encode results are very intriguing.

We've purchased the QNAP TS509 which should be here in a week or so and we'll compare. The Intel NAS has been load tested from multiple clients so we can compare to the QNAP using dual gigabit ports in the load balancing scenario. We're using a Gigabit smart switch that supports trunking (watered down 802.3ad) and have two workstations with dual gigabit onboard ports supporting teaming. Should be interesting

Cheers,
Dennis Wood

 

[thiggins]

Thanks for the info, Dennis. Are you saying that gigabit Ethernet is almost as good as direct attached?
You might want to look at the Fast Large File Transfers on Windows Shares? Jumbo Frames? article.

 

[8steve8]

correction

i happened to see a visitor from this page to a post i have with a song in it, seemed bizzare...

i think you have the wrong url:

ETHERNET PERformance:
http://www.alternativerecursion.info/?p=48

RAID Performance:
http://www.alternativerecursion.info/?p=31

 

[Dennis Wood]

Great read that. The link didn't work but google came up with this: http://www.alternativerecursion.info/?p=48 I have a G2S laptop with Vista SP1 which is not too impressive over the LAN...however the registry change suggested there is something we'll test.

In response to your question, yes, we're seeing 40MB/s transfer rates from the Intel NAS to a RAID 0 box (Asus P2N32 MB with dual gigabit LAN in team config) with Nvidia's version of RAID 0 striped across 3 drives. The only result that is better, is when writing from a RAID 5 to a RAID 0 on a workstation with the Intel based Asus P5W motherboard. That one uses an intel RAID chipset (we're using 3x300GB drives in RAID5) as well as a Jmicron used for the RAID 0 (two 320GB drives in RAID 0)

The measured test uses about 5.3GB consisting of 122 files taken directly from a Sony EX1 SxS card. These files are MP4 HD video files ranging in size from a 30MB to about 1 GB and the related metadata. The switch being used is a Dlink 1216 which supports jumbo frames, and all of our test workstations are config'd that way. I have no way of telling what the Intel NAS is using, but I'm assuming as per your review, it's not using jumbo frames. What's interesting is that one of the "cheap" workstations just uses a Dlink 530T card and we're measuring decent results. Here's a few numbers:

1.
Intel NAS to RAID 0 Workstation (Asus P5W, C2Duo, onboard Intel RAID 0, 2x300GB drives)
59.5MB/s file transfer, and ffmpeg encode at 32MB/s

RAID 0 Workstation (Asus P5W, C2Duo, onboard Intel RAID 0, 2x300GB drives) to Intel NAS
47.9MB/s file transfer, and ffmpeg encode at 21MB/s

2.
Single Drive workstation to Intel NAS
26MB/s file transfer, and ffmpeg encode at 18.7MB/s

Intel NAS to Single Drive Workstation
31MB/s file transfer, and ffmpeg encode at 19MB/s

3.
Intel RAID 5 to RAID 0 on same Workstation (Asus P5W, C2Duo, onboard Intel RAID 0, 2x300GB drives)
68MB/s file transfer, and ffmpeg encode at 51MB/s

Intel RAID 0 to RAID 5 on same Workstation (Asus P5W, C2Duo, onboard Intel RAID 0, 2x300GB drives)
60MB/s file transfer, and ffmpeg encode at 20.7MB/s

With both Nvidia onboard Raid 0 and Intel onboard Raid 0 (two different workstations) we're measuring nearly identical speeds of 40MB/s when writing from the Intel NAS to the respective workstation's RAID 0. The only thing faster we've found is from the RAID 5 Intel to an ESATA attached drive, or RAID 0 array...and that's only a bit faster ranging from 63MB/s to 70MB/s with ffmpeg encode rates exceeding 50MB/s. Because the ffmpeg script is splitting a 466MB file into two streams and writing them simulataneously, it's becoming obvious that a RAID 0 target is by far the best for this type of work.

 

[Dennis Wood]

I missed that RAID discussion Steve..and it's excellent. What we're seeing here in our testing is that RAID 5 absolutely kills our ffmpeg encode as two streams are written simulutaneously to an ICHR7 intel RAID5 (local). The Intel NAS we're testing handles the streams much better either due to the improved chipset it uses, or the fact that processing for the RAID5 writes is handled by the NAS, and not the workstation from which the encode originates. I'd agree...RAID 5 tests here showed similar dismal results during encodes: http://www.maximumpc.com/article/raid_controllers_compared?page=0,4

Being that some of our renders typically take 8 hours, the performance hit of RAID5 in this scenario makes RAID0 by far the best option as a target for the encoded/streamed files.

Cheers,
Dennis.

 

[thiggins]

You certainly have a strong stomach to be doing video renders directly to a NAS vs. local storage!

It makes sense that RAID 0 is going to give you higher performance than RAID 5, since that is what it is designed for.

If you really want to do RAID5, I'd look into a controller card.

 

[Dennis Wood]

The goal is to have a workable solution where a video project and its source files sit on the NAS so that multiple workstations could access and edit them, leaving them on the NAS. Renders would likely be run (given our test results) so that any encodes are written from the NAS source files to the local RAID 0 drives and then archived later back to the NAS. An XDcam EX1 outputs 35mbps variable bit rate files at 1920x1080 resolution which would require a maximum of 5MB/s or so for each stream in an edit. Theoretically, the NAS should be fine to serve up the typical 3-5 streams that would be edited simultaneously. I should point out that studios with large budgets would likely just use a 10GB SAN, but we're trying to work something applicable for our target market..the budget minded independent filmmaker.

This may all change once we load test with a few workstations hitting it simultaneously. This is where the Qnap's dual gigabit ports and 802.3ad supportive switch "should" make a difference.

From the RAID card review I linked to earlier, it looks like even high performing PCIe RAID cards slowed their render tests considerably over the onboard RAID 0 option..and the fastest encode was in fact to a RAID 0 array based on the test motherboard's intel raid controller. This is a great example of where the numbers tell a very different story than what may have been predicted. I gave up using HDtach as the numbers it was providing are way to optimistic once compared tests using actual files, CPU loads etc.

 

[8steve8]

I missed that RAID discussion Steve..and it's excellent. What we're seeing here in our testing is that RAID 5 absolutely kills our ffmpeg encode as two streams are written simulutaneously to an ICHR7 intel RAID5 (local). The Intel NAS we're testing handles the streams much better either due to the improved chipset it uses, or the fact that processing for the RAID5 writes is handled by the NAS, and not the workstation from which the encode originates. I'd agree...RAID 5 tests here showed similar dismal results during encodes: http://www.maximumpc.com/article/raid_controllers_compared?page=0,4

Being that some of our renders typically take 8 hours, the performance hit of RAID5 in this scenario makes RAID0 by far the best option as a target for the encoded/streamed files.

Cheers,
Dennis.

well i really dont like that maximumpc article, because it obviously tests the intel controller without proper caching... pay attention to the results of raid 5 with different caching schemes in my article..

 

[thiggins]

Steve,

What program are you running to gather your data in the articles?

 

[8steve8]

well depends on the article.


for raid,
iometer 2006.07.27 and a calculator.

 

[Dennis Wood]

Actually, having read that article, I agree with you Steve and would go with the Intel onboard RAID, ICHr7, 9 or 10 regardless of the read/write speeds of the "faster" discrete cards based solely on what they found with the real encoding test, and what we're finding at our end. Iometer, Iozone, HDtach etc are great tools but the results they provide don't account for other activity on the computer...like reading from one array while writing to another while the processor is taxed.

It's not overly elegant, but the combination of read/writes from both drives (or to/from a mapped drive), combined with Iozone, and then a multistream encode gives (at least for us) a realistic look at performance. We're just using a simple dos app called stopwatch to log the times at the beginning and end of each test. In excel, we've charted the MB transferred vs the logged elapsed time to get a look at the encode. Iozone is just run to compare synthetic vs actual tests...and although it provides better results with memory set at 4GB (all of our workstations have 2GB RAM), it also takes forever to complete. I decided to use 1G so our data set would emulate Tim's here.

Watching the tests, particulary the ffmpeg encode, frame rates drop down to 30fps going to any of our RAID5 targets (including the NAS) but will climb to nearly 300fps when writing to the RAID0 targets. Here's the batch file. Both target and destination drive host the lantest directory which has about 5.3 GB of files, including the iozone, stopwatch and our test files. The command to run test.bat is something like: "test c d raid_5_vs_raid_0" where "test" is the batch file name, "c" is the drive we start with, "d" is the target, and "raid_5_vs_raid_0" is text used to title the 3 log files generated. Note that the target and source drives are swapped in the process to test both scenarios. Btw, I make no claims to expertise in the batch file writing arena...so forgive the crudeness.

REM
REM *** Part 1 - Real World Test ***
REM
REM
REM Command follows format - test.bat [destination drive] [source drive] [description with no spaces] logfile is always on the first drive specified
REM
ECHO Lan Test from %1 to %2 %3 > "%1:\lantest\%3.txt"
rd %2:\lanout /s /q
md %2:\lanout
%1:
cd\
cd %1:\lantest
ECHO start copy >> "%1:\lantest\%3.txt"
%1:\lantest\stopwatch start >> "%1:\lantest\%3.txt"
xcopy %1:\lantest\undertow\*.* /s /y /e %2:\lanout
ECHO stop copy >> "%1:\lantest\%3.txt"
%1:\lantest\stopwatch start >> "%1:\lantest\%3.txt"
ECHO start encode >> "%1:\lantest\%3.txt"
%1:\lantest\stopwatch start >> "%1:\lantest\%3.txt"
%1:\lantest\ffmpeg -y -i %1:\lantest\test.mp4 -vcodec copy -f mpeg2video %2:\lanout\outputfile.m2v -acodec pcm_s16le %2:\lanout\outputfile.wav
ECHO finish encode >> "%1:\lantest\%3.txt"
rd %1:\lanout /s /q
md %1:\lanout
%1:
cd\
cd %2:\lantest
%1:\lantest\stopwatch start >> "%1:\lantest\%3.txt"
ECHO Lan Test from %2 to %1 %3 >> "%1:\lantest\%3.txt"
ECHO start copy >> "%1:\lantest\%3.txt"
%1:\lantest\stopwatch start >> "%1:\lantest\%3.txt"
xcopy %2:\lantest\undertow\*.* /s /y /e %1:\lanout
ECHO stop copy >> "%1:\lantest\%3.txt"
%1:\lantest\stopwatch start >> "%1:\lantest\%3.txt"
ECHO start encode >> "%1:\lantest\%3.txt"
%1:\lantest\stopwatch start >> "%1:\lantest\%3.txt"
%2:\lantest\ffmpeg -y -i %2:\lantest\test.mp4 -vcodec copy -f mpeg2video %1:\lanout\outputfile.m2v -acodec pcm_s16le %1:\lanout\outputfile.wav
ECHO finish encode >> "%1:\lantest\%3.txt"
%1:\lantest\stopwatch start >> "%1:\lantest\%3.txt"
REM
REM *** Part 2 - BEGIN IOZONE ***
REM
%1:
cd\
cd %1:\lantest
iozone -Rab %1:\lantest\%3_%1to%2.wks -y 64K -q 64k -S 4096 -L 64 -i 0 -i 1 -+u -f %2:\001.tst -q 64k -n 32M -g 1G -z
%2:
cd\
cd %2:\lantest
iozone -Rab %1:\lantest\%3_%2to%1.wks -y 64K -q 64k -S 4096 -L 64 -i 0 -i 1 -+u -f %1:\001.tst -q 64k -n 32M -g 1G -z

 

[Dennis Wood]

Just a quick note on some previous numbers. We had a file propogation issue with our test files, so a few of the tests were skewed. I've fixed the numbers up. Apologies for that.

At this point the best we can do (actual measured file size/time) with the Intel NAS on Windows XP is 40MB/s and this is only during writes from the NAS to a local raid 0 array. During our load testing, we saw some interesting results as the NAS kept up during writes back to the 2 workstations with no significant hit on performance when compared to a single workstation load (the two test workstations are identical). The big hit occured as the two workstations were writing to the NAS simultaneously to the tune of about 50%. This would suggest the Intel NAS cannot handle more than about 32MB/s for writes, but delivered about 64MB/s for reads. I have a feeling that the NAS cache made the difference there as the source files were the same.

In testing with a Vista SP1 Asus G2S laptop (gigabit) LAN, we officially set the speed record with the Intel NAS for downstream writes, delivering a measured 45MB/s to the laptop's drive. So this is a rare case where Vista SP1 is faster than XP SP3 ... at least when grabbing files from the LAN. This performance exceeded the numbers we saw writing from an ESATA drive to the Vista laptop.

The best we've seen so far from a networked system here is 50MB/s and this is coming from one workstation (Intel based Raid5) to another workstation (Nvidia based Raid0)...so basically our "own" XP workstation NAS. In other words, so far you're better off just building a box that used the Intel chipset for RAID..exactly as Tim seems to be doing right now.

At this point we've beaten a dead horse long enough so the next set of tests will be using the QNAP unit.

 

[ono]

Noise?

Which one is the quieter between the Synology 508 and the QNAP 509?

 

[Dennis Wood]

The TS509 is indeed using SAMBA in its OS. You can view this from the admin screen that indicates connected users. Samba is listed as one of the users from each connected machine.

RAID 0 results (5 Drives) are very good...up to 95MB/s read, 81MB/s write.

RAID 5 results (same 5 Drives) are not so good...particularly with files over 1 GB. The best we're seeing in our 5.3GB write test is 12MB/s write, and 42MB/s read.

More later...

 

[Unregistered]

I've also been testing this unit and helped Qnap troubleshoot the large file transfer issue. They have a fix that is not yet public.


I can now maintain transfer writes at 40MB/s. Reads are about 60MB/s. Both using samba and teracopy on vista sp1.

FTP of files are very fast. I can maintain writes on the nas at 80MB/s without any problems but I have to cap it at that speed or the nas becomes overloaded. I previously had to cap it at 20MB/s. Reads via ftp are about 120MB/s with bursts up to 180MB/s.

Looks like they tuned the cache pretty well but are still working on further tuning.

 

[Unregistered]

My tests were with 5 Seagate 750G drives in Raid 5.

 

[thiggins]

Which one is the quieter between the Synology 508 and the QNAP 509?

I don't do acoustic measurements with a meter, just subjective with my ear.
My test notes say the DS508 fan and drive noise was "low", while the TS509 Pro was rated "high".