NAS performance vs. guest performance

[kamina]

I've been considering getting a new fast NAS for some time now. I'm kind of hoping I'll be able to consolidate all my files on it - including photo's from my desktop (and hoping I won't notice a very negative impact vs. the local disks).

All the new hottest NAS devices seem to be managing reads of close to 100MB/s, but I'm starting to wonder how easy it would be to be limited on the client side before reaching such speeds. My desktop is a 4 year old iMac C2D, so basically a 4 year old mobile processor. While it's faster then what's inside the NAS it's probably not even close to as optimized for transferring fast over the network (which should be fast in my apartment).

So yeah, is there even a point in me looking at these newer fast Atom based NAS devices, or would something slower serve me just as well?

 

[teknojnky]

well, one thing I can say is that images tend to be one of the hardest things for any nas/network device to do well compared to local disks, since they are generally small and tend to be lots of them in individual folders.

small size/large quantity files tend to be much lower performance-wise than middle to large files.

couple that with wireless, and you have a recipe for disappointment.

if your just using the nas for storage and not image manipulation or other heavy work, and going over gigabit network, it may not be so bad, but still its unlikely you will be getting anywhere near 100mbs with that type of workload.

all that said, its still a good idea to be able to consolidate to one device, though you should still keep backups (raid is not a substutite for having multiple copies of your data).

 

[kamina]

well, one thing I can say is that images tend to be one of the hardest things for any nas/network device to do well compared to local disks, since they are generally small and tend to be lots of them in individual folders.

And that will be the case for me too. I think I have about 10000-15000 pictures in various folders. Mostly around 1-3MB but also containing raw files which are over 10MB in size.

small size/large quantity files tend to be much lower performance-wise than middle to large files.

couple that with wireless, and you have a recipe for disappointment.

Yeah, I won't be trying to use them over wireless. I do have .n wireless that gives about 10MB/s throughput (if I'm next to the base station), but I use that for browsing the web (I have a 100mbit fiber connection so it works out ok for general browsing).

My house has cat 5e cabling in place so I have a gigabit wired connection.

if your just using the nas for storage and not image manipulation or other heavy work, and going over gigabit network, it may not be so bad, but still its unlikely you will be getting anywhere near 100mbs with that type of workload.

But I'm hoping I'll be able to use it also for image manipulation and browsing photos in iPhoto / Lightroom etc.

all that said, its still a good idea to be able to consolidate to one device, though you should still keep backups (raid is not a substutite for having multiple copies of your data).

My main computer is an iMac. After getting a new workstation at the office (i7, 8GB ram and a fast SSD) the iMac is feeling very sluggish, and I'd like to swap the drive for an SSD. I can't do that without moving all large files away. So I'm really hoping to move almost everything to the network. While I'm at it I would also move all my movies from my HTPC to the NAS so everything is in one place.

Currently I take daily, weekly and monthly backups from the HTPC to an external disk using rsnapshot, and use timemachine on the mac. I also have separate disks I keep out of the house and update rarely (too rarely).

I'm also hoping the NAS would simplify this. That is, have all the data in one place, and take backups to an e-sata disk attached to the NAS. I could also replicate it with rsync to a disk attached to the HTPC so I would not have as much downtime if there was a problem with one of them.

I was testing the network throughput yesterday using the iMac and a macbook pro laptop. The method was to use netcat to echo text from one machine, and read it to the other. Using a direct cable between them I was only able to get 40MB/s throughput (I got the same speed when testing through the whole network) and the process was running at 100% cpu usage on the macbook.

This is actually where this whole thing came to my mind. Will I be cpu bound on the laptop / imac before being close to saturating the NAS? I'm guessing it's not exactly comparable, because I was not at 100% cpu usage due to the actual network throughput, rather due to generating the data that was sent through the network to test the throughput.

 

[00Roush]

If your C2D CPU is 1.8 Ghz or faster you have more than enough CPU power for 100 MB/sec. The main things that I have found to affect file transfer performance are hard drive speed in the client and NAS, OS being used on the client, NAS CPU power, and network cards used.

If your plan is to see 100 MB/sec on large file transfers you will need hard drives that can read and write at that speed to begin with. Next would be the network... PCIe based gigabit network cards are generally needed to support these speeds. The last thing is going to be the OS used on the client. Both Windows Vista SP1+ and Windows 7 have file copy optimizations that allow for high speed file copies using SMB. I am not sure how well OS X is setup to do fast file transfers so the OS might end up being a bottleneck.

I will tell you that with Vista SP1 and now Windows 7 I don't really notice much of a difference between local and network pictures. Average picture size is around 3 MB. With photo software things seem to work just as fast as well. It does depend on the program though... each piece of software has its own way of reading and writing files. Some have been optimized to read and write locally but not for network files. For the most part I have found that when doing image manipulation the file is copied to the machine you are doing the editing on and the network does not affect the editing speed. My guess is the only times you might notice a difference between network and local is when opening and saving a file.

If you looking to test your setup I would test with iperf if possible to test your raw network speed. This should give you a better idea of actual CPU usage during peak network load. You might give Xbench a try for testing the speed of file transfers.

00Roush