What's the speed secret to "Build Your Own Atom-based NAS - Part 2"?

[00Roush]

I was going by this article when trying to adjust the setting and edit the registry, as the article talks about editing the registry for the TCP Window Size. TCP Windows Size made a bit of difference in my reading (from 45 to 65 or so) but the real impact was adjusting the Buffer Length. I then checked this MS regedit guide but wasn't able to figure out exactly which parameters corresponded to Buffer Length in Jperf.

I will run a few more tests tonight with the Mbps output format and I'm also going to try and get FTP working.

My hope really was lying with the tuning of either the hard drives or networking in Ubuntu. When I had FreeNAS tuned, I was getting the same transfer rates between FTP & Samba. Those transfer rates were also very close to what I am currently getting with Ubuntu.

To my knowledge buffer length and window size in Iperf basically do the same thing. Now with Iperf the program specifically sets the window size based upon either the default or what you specify. With all other TCP/IP traffic Windows controls the window size. The window size setting in the registry sets the maximum possible size allowed. This does not mean that Windows will actually use that size. You can use a program like Wireshark or Microsoft Network Monitor to see the window sizes being used. In my testing I have usually seen it at 60k-64k for file copies across my LAN. If iperf is showing that your network is capable of over 100 MB/sec at a 64k window size I would say your network is good to go and no network setting changes are necessary.

Next up would be testing all of the disks involved to find out if they are the bottleneck. You already tested your drives on your server but what about the client? Also you had mentioned that your drives are showing up as SATA150 instead of SATA300. Could you check that your drives do not have a jumper set that has them in a SATA150 compatibility mode? If you really wanted you could also use Iozone to test your hard drives. On Ubuntu you should be able to install it by typing sudo apt-get install iozone3.

For reference here are my Iperf results:
C:\iperf>iperf -c 192.168.0.2 -P 1 -i 1 -p 5001 -f M -t 10
------------------------------------------------------------
Client connecting to 192.168.0.2, TCP port 5001
TCP window size: 0.01 MByte (default)
------------------------------------------------------------
[132] local 192.168.0.3 port 50788 connected with 192.168.0.2 port 5001
[ ID] Interval Transfer Bandwidth
[132] 0.0- 1.0 sec 49.9 MBytes 49.9 MBytes/sec
[132] 1.0- 2.0 sec 50.1 MBytes 50.1 MBytes/sec
[132] 2.0- 3.0 sec 49.7 MBytes 49.7 MBytes/sec
[132] 3.0- 4.0 sec 49.9 MBytes 49.9 MBytes/sec
[132] 4.0- 5.0 sec 50.0 MBytes 50.0 MBytes/sec
[132] 5.0- 6.0 sec 47.0 MBytes 47.0 MBytes/sec
[132] 6.0- 7.0 sec 49.5 MBytes 49.5 MBytes/sec
[132] 7.0- 8.0 sec 49.7 MBytes 49.7 MBytes/sec
[132] 8.0- 9.0 sec 49.6 MBytes 49.6 MBytes/sec
[132] 9.0-10.0 sec 49.3 MBytes 49.3 MBytes/sec
[132] 0.0-10.0 sec 495 MBytes 49.5 MBytes/sec


C:\iperf>iperf -c 192.168.0.2 -P 1 -i 1 -p 5001 -w 32.0K -l 64.0K -f M -t 10
------------------------------------------------------------
Client connecting to 192.168.0.2, TCP port 5001
TCP window size: 0.03 MByte
------------------------------------------------------------
[132] local 192.168.0.3 port 50789 connected with 192.168.0.2 port 5001
[ ID] Interval Transfer Bandwidth
[132] 0.0- 1.0 sec 113 MBytes 113 MBytes/sec
[132] 1.0- 2.0 sec 112 MBytes 112 MBytes/sec
[132] 2.0- 3.0 sec 112 MBytes 112 MBytes/sec
[132] 3.0- 4.0 sec 112 MBytes 112 MBytes/sec
[132] 4.0- 5.0 sec 112 MBytes 112 MBytes/sec
[132] 5.0- 6.0 sec 112 MBytes 112 MBytes/sec
[132] 6.0- 7.0 sec 112 MBytes 112 MBytes/sec
[132] 7.0- 8.0 sec 112 MBytes 112 MBytes/sec
[132] 8.0- 9.0 sec 112 MBytes 112 MBytes/sec
[132] 9.0-10.0 sec 112 MBytes 112 MBytes/sec
[132] 0.0-10.0 sec 1123 MBytes 112 MBytes/sec


C:\iperf>iperf -w 64k -c 192.168.0.2 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.0.2, TCP port 5001
TCP window size: 64.0 KByte
------------------------------------------------------------
[156] local 192.168.0.3 port 50853 connected with 192.168.0.2 port 5001
[ ID] Interval Transfer Bandwidth
[156] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec
[168] local 192.168.0.3 port 5001 connected with 192.168.0.2 port 1311
[ ID] Interval Transfer Bandwidth
[168] 0.0-10.0 sec 1.10 GBytes 944 Mbits/sec


00Roush

 

[milo]

To my knowledge buffer length and window size in Iperf basically do the same thing. Now with Iperf the program specifically sets the window size based upon either the default or what you specify. With all other TCP/IP traffic Windows controls the window size. The window size setting in the registry sets the maximum possible size allowed. This does not mean that Windows will actually use that size. You can use a program like Wireshark or Microsoft Network Monitor to see the window sizes being used. In my testing I have usually seen it at 60k-64k for file copies across my LAN. If iperf is showing that your network is capable of over 100 MB/sec at a 64k window size I would say your network is good to go and no network setting changes are necessary.

Next up would be testing all of the disks involved to find out if they are the bottleneck. You already tested your drives on your server but what about the client? Also you had mentioned that your drives are showing up as SATA150 instead of SATA300. Could you check that your drives do not have a jumper set that has them in a SATA150 compatibility mode? If you really wanted you could also use Iozone to test your hard drives. On Ubuntu you should be able to install it by typing sudo apt-get install iozone3.

00Roush

Thanks testing at MBytes/s and the info. I have checked the drives for jumpers and have even downloaded the Samsung Test/Configuration LiveCD to check the drives and ensure they are at SATA300, which they are. It's only during the boot messages that I see Ubuntu has them listed at SATA150.

Last night, I removed all my registry edits on my XP system and saw that it really made no difference. So I took a different direction all together and install XP Pro SP3 onto the server. I didn't have the copy activated but I don't think that would matter for this testing. I repeated my Laptop to Server tests and found my speeds dropped to about 20MB/s. Next, I tried drive to drive tests (on the now XP Pro server) and found that my speeds increased to 80-90MB/s. This makes me believe that my problem is somewhere in the drives using Ubuntu, or at least that Ubuntu could do better.

Would the speed of the OS drive matter? Currently it's a rather old laptop drive I pulled from a scrap laptop. Also, I will also use HDtach to test my laptop's hard drives but I think they should be good since they are fairly new.

Thanks

 

[00Roush]

Thanks testing at MBytes/s and the info. I have checked the drives for jumpers and have even downloaded the Samsung Test/Configuration LiveCD to check the drives and ensure they are at SATA300, which they are. It's only during the boot messages that I see Ubuntu has them listed at SATA150.

Last night, I removed all my registry edits on my XP system and saw that it really made no difference. So I took a different direction all together and install XP Pro SP3 onto the server. I didn't have the copy activated but I don't think that would matter for this testing. I repeated my Laptop to Server tests and found my speeds dropped to about 20MB/s. Next, I tried drive to drive tests (on the now XP Pro server) and found that my speeds increased to 80-90MB/s. This makes me believe that my problem is somewhere in the drives using Ubuntu, or at least that Ubuntu could do better.

Would the speed of the OS drive matter? Currently it's a rather old laptop drive I pulled from a scrap laptop. Also, I will also use HDtach to test my laptop's hard drives but I think they should be good since they are fairly new.

Thanks

Just remember that when XP is used as a server you need to set the system to use a large system cache for the best performance. Please take a look at this thread for more information.

Generally the OS drive would only affect your performance if it is the one being copied to/from. Could you do a hdparm (or HDtach) test on your server's OS drive?

00Roush

 

[milo]

Just remember that when XP is used as a server you need to set the system to use a large system cache for the best performance. Please take a look at this thread for more information.

Generally the OS drive would only affect your performance if it is the one being copied to/from. Could you do a hdparm (or HDtach) test on your server's OS drive?

00Roush

Sorry for the confusion, I didn't intend to use XP as a server software but just wanted to use it to test of the performance of the server system. Seeing that the disk to disk transfer speeds went up but the network transfer speeds went down helps me believe it's a configuration issue instead of a hardware issue.

I have since reinstalled Ubuntu 9.04 on the server and checked the OS drive with hdparm. The OS drive is a lot slower the "Timing cached reads" are about 574 MB/sec and the "Timing buffered disk reads" are about 14 MB/sec. The first reinstall I used the same setup as before except I decided to remove the swap partition since I thought the drive was slow. This didn't change anything. So I reinstalled again, this time with no swap and using the EXT4 filesystem on the OS and storage hard drives. The EXT4 filesystem gave me quite a boost. My drive to drive transfers went up to 77 MB/s and my network transfers went up as well. Using Vista 64-bit to the NAS, I see 55-60MB/s and using XP Pro to the NAS, I see 47-54MB/s.

I don't know if I can get much more from this system but moving to EXT4 from EXT3 did give me a good boost. I have a few more things to try from the reading I have been doing and I'll post any other results I get.

Thanks

Milo

 

[00Roush]

Glad to see your getting better performance.

Have you to gotten a chance to test your laptop's hard drive? I wonder if the speeds you are seeing now are the max speed of this drive?

00Roush

 

[milo]

Oops, sorry I forgot to mention those results. I downloaded the free version of HDTune and got the results shown below. Both hard drives have slightly higher average speeds than my current transfers so this might be the bottleneck now. The right (40 degree C) drive is the OS drive when running XP on my system.

Thanks again,

Milo

 

[bookmage]

Interesting stuff... I've been pondering using an Atom 330 for a mini-NAS for a while and this is helpful stuff...

On a slightly related note and cuz I like to crosscheck benchmarks w/lots of tools...
When I started using Linux, I was told to use Bonnie++ to check disk speeds. This recent article helps describe how: http://www.linux.com/archive/feature/139742

On the windows side, I used DiskBench which doesn't have a lot of options, but it was good for giving me real life numbers and pinpoint network or local bottlenecks.

I'll have to add your tools and configurations to my testing process...

 

[00Roush]

Interesting stuff... I've been pondering using an Atom 330 for a mini-NAS for a while and this is helpful stuff...

On a slightly related note and cuz I like to crosscheck benchmarks w/lots of tools...
When I started using Linux, I was told to use Bonnie++ to check disk speeds. This recent article helps describe how: http://www.linux.com/archive/feature/139742

On the windows side, I used DiskBench which doesn't have a lot of options, but it was good for giving me real life numbers and pinpoint network or local bottlenecks.

I'll have to add your tools and configurations to my testing process...

That is an excellent point.

Testing with HDtune or Hdparm basically gives you the raw disk speed and usually gives an idea of the maximum possible speed. The thing is that this is not the true real world speeds you will see. Most of the time actual read/write speeds are close to the raw speeds but it really depends on the file system being used. So using a program like Bonnie++ or DiskBench makes a little more sense. Iozone could also be used for this type of testing.

00Roush

 

[DarthV]

Has anyone used one of the dual core atoms to run a NAS? I'm looking at going with the dual core to get gigabit ethernet and slapping in a cheap 4 port SATA controller along with it. And probably going with a mimimal install of jaunty.

 

[aknab666]

Has anyone used one of the dual core atoms to run a NAS? I'm looking at going with the dual core to get gigabit ethernet and slapping in a cheap 4 port SATA controller along with it. And probably going with a mimimal install of jaunty.

Yes I have and it found it impressive given the power it uses. Unfortunately it took a power hit and is offline for the future.

 

[milo]

Has anyone used one of the dual core atoms to run a NAS? I'm looking at going with the dual core to get gigabit ethernet and slapping in a cheap 4 port SATA controller along with it. And probably going with a mimimal install of jaunty.

My setup listed in post #1 of this thread is working great as a NAS and FTP server. I still haven't gotten a Media Server installed on it, but that's only because of my lack of time and my lack of knowledge with Linux (Ubuntu).

I am just using the two on board SATA ports so I can't comment on a controller card.

 

[DarthV]

I'm planning to use the box as a media dump. So samba to my windows PC/laptop and NFS to my asrock ION 330 nettop running XBMC for my 52" LCD TV. Nice to see that you've got 50-60MB/s working with the onboard gigabit NIC.

 

[DarthV]

I've changed my mind, a friend wants to upgrade his HTPC, so I'm going to buy his current asus P5N7A-VM for a good price then throw in an intel e1400 + 4 port sata controller. Close to the same price as getting the dual core atom combo, but with extra PCI and PCI-e slots.