Gigabit Ethernet Need-to-Know

[rogerbinns]

If a Linux variant is typically the OS for the inexpensive NAS boxes, one wonders if Samba could be implemented on some of these boxes already?

An easy way to detect Samba is to use WireShark and record the connection (use a capture filter of "port 139 or port 445"). The client sends NegotiateRequest and then look in the NegotiateResponse sent by the server in the capabilities field. 5th from the end is "Unix extensions". If that is set then it is 99.999% certain to be Samba.

Given your budget you may find one of the open source NAS implementations work just as well - see FreeNAS and OpenFiler as examples. Or you could just install Ubuntu server edition which gives the additional advantage of other software such as web servers, ftp, graphics tools, batch processing etc being just a single command away. The latter is what I would choose in a similar situation to yours.

Other than Linux or BSD, a NAS may use Windows Storage Server but you have to be an OEM to buy it. If you love ZFS (and it has a lot to love) then Nexenta can supply it (OpenSolaris kernel, traditional Linux userspace programs).

 

[Dennis Wood]

Roger, you've got a whole lot of network knowledge piled into your cranial space This is excellent info...and I will test as you described on the Intel and QNAP NAS units.

I had looked at Ubuntu (but not tried it) and I'll admit that with more time on my hands I'd be very interested in giving it a try. In terms of recommendations for other creative types doing photo/video work, my tendency is to focus on the Mac and PC world not because it's better, but rather because the two OS's are standard in the editing/encoding world for the smaller outfits. A plug and playish NAS box with the OS installed, apps working, and some semblance of support fit's the bill exactly.

 

[rogerbinns]

(I've been doing this kind of stuff for many years both personally and professionally. It is certainly easier to pickup over long periods of time than just jumping in the deep end!)

I'd certainly focus on Mac and Windows for the clients. Other than backend render farms, there aren't any decent video tools for Linux.

However for the file server, Linux is a very good choice. The users should neither need to know or care what you are using. You should be aware that the Linux world is very different. One of the consequences of open source is that anyone can provide support and fixes at any time. I'd definitely shy away from low end vendors support simply because they can't provide it well. Simply look at what you are paying for the box minus what the raw box cost from the original vendor. You'll likely see a margin of a few hundred dollars. A US based employee (especially a California based one) costs up to $200k per year (fully loaded including facilities, benefits, employer taxes, equipment, building etc). You can work out just how many boxes they have to sell to pay for employees. Of course they could pay employees a lot less - one company I know was paying support people $10/hr which is less than burger flippers made in the area. (And the loaded cost is more than $10/hr in that case anyway).

So whether they are using Windows Storage Server, Linux or something else, budget noname vendors are not going to have good support. (They may have good customer service but that is different). For non-trivial issues you are going to have to support yourself. That means you want to use something that lots of other people do, and you want to use something that is open source so that they can make a setup exactly like what you have without having to worry about licensing issues. If you are using something very popular then the probability of Google searches solving your problem is greatly increased. There are also more likely to be small companies in the area providing hands on support.

In any event I'd recommend trying out Ubuntu and then getting familiar with Samba. Since the vast majority of NAS vendors are using it under the hood anyway you'll get a good understanding of what is possible, various quirks and pitfalls, Mac interoperability etc. Then at least you'll have better grounds for understanding just what the NAS vendors are doing and can see if they do things like hide advanced functionality, give good logs, troubleshooting etc.

 

[Dennis Wood]

I didn't have to look too hard on the QNAP TS509 for evidence of Samba...every connected machine has another connection listed as "Samba" in the admin menu listing current connections. This box was doing great with RAID0 on the large files, but it's definitely not as good in RAID5 configuration with files over 1GB in size.

When you do run into a head scratching experience l can definitely see the value in a box running Linux where you have more freedom to tweak.

 

[thiggins]

Roger,

In testing for the next "Fast NAS" installment, I have installed Vista Home Premium on the iozone system and found a significant increase in write speed, but virtually none for reads.

Can you provide any clues as to why? This is with Vista Home Premium on the iozone machine and WHS SP1 on the system being tested. Both have Intel Intel 82566DM onboard gigabit Ethernet, which I believe connects via PCIe.

 

[rogerbinns]

In testing for the next "Fast NAS" installment, I have installed Vista Home Premium on the iozone system and found a significant increase in write speed, but virtually none for reads.

I could only make guesses. To get the correct answer you need to run Wireshark and analyse the traces. I'd be happy to look at traces if you want. Email me a link to the traces or the traces themselves (my email server will accept mail up to 100MB in size).

As to the speculation, writes have always been a problem for Microsoft's SMB/CIFS stack. Their implementation lives in the kernel (as compared to user space for Samba). Kernel memory is not paged (well a tiny bit is for NT) and is a somewhat precious commodity. For SMB1 there are 6 different commands that do writes each that had different maximum size and other tweaks over time. The maximum size of a SMB1 write request is 64KB. Although the protocol supports it, Windows rarely does asynchronous write requests - ie it sends a write request and waits for a response before sending the next one. If the round trip time including server side processing took 10ms then a maximum of 100 requests could be processed per second. This is also exacerbated by programs like Explorer which often like to issue writes of 4kb.

SMB2 has exactly one write command and the maximum size of a request is theoretically bounded at 4GB. The server tells the client what number it wants and you can see that in the NegotiateProtocol response. The SMB2 stack also does lots more stuff asynchronously.

So the likely cause of performance increase is going to be some combination of increased request sizes as well as concurrency (asynchronous) in the requests. But it isn't possible to tell without Wireshark traces. Depending on your setup it could also be due to other factors such as in this fascinating article.

 

[Dennis Wood]

Tim, what are the read and write speeds from both ends? I've got enough test data here to likely figure out where the issue would be.

 

[thiggins]

Hi Roger,

Thanks for the offer to look at the WireShark traces. Do you want them from the machine that is doing the reading and writing (the iozone machine) or from the NAS (WHS) being tested?

That is a good article. File copying sure is complex, eh?

 

[rogerbinns]

Do you want them from the machine that is doing the reading and writing (the iozone machine) or from the NAS (WHS) being tested?

I want them from the machine performing the operations (aka iozone machine) since that shows what the user experiences. Captures from the NAS would show you the processing time for requests.

If possible also include captures from the previous slow setup as then I can tell you what the difference is.

That is a good article. File copying sure is complex, eh?

It also substantiates my earlier claim that you can tell me what results you want and I'll come up with the benchmark to get them For example you can pick file sizes based on how well the platform handles bulk reads/writes. In the SMB/CIFS space everyone used Netbench and optimized heavily for that. Then NetApp came along and claimed response time is more important than throughput. I wonder why

 

[Dennis Wood]

Roger I had read that article a few month ago while trying to figure out why Vista was so agonizing using an Esata drive. A good read indeed. In our case, SP1 and a few driver updates resolved the issue.

 

[raleighthings]

But what good is 10GigE if the limit is the hard drive speed?

Switch to switch and into a server. Into a server multiple requests can flow in and out independent of the disk speed. Especially with a decent caching arrangement.

 

[Dennis Wood]

Yes, and for serious video/film editing in networked environments, you'd have 10gig to the workstations too. Editing an uncompressed HD clip from a camera like RED at 4K, would tax even that network speed. 1 hour of footage would require about 4.5 TB of storage!

 

[raleighthings]

Yes, and for serious video/film editing in networked environments, you'd have 10gig to the workstations too. Editing an uncompressed HD clip from a camera like RED at 4K, would tax even that network speed. 1 hour of footage would require about 4.5 TB of storage!

Which was why Apple got into the RAID business. They wanted a plug and play way for video editing firms to switch to digital HD without hiring IBM services and such.

Now that RAID is much easier to buy and install they dropped their line and just have a recommended vendor.

 

[art2]

PCIe/PCI advantage?

Is there anything to be gained in a gigabit network by having a PCIe NIC on one side of a file transfer, if the other side has only a PCI NIC? As in the case where one computer has a free PCIe slot, and the other one does not. Any faster than using cheaper gigabit PCI adaptors in both computers?

 

[thiggins]

No. Connection speed will be limited by the slower NIC.

 

[rchapoteau]

Picked up another Netgear recently. That was kind of a mistake. They are so NOISY. The fans are ridiculously loud on these guys.

 

[dieter]

Measuring Network Performance - Jperf & TCP

In Doug Reid's article from 4/30/08, he attempts to show how to implement one's jperf findings by changing the MS Windows registry. I have tried to search the XP pro registry entry called "tcpwindowsize", but nothing came up. And what about the other Jperf parameter, buffer length?

Any thoughts?

Dieter

 

[dreid]

TCP Window Size

Thanks for the questions, I hope the below helps:

To adjust TCP Window Size in XP Pro:

1. Click Start, click Run, then type regedit.
2. Click the + sign next to HKEY_LOCAL_MACHINE.
3. Click the + sign next to SYSTEM
4. Click the + sign next to CurrentControlSet.
5. Click the + sign next to Services.
6. Click the + sign next to Tcpip.
7. Click on Parameters.
8. You should see a value to the right called TcpWindowSize. Double clicking on this value will let you change it. Note that the value is in hexadecimal, so you'll have to do the math.

I attached a screen shot to this post to demonstrate.

Regarding Buffer Length in Jperf, that is an option within Jperf itself, you'll see it on the left side of the Jperf window under the Transport layer options. I attached a screen shot for this as well.

Good luck!

 

[jalyst]

what's the difference between all the Intel PRO/1000's ?
Aside from PCI/PCIe interface...

http://www.intel.com/network/connec...rs.htm?iid=embed_portal+hdprod_deskt_adapters

http://www.intel.com/network/connec...s.htm?iid=embed_portal+hdprod_server_adapters

http://www.intel.com/products/ethernet/index.htm?iid=embed_portal+hdprod_ethernet

 

[thiggins]

what's the difference between all the Intel PRO/1000's ?
Aside from PCI/PCIe interface...

http://www.intel.com/network/connec...rs.htm?iid=embed_portal+hdprod_deskt_adapters

Look closely and you'll see the differences. These desktop adapters are single port.

http://www.intel.com/network/connectivity/products/server_adapters.htm?iid=embed_portal+hdprod_server_adapters

These server adapters are multi-ported. They may also perform more on-board processing to offload the host CPU.

http://www.intel.com/products/ethernet/index.htm?iid=embed_portal+hdprod_ethernet

These are embedded devices, i.e. chips.