Putting together the SNB RAID NAS Explorer

[thiggins]

Ok folks. I'm going to bite on the question that many of you have been asking: What are the factors in putting together a fast NAS?

So I'm assembling a testbed system that should allow me to explore the question.

Since there are so many great minds floating around here, I'm going to have you help me in selecting the components.

So far I have chosen a GIGABYTE GA-G33M-S2L mobo. Can handle the current Intel chipsets, has onboard PCI-e Ethernet (Realtek RTL8111B), Intel ICH9 Southbridge for up to RAID 5.

Anyone got something better? And what about CPU? Go with a cheapie or get at least a Core 2 Duo?

Memory suggestions?

Drive suggestions?

Have at it!

 

[Madwand]

I don't think that Gigabyte has a RAID 5 chipset. I suggest looking at Intel's Q-series chipsets, e.g. the Q35. Or when they're available, Q43 or Q45.

The Asus P5E-VM DO is a commonly-available Q35-based board. It has an ICH9x variant (ICH9DO) with 6 SATA ports supporting RAID 5, on-board video, and a somewhat uncommon and premium on-board Intel PCIe NIC.

One downside is that the NIC doesn't support jumbo frames, but that's more a nice-to-have than a critical requirement for the most part.

It supports the newer 45nm CPUs. I'd probably pair it with a budget dual-core, for a high-performance file server, but could even consider a quad core if I was planning on doing video encoding on it for example.

The Intel RAID chipset gives you several options -- on-board RAID under Windows, unix-based OS RAID, and Cirprico's RaidCode-based software RAID. I'd probably choose a PATA drive for the OS itself.

 

[jdabbs]

As Madwand alluded to, ICH9 is the non-hardware RAID variant.

Have you decided on the OS yet? Finding compatible hardware (BSD especially) should be the primary consideration in choosing components.

 

[thiggins]

Thanks for the mobo suggestion, Madwand. The Gigabyte board has the ICH9 southbridge. Doesn't that give you RAID 5 using the Intel Matrix Storage Console?

I would really like jumbo frame support, since that will help push throughput to the max. Thanks for the heads-up on the Intel chipset not supporting it. Suppose I could throw in a PCI-e-based NIC, but was hoping to avoid that.

jdabbs: As far as OS, that will be one of the things that I will be varying in search of maximum throughput. I don't think that I'm choosing a board with anything exotic.

I'll probably try Vista, WHS and maybe MS Server 2008, although its cost sort of rules it out as a reasonable alternative. Then some NAS distros and perhaps Ubuntu. I'll rely on your guys to guide me.

Do you see something in the Gigabyte or ASUS boards that would be a problem for open source distros?

 

[scotty]

Thanks for doing this. This is along the lines of what I want to test (or see tested).

- As far as OS - Definately Vista, WHS, and Server 08. It would be nice to see how something like OpenNAS compares, as well as a basic Linux distro like Ubuntu (seeing as Linux is hailed by many for its file serving robustness). I'm currently running Ubuntu 8.04 (the latest) and I've been very happy with its file serving capabilties. As far as open sourced distros and boards go, any board should work, but support for Nvidia's fancy features tends to be on and off in Linux. Going with an intel based board would probably be better. Ubuntu is probably just as good as any other distro to try.

- Memory probably wouldn't be an issue. Seeing how cheap 1GB DDR2 is these days I would doubt (although, it would be nice to prove for sure ) if more than 1GB in a NAS would matter much.

- Networking - Jumbo Frame On and Off would be nice. Testing 100 versus 1000mbit would be nice to see as well

- Testing one of 2 different procs would be nice (i.e. a low voltage Core2/Celeron versus a more powerful desktop proc - most NAS's like thecus' have celerons so it would be interesting to see).

- Testing an old junker PC would be nice to see too (i.e. an older P3/P4) in comparison.

- RAID - I'd be interested to see if it makes any difference. It'd be nice to see performance results from 1 drive, and 2/3 drives in various RAID configurations.

I'll be waiting to see the results with a lot of anticipation. Seeing as you can throw a pretty basic PC together these days for roughly the same price as a NAS, it would be really interesting to see how a PC fares.

 

[thiggins]

I'm talking with vendors to line up drives. It's been suggested that I might want to try SAS drives.

Any thoughts?

 

[Tamarin]

While SAS drives perform great, I think the majority of us wouldn't realistically use them. If you can do them as an additional test, it might be interesting to see what kind of difference we would see in using them over SATA, but I have a feeling that the price difference wouldn't justify it in most scenarios.

Tam

 

[Madwand]

ICH9 is not the same as ICH9R or ICH9DO, etc., however Intel sometimes uses ICH9 to denote the entire family, so this can be confusing. The plain ICH9 doesn't have the same feature set, and as can be seen from the motherboard specs. and Intel's, it doesn't support RAID 5.

I now see some contradictory info on jumbo frame support, suggesting perhaps that it wasn't supported at some time, and has since become supported. There have been contradictions in the past, too, and I recallthe "not supported" version being correct. I think you have to try and see to find out.

Intel: "Nope": http://www.intel.com/support/motherboards/desktop/sb/cs-022309.htm#jumbo

Intel" "Now with 100% more jumbo frames": http://www.intel.com/Assets/PDF/datasheet/316972.pdf

Trying the on-board NIC vs. an add-on PCIe one would be an interesting comparison, esp., if they're from the same vendor and family. Odds are however that the difference won't be clear cut and significant., except in some specific cases, e.g. with clients which may be struggling with throughput through a PCI bus.

Regarding SAS, I think the question is who is your audience. SAS and 2008 Server, etc., aren't really relevant to home users, and even to many small businesses, and you should keep the differences clearly in mind. It's likely that two different configurations would be more suitable for the business with money and specific performance goals vs. the typical household, with very different needs and budget constraints.

 

[jdabbs]

jdabbs: As far as OS, that will be one of the things that I will be varying in search of maximum throughput. I don't think that I'm choosing a board with anything exotic.

I'll probably try Vista, WHS and maybe MS Server 2008, although its cost sort of rules it out as a reasonable alternative. Then some NAS distros and perhaps Ubuntu. I'll rely on your guys to guide me.

Do you see something in the Gigabyte or ASUS boards that would be a problem for open source distros?

It's not so much brand issues, as it is verifying stable chipset/NIC support (accomplished with a few minutes of googling). Since you're going with Intel NICs, you dodge the notorious crab.

FreeNAS has confirmed support for only up to ICH8 (and that being determined by one line in a wiki). Linux has the lion's share of developer support, so BSD hardware support leaves much to be desired (ex: FreeBSD supports up to ICH7, which was released in early 2006). I don't consider myself a subject matter expert as my experience is derived from other, non-NAS distros (in this instance, pfSense).

 

[thiggins]

Thanks for the responses, guys. I hear you on the SAS, Windows Server cost issues. But to really drive the nail into this subject, I will try to explore even the higher-end options.

I'll start with the less expensive options and work upward.

 

[crashnburn]

I don't think that Gigabyte has a RAID 5 chipset. I suggest looking at Intel's Q-series chipsets, e.g. the Q35. Or when they're available, Q43 or Q45.

The Asus P5E-VM DO is a commonly-available Q35-based board. It has an ICH9x variant (ICH9DO) with 6 SATA ports supporting RAID 5, on-board video, and a somewhat uncommon and premium on-board Intel PCIe NIC.

One downside is that the NIC doesn't support jumbo frames, but that's more a nice-to-have than a critical requirement for the most part.

It supports the newer 45nm CPUs. I'd probably pair it with a budget dual-core, for a high-performance file server, but could even consider a quad core if I was planning on doing video encoding on it for example.

The Intel RAID chipset gives you several options -- on-board RAID under Windows, unix-based OS RAID, and Cirprico's RaidCode-based software RAID. I'd probably choose a PATA drive for the OS itself.

It would be interesting to see all these comparisons.

 

[Bart]

Thanks for doing this Tim.

I'd like to see it tested with both a cheaper CPU like the E2200 and a more expensive one like a E8500 or something.

I do agree with you that you should test it with windows server 2008 and WHS. I'd also like to see it compared with Open-e, OpenFiler, a regular Linux Distro (ubuntu?), FreeNAS and NASLite.

It also might not be such a good a good idea to try RAID5 on that Gigabyte board since that will most definitely be software RAID, and this will impact the speed of your NAS.

If possible, I would also like to see this system tested with the onboard network card vs an Intel NIC PCI-express card. I've been meaning to plug in an Intel NIC card in all my PC's because I've got a feeling those perform a lot better than the standard onboard NIC's.

 

[thiggins]

Hi Bart,
Most, if not all, commercial consumer NASes are doing software raid, too. So I think it will be good to look at it as part of this exercise.

I will also try one or more hardware RAID cards.

You should definitely see a throughput gain by using a PCI-Express NIC (or onboard Ethernet that connects via PCI-Express).

 

[Norcross]

I'm really looking forward to this as well, since it's the next piece in my home network setup. My linux experience is non-existent, so anything I put together would be a Windows flavor (XP or Vista).

Any thoughts about comparing 32bit vs 64 bit?

 

[thiggins]

Any thoughts about comparing 32bit vs 64 bit?

Just 32 bit.

 

[thiggins]

Resurrecting the thread

I'm resurrecting this thread because I need to choose a new motherboard to be able to test hardware RAID controllers. I have discovered that PCIe x1 won't cut it when you move to full-hardware controllers. Gotta have x8 to support pretty much anything that handles 4 drives.

Given that this is a focused application (NAS) and that the heavy lifting will be done in the controller, I'm thinking of a mini-ITX format board with maybe a VIA or other low-power processor to keep cost down. The controller will probably run in the $200 - $300 range.

Intel sent me a DG45FC, but its PCIe slot is only X1.

Suggestions?

 

[Tamarin]

Tim,

This isn't a Mini-ITX board, in fact, it's a full ATX one. But it would give you all the features you want. I know you want to keep your costs down, so perhaps the Gigabyte EP45-DS3R would be a board you'd want to consider. I got it from Newegg, seems like it was about 120. It has the ICH10R southbridge, so it should support all of Intel's software RAID options. I currently run it on my server at home, mirrored 1.5Tbytes. It has a x16 slot, as well as an x8 electrical, x16 physical slot. Additionally, dual Realtek Gig-e Ports, and 4 x1 PCI-e slots. And it's DDR2 so you won't have to shell out for DDR3. Basically, I've been really happy w/the board.

Tam

 

[00Roush]

I'm resurrecting this thread because I need to choose a new motherboard to be able to test hardware RAID controllers. I have discovered that PCIe x1 won't cut it when you move to full-hardware controllers. Gotta have x8 to support pretty much anything that handles 4 drives.

Given that this is a focused application (NAS) and that the heavy lifting will be done in the controller, I'm thinking of a mini-ITX format board with maybe a VIA or other low-power processor to keep cost down. The controller will probably run in the $200 - $300 range.

Intel sent me a DG45FC, but its PCIe slot is only X1.

Suggestions?

Doing a quick search on Newegg came up with no mini-ITX boards that have bigger than a x1 PCIe slot. Plenty of micro ATX boards that do though.

00Roush

 

[thiggins]

Thanks for the suggestion, Tamarin. I'd like to go no larger than Micro-ATX, however.

 

[Tamarin]

You might try this board then. It has dual x16 PCI-e slots, as well as 6 onboard ports, and the ICH10R Southbridge.

http://www.newegg.com/Product/Product.aspx?Item=N82E16813136055

Tam