Building a new home server for NAS / web hosting / etc. Hardware opinions?

[joker535]

After 6 years of 24/7 service my MSI wind nettop with an Atom 230 finally died (motherboard burned up) so its time for a new server. This machine was serving 4 websites (2 wordpress and 2 static sites one of which is a business site), serving NAS duty for storage and backups, serving music and videos to our other devices, and acting as an SSH tunnel. The old system had a pair of enterprise class sata drives in it but still managed to pull under 35W at full load. I want to be able to run a hypervisor with a couple of VMs on the new one but I want to keep the power use around 50W total. This means I need either an 8 core Atom C2750 (8 cores no HT and 20W) or a low power E3-1230Lv3 (4 cores with HT and 25W). Both are passively cooled, have similar ram power consumption, and will use the same drives so its down to MB/CPU decision time. The small price difference is not a concern.

If you had a choice between this combo:

Intel Xeon E3-1230Lv3

Supermicro X10SLM+_F-O - http://www.newegg.com/Product/Produc...82E16813182823

Or this one:

Supermicro C2750 Atom board - http://www.newegg.com/Product/Produc...82E16813182851

Which one would you choose and why?

Do you think the 4 core hyperthreaded Xeon will be faster/more powerful/more capable than the 8 core Atom in a hypervisor environment?

I am going to buy one of these 2 options and I am not interested at looking at anything else.

What hypervisor do you think would run best on either of these setups?

What NAS option would you suggest to run as a VM on top of the Type 1 hypervisor? It will handle files/backup for 5 android devices, 3 ubuntu based linux machines, and an occasional Windows 7 machine that visits.

I am leaning toward the Xeon and Xen but I have not decided yet. The Atom is easier to get and $100 cheaper.

Thanks in advance

 

[Balthazar-B]

If you're reasonably adventurous, and know your way around Linux/Unix, check out SmartOS as a platform for your server (https://smartos.org/). You get arguably the best, most reliable storage solution (ZFS) and a lot of flexibility with virtual machines, and the whole platform is very cleverly designed for hosting.

 

[L&LD]

With up to half of your power budget going towards the Intel Xeon E3-1230Lv3 based platform (25W TDP) it may seem excessive. But that is only 5W more (TDP) than the Atom based platform. So with all things being equal, the Atom based system squeaks a 'win' here.

However, when raw cpu performance is considered, the Xeon is in a different class and almost twice as powerful.

http://ark.intel.com/compare/75053,77987

The above link shows the differences between the two.

If you require more than 32GB of ram for your setup, the Atom based solution is the best choice here.

But if 32GB of ram or less is sufficient, I would choose the Xeon platform with the much higher performance / watt capability.


https://cpubenchmark.net/compare.php?cmp[]=2116&cmp[]=2185

The most telling indicator here is the Single Thread Rating numbers; 1595 vs. 581 (Xeon, Atom, respectively).

Almost 3x the performance for a mere 5W more.

And of course, the TDP is not power draw. That is simply the maximum the processor can dissipate safely.

 

[joker535]

Interesting. I have never looked at that one before. Can it be installed to disk instead of just running as a live instance in memory?

 

[joker535]

I am pretty sure I will be going with the Xeon once I locate another one. I had found a US supplier with them in stock but I posted a link to it on another forum and the entire stock the vendor had sold out the same day (stupid move on my part). The E3-1230Lv3 (and E3-1240Lv3) is very hard to find outside of OEM channels.

 

[coxhaus]

I have always been a big believer in ECC ram for servers. They both have that covered and both can use a maximum of 32 gig. I assume you have a plan for your RAID card as you don’t mention it.

I would buy the Xeon motherboard as it has Intel NICs built-in and Xeons are usually heavy hitters compared to ATOM CPUs.

 

[joker535]

Yep. I already have the raid card, cables, and drives. I will be using 16G of DDR3 1600 ECC unbuffered RAM to start with. If I need more I will jump to 32G.

 

[L&LD]

The Atom based processor can take up to 64GB of ECC ram.

I have always been a big believer in ECC ram for servers. They both have that covered and both can use a maximum of 32 gig. I assume you have a plan for your RAID card as you don’t mention it.

I would buy the Xeon motherboard as it has Intel NICs built-in and Xeons are usually heavy hitters compared to ATOM CPUs.

 

[coxhaus]

The Atom based processor can take up to 64GB of ECC ram.

The specs on NEWegg show 32 gig.

I still would rather have Intel NICs. Better through put.

 

[azazel1024]

What are your actual requirements? I see what you had been running and I also see you'd like to do hypervisor. What else? RAM requirements? Are you going to be running anything more demanding than what you were? Do you need significantly more performance than previously?

My two suggestions are, SSD boot/app drive and run the websites off of that (if you think you can manage to fit everything for the sites within the SSD). This alone can significantly decrease page load times, especially if you can't cache everything in RAM for the sites.

Next would be, don't get the Atom board or Xenon. Yes, they are great, but the cost associated just doesn't seem to be a good way to go unless you need ECC RAM or have a specific requirement of a server related board/product.

I'd look at a Haswell Celeron or Pentium chip probably in an H97 board.

I am currently running a G1610 Ivy Celeron in an H77 board with 8GB of RAM (DDR3 1600MHz G.Skill sniper in 2x4GB congifuration running at 1333MHz (because Ivy Celeron limitations) and 1.2V), a 60GB SSD boot drive and when I had a pair of 2x2TB HDDs in RAID0 (currently have only a single 3TB drive, will shortly be getting another to put it in to 2x3TB RAID0) and a pair of Intel Gigabit CT NICs with an Antec Earthwatts 380.

Base power consumption from the wall at idle is 21w, 33w doing low power things with the HDDs spun up, such as streaming music/movies. Under heavy system load (HDDs spun up, CPU at >90% load across both cores) I get about 51w at the wall for total system power consumption.

I'd imagine a Haswell Celeron or Pentium as well as relying on the onboard NICs could easily push that down to 12-15w idle and only 40-45w or so at full load (ignore the TDP on the processor, a fully loaded Ivy Celeron only uses about 20W and a full loaded Ivy Pentium is only about 25w. I'd assume a Haswell Celeron is more like 22W or so and a Haswell Pentium is only 25-28w with slightly better idle and lower use power consumption. A lot of the power savings is better board components required for Haswell).

A Haswell Celeron/Pentium is certainly going to destroy the Atom in single thread and probably be in the same ballpark for multithread performance though likely slower (because you have a core that is roughly 2-2.5x more capable at the same clocks, running at roughly 15-30% faster). Actually for single low multithreaded performance, the Haswell Celeron/Pentium is likely to be faster than the E3-1230Lv3 because of the higher clock speed and similar architecture, though it'll lag far behind in multithreaded performance and not have much more power consumption under heavy load (the E3-1230Lv3 from what I have seen under heavy load actually uses closer to 30w, despite a claimed 25w TDP. The C2750 with a 20W TDP seems actually closer to 17-18W under heavy load, the G3220 despite a 54w TDP seems closer to 30W of power consumption under heavy load)

So just a question of what you need. Do you need every single virtualization nit and ECC? Do you need absolutely barebones power consumption? If no and no, then a Haswell Celeron/Pentium based server is likely to save you $150-300 in up front costs and likely will have minimal ongoing cost of ownership difference (at most maybe $10 a year in extra electrical costs, likely les of a difference than that) and has some performance benefits over either server system, though more the Atom than the Xenon.

If you need some of the virtualization nits and a bit more performance, you could consider a Haswell i3s or an -S or -T i5 to limit maximum power consumption. Gets you the virtualization stuff that celeron/Pentium are missing, a lot more performance and still quite a bit lower cost than the Atom or Xenon based boards (though the price difference would be a lot less) and idle power consumption honestly wouldn't be too much more (maybe just an extra 1-3w). Load power consumption, especially on an -T i3/5 processor isn't going to be much more than the Xenon.

If you really don't want to look at anything else, I'd go with the Xenon for the much higher single thread performance.

 

[L&LD]

The specs on NEWegg show 32 gig.

I still would rather have Intel NICs. Better through put.

That is that specific board. The processor is what supports up to 64GB ram.

 

[stevech]

Can you start with choosing great NAS software?
It's hard to find
Hardware later.

 

[joker535]

Can you start with choosing great NAS software?
It's hard to find
Hardware later.

I have run FreeNAS in the past and then last year switched to a simple samba setup using Ubunut 14.04 LTS server. The samba setup gave me a lot better transfer rates than FreeNAS but was slightly more complicated to set up.

 

[azazel1024]

If you were in a windows setup, I'd say run Server 2013/Windows 8.1 on the server. Significantly faster network transfers to other Windows 8+ machines via SMB Multichannel (at least if you have >1NIC in the machines in question). Even to non-windows machines you get the redundancy and aggregate bandwidth of link aggregation/teaming, without having to futz with link aggregation and teaming (you just don't get the single transfer stream speed that SMB Multichannel between server and client can provide).

I've heard rumblings that Samba may pickup SMB Multichannel support at some point, but as it stands, it has been ~3 years since Microsoft added the feature in SMB3.0 in Windows 8 and no one has since added support for it in any of their SMB3.0 implementations.

 

[joker535]

What are your actual requirements? I see what you had been running and I also see you'd like to do hypervisor. What else? RAM requirements? Are you going to be running anything more demanding than what you were? Do you need significantly more performance than previously?

My two suggestions are, SSD boot/app drive and run the websites off of that (if you think you can manage to fit everything for the sites within the SSD). This alone can significantly decrease page load times, especially if you can't cache everything in RAM for the sites.

Next would be, don't get the Atom board or Xenon. Yes, they are great, but the cost associated just doesn't seem to be a good way to go unless you need ECC RAM or have a specific requirement of a server related board/product.

I'd look at a Haswell Celeron or Pentium chip probably in an H97 board.

I am currently running a G1610 Ivy Celeron in an H77 board with 8GB of RAM (DDR3 1600MHz G.Skill sniper in 2x4GB congifuration running at 1333MHz (because Ivy Celeron limitations) and 1.2V), a 60GB SSD boot drive and when I had a pair of 2x2TB HDDs in RAID0 (currently have only a single 3TB drive, will shortly be getting another to put it in to 2x3TB RAID0) and a pair of Intel Gigabit CT NICs with an Antec Earthwatts 380.

Base power consumption from the wall at idle is 21w, 33w doing low power things with the HDDs spun up, such as streaming music/movies. Under heavy system load (HDDs spun up, CPU at >90% load across both cores) I get about 51w at the wall for total system power consumption.

I'd imagine a Haswell Celeron or Pentium as well as relying on the onboard NICs could easily push that down to 12-15w idle and only 40-45w or so at full load (ignore the TDP on the processor, a fully loaded Ivy Celeron only uses about 20W and a full loaded Ivy Pentium is only about 25w. I'd assume a Haswell Celeron is more like 22W or so and a Haswell Pentium is only 25-28w with slightly better idle and lower use power consumption. A lot of the power savings is better board components required for Haswell).

A Haswell Celeron/Pentium is certainly going to destroy the Atom in single thread and probably be in the same ballpark for multithread performance though likely slower (because you have a core that is roughly 2-2.5x more capable at the same clocks, running at roughly 15-30% faster). Actually for single low multithreaded performance, the Haswell Celeron/Pentium is likely to be faster than the E3-1230Lv3 because of the higher clock speed and similar architecture, though it'll lag far behind in multithreaded performance and not have much more power consumption under heavy load (the E3-1230Lv3 from what I have seen under heavy load actually uses closer to 30w, despite a claimed 25w TDP. The C2750 with a 20W TDP seems actually closer to 17-18W under heavy load, the G3220 despite a 54w TDP seems closer to 30W of power consumption under heavy load)

So just a question of what you need. Do you need every single virtualization nit and ECC? Do you need absolutely barebones power consumption? If no and no, then a Haswell Celeron/Pentium based server is likely to save you $150-300 in up front costs and likely will have minimal ongoing cost of ownership difference (at most maybe $10 a year in extra electrical costs, likely les of a difference than that) and has some performance benefits over either server system, though more the Atom than the Xenon.

If you need some of the virtualization nits and a bit more performance, you could consider a Haswell i3s or an -S or -T i5 to limit maximum power consumption. Gets you the virtualization stuff that celeron/Pentium are missing, a lot more performance and still quite a bit lower cost than the Atom or Xenon based boards (though the price difference would be a lot less) and idle power consumption honestly wouldn't be too much more (maybe just an extra 1-3w). Load power consumption, especially on an -T i3/5 processor isn't going to be much more than the Xenon.

If you really don't want to look at anything else, I'd go with the Xenon for the much higher single thread performance.

I appreciate all the info you have put together here. I am going to have this machine hosting a VM that will be handling the same load as the previous machine plus have the ability to host a couple of extra VMs. I would like to keep a Ubuntu Server 14.04 LTS VM and a Server 2012 R2 VM (I use both of these in my job) available for testing purposes in a type 1 hypervisor so I don't have to run them in a type 2 on my workstation. Its very convenient to be able to fire up a VM to test something for work then revert it back to the base snapshot and shut it down.

I just ordered the MB, ram, and heatsink for the Xeon setup. Once the CPU comes back in stock I will be ready to go.

Thanks

 

[joker535]

If you were in a windows setup, I'd say run Server 2013/Windows 8.1 on the server. Significantly faster network transfers to other Windows 8+ machines via SMB Multichannel (at least if you have >1NIC in the machines in question). Even to non-windows machines you get the redundancy and aggregate bandwidth of link aggregation/teaming, without having to futz with link aggregation and teaming (you just don't get the single transfer stream speed that SMB Multichannel between server and client can provide).

I've heard rumblings that Samba may pickup SMB Multichannel support at some point, but as it stands, it has been ~3 years since Microsoft added the feature in SMB3.0 in Windows 8 and no one has since added support for it in any of their SMB3.0 implementations.

I prefer to keeps things a windows free as possible here. I fire up a Server 2012 R2 VM once in a while for testing and the occasional Windows laptop passes through but otherwise this is a M$ free household. I even considered using the free Hyper-V 2012 R2 core hypervisor but its a pain to manage without a Windows 8 machine on hand. I think I am going to try Xen.

 

[stevech]

... otherwise this is a M$ free household.

Is there a merit badge for that?

 

[joker535]

Is there a merit badge for that?

Yep. Better efficiency, better security, and less cost (no licenses).

 

[azazel1024]

Debatable, debatable and okay, sure.

Windows actually seems to have a more robust and feature rich SMB/CIFS stack than Linux does currently if that is the route you are going for network stuff. It is also deffinitely lower power currently from everything I have seen, at least comparing Ubuntu to Windows 8.1/server 2012 (Windows takes much better advantage of current generation processor low power states). Though it deffinitely has a much bigger storage footprint, unsure of RAM footprint.

Looking at current security vulns from the past year...Windows actually had fewer reported vulns than Android or Linux. By a fair margin. Granted it sometimes takes MS longer to patch vulns and/or they leave some open for months to years before they patch them (compared to Linux, compared to Android...there probably isn't a less secure operating system than android unless maybe you are talking about Windows XP). Running EMET with windows actually makes it a much tighter ship than probably any existing operating system (of course that has its own problems, mostly related to security being TOO strict and really having to play with settings to get what you need to work (IE not being to overly restrictive)).

Windows is certainly not free, so there is that.

 

[joker535]

If you're reasonably adventurous, and know your way around Linux/Unix, check out SmartOS as a platform for your server (https://smartos.org/). You get arguably the best, most reliable storage solution (ZFS) and a lot of flexibility with virtual machines, and the whole platform is very cleverly designed for hosting.

I decided to test out smartos on a piece of test hardware I had laying around. Its an HP DL380G5 with dual quad core xeons, 16G of ram, and a 3TB R10 MSA attached. It ran but it was so slow it was really unusable. If it runs that slow on a machine like that I would hate to see it on a single xeon machine.

I appreciate the info though.