NAS vs. a Desktop Server - Why Buy a NAS?

coxhaus said:

No jumbo frames. I don't like them. Won't use them. They are too much trouble.

Click to expand...

Exactly, remains of the 10/100 Mbps age. No need to use them in GigE or 10GigE era.



Sent from my iPad using Tapatalk

Wolf_666 said:

Exactly, remains of the 10/100 Mbps age. No need to use them in GigE or 10GigE era.



Sent from my iPad using Tapatalk

Click to expand...

Oh, they work fine on my network. I'll go turn them off because I want a 4% performance hit.

Sure, they aren't NEEDED, but they do reduce CPU overhead, switch overhead and increase bandwidth slightly. So terrible.

Yeah, jumbo frames can sometimes break things, but I've been running them fine on my network and no networking device has yet complained about it.

In a small micro environment it is easy to control jumbo frames. You can hand pick your devices and test them. Jump to a 5000 PC network with all the associated network equipment and there is no way to control things. There are already so many network errors going on you would never compound your problems by adding jumbo frames.

You know all your sent and received internet traffic the router has to parse and build 9K frames, from the LAN side the 9K frames hit the router and have to be buffered then broken down to normal Ethernet packets which are sent into the internet as the internet will not accept jumbo frames. The router has double duty as the router has to build all the jumbo frames for the LAN side as the internet only feeds the router with standard Ethernet frames.

coxhaus said:

In a small micro environment it is easy to control jumbo frames. You can hand pick your devices and test them. Jump to a 5000 PC network with all the associated network equipment and there is no way to control things. There are already so many network errors going on you would never compound your problems by adding jumbo frames.

You know all your sent and received internet traffic the router has to parse and build 9K frames, from the LAN side the 9K frames hit the router and have to be buffered then broken down to normal Ethernet packets which are sent into the internet as the internet will not accept jumbo frames. The router has double duty as the router has to build all the jumbo frames for the LAN side as the internet only feeds the router with standard Ethernet frames.

Click to expand...

Actually no, the client NIC will send 1500MTU packets to the router, the router doesn't have to do any work to break down packets to send to WAN. Just because 9k jumbo is enabled does not mean that every single packet sent by the client NIC is going to be 1500MTU. It'll only send packets up to 9000MTU in size to recipients that will accept packets of up to that size. If 4k is the limit, then it will downsize its packets to 4k and so on.

Router is only going to accept 1500MTU in and out.

I've wiresharked traffic to my router...1500MTU only goes out from my computer to the router. Send stuff to my server and magically it is 9000MTU (most of them), to my tablet with 4k jumbo enabled...they are 4k in size.

I'd agree, in huge networks it probably isn't the best idea to muck with optional things. That said, I have seen jumbo frames working fine on ~400 client networks with nothing breaking.

Also considering that a number of people on here are talking about their own personal network of a small number of computers and network devices...pretty easy to trouble shoot.

"I won't use it because there may only be minimal gains and I hear it can break things" is often a bad argument for something. Especially in personal networking. "I can't chance bringing down my 3,000 computer campus" is an excellent reason however.

I always thought if you implemented jumbo frames it needed to be implemented on everything or there would be problems.

Nope. What will generally break things is if the jumbo frame implementation is somehow broken. I have run in to switches that don't properly accept something like 9k jumbo (as an example) even though they are supposed to. I've run 9k jumbo through switches, access points, etc. that do not accept jumbo at all, the client NIC always scales back to the proper MTU size, 1500. It just seems to be if the networking device improperly implements jumbo frames, that is when problems occur.

TCP windowing is (generally) a beautiful thing.

My current Cisco switches require you to turn on jumbo frames if you want to use them. So I assume you need to have jumbo frames everywhere. I still do not see how the devices know to ship small frames to the router when the packet size is 9K. Do the devices have variable packet sizes. I have not heard of this. I far as I know all packet sizes are the same fixed packet size but how much data is loaded into them is what varies. If you send 1 byte of data the frame is 9K with 1 byte of data in it.

http://en.wikipedia.org/wiki/Path_MTU_discovery

Path MTU discovery is what is used to determine the min/max MTU size on the network path. You get an ICMP message back for fragmentation if the MTU is too large, then the client knows to reduce the MTU size and then try again. Host OS then know on each path how large it can send packets.

The problem can come in if that ICMP fragmentation packet isn't transmitted back to the host, or if for some reason it gets blocked or the host doesn't responsed appropriately and reduce the frame size below the threshold. All of those can go wrong. They generally do not, but it can happen.

TCP windowing is then used to determine how many bytes/packets can then be sent along the path before an acknowledgement packet must be received.

Also, no, you can send frames of whatever size you want, so long as the frame has all of the header information there. You could send 250 byte packets if you wanted to. The header would inform the recipient of what data to expect based on the parity bits, so long as the payload and parity match, the recipient is happy.

Here is what I have found so far. This is an article in InfoWorld. I will keep looking.

The key concept to keep in mind is that all the network devices along the communication path must support jumbo frames. Jumbo frames needs to be configured to work on the ingress and egress interface of each device along the end-to-end transmission path. Furthermore, all devices in the topology must also agree on the maximum jumbo frame size. If there are devices along the transmission path that have varying frame sizes, then you can also end up with fragmentation problems. Also, if a device along the path does not support jumbo frames and it receives one, it will drop it.
http://www.networkworld.com/article/2224722/cisco-subnet/jumbo-frames.html

Here is something I found. I used the term buffer but the correct term is fragmentation when discussing the router

Fragmentation, which can occur in routers, as I'll discuss below, is a Layer 3 function. This leads us to another key point:

Key Point #3: For a jumbo packet to pass through a router, both the ingress and egress interfaces must support the larger packet size. Otherwise, the packets will be dropped or fragmented.

http://www.smallnetbuilder.com/lanw...o-know-jumbo-frames-in-small-networks?start=2

The process of utilizing the DF bit and ICMP messaging is referred to as PMTUD, or Packet MTU Discovery. This process applies to TCP flows, which provides retransmission. If the jumbo frames are utilizing UDP, there is no retransmission. If the sender sends jumbo frames utilizing UDP across a network that does support jumbo frames, and the receiver doesn't accept jumbo frames, the packets will be dropped. If the sender sends jumbo frames utilizing UDP across a network that doesn't support jumbo frames, the packets will either be dropped at Layer 2 or fragmented at Layer 3.

To summarize, there are many factors working against jumbo frame transmissions. Every device from end to end must support jumbo frames of the same frame size. The maximum frame size in a transmission is determined by the smallest maximum frame size supported end to end. Security restrictions on a network may cause black hole occurrences. Dropped packets will degrade connections. Finally, fragmentation and dropped packets can result in excessive retransmissions of data, defeating throughput gains.

I have to confess the last time I played with jumbo frames was in the 10/100 world before gig Ethernet. Jumbo frames have come a long ways since then.

Yes, with TCP jumbo frames is effectively a non-issue unless something has a broken implementation of various standards. If a frame of too large a size is sent, an ICMP fragmentation message will be sent back and the sender will reduce the frame size until the MTU is discovered and the sender will then use that.

If the sender doesn't implement MTU discovery correctly, it will not reduce the frame size, resulting in fragmentation/dropped packets. If something along the path does not implement TCP fragmentation correctly, it won't send back an ICMP fragmentation message if the frame size is exceeded.

In my experience, at least for networking products of the last 10 years, everything has implemented both ICMP fragmentation and MTU discovery properly (across at least 50-60 different switches and NICs I've used from various manufacturers). This means either jumbo frames being supported, or if they aren't or a smaller sized jumbo is supported, the sender correctly gets and reacts to ICMP fragmentation messages and reduces the frame size to the maximum allowable on the path.

Also in my experience, at least on NICs, enabling jumbo frames is ONLY enabling it for TCP/IP. It is NOT enabling it for UDP (Intel specifically calls this out that enabling jumbo frames on their NICs is only enabling TCP/IP jumbo frames), in part because of the mentioned issues with no way to react if the packet is too large other than relying on the networking services using UDP to have a built in method to react to fragmented frames.

You also have the issue that if ICMP is blocked internally, you'll never get fragmentation messages back if a device does not support the frame size you are attempting to use (which is why I don't have my switches blocking ICMP).

I think ICMP is not going to work across the internet as everybody blocks ping. It is a normal thing to block ping because of DDOS attacks. So for the little speed increase local you are going to degrade your internet access by slapping all the extra baggage of deciding how to transmit and waiting for responses and possibly fragmenting your internet packets. I still don't see it but it 's better implemented than in the old days.

Have you missed the many times I've said "your router is not going to accept jumbo frames"
You send a jumbo frame to your router (well, most routers I have seen) and it is going to kick back an ICMP fragmentation message until your computer reduces the frame size to 1500MTU.

AND/OR it will break up the packets in to 1500MTU (or smaller, as some WAN connections are a bit thriftier) packets to head out to the wilds of the internet.

I've ran wireshark a bunch on my network with several different routers. I see a standard 1500MTU packets being sent to my router and sent back from it even with jumbo frames on. I have on a couple of occasions seen a jumbo frame go out, ICMP fragmentation packet come back, smaller jumbo go out, rinse and repeat a few times (I want to say there were 4-6 send/responses before my computer just sent 1500MTU packets). Not sure how long Windows "remembers" the frame size on a path. At the very least, I don't think I've seen it on every new IP connection through my router, but I have seen it a few times.

I've seen it with...I think I've bothered looking on 3 different routers over the last few years maybe?

I know you don't see it, but since it works perfectly fine in practice 99% of the time...

The 1% can be a REAL B in an enterprise environment, but I've seen jumbo frames enabled on large networks too, and the networks had not come crashing down.

I still don't want the overhead associated with calculating and reducing the frame size to a standard Ethernet frame when I can just transmit a standard frame without any overhead by using the standard Ethernet frame size.

PS
I think this would have a negative impact for gaming but I have not tried it.

No, gaming is almost exclusively UDP and in general, jumbo frames being enabled is on the TCP/IP interface ONLY.

The overhead associated with calculating and reducing on a path that does not support jumbo frames is the first few miliseconds of the connection. You'd end up soaking up a fraction of one percent of a second's worth of bandwidth. Considering that the net increase is in the range of 4% or so, plus reduced CPU overhead...

The penalty is massively smaller than the potential pay offs. MTU discovery is done, at most, at the begining of a TCP/IP connection (and from some wiresharking, it looks like it may only be done on a path the first time a path is used if a ICMP fragmentation packet gets sent back. After that, the network layer on the host remembers that the path has an MTU limit of [insert limit here]).

azazel1024 said:

The overhead associated with calculating and reducing on a path that does not support jumbo frames is the first few miliseconds of the connection. You'd end up soaking up a fraction of one percent of a second's worth of bandwidth. Considering that the net increase is in the range of 4% or so, plus reduced CPU overhead...

Click to expand...

There is no net increase if you end up calculating down to a standard frame. You ended up with a net overhead loss. So all your internet traffic will fit in this class of transmission.

Except last I checked, I have a 75Mbps internet connection and a 1Gbps link to my router. Since WAN to LAN is slower than my LAN to LAN link to the router, there is zero loss.

Also as I mentioned (please read my posts careful, I can't help feeling like you are not). The MTU calculstion is ONLY done initially on the path or at worst, for each new TCP/IP connection. This means over the life of any given internet transfer (webpage, file transfer, etc.) you are talking at MOST hundreths of a percent impact on performance. In a lot of cases, LESS. Supposing your LAN to WAN was equal to your LAN to LAN, since LAN to LAN is significantly faster for most people, there is effectively no bandwidth impact at any point and MIGHT possibly be an initial lag of a few miliseconds on the connection, and only TCP/IP connections, and generally only on the first pathing to the gateway, not on every TCP/IP connection (I am not seeing MTU discovery to my router most of the time, I've only seen it a couple of times, which says to me that it doesn't do it on every TCP/IP connection, but only occasionally on a known path).

This will not happen at all on UDP, ICMP, etc. Locally, you've got that 4% performance boost as well as lower CPU overhead.

All of your internet traffic will fit in to "effectively no impact at all".

I think you will incur latency which I chase as much or more than bandwidth. I think there are lots of connections going on and I think this is happening more than you think. I don't think this is something I will ever be sold on. I am happy you like it and think the benefits are worth it. That 1% problems you talked about earlier are like some of the network problems I used to have to chase. They usually were Cisco IOS issues. If I have a network I have to resolve all the issues. I can't stand to have any faults in my network.

If you never negotiate MTU size, then there is no latency impact (other than the fractions of a MICROsecond, that larger packets incur with store and forward switches).

I just mentioned I've watched wireshark for awhile, and I've only ever seen a tiny number of MTU size operations performed on transfers to my router. This is when examining fairly large logs of data (several hours of traffic). Since that stuff isn't going over the internet, it is back to local latency, which since it only takes a few back and forths, is on rare occasions possibly adding 1-2ms of latency when MTU size determination has to be done. On rare, rare occasions (it is NOT every TCP/IP connection). And it never occurs on UDP, which is generally where you are going to care about latency (VPN, gaming, VOIP).

I understand I can't convince you, but I still can't help feeling like despite all of my explinations, you just don't understand how L1/L2/L3 work/my explinations. To me the only time you would not want to use jumbo frames is if

1) For whatever crazy reason, you do not want to spend anytime troubleshooting your network if they could possibly cause an issue (very small chance of issue occuring. Extremely simple to back-out jumbo frames if enabling it suddenly causes problems, doesn't take that long to see if stuff suddenly starts breaking). I can see this possibly being a concern on LARGE networks, small home networks however...

2) You do not do ANY LAN file transfers what-so-ever, and or ALL LAN file transfers are actually conducted wirelessly (IE nothing is done on the wire). In which case, you'll be limited to MTU restrictions of 802.11, which means no jumbo frames anyway.

3) Your networking gear does not support jumbo frames (most even dumb switches support up to 9k jumbo frames).

Other than that, it is pretty much just "it makes me unconfortable for unspecified reasons"

Intra-LAN, (not out the the Internet) and if all LAN nodes you wish to exchange traffic with, and all LAN switches support jumbo frames.

Too much of a PITA for me. Esp. with typical data traffic coming no where close the the capacity of even standard MTUs on gigE LANs.