Jumbo Frames, Interupt Moderation

[BTRY B 529th FA BN]

If one PC on the LAN is using Jumbo Frames do all others have-to to avoid problems with the Router?

EDIT:

I am also curious how to reduce my interrupt moderation as much as possible without increasing the latency in sending packets.

I've got a Intel Pro PT Dual NIC, and a Intel Gigabit Intel CT Desktop adapter as a backup.

Thanks

 

[spidey07]

TCP will be fine as the max segment side will be agreed upon. UDP or any other protocol then all bets are off.

It's generally a "bad idea" to have all machines on a segment with different MTUs.

 

[BTRY B 529th FA BN]

Do I have to manually set the MTU to 9k, or by setting 9k (Jumbo Frames) in the Advanced adapter properties does it take care of this?

? is there a way to adjust UDP ? I thought I read something about FastSendDatagramThreshold

 

[imagoon]

Jumbo Frames are pointless except in a SAN situation and even then the result today is minimal. You have to set every device to MTU 9000, 9015, 9018, 9218 depending on how the device represents jumbo frames. UDP is not MTU aware and will get "lost" or fragmented at the MTU transition. This typically wrecks UDP performance. If you're doing this for "gaming" then stop because you are doing nothing other than hurting yourself.

 

[BTRY B 529th FA BN]

Then how do you set everything up for maximum throughput + low system latency (not internet latency/ping) ? Are you a network guru?

 

[imagoon]

Then how do you set everything up for maximum throughput + low system latency (not internet latency/ping) ? Are you a network guru?

You're question is pretty broad.

For most home users there is minimal you can do to improve latency because is most cases your connection to the router is in the sub 1 millisecond range. There periodically is tweaks specific to certain games such as WoW and the TCP ack frequency. It is very dependent on how the app is built. In most cases having a halfway decent NIC and a half decent router is really all you need. The half way decent Nic will handle stacking frames in interrupt groups (vs 1 interrupt per frame), the 1/2 decent router handles keeping interface to interface latency down.

I am still not clear on this system latency you seem to be a bit obsessed with lately here in the networking forum. We have servers handling multiple 10gigE connections not going above 2-3% CPU for network layer interrupt handling [on single core]. Those servers are seeing more than a level magnitude of interrupt activity you home computer will be seeing.

Short of SAN's we don't bother with Jumbo frames because the CPU savings isn't work the time to configure it all and tweak it. Add in the varying level of support in everything from the NIC, routers and switches it isn't worth it to gain 1% CPU on one core on one system.

As for the guru thing, I'll leave it up to you. I have built and deployed multinational networks for a Fortune 500. Take that as you will.

 

[dawks]

Yea all these little tweaks aren't going to have much of an impact on ping, if any at all. Many of the changes you are looking at simply reduce CPU utilization. It might be beneficial if you were running large internet servers with thousands of clients, and millions of packets per second.. But if you're running any dual core 2Ghz CPU from the last 5 years, its really inconsequential in terms of gaming performance. If you want a different ping, choose a different server, or try a different ISP. Jumbo frames wont change you're gaming experience, but if anything it will degrade it, as your router has to break the packets down and push them over the internet (since jumbo frames arent really supported on the public internet).

 

[imagoon]

add on to Dawks:

Most major game provides as send fragmented packets from jumbo frames to /dev/null also.

 

[JackMDS]

It might be beneficial if you were running large internet servers with thousands of clients, and millions of packets per second.

+1
----------

Most users (Including Enthusiasts) are basically Ignorant about Networking (posting twice a day the word stable & DD-WRT does not make One Network savvy :sneaky: - ).

Users carry the mundane knowledge of CPU OC and Video frames pushing into Networking which is causing more harm than being useful.

Unless there is a special situation similar to what described in the Quote.

Leave the NIC configuration and other Networking parameters at Default.

 

[Jamsan]

Deleted.

-Jack
Moderator.

 

[BTRY B 529th FA BN]

Aowe, he's not!? This is BS!


:whiste:

 

[cmetz]

BTRY, as a general rule, you must configure all L2 domains with the same value for MTU and MRU for all stations. (Yes, advanced topic, you can build a mixed network and you can make it work with the right configuration. Don't. Just don't.)

With the cards you have, make sure you have MSI or MSI-X in use. If you want lowest latency, you should disable interrupt coalescing, while if you want highest throughput, you should leave it on or adaptive. These are all receive-side issues; as far as I know there is nothing you can or should need to tweak on the send-side.

 

[BTRY B 529th FA BN]

BTRY, as a general rule, you must configure all L2 domains with the same value for MTU and MRU for all stations. (Yes, advanced topic, you can build a mixed network and you can make it work with the right configuration. Don't. Just don't.)

How is the MRU edited? Just curious

With the cards you have, make sure you have MSI or MSI-X in use. If you want lowest latency, you should disable interrupt coalescing, while if you want highest throughput, you should leave it on or adaptive. These are all receive-side issues; as far as I know there is nothing you can or should need to tweak on the send-side.

Ok. :hmm: I've tried doin a little Googgling to get familiar with some terminology. I'm not sure what MSI or MSI-X means in total. I'm guessing it refers to Interrupt Moderation, not Interrupt Moderation Rate


There are 2 different Interrupt Moderation settings.

Interrupt Moderation = Enabled, or Disabled
Interrupt Moderation Rate = Off, Minimal, Low, Medium, High, Extreme, Adaptive.

 

[cmetz]

BTRY, as far as I know all modern OSs set the MRU to be the same as the MTU of the interface. While technically possible for the values to be different, that would cause only badness in practice, so in practice MRU==MTU.

MSI/MSI-X - see:

https://en.wikipedia.org/wiki/Message_Signaled_Interrupts

Not really well covered in this article - old-fashioned out of band PCI interrupts had a terribly high latency and were just plain bad for performance. MSI helps with that. Also, PCI-E is just plain better for performance than the old PCI bus.

Interrupt coalescing (interrupt moderation) is a trick whereby the receiver side of the controller only interrupts the system when it has received a certain number of packets or a certain amount of time has gone by since the first receive. It increases throughput but also increases latency. It was a bigger deal back in the PCI days, when the interrupt latency was high and there was only one bus and it had to be totally interrupted to service a device interrupt. In modern PCI-E systems, while it does help, it's not such a big deal.

You can play with the values, but I would suggest enabled/adaptive. If the driver is well written (I believe Intel's NIC drivers are), it should do a good job of automatically flipping you between lower latency and higher throughput configurations based on current need.