Jumbo Frames - Enable them!

[Emulex]

yeah IIRC I've got a 10gbe nic that only supports 8000mtu as large jumbo frames. really throws a kink when the rest of jumbo assumes MTU of ~9K .

But ISCSI writes packets larger than 9K - so the nic acceleration will break them into pieces and reorganize them back offloaded. Jumbo really causes many low end switches to fail with FLOW control and jumbo packet size on at the same time due to lack of buffers!

 

[azazel1024]

Actually the flow control is an important point, I've found with jumbo frames having flow control ON is very important.

If I leave it off with jumbo frames on every single one of my non-Intel NICs fails spectacularly. If I have flow control on on my NICs and switches, then everything works great with jumbo frames on.

With the Intel NICs and jumbo frames, WITHIN the same switch it works fine with flow control off, across switches and again problems. With flow control on, everything works wonderfully.

 

[imagoon]

Actually the flow control is an important point, I've found with jumbo frames having flow control ON is very important.

If I leave it off with jumbo frames on every single one of my non-Intel NICs fails spectacularly. If I have flow control on on my NICs and switches, then everything works great with jumbo frames on.

With the Intel NICs and jumbo frames, WITHIN the same switch it works fine with flow control off, across switches and again problems. With flow control on, everything works wonderfully.

Flow control doesn't mean a whole lot when you don't have the buffer space to store the in flight frames.

 

[Emulex]

Precisely most lower end switches cannot allocate enough buffers to handle a flow control pause frame for long without dropping packets.

Kills ISCSI if you have any sort of tcp packet loss on the network!

 

[azazel1024]

I don't disagree. Of course that also enters in to a discussion of why one shouldn't use crappy low end switches in most cases.

 

[RadiclDreamer]

I don't disagree. Of course that also enters in to a discussion of why one shouldn't use crappy low end switches in most cases.

Not to mention crappy low end (non intel) network cards.