Disabling Offloading on Ubuntu 18.04 with Netplan and/or systemd-networkd
原文： https://michael.mulqueen.me.uk/2018/08/disable-offloading-netplan-ubuntu/
发布于：21 Aug 2018
Why Disable Offloading?
Firstly, why not disable offloading? Offloading is a way for the operating system and your CPU to offload some of the work involved in transmitting packets (which can be significant, especially at higher speeds. The two main cases that I’ve come across for this are offloading the calculation of TCP checksums and handling fragmentation. Hardware offloading should be a good thing and it often is, I’ve only rarely found a need to disable it.
However, sometimes it just seems to break things and you can’t find a better work around. For example, I had a problem earlier where a particular host of a network was sending TCP segments and they were being routed to their destinations but then being discarded. I set up a packet capture on another remote host under our control and I was seeing a SYN arriving and then more SYNs as retransmission was attempted and no SYN-ACK was being sent back. This was isolated to this host, no other hosts on the sending network had this issue. I ruled out the other usual suspects - NAT, firewall rules etc.
Finally, I decided there was nothing left to try other than disable offloading and like that, the issue was gone! I had to disable checksum offloading, but you may have a slightly different issue.
Disabling Offloading Temporarily
This should work whatever you’re using to manage your network.
To do this, you’ll need to have ethtool installed:
sudo apt install ethtool
Once it’s installed, work out what interface is affected. For the sake of this post, we’ll say it’s eth0, but it could be something completely different (e.g. br1, vnet2, enp1s2)
My problem was caused by checksum offloading, which can be disabled by running this command:
sudo ethtool --offload eth0 rx off tx off
rx off disables for things you’re receiving. tx off disables it for things you’re transmitting.
If the problem goes away, then you’ve found the cause. Great. Bear in mind though, when you next reboot, these settings will be reset to default and the problem will come back.
If this didn’t solve the problem, it may be worth looking at some of the features that can be enabled/disabled:
-
rx- receive (RX) checksumming -
tx- transmit (TX) checksumming -
sg- scatter-gather -
tso- TCP segmentation offload -
ufo- UDP fragmentation offload -
gso- generic segmentation offload -
gro- generic receive offload -
lro- large receive offload -
rxvlan- receive (RX) VLAN acceleration -
txvlan- transmit (TX) VLAN acceleration -
ntuple- receive (RX) ntuple filters and actions -
rxhash- receive hashing offload
In general, it’s best to disable the minimum amount of features you need to resolve your issue rather than blanket disabling everything.
Netplan and systemd-networkd
Ubuntu 18.04 brought with it Canonical’s new answer to the problem of network configuration, netplan. I don’t particularly like netplan, but I also like to use the defaults wherever possible. On a server, the best choice is to use the systemd-networkd backend to netplan (the other choice is NetworkManager). You might argue that an even better choice is to cut netplan out of the loop and use systemd-networkd directly. In either case, I’m assuming in this post that you’re either using netplan and systemd-networkd together or systemd-networkd alone.
If you’ve used netplan, you’ll realise that for basic uses, it’s quite nice and neat, but for more advanced uses, you start to find the bugs and the missing features (though it is continuing to improve).
Disabling Offloading Permanently
Netplan doesn’t offer a way to turn off offloading within its configuration. Systemd-networkd does provide some options to disable offloading, although I wasn’t able to find options for all of the features available in ethtool, for this reason I’ve not investigated it further.
Luckily, Ubuntu 18.04 comes with the networkd-dispatcher package. You can use this to run scripts (like legacy ifup/ifdown scripts).
If you’re missing the package:
sudo apt install networkd-dispatcher
We need to create a script to run as the network comes up:
sudo nano /usr/lib/networkd-dispatcher/routable.d/10-disable-offloading
I put the following content in it:
#!/bin/sh
ethtool --offload eth0 rx off tx off
You’ll need to change this if you’ve used other options or a different network interface.
Finally, ensure that it’s executable:
sudo chmod +x /usr/lib/networkd-dispatcher/routable.d/10-disable-offloading
If everything has worked, you’ll now be able to boot without issue.
Please note, I’ve located this configuration file in /usr/lib/networkd-dispatcher due to LP #1765152, you may see /etc/networkd-dispatcher in other distributions or other documentation. The correct location in Ubuntu 18.04 is in /usr/lib/networkd-dispatcher, even if that seems a bit odd.
追加说明： LP #1765152BUG解决后, Scripts can be installed into these directories under /usr/lib/networkd-dispatcher for system packages, and /etc/networkd-dispatcher for local overrides
红帽Linux技术支持页面给出的解决方案
How to write a NetworkManager dispatcher script to apply ethtool commands?
SOLUTION
环境
- Red Hat Enterprise Linux 8
- Red Hat Enterprise Linux 7
- NetworkManager used to manage network connections
- Network interface supporting configurable offload via ethtool -K commands
问题
How to write a NetworkManager dispatcher script to apply ethtool commands?
Need to apply ethtool offloading to a network interface at boot or startup?
What is the correct syntax for a NM dispatcher script to enable or disable offloading like checksumming or large receive (GSO/LRO) or segmentation (GRO/LSO)?
决议
Create a unique executable file in the /etc/NetworkManager/dispatcher.d/ directory, for example:
sudo su
touch /etc/NetworkManager/dispatcher.d/30-ethtool
chmod +x /etc/NetworkManager/dispatcher.d/30-ethtool
For a standalone network interface:
#!/bin/bash
if [ "$1" == "eth0" ] && [ "$2" == "up" ]; then
ethtool -K "$1" rx off gro off lro off
fi
For 2 standalone network interfaces:
#!/bin/bash
if [ "$1" == "eth0" -a "$2" == "up" ] || [ "$1" == "eth1" -a "$2" == "up" ] ; then
ethtool -K "$1" rx off gro off lro off
fi
For a bonding or teaming network interface:
#!/bin/bash
if [ "$1" == "bond0" ] && [ "$2" == "up" ]; then
for INTERFACE in bond0 eth0 eth1; do
ethtool -K "$INTERFACE" rx off gro off lro off
done
fi
Modify the list of interfaces as required, ensure the parent bond/team device and all slave devices have settings applied to them.
For RHEL7 in /etc/sysconfig/network-scripts/ifcfg-* you may have:
ETHTOOL_OPTS="-K ${DEVICE} gso off gro off tso off"
if more options then use like
ETHTOOL_OPTS="-K ${DEVICE} gso off;-K ${DEVICE} gro off;-K ${DEVICE} tso off"
you have to have naturally DEVICE defined in your ifcfg file.
No rc.local nor extra ifup scripts needed. Easy to you in generic deployment systems.
但如果你的Linux虚拟机是带图形界面的Desktop版本, 还需要手动创建一个Network Manager配置文件/etc/NetworkManager/dispatcher.d/20-ethtool
If you are on RHEL7 (or alike) and use Network Manager instead of /etc/init.d/network to control your interfaces the proposed answer will not work, since /sbin/ifup-local (as well as ifdown-pre-local and ifdown-local) will never get executed.
Instead put your scripts into /etc/NetworkManager/dispatcher.d/ and make sure the NetworkManager-dispatcher service is enabled
systemctl enable NetworkManager-dispatcher
Hint: The dispatcher will only kick in once NetworkManager makes changes to an interface, it doesn't need to be running or anything, so if the status reads
Active: inactive (dead)
that's completely fine!
Also make sure your script is:
- executable (chmod +x)
- owned by root (chown root:root)
- writable by root only (chmod 755)
Now NetworkManager will pass two (2) variables to the dispatcher:
$1 being the interface (eno16777984, eth0, ppp0, etc...)
$2 holding the status (either up or down)
and it allows chaining the scripts (just like /etc/rc...) to have some control over the order in which the dispatcher will execute them:
10-first, 20-second and so on...
The order will be ascending on a connect
if [$2 = "up"] its 10-first followed by 20-second
and descending on a disconnect
if [$2 = "down"] you get 20-second followed by 10-first
and so on.
So to accomplish what OP was looking for you could put something like this:
#!/bin/bash
# 注意bash脚本中括号要写 [[ ]]否则逻辑错误
if [[ "$1" = "eth0" && "$2" = "up" ]]; then
/sbin/ethtool --offload "$1" gro off
fi
in /etc/NetworkManager/dispatcher.d/20-ethtool
参考：
- https://serverfault.com/questions/463111/how-to-persist-ethtool-settings-through-reboot