K8S 不同节点无法访问Pod

#只能运行pod的node访问此pod id

报错的信息：

#curl 10.96.16.213
curl: (7) Failed connect to 10.96.16.213:80; Connection refused

查看日志发现有报错

Jul 14 11:11:50 k8s-master NetworkManager[707]:   [1657768310.9063] device (tunl0): state change: unavailable -> unmanaged (reason 'connection-assumed', sys-iface-state: 'managed')
Jul 14 11:12:00 k8s-master NetworkManager[707]:   [1657768320.9023] device (tunl0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'managed')
Jul 14 11:12:00 k8s-master NetworkManager[707]:   [1657768320.9055] device (tunl0): state change: unavailable -> unmanaged (reason 'connection-assumed', sys-iface-state: 'managed')
Jul 14 11:12:10 k8s-master NetworkManager[707]:   [1657768330.9055] device (tunl0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'managed')
Jul 14 11:12:10 k8s-master NetworkManager[707]:   [1657768330.9096] device (tunl0): state change: unavailable -> unmanaged (reason 'connection-assumed', sys-iface-state: 'managed')
Jul 14 11:12:20 k8s-master NetworkManager[707]:   [1657768340.9071] device (tunl0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'managed')
Jul 14 11:12:20 k8s-master NetworkManager[707]:   [1657768340.9101] device (tunl0): state change: unavailable -> unmanaged (reason 'connection-assumed', sys-iface-state: 'managed')
Jul 14 11:12:30 k8s-master NetworkManager[707]:   [1657768350.9077] device (tunl0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'managed')
Jul 14 11:12:30 k8s-master NetworkManager[707]:   [1657768350.9124] device (tunl0): state change: unavailable -> unmanaged (reason 'connection-assumed', sys-iface-state: 'managed')
Jul 14 11:12:40 k8s-master NetworkManager[707]:   [1657768360.9099] device (tunl0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'managed')
Jul 14 11:12:40 k8s-master NetworkManager[707]:   [1657768360.9134] device (tunl0): state change: unavailable -> unmanaged (reason 'connection-assumed', sys-iface-state: 'managed')

查看NW

systemctl status NetworkManager
● NetworkManager.service - Network Manager
   Loaded: loaded (/usr/lib/systemd/system/NetworkManager.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2022-07-13 14:54:07 CST; 20h ago
     Docs: man:NetworkManager(8)
 Main PID: 707 (NetworkManager)
    Tasks: 3
   Memory: 5.0M
   CGroup: /system.slice/NetworkManager.service
           └─707 /usr/sbin/NetworkManager --no-daemon

Jul 14 11:08:10 k8s-master NetworkManager[707]:   [1657768090.8401] device (tunl0): state change: un...ed')
Jul 14 11:08:10 k8s-master NetworkManager[707]:   [1657768090.8437] device (tunl0): state change: un...ed')
Jul 14 11:08:20 k8s-master NetworkManager[707]:   [1657768100.8419] device (tunl0): state change: un...ed')
Jul 14 11:08:20 k8s-master NetworkManager[707]:   [1657768100.8447] device (tunl0): state change: un...ed')
Jul 14 11:08:30 k8s-master NetworkManager[707]:   [1657768110.8445] device (tunl0): state change: un...ed')
Jul 14 11:08:30 k8s-master NetworkManager[707]:   [1657768110.8478] device (tunl0): state change: un...ed')
Jul 14 11:08:40 k8s-master NetworkManager[707]:   [1657768120.8474] device (tunl0): state change: un...ed')
Jul 14 11:08:40 k8s-master NetworkManager[707]:   [1657768120.8496] device (tunl0): state change: un...ed')
Jul 14 11:08:50 k8s-master NetworkManager[707]:   [1657768130.8512] device (tunl0): state change: un...ed')
Jul 14 11:08:50 k8s-master NetworkManager[707]:   [1657768130.8548] device (tunl0): state change: un...ed')

继续查看，反正不理解什么意思

#nmcli dev status
DEVICE           TYPE      STATE      CONNECTION 
ens32            ethernet  connected  ens32      
docker0          bridge    unmanaged  --         
cali2c41246af73  ethernet  unmanaged  --         
cali63da9f00f44  ethernet  unmanaged  --         
cali64fde1b9429  ethernet  unmanaged  --         
cali8fd0d8937fc  ethernet  unmanaged  --         
caliac11e9a8d49  ethernet  unmanaged  --         
vethf100e4e      ethernet  unmanaged  --         
tunl0            iptunnel  unmanaged  --         
lo               loopback  unmanaged  --  

把NW禁用吧

 /etc/sysconfig/network-scripts/ifcfg-ens32 

加入
NM_CONTROLLED=no

重启NW再看

#nmcli dev status
DEVICE           TYPE      STATE         CONNECTION 
tunl0            iptunnel  disconnected  --         
docker0          bridge    unmanaged     --         
cali2c41246af73  ethernet  unmanaged     --         
cali63da9f00f44  ethernet  unmanaged     --         
cali64fde1b9429  ethernet  unmanaged     --         
cali8fd0d8937fc  ethernet  unmanaged     --         
caliac11e9a8d49  ethernet  unmanaged     --         
ens32            ethernet  unmanaged     --         
vethf100e4e      ethernet  unmanaged     --         
lo               loopback  unmanaged     --      

ok  curl 就通了

为什么呢？