calico网络学习一(容器和calico的veth pair通讯过程)
容器和calico的veth pair通讯过程
本文章分析的前提是在calico部署在k8s上，并开启了ipip的功能
root@test-master-113 ~]# calicoctl get ippool -oyaml
- apiVersion: v1
kind: ipPool
metadata:
cidr: 192.168.0.0/16
spec:
ipip:
enabled: true
nat-outgoing: true每一个容器通过calico cni-plugin 插件设置默认的路由，都会在指定的network-ns创建veth-pair,位于容器内的veth将会被设置成169.254.1.1
# ip route
default via 169.254.1.1 dev eth0
169.254.1.1 dev eth0 scope link容器内默认路由的mac地址被设置成主机节点上的某个一cali开头的网卡
# arp
Address HWtype HWaddress Flags Mask Iface
169.254.1.1 ether 4e:e1:53:f4:e0:06 C eth0
# ip neigh
169.254.1.1 dev eth0 lladdr 4e:e1:53:f4:e0:06 REACHABLEmac地址为4e:e1:53:f4:e0:06
cni-plugin创建了endpoint之后，会将其保存到etcd中，felix从而感知到endpoint的变化。felix会在host端设置一条静态arp:
root@test-slave-114 ~]# ip link show cali34d45bc515f
18704: cali34d45bc515f@if4: mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 4e:e1:53:f4:e0:06 brd ff:ff:ff:ff:ff:ff link-netnsid 14 查看容器的网卡信息
[root@test-master-113 ~]# calicoctl get workloadendpoint --workload=tanyanliao.beegotest-3324938723-l2tkm -oyaml
- apiVersion: v1
kind: workloadEndpoint
metadata:
labels:
ClusterID: CID-ca4135da3326
UserID: "102"
calico/k8s_ns: tanyanliao
diskType: ""
name: beegotest
pod-template-hash: "3324938723"
tenxcloud.com/appName: beegotest
tenxcloud.com/svcName: beegotest
name: eth0
node: test-slave-114
orchestrator: k8s
workload: tanyanliao.beegotest-3324938723-l2tkm
spec:
interfaceName: cali34d45bc515f
ipNetworks:
- 192.168.134.64/32
mac: d6:e5:13:88:83:8c
profiles:
- k8s_ns.tanyanliao可以看到这个容器tanyanliao.beegotest-3324938723-l2tkm的网卡名称为cali34d45bc515f,在主机上查看对应的网卡cali34d45bc515f的信息
[root@test-slave-114 ~]# ip link show cali34d45bc515f
18704: cali34d45bc515f@if4: mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 4e:e1:53:f4:e0:06 brd ff:ff:ff:ff:ff:ff link-netnsid 14 可以看到网卡的mac地址是4e:e1:53:f4:e0:06跟在容器内执行arp命令查询出来的mac地址信息是一致的，也就是说容器内的流量不管是流进还是流出都是通过默认的路由流进流出的
容器内的网卡mac地址
# ip addr
1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: tunl0@NONE: mtu 1480 qdisc noop state DOWN group default qlen 1
link/ipip 0.0.0.0 brd 0.0.0.0
4: eth0@if18704: mtu 1500 qdisc noqueue state UP group default
link/ether d6:e5:13:88:83:8c brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 192.168.134.64/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::d4e5:13ff:fe88:838c/64 scope link
valid_lft forever preferred_lft forever
以上说明：
cni-plugin插件在容器生成了默认的路由，由于默认的IP169.254.1.1是无效的路由，因此，该mac地址设置成了该容器在node节点上的mac地址4e:e1:53:f4:e0:06(这一步是通过calico felix完成的)，也就是说,容器内的veth mac地址为d6:e5:13:88:83:8c的流量转发是通过默认路由转发到主机生对应的网卡，然后通过ipip隧道走出去的(前提是开了ipip功能的calico)，IPIP隧道是通过tunl0，通讯的
[root@test-slave-114 ~]# ip addr |grep tun
4: tunl0@NONE: mtu 1440 qdisc noqueue state UNKNOWN qlen 1
inet 192.168.134.113/32 brd 192.168.134.113 scope global tunl0 参考:
calico的架构设计与组件交互过程