kubernetes网络方案calico

环境

系统：centos7.3
192.168.40.50 local-master
192.168.40.51 local-node1
192.168.40.52 local-node2

master:
kubernetes-master-1.5.2-0.2.gitc55cf2b.el7.x86_64
kubernetes-client-1.5.2-0.2.gitc55cf2b.el7.x86_64
etcd-3.1.0-2.el7.x86_64
docker-client-1.12.6-11.el7.centos.x86_64
docker-1.12.6-11.el7.centos.x86_64
docker-common-1.12.6-11.el7.centos.x86_64

node:
kubernetes-client-1.5.2-0.2.gitc55cf2b.el7.x86_64
kubernetes-node-1.5.2-0.2.gitc55cf2b.el7.x86_64
docker-client-1.12.6-11.el7.centos.x86_64
docker-1.12.6-11.el7.centos.x86_64
docker-common-1.12.6-11.el7.centos.x86_64

一. 修改各节点docker ip

master:

vi  /usr/lib/systemd/system/docker.service
--bip=172.25.0.1/24 \
# systemctl  daemon-reload
# systemctl  restart docker

node1：

vi  /usr/lib/systemd/system/docker.service
--bip=172.25.1.1/24 \
# systemctl  daemon-reload
# systemctl  restart docker

node2：

vi  /usr/lib/systemd/system/docker.service
--bip=172.25.2.1/24 \
# systemctl  daemon-reload
# systemctl  restart docker

二.下载插件

master：

# wget https://github.com/projectcalico/calicoctl/releases/download/v1.1.0/calicoctl
# chmod +x calicoctl
# mv calicoctl  /usr/bin/
# docker pull docker.io/calico/node:v1.1.0
# docker tag  docker.io/calico/node:v1.1.0  quay.io/calico/node:v1.1.0
# wget -N -P /opt/cni/bin/ https://github.com/projectcalico/calico-cni/releases/download/v1.6.0/calico
# wget -N -P /opt/cni/bin/ https://github.com/projectcalico/calico-cni/releases/download/v1.6.0/calico-ipam
# chmod +x /opt/cni/bin/calico /opt/cni/bin/calico-ipam

node

# docker pull docker.io/calico/node:v1.1.0
# docker tag  docker.io/calico/node:v1.1.0  quay.io/calico/node:v1.1.0
# wget -N -P /opt/cni/bin/ https://github.com/projectcalico/calico-cni/releases/download/v1.6.0/calico
# wget -N -P /opt/cni/bin/ https://github.com/projectcalico/calico-cni/releases/download/v1.6.0/calico-ipam
# chmod +x /opt/cni/bin/calico /opt/cni/bin/calico-ipam

三. 配置文件

参考：http://docs.projectcalico.org/v2.1/usage/configuration/as-service
集群所有主机都需要配置以下三个文件
创建calico-node启动文件

# vim /usr/lib/systemd/system/calico-node.service

[Unit]
Description=calico-node
After=docker.service
Requires=docker.service
[Service]
EnvironmentFile=/etc/calico/calico.env
ExecStartPre=-/usr/bin/docker rm -f calico-node
ExecStart=/usr/bin/docker run --net=host --privileged \
 --name=calico-node \
 -e NODENAME=${CALICO_NODENAME} \
 -e IP=${CALICO_IP} \
 -e IP6=${CALICO_IP6} \
 -e CALICO_NETWORKING_BACKEND=${CALICO_NETWORKING_BACKEND} \
 -e AS=${CALICO_AS} \
 -e NO_DEFAULT_POOLS=${CALICO_NO_DEFAULT_POOLS} \
 -e CALICO_LIBNETWORK_ENABLED=${CALICO_LIBNETWORK_ENABLED} \
 -e ETCD_ENDPOINTS=${ETCD_ENDPOINTS} \
 -e ETCD_CA_CERT_FILE=${ETCD_CA_CERT_FILE} \
 -e ETCD_CERT_FILE=${ETCD_CERT_FILE} \
 -e ETCD_KEY_FILE=${ETCD_KEY_FILE} \
 -v /var/log/calico:/var/log/calico \
 -v /run/docker/plugins:/run/docker/plugins \
 -v /lib/modules:/lib/modules \
 -v /var/run/calico:/var/run/calico \
 quay.io/calico/node:v1.1.0
ExecStop=-/usr/bin/docker stop calico-node

[Install]
WantedBy=multi-user.target

创建calico-node服务的环境文件

# vi /etc/calico/calico.env
ETCD_ENDPOINTS=http://local-master:2379
ETCD_CA_FILE=""
ETCD_CERT_FILE=""
ETCD_KEY_FILE=""
CALICO_NODENAME=""
CALICO_NO_DEFAULT_POOLS=""
#CALICO_IP=""
CALICO_IP=""
CALICO_IP6=""
CALICO_AS=""
CALICO_LIBNETWORK_ENABLED=true
CALICO_NETWORKING_BACKEND=bird

创建CNI网络配置文件，用于通知kubernetes创建一个名为calico-k8s-network的网络，并对此网络使用calico插件

# mkdir /etc/cni/net.d
# vi /etc/cni/net.d/10-calico.conf
{
    "name": "calico-k8s-network",
    "type": "calico",
    "etcd_endpoints": "http://local-master:2379",
    "log_level": "info",
    "ipam": {
        "type": "calico-ipam"
    },
    "policy": {
        "type": "k8s"
         k8s_api_root": "http://local-master:8080"
    }
}

master主机上修改

# vim /etc/etcd/etcd.conf
# [member]
ETCD_NAME=default
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="http://local-master:2380"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
#[cluster]
ETCD_INITIAL_CLUSTER="default=http://local-master:2380"
ETCD_ADVERTISE_CLIENT_URLS=http://local-master:2379

master 主机上
wget http://docs.projectcalico.org/v1.5/getting-started/kubernetes/installation/policy-controller.yaml
修改 policy-controller.yaml文件里的etcd的地址

图片.png

四. 启动

master

# serivce etcd restart
# kubectl  create -f policy-controller.yaml

每个节点上启动calico-node服务

# systemctl enable calico-node
# systemctl start calico-node
# export ETCD_AUTHORITY=local-master:2379

五.验证calico是否启动正常

calicoctl node status
calicoctl get nodes --out=wide

图片.png

图片.png

六.添加子网

[root@local-master v1.1.0]# cat pool.yaml 
apiVersion: v1
kind: ipPool
metadata:
  cidr: 172.25.0.0/16
spec:
  ipip:
    enabled: true
    mode: always
  nat-outgoing: true
[root@local-master v1.1.0]# calicoctl  apply -f pool.yaml

查看子网

图片.png

七.修改node上的配置

设置kube-proxy服务的proxy-mode配置为iptables，修改配置文件/etc/kubernetes/proxy：
KUBE_PROXY_ARGS=" --proxy-mode=iptables"
service kube-proxy restart

八.测试

两个node分别创建一个容器，两个容器内能互ping成功
相关：
http://docs.projectcalico.org/v2.1/introduction/
http://www.tuicool.com/articles/RVba2yr
http://blog.dataman-inc.com/shurenyun-docker-133/