《kubernetes-1.8.0》06-addon-calico
《kubernetes-1.8.0》06-addon-calico
《kubernetes 1.8.0 测试环境安装部署》
时间：2017-11-23
一、修改calico配置
在mritd.me/部署 Calico中提及：
官方文档中直接创建的 calico.yml 文件中，使用 DaemonSet 方式启动 calico-node，同时 calico-node 的 IP 设置和 NODENAME 设置均为空，此时 calico-node 会进行自动获取，网络复杂情况下获取会出现问题；比如 IP 拿到了 docker 网桥的 IP，NODENAME 获取不正确等，最终导致出现很奇怪的错误
经测试 2.6.1 calico-node镜像版本确实有这样的问题，漠然的方法是calico node采用systemd的方式控制，其他组件通过daemonset安装。后续calico node镜像升级成 2.6.2 该问题就没有再出现。
获取最新的calico.yaml：
$sudo mkdir ~/calico/
$cd ~/calico/
$wget https://docs.projectcalico.org/v2.6/getting-started/kubernetes/installation/hosted/calico.yaml查看calico-node所采用的镜像版本：
修改calico.yaml文件：
# 替换 Etcd 地址
sed -i 's@.*etcd_endpoints:.*@\ \ etcd_endpoints:\ \"https://172.18.169.131:2379,https://172.18.169.132:2379,https://172.18.169.133:2379\"@gi' calico.yaml
# 替换 Etcd 证书
export ETCD_CERT=`cat /etc/etcd/ssl/etcd.pem | base64 | tr -d '\n'`
export ETCD_KEY=`cat /etc/etcd/ssl/etcd-key.pem | base64 | tr -d '\n'`
export ETCD_CA=`cat /etc/etcd/ssl/etcd-root-ca.pem | base64 | tr -d '\n'`
sed -i "s@.*etcd-cert:.*@\ \ etcd-cert:\ ${ETCD_CERT}@gi" calico.yaml
sed -i "s@.*etcd-key:.*@\ \ etcd-key:\ ${ETCD_KEY}@gi" calico.yaml
sed -i "s@.*etcd-ca:.*@\ \ etcd-ca:\ ${ETCD_CA}@gi" calico.yaml
sed -i 's@.*etcd_ca:.*@\ \ etcd_ca:\ "/calico-secrets/etcd-ca"@gi' calico.yaml
sed -i 's@.*etcd_cert:.*@\ \ etcd_cert:\ "/calico-secrets/etcd-cert"@gi' calico.yaml
sed -i 's@.*etcd_key:.*@\ \ etcd_key:\ "/calico-secrets/etcd-key"@gi' calico.yaml二、修改kubelet配置
根据官方文档要求 kubelet 配置必须增加--network-plugin=cni选项，所以需要修改 kubelet 配置:
###
# kubernetes kubelet (minion) config
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=172.18.169.131"
# The port for the info server to serve on
# KUBELET_PORT="--port=10250"
# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=node.131"
# location of the api-server
# KUBELET_API_SERVER=""
# Add your own!
KUBELET_ARGS="--cgroup-driver=cgroupfs \
--network-plugin=cni \
--cluster-dns=10.254.0.2 \
--resolv-conf=/etc/resolv.conf \
--experimental-bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig \
--kubeconfig=/etc/kubernetes/kubelet.kubeconfig \
--fail-swap-on=false \
--cert-dir=/etc/kubernetes/ssl \
--cluster-domain=cluster.local. \
--hairpin-mode=promiscuous-bridge \
--serialize-image-pulls=false \
--pod-infra-container-image=gcr.io/google_containers/pause-amd64:3.0"分别重启4个节点的kubelet：
systemctl daemon-reload
systemctl restart kubelet查看节点状态：
[root@node-131 calico]# kubectl get node
NAME STATUS ROLES AGE VERSION
node.131 NotReady <none> 12h v1.8.0
node.132 NotReady <none> 12h v1.8.0
node.133 NotReady <none> 12h v1.8.0
node.134 NotReady <none> 12h v1.8.0此时执行 kubectl get node 会看到 Node 为 NotReady 状态，属于正常情况
三、创建calico Daemonset
# 先创建 RBAC
kubectl apply -f https://docs.projectcalico.org/v2.6/getting-started/kubernetes/installation/rbac.yaml
# 再创建 Calico Daemonset
kubectl create -f calico.yamlquay.io仓库的镜像还是拖的动的，这里就不docker load了，除了calico-node image，其他的镜像可以通过mritd提供的tarball进行load。
检查Daemonset和相应pod运行情况：
[root@node-131 images]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-94b7cb897-krckw 1/1 Running 0 29m
calico-node-5dc8z 2/2 Running 0 29m
calico-node-gm9k8 2/2 Running 0 29m
calico-node-kt5fk 2/2 Running 0 29m
calico-node-xds45 2/2 Running 0 29m
[root@node-131 images]# kubectl get ds -n kube-system
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
calico-node 4 4 4 4 4 29m 重启kubelet、docker：
systemctl restart kubelet
systemctl restart docker四、测试跨主机通讯
创建测试实例：
## 创建 deployment
$ mkdir ~/demo
$ cd ~/demo
$ cat << EOF >> demo.deploy.yml
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: demo-deployment
spec:
replicas: 4
selector:
matchLabels:
app: demo
template:
metadata:
labels:
app: demo
spec:
containers:
- name: demo
image: mritd/demo
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
EOF
$ kubectl create -f demo.deploy.yml
验证通信：
[root@node-131 images]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE
demo-deployment-5fc9c54fb4-5pgfk 1/1 Running 0 2m 192.168.177.65 node.132
demo-deployment-5fc9c54fb4-5svgl 1/1 Running 0 2m 192.168.33.193 node.131
demo-deployment-5fc9c54fb4-dfcfd 1/1 Running 0 2m 192.168.188.1 node.133
demo-deployment-5fc9c54fb4-dttvb 1/1 Running 0 2m 192.168.56.65 node.134
[root@node-131 images]# kubectl exec -ti demo-deployment-5fc9c54fb4-5svgl bash
bash-4.3# ping 192.168.56.66
PING 192.168.56.66 (192.168.56.66): 56 data bytes
64 bytes from 192.168.56.66: seq=0 ttl=62 time=0.407 ms
^C
--- 192.168.56.66 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.407/0.407/0.407 ms至此，群集网络组件calico搭建完成
本系列其他内容：
01-环境准备
02-etcd群集搭建
03-kubectl管理工具
04-master搭建
05-node节点搭建
06-addon-calico
07-addon-kubedns
08-addon-dashboard
09-addon-kube-prometheus
10-addon-EFK
11-addon-Harbor
12-addon-ingress-nginx
13-addon-traefik
参考链接：
https://mritd.me/2017/10/09/set-up-kubernetes-1.8-ha-cluster/
https://docs.projectcalico.org/v2.6/getting-started/kubernetes/