部署CalicoNode（所有节点）报错：
Oct 10 21:04:55 smallsoup docker[76907]: bird: BGP: Unexpected connect from unknown address 192.168.1.104 (port 58153)
Oct 10 21:04:59 smallsoup docker[76907]: bird: BGP: Unexpected connect from unknown address 192.168.1.104 (port 40415)
Oct 10 21:05:00 smallsoup docker[76907]: 2018-10-10 13:05:00.213 [INFO][87] int_dataplane.go 690: Applying dataplane updates
Oct 10 21:05:00 smallsoup docker[76907]: 2018-10-10 13:05:00.213 [INFO][87] ipsets.go 224: Asked to resync with the dataplane on next update. family="inet"
Oct 10 21:05:00 smallsoup docker[76907]: 2018-10-10 13:05:00.213 [INFO][87] ipsets.go 255: Resyncing ipsets with dataplane. family="inet"
Oct 10 21:05:00 smallsoup docker[76907]: 2018-10-10 13:05:00.218 [INFO][87] ipsets.go 297: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=4.542824ms
Oct 10 21:05:00 smallsoup docker[76907]: 2018-10-10 13:05:00.218 [INFO][87] int_dataplane.go 705: Finished applying updates to dataplane. msecToApply=5.302726
查看节点运行情况，
$ calicoctl node status
Calico process is running.
IPv4 BGP status
+---------------+-------------------+-------+----------+--------------------------------+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+---------------+-------------------+-------+----------+--------------------------------+
| 192.168.1.103 | node-to-node mesh | start | 00:01:41 | Active Socket: Connection |
| | | | | closed |
+---------------+-------------------+-------+----------+--------------------------------+
IPv6 BGP status
No IPv6 peers found.
节点信息如下：
系统类型 IP地址 节点角色 CPU Memory Hostname
Centos7.2 192.168.1.103 master 1 2G server01
Centos7.2 192.168.1.102 worker 1 2G server02
Centos7.2 192.168.1.104 worker 1 2G server03
是因为/lib/systemd/system/kube-calico.service没ExecStart后面没加--IP参数，值为当前节点IP，应该是这样：
[Unit]
Description=calico node
After=docker.service
Requires=docker.service
[Service]
User=root
PermissionsStartOnly=true
ExecStart=/usr/bin/docker run --net=host --privileged --name=calico-node \
-e ETCD_ENDPOINTS=http://192.168.1.103:2379 \
-e CALICO_LIBNETWORK_ENABLED=true \
-e CALICO_NETWORKING_BACKEND=bird \
-e CALICO_DISABLE_FILE_LOGGING=true \
-e CALICO_IPV4POOL_CIDR=172.20.0.0/16 \
-e CALICO_IPV4POOL_IPIP=off \
-e FELIX_DEFAULTENDPOINTTOHOSTACTION=ACCEPT \
-e FELIX_IPV6SUPPORT=false \
-e FELIX_LOGSEVERITYSCREEN=info \
-e FELIX_IPINIPMTU=1440 \
-e FELIX_HEALTHENABLED=true \
-e IP=192.168.1.103 \
-v /var/run/calico:/var/run/calico \
-v /lib/modules:/lib/modules \
-v /run/docker/plugins:/run/docker/plugins \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /var/log/calico:/var/log/calico \
registry.cn-hangzhou.aliyuncs.com/imooc/calico-node:v2.6.2
ExecStop=/usr/bin/docker rm -f calico-node
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
结果：
[root@Server02 kubernetes-starter]# calicoctl node status
Calico process is running.
IPv4 BGP status
+---------------+-------------------+-------+----------+-------------+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+---------------+-------------------+-------+----------+-------------+
| 192.168.1.102 | node-to-node mesh | up | 13:05:42 | Established |
| 192.168.1.103 | node-to-node mesh | up | 13:10:56 | Established |
+---------------+-------------------+-------+----------+-------------+
IPv6 BGP status
No IPv6 peers found.
参考：Kubernetes集成Calico + 遇到的问题
k8s集群搭建好以后，小试牛刀：
[root@smallsoup net.d]# kubectl run kubernetes-bootcamp --image=jocatalin/kubernetes-bootcamp:v1 --port=8080
deployment "kubernetes-bootcamp" created
[root@smallsoup net.d]#
[root@smallsoup net.d]# kubectl get deployments
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kubernetes-bootcamp 1 1 1 1 4m
[root@smallsoup net.d]#
[root@smallsoup net.d]#
[root@smallsoup net.d]# kubectl get pods
NAME READY STATUS RESTARTS AGE
kubernetes-bootcamp-6b7849c495-vrcc2 1/1 Running 0 6m
[root@smallsoup net.d]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
kubernetes-bootcamp-6b7849c495-vrcc2 1/1 Running 0 6m 172.20.173.192 192.168.1.103
[root@smallsoup net.d]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
56165c91587d jocatalin/kubernetes-bootcamp "/bin/sh -c 'node se…" 5 minutes ago Up 5 minutes k8s_kubernetes-bootcamp_kubernetes-bootcamp-6b7849c495-vrcc2_default_878e7b6f-cc92-11e8-bd34-000c295db389_0
73ece5042794 registry.cn-hangzhou.aliyuncs.com/imooc/pause-amd64:3.0 "/pause" 7 minutes ago Up 7 minutes k8s_POD_kubernetes-bootcamp-6b7849c495-vrcc2_default_878e7b6f-cc92-11e8-bd34-000c295db389_0
a9b502648a37 registry.cn-hangzhou.aliyuncs.com/imooc/calico-node:v2.6.2 "start_runit" 41 minutes ago Up 40 minutes calico-node
[root@smallsoup net.d]#
[root@smallsoup net.d]# kubectl proxy
Starting to serve on 127.0.0.1:8001
[root@smallsoup ~]# curl http://127.0.0.1:8001/api/v1/proxy/namespaces/default/pods/kubernetes-bootcamp-6b7849c495-vrcc2/
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-6b7849c495-vrcc2 | v=1
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl scale deploy kubernetes-bootcamp --replicas=4
deployment "kubernetes-bootcamp" scaled
[root@smallsoup ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
kubernetes-bootcamp-6b7849c495-mpp5k 1/1 Running 0 4m 172.20.173.193 192.168.1.103
kubernetes-bootcamp-6b7849c495-nbbk5 1/1 Running 0 4m 172.20.0.192 192.168.1.102
kubernetes-bootcamp-6b7849c495-tgscr 1/1 Running 0 4m 172.20.0.128 192.168.1.102
kubernetes-bootcamp-6b7849c495-vrcc2 1/1 Running 0 32m 172.20.173.192 192.168.1.103
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kubernetes-bootcamp 4 4 4 4 32m
[root@smallsoup ~]# kubectl scale deploy kubernetes-bootcamp --replicas=2
deployment "kubernetes-bootcamp" scaled
[root@smallsoup ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
kubernetes-bootcamp-6b7849c495-mpp5k 1/1 Running 0 5m 172.20.173.193 192.168.1.103
kubernetes-bootcamp-6b7849c495-nbbk5 1/1 Terminating 0 5m 172.20.0.192 192.168.1.102
kubernetes-bootcamp-6b7849c495-tgscr 1/1 Terminating 0 5m 172.20.0.128 192.168.1.102
kubernetes-bootcamp-6b7849c495-vrcc2 1/1 Running 0 34m 172.20.173.192 192.168.1.103
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl describe deploy
Name: kubernetes-bootcamp
Namespace: default
CreationTimestamp: Wed, 10 Oct 2018 21:43:52 +0800
Labels: run=kubernetes-bootcamp
Annotations: deployment.kubernetes.io/revision=1
Selector: run=kubernetes-bootcamp
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 1 max unavailable, 1 max surge
Pod Template:
Labels: run=kubernetes-bootcamp
Containers:
kubernetes-bootcamp:
Image: jocatalin/kubernetes-bootcamp:v1
Port: 8080/TCP
Environment:
Mounts:
Volumes:
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
OldReplicaSets:
NewReplicaSet: kubernetes-bootcamp-6b7849c495 (2/2 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 38m deployment-controller Scaled up replica set kubernetes-bootcamp-6b7849c495 to 1
Normal ScalingReplicaSet 10m deployment-controller Scaled up replica set kubernetes-bootcamp-6b7849c495 to 4
Normal ScalingReplicaSet 4m deployment-controller Scaled down replica set kubernetes-bootcamp-6b7849c495 to 2
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl set image deploy kubernetes-bootcamp kubernetes-bootcamp=jocatalin/kubernetes-bootcamp:v2
deployment "kubernetes-bootcamp" image updated
[root@smallsoup ~]# kubectl rollout status deploy kubernetes-bootcamp
Waiting for rollout to finish: 1 old replicas are pending termination...
Waiting for rollout to finish: 1 old replicas are pending termination...
Waiting for rollout to finish: 1 old replicas are pending termination...
Waiting for rollout to finish: 1 of 2 updated replicas are available...
^C[root@smallsoup ~]#
[root@smallsoup ~]# kubectl rollout status deploy kubernetes-bootcamp
Waiting for rollout to finish: 1 of 2 updated replicas are available...
[root@smallsoup ~]# kubectl describe deploy
Name: kubernetes-bootcamp
Namespace: default
CreationTimestamp: Wed, 10 Oct 2018 21:43:52 +0800
Labels: run=kubernetes-bootcamp
Annotations: deployment.kubernetes.io/revision=2
Selector: run=kubernetes-bootcamp
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 1 max unavailable, 1 max surge
Pod Template:
Labels: run=kubernetes-bootcamp
Containers:
kubernetes-bootcamp:
Image: jocatalin/kubernetes-bootcamp:v2
Port: 8080/TCP
Environment:
Mounts:
Volumes:
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
OldReplicaSets:
NewReplicaSet: kubernetes-bootcamp-7689dc585d (2/2 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 41m deployment-controller Scaled up replica set kubernetes-bootcamp-6b7849c495 to 1
Normal ScalingReplicaSet 13m deployment-controller Scaled up replica set kubernetes-bootcamp-6b7849c495 to 4
Normal ScalingReplicaSet 7m deployment-controller Scaled down replica set kubernetes-bootcamp-6b7849c495 to 2
Normal ScalingReplicaSet 2m deployment-controller Scaled up replica set kubernetes-bootcamp-7689dc585d to 1
Normal ScalingReplicaSet 2m deployment-controller Scaled down replica set kubernetes-bootcamp-6b7849c495 to 1
Normal ScalingReplicaSet 2m deployment-controller Scaled up replica set kubernetes-bootcamp-7689dc585d to 2
Normal ScalingReplicaSet 1m deployment-controller Scaled down replica set kubernetes-bootcamp-6b7849c495 to 0
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl rollout status deploy kubernetes-bootcamp
deployment "kubernetes-bootcamp" successfully rolled out
[root@smallsoup ~]#
#如果一不小心版本写错了，可以回滚：
[root@smallsoup ~]# kubectl set image deploy kubernetes-bootcamp kubernetes-bootcamp=jocatalin/kubernetes-bootcamp:v20deployment "kubernetes-bootcamp" image updated
[root@smallsoup ~]#
[root@smallsoup ~]# kubectl rollout status deploy kubernetes-bootcamp
Waiting for rollout to finish: 1 old replicas are pending termination...
^C[root@smallsoup ~]# kubectl rollout undo deploy kubernetes-bootcamp
deployment "kubernetes-bootcamp"
[root@smallsoup ~]#
[root@smallsoup services]# vim nginx-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
[root@smallsoup services]# kubectl create -f nginx-pod.yaml
pod "nginx" created
[root@smallsoup services]# kubectl get pods
NAME READY STATUS RESTARTS AGE
kubernetes-bootcamp-7689dc585d-45w6d 1/1 Running 0 5m
kubernetes-bootcamp-7689dc585d-zxxdf 1/1 Running 0 10m
nginx 0/1 ContainerCreating 0 5s
[root@smallsoup services]# kubectl get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kubernetes-bootcamp 2 2 2 2 49m
[root@smallsoup services]# kubectl get pods
NAME READY STATUS RESTARTS AGE
kubernetes-bootcamp-7689dc585d-45w6d 1/1 Running 0 12m
kubernetes-bootcamp-7689dc585d-zxxdf 1/1 Running 0 16m
nginx 1/1 Running 0 6m
[root@smallsoup net.d]# kubectl proxy
Starting to serve on 127.0.0.1:8001
[root@smallsoup services]# curl http://127.0.0.1:8001/api/v1/proxy/namespaces/default/pods/nginx/
Welcome to nginx!
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
创建deployment：
[root@smallsoup services]# vim nginx-deployments.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 2
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
[root@smallsoup services]# kubectl create -f nginx-deployments.yaml
deployment "nginx-deployment" created
[root@smallsoup services]# kubectl get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kubernetes-bootcamp 2 2 2 2 1h
nginx-deployment 2 2 2 2 3m
[root@smallsoup services]# kubectl get pods -owide
NAME READY STATUS RESTARTS AGE IP NODE
kubernetes-bootcamp-7689dc585d-45w6d 1/1 Running 0 23m 172.20.173.197 192.168.1.103
kubernetes-bootcamp-7689dc585d-zxxdf 1/1 Running 0 27m 172.20.0.129 192.168.1.102
nginx 1/1 Running 0 17m 172.20.0.132 192.168.1.102
nginx-deployment-6c54bd5869-94bgh 1/1 Running 0 3m 172.20.0.133 192.168.1.102
nginx-deployment-6c54bd5869-js2r8 1/1 Running 0 3m 172.20.173.198 192.168.1.103
[root@smallsoup services]# kubectl get pods -o wide -l app=nginx
NAME READY STATUS RESTARTS AGE IP NODE
nginx-deployment-6c54bd5869-94bgh 1/1 Running 0 4m 172.20.0.133 192.168.1.102
nginx-deployment-6c54bd5869-js2r8 1/1 Running 0 4m 172.20.173.198 192.168.1.103
#Services
[root@smallsoup kubernetes-starter]# kubectl expose deploy kubernetes-bootcamp --type="NodePort" --target-port=8080 --port=80
service "kubernetes-bootcamp" exposed
[root@smallsoup kubernetes-starter]# kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.68.0.1 443/TCP 15h
kubernetes-bootcamp NodePort 10.68.194.69 80:27543/TCP 16s
[root@smallsoup kubernetes-starter]#
[root@smallsoup kubernetes-starter]# netstat -ntlp | grep 27543
tcp6 0 0 :::27543 :::* LISTEN 90067/kube-proxy
[root@smallsoup kubernetes-starter]# curl 192.168.1.103:27543
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-7689dc585d-45w6d | v=2
[root@smallsoup kubernetes-starter]# curl 192.168.1.102:27543
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-7689dc585d-45w6d | v=2
[root@smallsoup kubernetes-starter]# curl 192.168.1.104:27543
curl: (7) Failed connect to 192.168.1.104:27543; Connection refused
[root@smallsoup kubernetes-starter]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
kubernetes-bootcamp-7689dc585d-45w6d 1/1 Running 0 52m 172.20.173.197 192.168.1.103
kubernetes-bootcamp-7689dc585d-zxxdf 1/1 Running 0 57m 172.20.0.129 192.168.1.102
nginx 1/1 Running 0 47m 172.20.0.132 192.168.1.102
nginx-deployment-6c54bd5869-94bgh 1/1 Running 0 33m 172.20.0.133 192.168.1.102
nginx-deployment-6c54bd5869-js2r8 1/1 Running 0 33m 172.20.173.198 192.168.1.103
[root@smallsoup kubernetes-starter]# docker ps -a|grep bootcamp
230582f46c49 jocatalin/kubernetes-bootcamp "/bin/sh -c 'node se…" About an hour ago Up About an hour k8s_kubernetes-bootcamp_kubernetes-bootcamp-7689dc585d-45w6d_default_a7c9c95c-cc98-11e8-bd34-000c295db389_0
48c8ca188409 registry.cn-hangzhou.aliyuncs.com/imooc/pause-amd64:3.0 "/pause" About an hour ago Up About an hour k8s_POD_kubernetes-bootcamp-7689dc585d-45w6d_default_a7c9c95c-cc98-11e8-bd34-000c295db389_0
[root@smallsoup kubernetes-starter]# docker exec -ti 230582f46c49 bash
root@kubernetes-bootcamp-7689dc585d-45w6d:/# curl 10.68.194.69
Hello Kubernetes bootcamp! | Running on: kubernetes-bootcamp-7689dc585d-45w6d | v=2
root@kubernetes-bootcamp-7689dc585d-45w6d:/#
[root@smallsoup services]# vim nginx-service.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
ports:
- port: 8080
targetPort: 80
nodePort: 20000
selector:
app: nginx
type: NodePort
[root@smallsoup services]# kubectl create -f nginx-service.yaml
service "nginx-service" created
[root@smallsoup services]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.68.0.1 443/TCP 15h
kubernetes-bootcamp NodePort 10.68.194.69 80:27543/TCP 10m
nginx-service NodePort 10.68.80.238 8080:20000/TCP 2m
[root@smallsoup services]# curl 10.68.80.238:8080
[root@smallsoup services]# curl 192.168.1.103:20000
[root@smallsoup services]# curl 192.168.1.102:20000
Welcome to nginx!
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
dns服务：
[root@smallsoup kubernetes-starter]# kubectl create -f target/services/kube-dns.yaml
configmap "kube-dns" created
serviceaccount "kube-dns" created
service "kube-dns" created
deployment "kube-dns" created
[root@smallsoup kubernetes-starter]#
[root@smallsoup kubernetes-starter]# kubectl -n kube-system get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.68.0.2 53/UDP,53/TCP 48s
[root@smallsoup kubernetes-starter]#
[root@smallsoup kubernetes-starter]# kubectl -n kube-system get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kube-dns 1 1 1 1 1m
[root@smallsoup kubernetes-starter]# kubectl -n kube-system get pods
NAME READY STATUS RESTARTS AGE
kube-dns-74cdd9c8-smtld 3/3 Running 0 1m
[root@smallsoup kubernetes-starter]# kubectl -n kube-system get pods -owide
NAME READY STATUS RESTARTS AGE IP NODE
kube-dns-74cdd9c8-smtld 3/3 Running 0 1m 172.20.173.199 192.168.1.103
验证域名解析：
root@kubernetes-bootcamp-7689dc585d-45w6d:/#
root@kubernetes-bootcamp-7689dc585d-45w6d:/#
root@kubernetes-bootcamp-7689dc585d-45w6d:/# cat /etc/resolv.conf
nameserver 10.68.0.2
search default.svc.cluster.local. svc.cluster.local. cluster.local. DHCP HOST
options ndots:5
root@kubernetes-bootcamp-7689dc585d-45w6d:/#
root@kubernetes-bootcamp-7689dc585d-45w6d:/# curl nginx-service:8080