快速部署单机版kubernetes 1.15.0

单机版本安装kubernetes1.15.0基本在1.13.4的脚本基础上进行，相关信息memo如下。

• 安装单机版kubernetes离线设定文件，使用如下步骤即可。

步骤1：下载easypack

[root@host131 tmp]# git clone https://github.com/liumiaocn/easypack.git
Cloning into ‘easypack’…
…省略
[root@host131 tmp]#

步骤2: 下载相关离线二进制文件

• 修改并确定要下砸的各组件的版本号

[root@host131 tmp]# cd easypack/k8s/shell/
[root@host131 shell]# head -n11 step0-get-binaries.sh 
#!/bin/sh

TYPE_OS=linux
TYPE_ARCH=amd64
VER_CFSSL=R1.2
#VER_KUBERNETES=v1.13.4
VER_KUBERNETES=v1.15.0
VER_ETCD=v3.3.13
VER_DOCKER=18.09.7
VER_CNI=v0.7.4
VER_FLANNEL=v0.11.0
DIR_DOWNLOAD=/tmp
[root@host131 shell]#

执行如下命令即可完成下载

sh step0-get-binaries.sh

缺省的情况下会在/tmp/binaries下进行二进制的收集，可自行修改

[root@host131 shell]# head -n10 install.cfg 
#!/bin/sh

#######BEGIN: SETTING: BASIC#########
ENV_HOME_BINARY=/tmp/binaries
ENV_HOME_CFSSL=${ENV_HOME_BINARY}/cfssl
ENV_HOME_ETCD=${ENV_HOME_BINARY}/etcd
ENV_HOME_FLANNEL=${ENV_HOME_BINARY}/flannel
ENV_HOME_DOCKER=${ENV_HOME_BINARY}/docker
ENV_CURRENT_HOSTIP="192.168.163.131"
#######END  : SETTING: BASIC#########
[root@host131 shell]#

注：下载请自行完成

步骤3: 调整配置

最简单的方式，就是把如下的配置文件中的IP进行全文替换应该即可：192.168.163.131

[root@host131 shell]# ls install.cfg 
install.cfg
[root@host131 shell]#

步骤4: 清空数据并安装

管理脚本的说明

目前的功能，稍微包了一下，参看如下usage即可知道：

[root@host131 shell]# sh all-k8s-mgnt.sh 
Usage: all-k8s-mgnt.sh ACTION TYPE
       ACTION:start|stop|restart|status|install|clear
       TYPE:master|node|docker|ssl|apiserver|scheduler|controller
            kubelet|kubeproxy|flannel|etcd

[root@host131 shell]#

主要是一行行的systemctl和执行各个脚本太繁琐，稍作简化。需要说明的是clear会删除所有的临时文件和设定文件，请各位执行之前务必确认不要删错

if [ _"$ACTION" = _"clear" ]; then
  # in order to avoid rm -rf / : here hard coding for default dir
  echo "## data dir clear operation begins..."
  echo " # clear ssl dirs "
  rm -rf /etc/ssl/{ca,etcd,flannel,k8s} 
  echo " # clear etc dirs " 
  rm -rf /etc/{docker,flannel,k8s,etcd,kubernetes}
  echo " # clear log dirs "
  rm -rf /var/log/kubernetes
  echo " # clear working dirs or data dirs"
  rm -rf /var/lib/kubelet /var/lib/k8s /var/lib/docker /var/lib/etcd 
  echo "## data dir clear operation ends  ..."
  exit 0
fi

执行命令：
sh all-k8s-mgnt.sh stop all; sh all-k8s-mgnt.sh clear all; sh all-k8s-mgnt.sh install all

注：可多次执行，如果提示错误，上述命令可再执行一遍即可。

• 执行日志

[root@host131 shell]# sh all-k8s-mgnt.sh stop all; sh all-k8s-mgnt.sh clear all; sh all-k8s-mgnt.sh install all
## Sun Jun 30 03:46:13 EDT 2019 ACTION: stop  Service: ssl begins ...
## Sun Jun 30 03:46:13 EDT 2019 ACTION: stop  Service: ssl ends  ...

## Sun Jun 30 03:46:13 EDT 2019 ACTION: stop  Service: etcd begins ...
## Sun Jun 30 03:46:20 EDT 2019 ACTION: stop  Service: etcd ends  ...

## Sun Jun 30 03:46:20 EDT 2019 ACTION: stop  Service: apiserver begins ...
## Sun Jun 30 03:46:40 EDT 2019 ACTION: stop  Service: apiserver ends  ...

## Sun Jun 30 03:46:40 EDT 2019 ACTION: stop  Service: scheduler begins ...
## Sun Jun 30 03:46:41 EDT 2019 ACTION: stop  Service: scheduler ends  ...

## Sun Jun 30 03:46:41 EDT 2019 ACTION: stop  Service: controller begins ...
## Sun Jun 30 03:46:41 EDT 2019 ACTION: stop  Service: controller ends  ...

## Sun Jun 30 03:46:41 EDT 2019 ACTION: stop  Service: flannel begins ...
## Sun Jun 30 03:46:42 EDT 2019 ACTION: stop  Service: flannel ends  ...

## Sun Jun 30 03:46:42 EDT 2019 ACTION: stop  Service: docker begins ...
## Sun Jun 30 03:46:42 EDT 2019 ACTION: stop  Service: docker ends  ...

## Sun Jun 30 03:46:42 EDT 2019 ACTION: stop  Service: kubelet begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: kubelet ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: kubeproxy begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: kubeproxy ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: init begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: init ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: coredns begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: coredns ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: ssl begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: ssl ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: etcd begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: etcd ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: apiserver begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: apiserver ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: scheduler begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: scheduler ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: controller begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: controller ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: flannel begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: flannel ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: docker begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: docker ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: kubelet begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: kubelet ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: kubeproxy begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: kubeproxy ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: init begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: init ends  ...

## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: coredns begins ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: stop  Service: coredns ends  ...

## data dir clear operation begins...
 # clear ssl dirs 
 # clear etc dirs 
 # clear log dirs 
 # cler ~/.kube
 # clear working dirs or data dirs
#   umount /var/lib/kubelet/pods/4587c7c7-0360-46b5-8b8f-77ec77949a47/volumes/kubernetes.io~secret/coredns-token-n7d6d
## data dir clear operation ends  ...
## Sun Jun 30 03:46:43 EDT 2019 ACTION: install  Service: ssl begins ...
2019/06/30 03:46:43 [INFO] generating a new CA key and certificate from CSR
2019/06/30 03:46:43 [INFO] generate received request
2019/06/30 03:46:43 [INFO] received CSR
2019/06/30 03:46:43 [INFO] generating key: rsa-2048
2019/06/30 03:46:44 [INFO] encoded CSR
2019/06/30 03:46:44 [INFO] signed certificate with serial number 391210733936267787210777050101422802648649410405
/etc/ssl/ca/ca-key.pem	/etc/ssl/ca/ca.pem

2019/06/30 03:46:44 [INFO] generate received request
2019/06/30 03:46:44 [INFO] received CSR
2019/06/30 03:46:44 [INFO] generating key: rsa-2048
2019/06/30 03:46:44 [INFO] encoded CSR
2019/06/30 03:46:44 [INFO] signed certificate with serial number 698210269964151294075521146916998572452179503191
2019/06/30 03:46:44 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
/etc/ssl/etcd/cert-etcd-key.pem  /etc/ssl/etcd/cert-etcd.pem
2019/06/30 03:46:44 [INFO] generate received request
2019/06/30 03:46:44 [INFO] received CSR
2019/06/30 03:46:44 [INFO] generating key: rsa-2048
2019/06/30 03:46:44 [INFO] encoded CSR
2019/06/30 03:46:44 [INFO] signed certificate with serial number 405583254905228665131560472034390939451813225368
2019/06/30 03:46:44 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
## cert for kube-apiserver
/etc/ssl/k8s/cert-k8s-key.pem  /etc/ssl/k8s/cert-k8s.pem
2019/06/30 03:46:44 [INFO] generate received request
2019/06/30 03:46:44 [INFO] received CSR
2019/06/30 03:46:44 [INFO] generating key: rsa-2048
2019/06/30 03:46:44 [INFO] encoded CSR
2019/06/30 03:46:44 [INFO] signed certificate with serial number 302309737400960595766603073911338465948774571617
2019/06/30 03:46:44 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
## cert for kube-controller-manager
/etc/ssl/k8s/cert-k8scm-key.pem  /etc/ssl/k8s/cert-k8scm.pem
2019/06/30 03:46:44 [INFO] generate received request
2019/06/30 03:46:44 [INFO] received CSR
2019/06/30 03:46:44 [INFO] generating key: rsa-2048
2019/06/30 03:46:44 [INFO] encoded CSR
2019/06/30 03:46:44 [INFO] signed certificate with serial number 414169594299364704083157390092074355042106563795
2019/06/30 03:46:44 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
## cert for kube-scheduler
/etc/ssl/k8s/cert-k8sch-key.pem  /etc/ssl/k8s/cert-k8sch.pem

## create kubeconfig for kube-controller-manager
Cluster "kubernetes" set.
User "system:kube-controller-manager" set.
Context "system:kube-controller-manager" created.
Switched to context "system:kube-controller-manager".
## create kubeconfig for kube-scheduler
Cluster "kubernetes" set.
User "system:kube-scheduler" set.
Context "system:kube-scheduler" created.
Switched to context "system:kube-scheduler".
2019/06/30 03:46:45 [INFO] generate received request
2019/06/30 03:46:45 [INFO] received CSR
2019/06/30 03:46:45 [INFO] generating key: rsa-2048
2019/06/30 03:46:45 [INFO] encoded CSR
2019/06/30 03:46:45 [INFO] signed certificate with serial number 31679333033892674869399652825878046257531985858
2019/06/30 03:46:45 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
/etc/ssl/ca/kubeadmin-key.pem  /etc/ssl/ca/kubeadmin.pem
Cluster "kubernetes" set.
User "admin" set.
Context "kubernetes" created.
Switched to context "kubernetes".
## copy kubectl.kubeconfig to ~/.kube/config
## setting : firewall
##  setting: selinux
##  setting: kernel
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
net.ipv4.tcp_tw_recycle = 0
vm.overcommit_memory = 1
vm.panic_on_oom = 0
fs.inotify.max_user_instances = 8192
fs.inotify.max_user_watches = 1048576
fs.file-max = 52706963
fs.nr_open = 52706963
net.ipv6.conf.all.disable_ipv6 = 1
net.netfilter.nf_conntrack_max = 2310720
## Sun Jun 30 03:46:46 EDT 2019 ACTION: install  Service: ssl ends  ...

## Sun Jun 30 03:46:46 EDT 2019 ACTION: install  Service: etcd begins ...

##  stop etcd service

##  daemon reload service 

##  start etcd service 

##  enable etcd service 

##  check  etcd status
● etcd.service - Etcd Server
   Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-06-30 03:46:47 EDT; 80ms ago
   CGroup: /system.slice/etcd.service
           └─13447 /usr/local/bin/etcd --name=etcd-01 --data-dir=/var/lib/etcd//default.etcd --listen-peer-urls=https://192.168.163.131:2380 --listen-client-urls=https://192.168.163.131:2379,http://127.0.0.1:2379 --advertise-client-urls=https://192.168.163.131:2379 --initial-advertise-peer-urls=https://192.168.163.131:2380 --initial-cluster=etcd-01=https://192.168.163.131:2380 --initial-cluster-token=etcd-cluster --initial-cluster-state=new --cert-file=/etc/ssl/etcd/cert-etcd.pem --key-file=/etc/ssl/etcd/cert-etcd-key.pem --peer-cert-file=/etc/ssl/etcd/cert-etcd.pem --peer-key-file=/etc/ssl/etcd/cert-etcd-key.pem --trusted-ca-file=/etc/ssl/ca/ca.pem --peer-trusted-ca-file=/etc/ssl/ca/ca.pem

##  etcd version
etcd Version: 3.3.13
Git SHA: 98d3084
Go Version: go1.10.8
Go OS/Arch: linux/amd64

##  etcd cluster health
https://192.168.163.131:2379 is healthy: successfully committed proposal: took = 931.033µs
## Sun Jun 30 03:46:47 EDT 2019 ACTION: install  Service: etcd ends  ...

## Sun Jun 30 03:46:47 EDT 2019 ACTION: install  Service: apiserver begins ...

##  kube-apiserver service

##  daemon reload service 

##  start kube-apiserver service 

##  enable kube-apiserver service 

##  check  kube-apiserver status
● kube-apiserver.service - Kubernetes API Server
   Loaded: loaded (/usr/lib/systemd/system/kube-apiserver.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-06-30 03:46:47 EDT; 209ms ago
   CGroup: /system.slice/kube-apiserver.service
           └─13529 /usr/local/bin/kube-apiserver --logtostderr=false --v=2 --log-dir=/var/log/kubernetes --etcd-servers=https://192.168.163.131:2379 --authorization-mode=RBAC,Node --enable-admission-plugins=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota,NodeRestriction --anonymous-auth=false --bind-address=192.168.163.131 --kubelet-https=true --insecure-port=0 --runtime-config=api/all=true --advertise-address=192.168.163.131 --allow-privileged=true --service-cluster-ip-range=10.254.0.0/16 --service-node-port-range=30000-50000 --enable-bootstrap-token-auth --token-auth-file=/etc/k8s/token.csv --tls-cert-file=/etc/ssl/k8s/cert-k8s.pem --tls-private-key-file=/etc/ssl/k8s/cert-k8s-key.pem --client-ca-file=/etc/ssl/ca/ca.pem --service-account-key-file=/etc/ssl/ca/ca-key.pem --kubelet-certificate-authority=/etc/ssl/ca/ca.pem --kubelet-client-certificate=/etc/ssl/k8s/cert-k8s.pem --kubelet-client-key=/etc/ssl/k8s/cert-k8s-key.pem --etcd-cafile=/etc/ssl/ca/ca.pem --etcd-certfile=/etc/ssl/etcd/cert-etcd.pem --etcd-keyfile=//etc/ssl/etcd/cert-etcd-key.pem
Jun 30 03:46:47 host131 kube-apiserver[13529]: Flag --insecure-port has been deprecated, This flag will be removed in a future version.

##  kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

##  get cs
NAME                 STATUS      MESSAGE                                                                                     ERROR
scheduler            Unhealthy   Get http://127.0.0.1:10251/healthz: dial tcp 127.0.0.1:10251: connect: connection refused   
controller-manager   Unhealthy   Get http://127.0.0.1:10252/healthz: dial tcp 127.0.0.1:10252: connect: connection refused   
etcd-0               Healthy     {"health":"true"}                                                                           
## set kubectl exec privilleges
clusterrolebinding.rbac.authorization.k8s.io/kube-apiserver:kubelet-apis created
## Sun Jun 30 03:46:52 EDT 2019 ACTION: install  Service: apiserver ends  ...

## Sun Jun 30 03:46:52 EDT 2019 ACTION: install  Service: scheduler begins ...

##  kube-scheduler service

##  daemon reload service 

##  start kube-scheduler service 

##  enable kube-scheduler service 

##  check  kube-scheduler status
● kube-scheduler.service - Kubernetes Scheduler
   Loaded: loaded (/usr/lib/systemd/system/kube-scheduler.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-06-30 03:46:52 EDT; 240ms ago
   CGroup: /system.slice/kube-scheduler.service
           └─13619 /usr/local/bin/kube-scheduler --logtostderr=false --v=2 --log-dir=/var/log/kubernetes --kubeconfig=/etc/ssl/k8s/kube-scheduler.kubeconfig

##  get cs
NAME                 STATUS      MESSAGE                                                                                     ERROR
controller-manager   Unhealthy   Get http://127.0.0.1:10252/healthz: dial tcp 127.0.0.1:10252: connect: connection refused   
scheduler            Healthy     ok                                                                                          
etcd-0               Healthy     {"health":"true"}                                                                           
## Sun Jun 30 03:46:54 EDT 2019 ACTION: install  Service: scheduler ends  ...

## Sun Jun 30 03:46:54 EDT 2019 ACTION: install  Service: controller begins ...

##  kube-controller-manager service

##  daemon reload service 

##  start kube-controller-manager service 

##  enable kube-controller-manager service 

##  check  kube-controller-manager status
● kube-controller-manager.service - Kubernetes Controller Manager
   Loaded: loaded (/usr/lib/systemd/system/kube-controller-manager.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-06-30 03:46:55 EDT; 227ms ago
   CGroup: /system.slice/kube-controller-manager.service
           └─13690 /usr/local/bin/kube-controller-manager --logtostderr=false --v=2 --log-dir=/var/log/kubernetes --kubeconfig=/etc/ssl/k8s/kube-controller-manager.kubeconfig --authentication-kubeconfig=/etc/ssl/k8s/kube-controller-manager.kubeconfig --authorization-kubeconfig=/etc/ssl/k8s/kube-controller-manager.kubeconfig --leader-elect=true --service-cluster-ip-range=10.254.0.0/16 --cluster-name=kubernetes --cluster-signing-cert-file=/etc/ssl/ca/ca.pem --cluster-signing-key-file=/etc/ssl/ca/ca-key.pem --root-ca-file=/etc/ssl/ca/ca.pem --service-account-private-key-file=/etc/ssl/ca/ca-key.pem --controllers=*,bootstrapsigner,tokencleaner --horizontal-pod-autoscaler-use-rest-clients=true --horizontal-pod-autoscaler-sync-period=10s --tls-cert-file=/etc/ssl/k8s/cert-k8scm.pem --tls-private-key-file=/etc/ssl/k8s/cert-k8scm-key.pem --use-service-account-credentials=true
Jun 30 03:46:55 host131 kube-controller-manager[13690]: Flag --horizontal-pod-autoscaler-use-rest-clients has been deprecated, Heapster is no longer supported as a source for Horizontal Pod Autoscaler metrics.

##  get cs
NAME                 STATUS    MESSAGE             ERROR
scheduler            Healthy   ok                  
controller-manager   Healthy   ok                  
etcd-0               Healthy   {"health":"true"}   
## Sun Jun 30 03:46:57 EDT 2019 ACTION: install  Service: controller ends  ...

## Sun Jun 30 03:46:57 EDT 2019 ACTION: install  Service: flannel begins ...
2019/06/30 03:46:57 [INFO] generate received request
2019/06/30 03:46:57 [INFO] received CSR
2019/06/30 03:46:57 [INFO] generating key: rsa-2048
2019/06/30 03:46:57 [INFO] encoded CSR
2019/06/30 03:46:57 [INFO] signed certificate with serial number 244938635491419017685207615512005033652743786220
2019/06/30 03:46:57 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
/etc/ssl/flannel/flanneld-key.pem  /etc/ssl/flannel/flanneld.pem
{"Network":"10.254.0.0/16", "SubnetLen": 21, "Backend": {"Type": "vxlan"}}

##  flanneld service

##  daemon reload service 

##  start flannel service 

##  enable flannel service 
● flanneld.service - Flanneld Service
   Loaded: loaded (/usr/lib/systemd/system/flanneld.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-06-30 03:46:58 EDT; 145ms ago
     Docs: https://github.com/coreos/flannel
 Main PID: 13829 (flanneld)
   CGroup: /system.slice/flanneld.service
           └─13829 /usr/local/bin/flanneld -etcd-cafile=/etc/ssl/ca/ca.pem -etcd-certfile=/etc/ssl/flannel/flanneld.pem -etcd-keyfile=/etc/...

Jun 30 03:46:58 host131 flanneld[13829]: I0630 03:46:58.308767   13829 main.go:244] Created subnet manager: Etcd Local Manager with...4.8.0/21
Jun 30 03:46:58 host131 flanneld[13829]: I0630 03:46:58.308775   13829 main.go:247] Installing signal handlers
Jun 30 03:46:58 host131 flanneld[13829]: I0630 03:46:58.320728   13829 main.go:386] Found network config - Backend type: vxlan
Jun 30 03:46:58 host131 flanneld[13829]: I0630 03:46:58.320764   13829 vxlan.go:120] VXLAN config: VNI=1 Port=0 GBP=false DirectRouting=false
Jun 30 03:46:58 host131 systemd[1]: Started Flanneld Service.
Jun 30 03:46:58 host131 flanneld[13829]: I0630 03:46:58.328734   13829 local_manager.go:147] Found lease (10.254.8.0/21) for curren... reusing
Jun 30 03:46:58 host131 flanneld[13829]: I0630 03:46:58.331193   13829 main.go:317] Wrote subnet file to /run/flannel/subnet.env
Jun 30 03:46:58 host131 flanneld[13829]: I0630 03:46:58.331206   13829 main.go:321] Running backend.
Jun 30 03:46:58 host131 flanneld[13829]: I0630 03:46:58.341031   13829 vxlan_network.go:60] watching for new subnet leases
Jun 30 03:46:58 host131 flanneld[13829]: I0630 03:46:58.341509   13829 main.go:429] Waiting for 22h59m59.987517957s to renew lease
Hint: Some lines were ellipsized, use -l to show in full.
## Sun Jun 30 03:46:58 EDT 2019 ACTION: install  Service: flannel ends  ...

## Sun Jun 30 03:46:58 EDT 2019 ACTION: install  Service: docker begins ...

##  docker service

##  daemon reload service 

##  start docker service 

##  enable docker service 

##  check  docker status
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-06-30 03:46:58 EDT; 220ms ago
   CGroup: /system.slice/docker.service
           ├─13921 /usr/local/bin/dockerd --bip=10.254.8.1/21 --ip-masq=false --mtu=1450 --registry-mirror=https://hub-mirror.c.163.com -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock --selinux-enabled=false --log-opt max-size=1g
           └─13926 containerd --config /var/run/docker/containerd/containerd.toml --log-level info

##  check docker version
Client: Docker Engine - Community
 Version:           18.09.7
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        2d0083d
 Built:             Thu Jun 27 17:54:15 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.7
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       2d0083d
  Built:            Thu Jun 27 18:01:17 2019
  OS/Arch:          linux/amd64
  Experimental:     false

##  load pause imgage for kubelet
e17133b79956: Loading layer [==================================================>]  744.4kB/744.4kB
Loaded image: gcr.io/google_containers/pause-amd64:3.1
## load all other images needed
## Sun Jun 30 03:46:59 EDT 2019 ACTION: install  Service: docker ends  ...

## Sun Jun 30 03:46:59 EDT 2019 ACTION: install  Service: kubelet begins ...
2019/06/30 03:46:59 [INFO] generate received request
2019/06/30 03:46:59 [INFO] received CSR
2019/06/30 03:46:59 [INFO] generating key: rsa-2048
2019/06/30 03:47:00 [INFO] encoded CSR
2019/06/30 03:47:00 [INFO] signed certificate with serial number 724498985882347033535102661451612181859632274630
2019/06/30 03:47:00 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
/etc/ssl/k8s/cert-kubeproxy-key.pem  /etc/ssl/k8s/cert-kubeproxy.pem
Cluster "kubernetes" set.
User "kubelet-bootstrap" set.
Context "default" created.
Switched to context "default".
Cluster "kubernetes" set.
User "kube-proxy" set.
Context "default" created.
Switched to context "default".
clusterrolebinding.rbac.authorization.k8s.io/kubelet-bootstrap created

##  kubelet service

##  daemon reload service 

##  start kubelet service 

##  enable kubelet service 

##  check  kubelet status
● kubelet.service - Kubernetes Kubelet Service
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-06-30 03:47:01 EDT; 233ms ago
   CGroup: /system.slice/kubelet.service
           └─14159 /usr/local/bin/kubelet --logtostderr=false --v=2 --log-dir=/var/log/kubernetes --root-dir=/var/lib/kubelet --cert-dir=/etc/ssl/k8s --fail-swap-on=false --hostname-override=192.168.163.131 --bootstrap-kubeconfig=/etc/ssl/k8s/bootstrap.kubeconfig --kubeconfig=/etc/k8s/kubelet.kubeconfig --config=/etc/k8s/kubelet-config.yaml --pod-infra-container-image=gcr.io/google_containers/pause-amd64:3.1 --event-qps=0 --kube-api-qps=1000 --kube-api-burst=2000 --registry-qps=0 --image-pull-progress-deadline=30m
Jun 30 03:47:01 host131 kubelet[14159]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 30 03:47:01 host131 kubelet[14159]: Flag --event-qps has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 30 03:47:01 host131 kubelet[14159]: Flag --kube-api-qps has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 30 03:47:01 host131 kubelet[14159]: Flag --kube-api-burst has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 30 03:47:01 host131 kubelet[14159]: Flag --registry-qps has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 30 03:47:01 host131 kubelet[14159]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 30 03:47:01 host131 kubelet[14159]: Flag --event-qps has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 30 03:47:01 host131 kubelet[14159]: Flag --kube-api-qps has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 30 03:47:01 host131 kubelet[14159]: Flag --kube-api-burst has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 30 03:47:01 host131 kubelet[14159]: Flag --registry-qps has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.


##  get csr information
NAME        AGE   REQUESTOR           CONDITION
csr-jmzrx   1s    kubelet-bootstrap   Pending
##  kubectl get nodes 
No resources found.
## Sun Jun 30 03:47:02 EDT 2019 ACTION: install  Service: kubelet ends  ...

## Sun Jun 30 03:47:02 EDT 2019 ACTION: install  Service: kubeproxy begins ...

##  kube-proxy service

##  daemon reload service 

##  start kube-proxy service 

##  enable kube-proxy service 

##  check  kube-proxy status
● kube-proxy.service - Kubernetes Kube-Proxy Service
   Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-06-30 03:47:02 EDT; 223ms ago
   CGroup: /system.slice/kube-proxy.service
           ‣ 14263 /usr/local/bin/kube-proxy --logtostderr=false --v=2 --log-dir=/var/log/kubernetes --config=/etc/k8s/kube-proxy-config.yaml
## Sun Jun 30 03:47:02 EDT 2019 ACTION: install  Service: kubeproxy ends  ...

## Sun Jun 30 03:47:02 EDT 2019 ACTION: install  Service: init begins ...
## auto approve csr-jmzrx
certificatesigningrequest.certificates.k8s.io/csr-jmzrx approved
## sleep 3s for auto csr approve
## kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

## kubectl cluster-info
Kubernetes master is running at https://192.168.163.131:6443

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

## kubectl get node
No resources found.

## kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
scheduler            Healthy   ok                  
controller-manager   Healthy   ok                  
etcd-0               Healthy   {"health":"true"}   

## Sun Jun 30 03:47:06 EDT 2019 ACTION: install  Service: init ends  ...

## Sun Jun 30 03:47:06 EDT 2019 ACTION: install  Service: coredns begins ...
## please make sure you can get the following images
        image: k8s.gcr.io/coredns:1.2.6
## the following keyword needs to be replaced

## replace __PILLAR__DNS__DOMAIN__
## replace __PILLAR__DNS__SERVER__
## delete service first
/root/easypack/k8s/shell/etc/plugins/coredns

## create service 
serviceaccount/coredns created
clusterrole.rbac.authorization.k8s.io/system:coredns created
clusterrolebinding.rbac.authorization.k8s.io/system:coredns created
configmap/coredns created
deployment.extensions/coredns created
service/kube-dns created

## begin check coredns, wait for 3s ...









NAME                          READY   STATUS    RESTARTS   AGE
pod/coredns-b7d8c5745-hj2v8   0/1     Pending   0          3s
NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
service/kube-dns   ClusterIP   10.254.0.2           53/UDP,53/TCP   3s
NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/coredns   0/1     1            0           3s
NAME                                DESIRED   CURRENT   READY   AGE
replicaset.apps/coredns-b7d8c5745   1         1         0       3s
## Sun Jun 30 03:47:09 EDT 2019 ACTION: install  Service: coredns ends  ...

[root@host131 shell]#

结果确认

• 版本确认

[root@host131 shell]# kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
[root@host131 shell]#

• 节点确认

[root@host131 shell]# kubectl get nodes -o wide
NAME              STATUS   ROLES    AGE    VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION          CONTAINER-RUNTIME
192.168.163.131   Ready       102s   v1.15.0   192.168.163.131           CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://18.9.7
[root@host131 shell]# 

整体操作

• https://blog.csdn.net/liumiaocn/article/details/88413428