Dotbalo

CentOS 8 二进制高可用安装 k8s 1.16.x

基本说明

 　　本文章将演示CentOS 8二进制方式安装高可用k8s 1.16.x，相对于其他版本，二进制安装方式并无太大区别。CentOS 8相对于CentOS 7操作更加方便，比如一些服务的关闭，无需修改配置文件即可永久生效，CentOS 8默认安装的内核版本是4.18，所以在安装k8s的过程中也无需在进行内核升级，系统环境也可按需升级，如果下载的是最新版的CentOS 8，系统升级也可省略。

基本环境配置

　　主机信息

192.168.1.19 k8s-master01
192.168.1.18 k8s-master02
192.168.1.20 k8s-master03
192.168.1.88 k8s-master-lb
192.168.1.21 k8s-node01
192.168.1.22 k8s-node02
　　系统环境

[root@k8s-master01 ~]# cat /etc/redhat-release 
CentOS Linux release 8.0.1905 (Core) 
[root@k8s-master01 ~]# uname -a
Linux k8s-master01 4.18.0-80.el8.x86_64 #1 SMP Tue Jun 4 09:19:46 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

　　配置所有节点hosts文件

[root@k8s-master01 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.1.19 k8s-master01
192.168.1.18 k8s-master02
192.168.1.20 k8s-master03
192.168.1.88 k8s-master-lb
192.168.1.21 k8s-node01
192.168.1.22 k8s-node02
　　所有节点关闭firewalld 、dnsmasq、NetworkManager、selinux

systemctl disable --now firewalld 
systemctl disable --now dnsmasq
setenforce 0
　　所有节点关闭swap分区

[root@k8s-master01 ~]# swapoff -a && sysctl -w vm.swappiness=0
vm.swappiness = 0
　　所有节点同步时间

ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
echo 'Asia/Shanghai' >/etc/timezone
ntpdate time2.aliyun.com
　　Master01节点生成ssh key

[root@k8s-master01 ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:6uz2kI+jcMJIUQWKqRcDRbvpVxhCW3Tmqn0NKS+lT3U root@k8s-master01
The key's randomart image is:
+---[RSA 2048]----+
|.o++=.o          |
|.+o+ +           |
|oo* . .          |
|. .* + .         |
|..+ + = S E      |
|.= o * * .       |
|. * * B .        |
|   = Bo+         |
|    .+*oo        |
+----[SHA256]-----+
　　Master01配置免密码登录其他节点

[root@k8s-master01 ~]# for i in k8s-master01 k8s-master02 k8s-master03 k8s-node01 k8s-node02;do ssh-copy-id -i .ssh/id_rsa.pub $i;done
　　所有节点安装基本工具

yum install wget jq psmisc vim net-tools yum-utils device-mapper-persistent-data lvm2 git -y
 　　Master01下载安装文件

[root@k8s-master01 ~]# git clone https://github.com/dotbalo/k8s-ha-install.git
Cloning into 'k8s-ha-install'...
remote: Enumerating objects: 12, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 461 (delta 2), reused 5 (delta 1), pack-reused 449
Receiving objects: 100% (461/461), 19.52 MiB | 4.04 MiB/s, done.
Resolving deltas: 100% (163/163), done.
　　切换到1.16.x分支

git checkout manual-installation-v1.16.x

基本组件安装

　　配置Docker yum源

[root@k8s-master01 k8s-ha-install]# curl https://download.docker.com/linux/centos/docker-ce.repo -o /etc/yum.repos.d/docker-ce.repo
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2424 100 2424 0 0 3639 0 --:--:-- --:--:-- --:--:-- 3645
[root@k8s-master01 k8s-ha-install]# yum makecache
CentOS-8 - AppStream 559 kB/s | 6.3 MB 00:11
CentOS-8 - Base 454 kB/s | 7.9 MB 00:17
CentOS-8 - Extras 592 B/s | 2.1 kB 00:03
Docker CE Stable - x86_64 5.8 kB/s | 20 kB 00:03
Last metadata expiration check: 0:00:01 ago on Sat 02 Nov 2019 02:46:29 PM CST.
Metadata cache created.
　　所有节点安装新版containerd

[root@k8s-master01 k8s-ha-install]# wget https://download.docker.com/linux/centos/7/x86_64/edge/Packages/containerd.io-1.2.6-3.3.el7.x86_64.rpm
--2019-11-02 15:00:20-- https://download.docker.com/linux/centos/7/x86_64/edge/Packages/containerd.io-1.2.6-3.3.el7.x86_64.rpm
Resolving download.docker.com (download.docker.com)... 13.225.103.32, 13.225.103.65, 13.225.103.10, ...
Connecting to download.docker.com (download.docker.com)|13.225.103.32|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 27119348 (26M) [binary/octet-stream]
Saving to: ‘containerd.io-1.2.6-3.3.el7.x86_64.rpm’

containerd.io-1.2.6-3.3.el7.x86_64.rpm 100%[===================================================================================================================>] 25.86M 1.55MB/s in 30s

2019-11-02 15:00:51 (887 KB/s) - ‘containerd.io-1.2.6-3.3.el7.x86_64.rpm’ saved [27119348/27119348]

[root@k8s-master01 k8s-ha-install]# yum -y install containerd.io-1.2.6-3.3.el7.x86_64.rpm
Last metadata expiration check: 0:14:35 ago on Sat 02 Nov 2019 02:46:29 PM CST.
　　所有节点安装最新版Docker

[root@k8s-master01 k8s-ha-install]# yum install docker-ce -y
　　所有节点开启Docker并设置开机自启动

[root@k8s-master01 k8s-ha-install]# systemctl enable --now docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
[root@k8s-master01 k8s-ha-install]# docker version
Client: Docker Engine - Community
Version: 19.03.4
API version: 1.40
Go version: go1.12.10
Git commit: 9013bf583a
Built: Fri Oct 18 15:52:22 2019
OS/Arch: linux/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 19.03.4
API version: 1.40 (minimum version 1.12)
Go version: go1.12.10
Git commit: 9013bf583a
Built: Fri Oct 18 15:50:54 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.6
GitCommit: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc:
Version: 1.0.0-rc8
GitCommit: 425e105d5a03fabd737a126ad93d62a9eeede87f
docker-init:
Version: 0.18.0
GitCommit: fec3683


4. k8s组件安装

　　下载kubernetes 1.16.x安装包

https://dl.k8s.io/v1.16.2/kubernetes-server-linux-amd64.tar.gz
　　下载etcd 3.3.15安装包

[root@k8s-master01 ~]# wget https://github.com/etcd-io/etcd/releases/download/v3.3.15/etcd-v3.3.15-linux-amd64.tar.gz
　　解压kubernetes安装文件

[root@k8s-master01 ~]# tar -xf kubernetes-server-linux-amd64.tar.gz --strip-components=3 -C /usr/local/bin kubernetes/server/bin/kube{let,ctl,-apiserver,-controller-manager,-scheduler,-proxy}
　　解压etcd安装文件

[root@k8s-master01 ~]# tar -zxvf etcd-v3.3.15-linux-amd64.tar.gz --strip-components=1 -C /usr/local/bin etcd-v3.3.15-linux-amd64/etcd{,ctl}
　　版本查看

[root@k8s-master01 ~]# etcd --version
etcd Version: 3.3.15
Git SHA: 94745a4ee
Go Version: go1.12.9
Go OS/Arch: linux/amd64
[root@k8s-master01 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:18:23Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?

　　将组件发送到其他节点

MasterNodes='k8s-master02 k8s-master03'
WorkNodes='k8s-node01 k8s-node02'
for NODE in $MasterNodes; do echo $NODE; scp /usr/local/bin/kube{let,ctl,-apiserver,-controller-manager,-scheduler,-proxy} $NODE:/usr/local/bin/; scp /usr/local/bin/etcd* $NODE:/usr/local/bin/; done
for NODE in $WorkNodes; do scp /usr/local/bin/kube{let,-proxy} $NODE:/usr/local/bin/ ; done

　　CNI安装，下载CNI组件

wget https://github.com/containernetworking/plugins/releases/download/v0.7.5/cni-plugins-amd64-v0.7.5.tgz
　　所有节点创建/opt/cni/bin目录

mkdir -p /opt/cni/bin
　　解压cni并发送至其他节点

tar -zxf cni-plugins-amd64-v0.7.5.tgz -C /opt/cni/bin
for NODE in $MasterNodes; do ssh $NODE 'mkdir -p /opt/cni/bin'; scp /opt/cni/bin/ $NODE:/opt/cni/bin/; done
for NODE in $WorkNodes; do ssh $NODE 'mkdir -p /opt/cni/bin'; scp /opt/cni/bin/ $NODE:/opt/cni/bin/; done


5. 生成证书

　　下载生成证书工具

wget "https://pkg.cfssl.org/R1.2/cfssl_linux-amd64" -O /usr/local/bin/cfssl
wget "https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64" -O /usr/local/bin/cfssljson
chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson
　　所有Master节点创建etcd证书目录

mkdir /etc/etcd/ssl -p

　　Master01节点生成etcd证书

[root@k8s-master01 pki]# pwd
/root/k8s-ha-install/pki

[root@k8s-master01 pki]# cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare /etc/etcd/ssl/etcd-ca
2019/11/02 20:39:26 [INFO] generating a new CA key and certificate from CSR
2019/11/02 20:39:26 [INFO] generate received request
2019/11/02 20:39:26 [INFO] received CSR
2019/11/02 20:39:26 [INFO] generating key: rsa-2048
2019/11/02 20:39:27 [INFO] encoded CSR
2019/11/02 20:39:27 [INFO] signed certificate with serial number 603981780847439086338945186785919518080443607000

[root@k8s-master01 pki]# cfssl gencert \

-ca=/etc/etcd/ssl/etcd-ca.pem \
-ca-key=/etc/etcd/ssl/etcd-ca-key.pem \
-config=ca-config.json \
-hostname=127.0.0.1,k8s-master01,k8s-master02,k8s-master03,192.168.1.19,192.168.1.18,192.168.1.20 \
json | cfssljson -bare /etc/etcd/ssl/etcd
-profile=kubernetes \
etcd-csr.json | cfssljson -bare /etc/etcd/ssl/etcd
2019/11/02 20:48:51 [INFO] generate received request
2019/11/02 20:48:51 [INFO] received CSR
2019/11/02 20:48:51 [INFO] generating key: rsa-2048

2019/11/02 20:48:52 [INFO] encoded CSR
2019/11/02 20:48:52 [INFO] signed certificate with serial number 584188914871773268065873129590581350709445633441

　　将证书复制到其他节点

[root@k8s-master01 pki]# MasterNodes='k8s-master02 k8s-master03'
[root@k8s-master01 pki]# WorkNodes='k8s-node01 k8s-node02'
[root@k8s-master01 pki]#
[root@k8s-master01 pki]#
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# for NODE in $MasterNodes; do

ssh $NODE "mkdir -p /etc/etcd/ssl"
for FILE in etcd-ca-key.pem  etcd-ca.pem  etcd-key.pem  etcd.pem; do
  scp /etc/etcd/ssl/${FILE} $NODE:/etc/etcd/ssl/${FILE}
done

done
etcd-ca-key.pem 100% 1675 424.0KB/s 00:00

etcd-ca.pem 100% 1363 334.4KB/s 00:00
etcd-key.pem 100% 1679 457.4KB/s 00:00
etcd.pem 100% 1505 254.5KB/s 00:00
etcd-ca-key.pem 100% 1675 308.3KB/s 00:00
etcd-ca.pem 100% 1363 479.0KB/s 00:00
etcd-key.pem 100% 1679 208.1KB/s 00:00
etcd.pem 100% 1505 398.1KB/s 00:00

　　生成kubernetes证书

　　所有节点创建kubernetes相关目录

mkdir -p /etc/kubernetes/pki
[root@k8s-master01 pki]# cfssl gencert -initca ca-csr.json | cfssljson -bare /etc/kubernetes/pki/ca
2019/11/02 20:56:39 [INFO] generating a new CA key and certificate from CSR
2019/11/02 20:56:39 [INFO] generate received request
2019/11/02 20:56:39 [INFO] received CSR
2019/11/02 20:56:39 [INFO] generating key: rsa-2048
2019/11/02 20:56:39 [INFO] encoded CSR
2019/11/02 20:56:39 [INFO] signed certificate with serial number 204375274212114571715224985915861594178740016435

[root@k8s-master01 pki]# cfssl gencert -ca=/etc/kubernetes/pki/ca.pem -ca-key=/etc/kubernetes/pki/ca-key.pem -config=ca-config.json -hostname=10.96.0.1,192.168.1.88,127.0.0.1,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,192.168.1.19,192.168.1.18,192.168.1.20 -profile=kubernetes apiserver-csr.json | cfssljson -bare /etc/kubernetes/pki/apiserver
2019/11/02 20:58:37 [INFO] generate received request
2019/11/02 20:58:37 [INFO] received CSR
2019/11/02 20:58:37 [INFO] generating key: rsa-2048
2019/11/02 20:58:37 [INFO] encoded CSR
2019/11/02 20:58:37 [INFO] signed certificate with serial number 530242414195398724956103410222477752498137496229

[root@k8s-master01 pki]# cfssl gencert -initca front-proxy-ca-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-ca
netes front-proxy-client-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-client
2019/11/02 20:59:28 [INFO] generating a new CA key and certificate from CSR
2019/11/02 20:59:28 [INFO] generate received request
2019/11/02 20:59:28 [INFO] received CSR
2019/11/02 20:59:28 [INFO] generating key: rsa-2048
2019/11/02 20:59:29 [INFO] encoded CSR
2019/11/02 20:59:29 [INFO] signed certificate with serial number 400365003875188640769336046625866020115935574187
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# cfssl gencert -ca=/etc/kubernetes/pki/front-proxy-ca.pem -ca-key=/etc/kubernetes/pki/front-proxy-ca-key.pem -config=ca-config.json -profile=kubernetes front-proxy-client-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-client
2019/11/02 20:59:29 [INFO] generate received request
2019/11/02 20:59:29 [INFO] received CSR
2019/11/02 20:59:29 [INFO] generating key: rsa-2048
2019/11/02 20:59:31 [INFO] encoded CSR
2019/11/02 20:59:31 [INFO] signed certificate with serial number 714876805420065531759406103649716563367283489841

[root@k8s-master01 pki]# cfssl gencert \

-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
manager-csr.json | cfssljson -bare /etc/kubernetes/pki/controller-manager
2019/11/02 20:59:56 [INFO] generate received request
2019/11/02 20:59:56 [INFO] received CSR
2019/11/02 20:59:56 [INFO] generating key: rsa-2048

2019/11/02 20:59:56 [INFO] encoded CSR
2019/11/02 20:59:56 [INFO] signed certificate with serial number 365163931170718211641188236585237940196007847195
2019/11/02 20:59:56 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes \

--certificate-authority=/etc/kubernetes/pki/ca.pem \
--embed-certs=true \
--server=https://192.168.1.88:8443 \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig

kubectl config set-context system:kube-controller-manager@kubernetes \
--cluster=kubernetes \
--user=system:kube-controller-manager \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig

kubectl config use-context system:kube-controller-manager@kubernetes \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
Cluster "kubernetes" set.
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# kubectl config set-credentials system:kube-controller-manager \

--client-certificate=/etc/kubernetes/pki/controller-manager.pem \
--client-key=/etc/kubernetes/pki/controller-manager-key.pem \
--embed-certs=true \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
User "system:kube-controller-manager" set.
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# kubectl config set-context system:kube-controller-manager@kubernetes \
--cluster=kubernetes \
--user=system:kube-controller-manager \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
Context "system:kube-controller-manager@kubernetes" created.
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# kubectl config use-context system:kube-controller-manager@kubernetes \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
Switched to context "system:kube-controller-manager@kubernetes".

[root@k8s-master01 pki]# cfssl gencert \

-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
scheduler-csr.json | cfssljson -bare /etc/kubernetes/pki/scheduler
2019/11/02 21:00:48 [INFO] generate received request
2019/11/02 21:00:48 [INFO] received CSR
2019/11/02 21:00:48 [INFO] generating key: rsa-2048
2019/11/02 21:00:49 [INFO] encoded CSR
2019/11/02 21:00:49 [INFO] signed certificate with serial number 536666325566380578973549296433116161533422391008
2019/11/02 21:00:49 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").

[root@k8s-master01 pki]# kubectl config set-cluster kubernetes \

--certificate-authority=/etc/kubernetes/pki/ca.pem \
--embed-certs=true \
--server=https://192.168.1.88:8443 \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
heduler@kubernetes \
--cluster=kubernetes \
--user=system:kube-scheduler \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig

kubectl config use-context system:kube-scheduler@kubernetes \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
Cluster "kubernetes" set.
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# kubectl config set-credentials system:kube-scheduler \

--client-certificate=/etc/kubernetes/pki/scheduler.pem \
--client-key=/etc/kubernetes/pki/scheduler-key.pem \
--embed-certs=true \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
User "system:kube-scheduler" set.
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# kubectl config set-context system:kube-scheduler@kubernetes \
--cluster=kubernetes \
--user=system:kube-scheduler \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
Context "system:kube-scheduler@kubernetes" created.
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# kubectl config use-context system:kube-scheduler@kubernetes \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
Switched to context "system:kube-scheduler@kubernetes".

[root@k8s-master01 pki]# cfssl gencert \

-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
admin-csr.json | cfssljson -bare /etc/kubernetes/pki/admin
2019/11/02 21:01:56 [INFO] generate received request
2019/11/02 21:01:56 [INFO] received CSR
2019/11/02 21:01:56 [INFO] generating key: rsa-2048
2019/11/02 21:01:56 [INFO] encoded CSR
2019/11/02 21:01:56 [INFO] signed certificate with serial number 596648060489336426936623033491721470152303339581
2019/11/02 21:01:56 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").

[root@k8s-master01 pki]# kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://192.168.1.88:8443 --kubeconfig=/etc/kubernetes/admin.kubeconfig
s --user=kubernetes-admin --kubeconfig=/etc/kubernetes/admin.kubeconfig

kubectl config use-context kubernetes-admin@kubernetes --kubeconfig=/etc/kubernetes/admin.kubeconfig
Cluster "kubernetes" set.
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# kubectl config set-credentials kubernetes-admin --client-certificate=/etc/kubernetes/pki/admin.pem --client-key=/etc/kubernetes/pki/admin-key.pem --embed-certs=true --kubeconfig=/etc/kubernetes/admin.kubeconfig
User "kubernetes-admin" set.
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# kubectl config set-context kubernetes-admin@kubernetes --cluster=kubernetes --user=kubernetes-admin --kubeconfig=/etc/kubernetes/admin.kubeconfig
Context "kubernetes-admin@kubernetes" created.
[root@k8s-master01 pki]#
[root@k8s-master01 pki]# kubectl config use-context kubernetes-admin@kubernetes --kubeconfig=/etc/kubernetes/admin.kubeconfig
Switched to context "kubernetes-admin@kubernetes".

[root@k8s-master01 pki]# for NODE in k8s-master01 k8s-master02 k8s-master03; do

\cp kubelet-csr.json kubelet-$NODE-csr.json;
sed -i "s/\$NODE/$NODE/g" kubelet-$NODE-csr.json;
cfssl gencert \
  -ca=/etc/kubernetes/pki/ca.pem \
  -ca-key=/etc/kubernetes/pki/ca-key.pem \
  -config=ca-config.json \
  -hostname=$NODE \
  -profile=kubernetes \
  kubelet-$NODE-csr.json | cfssljson -bare /etc/kubernetes/pki/kubelet-$NODE;
rm -f kubelet-$NODE-csr.json
done
2019/11/02 21:03:19 [INFO] generate received request
2019/11/02 21:03:19 [INFO] received CSR
2019/11/02 21:03:19 [INFO] generating key: rsa-2048
2019/11/02 21:03:20 [INFO] encoded CSR
2019/11/02 21:03:20 [INFO] signed certificate with serial number 537071233304209996689594645648717291884343909273
2019/11/02 21:03:20 [INFO] generate received request
2019/11/02 21:03:20 [INFO] received CSR
2019/11/02 21:03:20 [INFO] generating key: rsa-2048
2019/11/02 21:03:20 [INFO] encoded CSR
2019/11/02 21:03:20 [INFO] signed certificate with serial number 93675644052244007817881327407761955585316699106
2019/11/02 21:03:20 [INFO] generate received request
2019/11/02 21:03:20 [INFO] received CSR
2019/11/02 21:03:20 [INFO] generating key: rsa-2048
2019/11/02 21:03:21 [INFO] encoded CSR
2019/11/02 21:03:21 [INFO] signed certificate with serial number 369432006714320793581219965286097236027135622831

[root@k8s-master01 pki]# for NODE in k8s-master01 k8s-master02 k8s-master03; do

ssh $NODE "mkdir -p /etc/kubernetes/pki"
scp /etc/kubernetes/pki/ca.pem $NODE:/etc/kubernetes/pki/ca.pem
scp /etc/kubernetes/pki/kubelet-$NODE-key.pem $NODE:/etc/kubernetes/pki/kubelet-key.pem
scp /etc/kubernetes/pki/kubelet-$NODE.pem $NODE:/etc/kubernetes/pki/kubelet.pem
rm -f /etc/kubernetes/pki/kubelet-$NODE-key.pem /etc/kubernetes/pki/kubelet-$NODE.pem
done
ca.pem 100% 1407 494.1KB/s 00:00
kubelet-k8s-master01-key.pem 100% 1679 549.9KB/s 00:00
kubelet-k8s-master01.pem 100% 1501 535.5KB/s 00:00
ca.pem 100% 1407 358.2KB/s 00:00
kubelet-k8s-master02-key.pem 100% 1675 539.8KB/s 00:00
kubelet-k8s-master02.pem 100% 1501 375.5KB/s 00:00
ca.pem 100% 1407 464.7KB/s 00:00
kubelet-k8s-master03-key.pem 100% 1679 566.2KB/s 00:00
kubelet-k8s-master03.pem 100% 1501 515.1KB/s 00:00

[root@k8s-master01 pki]# for NODE in k8s-master01 k8s-master02 k8s-master03; do

ssh $NODE "cd /etc/kubernetes/pki && \
  kubectl config set-cluster kubernetes \
    --certificate-authority=/etc/kubernetes/pki/ca.pem \
    --embed-certs=true \
    --server=https://192.168.1.88:8443 \
    --kubeconfig=/etc/kubernetes/kubelet.kubeconfig && \
  kubectl config set-credentials system:node:${NODE} \
    --client-certificate=/etc/kubernetes/pki/kubelet.pem \
    --client-key=/etc/kubernetes/pki/kubelet-key.pem \
    --embed-certs=true \
    --kubeconfig=/etc/kubernetes/kubelet.kubeconfig && \
  kubectl config set-context system:node:${NODE}@kubernetes \
    --cluster=kubernetes \
    --user=system:node:${NODE} \
    --kubeconfig=/etc/kubernetes/kubelet.kubeconfig && \
  kubectl config use-context system:node:${NODE}@kubernetes \
    --kubeconfig=/etc/kubernetes/kubelet.kubeconfig"

done

Cluster "kubernetes" set.
User "system:node:k8s-master01" set.
Context "system:node:k8s-master01@kubernetes" created.
Switched to context "system:node:k8s-master01@kubernetes".
Cluster "kubernetes" set.
User "system:node:k8s-master02" set.
Context "system:node:k8s-master02@kubernetes" created.
Switched to context "system:node:k8s-master02@kubernetes".
Cluster "kubernetes" set.
User "system:node:k8s-master03" set.
Context "system:node:k8s-master03@kubernetes" created.
Switched to context "system:node:k8s-master03@kubernetes".

　　创建ServiceAccount Key

[root@k8s-master01 pki]# openssl genrsa -out /etc/kubernetes/pki/sa.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
...................................................................................+++++
...............+++++
e is 65537 (0x010001)
[root@k8s-master01 pki]# openssl rsa -in /etc/kubernetes/pki/sa.key -pubout -out /etc/kubernetes/pki/sa.pub
writing RSA key
[root@k8s-master01 pki]# for NODE in k8s-master02 k8s-master03; do

for FILE in $(ls /etc/kubernetes/pki | grep -v etcd); do
fig; do
scp /etc/kubernetes/${FILE} $NODE:/etc/kubernetes/${FILE}
done
done
scp /etc/kubernetes/pki/${FILE} $NODE:/etc/kubernetes/pki/${FILE}
done
for FILE in admin.kubeconfig controller-manager.kubeconfig scheduler.kubeconfig; do
scp /etc/kubernetes/${FILE} $NODE:/etc/kubernetes/${FILE}
done
done
admin.csr 100% 1025 303.5KB/s 00:00
admin-key.pem 100% 1675 544.2KB/s 00:00
admin.pem 100% 1440 447.6KB/s 00:00
apiserver.csr 100% 1029 251.9KB/s 00:00
apiserver-key.pem 100% 1679 579.5KB/s 00:00
apiserver.pem 100% 1692 592.0KB/s 00:00
ca.csr 100% 1025 372.6KB/s 00:00
ca-key.pem 100% 1679 535.5KB/s 00:00
ca.pem 100% 1407 496.0KB/s 00:00
controller-manager.csr 100% 1082 273.4KB/s 00:00
controller-manager-key.pem 100% 1679 552.8KB/s 00:00
controller-manager.pem 100% 1497 246.7KB/s 00:00
front-proxy-ca.csr 100% 891 322.2KB/s 00:00
front-proxy-ca-key.pem 100% 1679 408.7KB/s 00:00
front-proxy-ca.pem 100% 1143 436.9KB/s 00:00
front-proxy-client.csr 100% 903 283.5KB/s 00:00
front-proxy-client-key.pem 100% 1679 265.2KB/s 00:00
front-proxy-client.pem 100% 1188 358.8KB/s 00:00
kubelet-k8s-master01.csr 100% 1050 235.6KB/s 00:00
kubelet-k8s-master02.csr 100% 1050 385.3KB/s 00:00
kubelet-k8s-master03.csr 100% 1050 319.8KB/s 00:00
kubelet-key.pem 100% 1679 562.2KB/s 00:00
kubelet.pem 100% 1501 349.4KB/s 00:00
sa.key 100% 1679 455.2KB/s 00:00
sa.pub 100% 451 152.3KB/s 00:00
scheduler.csr 100% 1058 402.0KB/s 00:00
scheduler-key.pem 100% 1675 291.8KB/s 00:00
scheduler.pem 100% 1472 426.2KB/s 00:00
admin.kubeconfig 100% 6432 1.5MB/s 00:00
controller-manager.kubeconfig 100% 6568 1.1MB/s 00:00
scheduler.kubeconfig 100% 6496 601.6KB/s 00:00
admin.csr 100% 1025 368.2KB/s 00:00
admin-key.pem 100% 1675 542.3KB/s 00:00
admin.pem 100% 1440 247.7KB/s 00:00
apiserver.csr 100% 1029 212.4KB/s 00:00
apiserver-key.pem 100% 1679 530.8KB/s 00:00
apiserver.pem 100% 1692 557.3KB/s 00:00
ca.csr 100% 1025 266.4KB/s 00:00
ca-key.pem 100% 1679 408.6KB/s 00:00
ca.pem 100% 1407 404.6KB/s 00:00
controller-manager.csr 100% 1082 248.9KB/s 00:00
controller-manager-key.pem 100% 1679 495.4KB/s 00:00
controller-manager.pem 100% 1497 262.1KB/s 00:00
front-proxy-ca.csr 100% 891 140.9KB/s 00:00
front-proxy-ca-key.pem 100% 1679 418.7KB/s 00:00
front-proxy-ca.pem 100% 1143 362.3KB/s 00:00
front-proxy-client.csr 100% 903 305.6KB/s 00:00
front-proxy-client-key.pem 100% 1679 342.7KB/s 00:00
front-proxy-client.pem 100% 1188 235.1KB/s 00:00
kubelet-k8s-master01.csr 100% 1050 215.1KB/s 00:00
kubelet-k8s-master02.csr 100% 1050 354.0KB/s 00:00
kubelet-k8s-master03.csr 100% 1050 243.0KB/s 00:00
kubelet-key.pem 100% 1679 591.3KB/s 00:00
kubelet.pem 100% 1501 557.0KB/s 00:00
sa.key 100% 1679 387.5KB/s 00:00
sa.pub 100% 451 154.0KB/s 00:00
scheduler.csr 100% 1058 191.5KB/s 00:00
scheduler-key.pem 100% 1675 426.3KB/s 00:00
scheduler.pem 100% 1472 561.3KB/s 00:00
admin.kubeconfig 100% 6432 1.3MB/s 00:00
controller-manager.kubeconfig 100% 6568 1.7MB/s 00:00
scheduler.kubeconfig 100% 6496 1.3MB/s 00:00


6. Kubernetes系统组件配置

　　etcd配置大致相同，注意修改每个Master节点的etcd配置的主机名和IP地址

name: 'k8s-master01'
data-dir: /var/lib/etcd
wal-dir: /var/lib/etcd/wal
snapshot-count: 5000
heartbeat-interval: 100
election-timeout: 1000
quota-backend-bytes: 0
listen-peer-urls: 'https://192.168.1.19:2380'
listen-client-urls: 'https://192.168.1.19:2379,http://127.0.0.1:2379'
max-snapshots: 3
max-wals: 5
cors:
initial-advertise-peer-urls: 'https://192.168.1.19:2380'
advertise-client-urls: 'https://192.168.1.19:2379'
discovery:
discovery-fallback: 'proxy'
discovery-proxy:
discovery-srv:
initial-cluster: 'k8s-master01=https://192.168.1.19:2380,k8s-master02=https://192.168.1.18:2380,k8s-master03=https://192.168.1.20:2380'
initial-cluster-token: 'etcd-k8s-cluster'
initial-cluster-state: 'new'
strict-reconfig-check: false
enable-v2: true
enable-pprof: true
proxy: 'off'
proxy-failure-wait: 5000
proxy-refresh-interval: 30000
proxy-dial-timeout: 1000
proxy-write-timeout: 5000
proxy-read-timeout: 0
client-transport-security:
ca-file: '/etc/kubernetes/pki/etcd/etcd-ca.pem'
cert-file: '/etc/kubernetes/pki/etcd/etcd.pem'
key-file: '/etc/kubernetes/pki/etcd/etcd-key.pem'
client-cert-auth: true
trusted-ca-file: '/etc/kubernetes/pki/etcd/etcd-ca.pem'
auto-tls: true
peer-transport-security:
ca-file: '/etc/kubernetes/pki/etcd/etcd-ca.pem'
cert-file: '/etc/kubernetes/pki/etcd/etcd.pem'
key-file: '/etc/kubernetes/pki/etcd/etcd-key.pem'
peer-client-cert-auth: true
trusted-ca-file: '/etc/kubernetes/pki/etcd/etcd-ca.pem'
auto-tls: true
debug: false
log-package-levels:
log-output: default
force-new-cluster: false

　　所有Master节点创建etcd service并启动

[root@k8s-master01 pki]# cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Service
Documentation=https://coreos.com/etcd/docs/latest/
After=network.target

[Service]
Type=notify
ExecStart=/usr/local/bin/etcd --config-file=/etc/etcd/etcd.config.yml
Restart=on-failure
RestartSec=10
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
Alias=etcd3.service

[root@k8s-master01 pki]# mkdir /etc/kubernetes/pki/etcd
[root@k8s-master01 pki]# ln -s /etc/etcd/ssl/* /etc/kubernetes/pki/etcd/
[root@k8s-master01 pki]# systemctl daemon-reload
[root@k8s-master01 pki]# systemctl enable --now etcd
Created symlink /etc/systemd/system/etcd3.service → /usr/lib/systemd/system/etcd.service.
Created symlink /etc/systemd/system/multi-user.target.wants/etcd.service → /usr/lib/systemd/system/etcd.service.

　　高可用配置

　　所有Master节点安装keepalived和haproxy

yum install keepalived haproxy -y
　　HAProxy配置

[root@k8s-master01 pki]# cat /etc/haproxy/haproxy.cfg
global
maxconn 2000
ulimit-n 16384
log 127.0.0.1 local0 err
stats timeout 30s

defaults
log global
mode http
option httplog
timeout connect 5000
timeout client 50000
timeout server 50000
timeout http-request 15s
timeout http-keep-alive 15s

frontend monitor-in
bind *:33305
mode http
option httplog
monitor-uri /monitor

listen stats
bind *:8006
mode http
stats enable
stats hide-version
stats uri /stats
stats refresh 30s
stats realm Haproxy\ Statistics
stats auth admin:admin

frontend k8s-master
bind 0.0.0.0:8443
bind 127.0.0.1:8443
mode tcp
option tcplog
tcp-request inspect-delay 5s
default_backend k8s-master

backend k8s-master
mode tcp
option tcplog
option tcp-check
balance roundrobin
default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
server k8s-master01 192.168.1.19:6443 check
server k8s-master02 192.168.1.18:6443 check
server k8s-master03 192.168.1.20:6443 check

　　KeepAlived配置

! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 2
weight -5
fall 3
rise 2
}
vrrp_instance VI_1 {
state MASTER
interface ens160
mcast_src_ip 192.168.1.19
virtual_router_id 51
priority 100
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {
192.168.1.88
}
track_script {
chk_apiserver
} }
　　健康检查配置

[root@k8s-master01 keepalived]# cat /etc/keepalived/check_apiserver.sh
#!/bin/bash

err=0
for k in $(seq 1 5)
do
check_code=$(pgrep kube-apiserver)
if [[ $check_code == "" ]]; then
err=$(expr $err + 1)
sleep 5
continue
else
err=0
break
fi
done

if [[ $err != "0" ]]; then
echo "systemctl stop keepalived"
/usr/bin/systemctl stop keepalived
exit 1
else
exit 0
fi
　　启动HAProxy和KeepAlived

[root@k8s-master01 keepalived]# systemctl enable --now haproxy
[root@k8s-master01 keepalived]# systemctl enable --now keepalived
　　VIP测试

[root@k8s-master01 pki]# ping 192.168.1.88
PING 192.168.1.88 (192.168.1.88) 56(84) bytes of data.
64 bytes from 192.168.1.88: icmp_seq=1 ttl=64 time=1.39 ms
64 bytes from 192.168.1.88: icmp_seq=2 ttl=64 time=2.46 ms
64 bytes from 192.168.1.88: icmp_seq=3 ttl=64 time=1.68 ms
64 bytes from 192.168.1.88: icmp_seq=4 ttl=64 time=1.08 ms

　　Kubernetes组件配置

　　所有节点创建相关目录

[root@k8s-master01 pki]# mkdir -p /etc/kubernetes/manifests/ /etc/systemd/system/kubelet.service.d /var/lib/kubelet /var/log/kubernetes
　　所有Master节点创建kube-apiserver service

[root@k8s-master01 pki]# cat /usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
ExecStart=/usr/local/bin/kube-apiserver \
--v=2 \
--logtostderr=true \
--allow-privileged=true \
--bind-address=0.0.0.0 \
--secure-port=6443 \
--insecure-port=0 \
--advertise-address=192.168.1.88 \
--service-cluster-ip-range=10.96.0.0/12 \
--service-node-port-range=30000-32767 \
--etcd-servers=https://192.168.1.19:2379,https://192.168.1.18:2379,https://192.168.1.20:2379 \
--etcd-cafile=/etc/etcd/ssl/etcd-ca.pem \
--etcd-certfile=/etc/etcd/ssl/etcd.pem \
--etcd-keyfile=/etc/etcd/ssl/etcd-key.pem \
--client-ca-file=/etc/kubernetes/pki/ca.pem \
--tls-cert-file=/etc/kubernetes/pki/apiserver.pem \
--tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem \
--kubelet-client-certificate=/etc/kubernetes/pki/apiserver.pem \
--kubelet-client-key=/etc/kubernetes/pki/apiserver-key.pem \
--service-account-key-file=/etc/kubernetes/pki/sa.pub \
--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota \
--authorization-mode=Node,RBAC \
--enable-bootstrap-token-auth=true \
--requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem \
--proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.pem \
--proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client-key.pem \
--requestheader-allowed-names=aggregator \
--requestheader-group-headers=X-Remote-Group \
--requestheader-extra-headers-prefix=X-Remote-Extra- \
--requestheader-username-headers=X-Remote-User \
--token-auth-file=/etc/kubernetes/token.csv

Restart=on-failure
RestartSec=10s
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target
[root@k8s-master01 pki]# vim /etc/kubernetes/token.csv
[root@k8s-master01 pki]# cat !$
cat /etc/kubernetes/token.csv
d7d356746b508a1a478e49968fba7947,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
　　所有Master节点开启kube-apiserver

[root@k8s-master01 pki]# systemctl enable --now kube-apiserver

　　所有Master节点配置kube-controller-manager service

[root@k8s-master01 pki]# cat /usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
ExecStart=/usr/local/bin/kube-controller-manager \
--v=2 \
--logtostderr=true \
--address=127.0.0.1 \
--root-ca-file=/etc/kubernetes/pki/ca.pem \
--cluster-signing-cert-file=/etc/kubernetes/pki/ca.pem \
--cluster-signing-key-file=/etc/kubernetes/pki/ca-key.pem \
--service-account-private-key-file=/etc/kubernetes/pki/sa.key \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig \
--leader-elect=true \
--use-service-account-credentials=true \
--node-monitor-grace-period=40s \
--node-monitor-period=5s \
--pod-eviction-timeout=2m0s \
--controllers=*,bootstrapsigner,tokencleaner \
--allocate-node-cidrs=true \
--cluster-cidr=10.244.0.0/16 \
--requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem \
--node-cidr-mask-size=24

Restart=always
RestartSec=10s

[Install]
WantedBy=multi-user.target

　　所有Master节点启动kube-controller-manager

[root@k8s-master01 pki]# systemctl daemon-reload

[root@k8s-master01 pki]# systemctl enable --now kube-controller-manager
Created symlink /etc/systemd/system/multi-user.target.wants/kube-controller-manager.service → /usr/lib/systemd/system/kube-controller-manager.service.

　　所有Master节点配置kube-scheduler service

[root@k8s-master01 pki]# cat /usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
ExecStart=/usr/local/bin/kube-scheduler \
--v=2 \
--logtostderr=true \
--address=127.0.0.1 \
--leader-elect=true \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig

Restart=always
RestartSec=10s

[Install]
WantedBy=multi-user.target

[root@k8s-master01 pki]# systemctl daemon-reload

[root@k8s-master01 pki]# systemctl enable --now kube-scheduler
Created symlink /etc/systemd/system/multi-user.target.wants/kube-scheduler.service → /usr/lib/systemd/system/kube-scheduler.service.


7.  TLS Bootstrapping配置

　　在Master01创建bootstrap

kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://192.168.1.88:8443 --kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig
kubectl config set-credentials tls-bootstrap-token-user --token=c8ad9c.2e4d610cf3e7426e --kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig
kubectl config set-context tls-bootstrap-token-user@kubernetes --cluster=kubernetes --user=tls-bootstrap-token-user --kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig
kubectl config use-context tls-bootstrap-token-user@kubernetes --kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig
　　

[root@k8s-master01 bootstrap]# pwd
/root/k8s-ha-install/bootstrap

[root@k8s-master01 bootstrap]# kubectl create -f bootstrap.secret.yaml
secret/bootstrap-token-c8ad9c created
clusterrolebinding.rbac.authorization.k8s.io/kubelet-bootstrap created
clusterrolebinding.rbac.authorization.k8s.io/node-autoapprove-bootstrap created
clusterrolebinding.rbac.authorization.k8s.io/node-autoapprove-certificate-rotation created
clusterrole.rbac.authorization.k8s.io/system:kube-apiserver-to-kubelet created
clusterrolebinding.rbac.authorization.k8s.io/system:kube-apiserver created


8. Node节点配置

　　复制证书至Node节点

[root@k8s-master01 bootstrap]# for NODE in k8s-node01 k8s-node02; do

ssh $NODE mkdir -p /etc/kubernetes/pki /etc/etcd/ssl /etc/etcd/ssl
for FILE in etcd-ca.pem etcd.pem etcd-key.pem; do
  scp /etc/etcd/ssl/$FILE $NODE:/etc/etcd/ssl/
done
for FILE in pki/ca.pem pki/ca-key.pem pki/front-proxy-ca.pem bootstrap-kubelet.kubeconfig; do
  scp /etc/kubernetes/$FILE $NODE:/etc/kubernetes/${FILE}

done
done

etcd-ca.pem 100% 1363 314.0KB/s 00:00
etcd.pem 100% 1505 429.1KB/s 00:00
etcd-key.pem 100% 1679 361.9KB/s 00:00
ca.pem 100% 1407 459.5KB/s 00:00
ca-key.pem 100% 1679 475.2KB/s 00:00
front-proxy-ca.pem 100% 1143 214.5KB/s 00:00
bootstrap-kubelet.kubeconfig 100% 2291 695.1KB/s 00:00
etcd-ca.pem 100% 1363 325.5KB/s 00:00
etcd.pem 100% 1505 301.2KB/s 00:00
etcd-key.pem 100% 1679 260.9KB/s 00:00
ca.pem 100% 1407 420.8KB/s 00:00
ca-key.pem 100% 1679 398.0KB/s 00:00
front-proxy-ca.pem 100% 1143 224.9KB/s 00:00
bootstrap-kubelet.kubeconfig 100% 2291 685.4KB/s 00:00

　　所有Node节点创建相关目录

mkdir -p /var/lib/kubelet /var/log/kubernetes /etc/systemd/system/kubelet.service.d /etc/kubernetes/manifests/
　　所有节点配置kubelet service（Master节点不部署Pod也可无需配置）

[root@k8s-master01 bootstrap]# vim /usr/lib/systemd/system/kubelet.service
[root@k8s-master01 bootstrap]# cat !$
cat /usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/kubernetes/kubernetes
After=docker.service
Requires=docker.service

[Service]
ExecStart=/usr/local/bin/kubelet

Restart=always
StartLimitInterval=0
RestartSec=10

[Install]
WantedBy=multi-user.target

[root@k8s-master01 bootstrap]# vim /etc/systemd/system/kubelet.service.d/10-kubelet.conf
[root@k8s-master01 bootstrap]# cat !$
cat /etc/systemd/system/kubelet.service.d/10-kubelet.conf
[Service]
Environment=" KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig --kubeconfig=/etc/kubernetes/kubelet.kubeconfig"
Environment="KUBELET_SYSTEM_ARGS=--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin"
Environment="KUBELET_CONFIG_ARGS=--config=/etc/kubernetes/kubelet-conf.yml"
Environment="KUBELET_EXTRA_ARGS=--node-labels=node.kubernetes.io/node='' --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1"
ExecStart=
ExecStart=/usr/local/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_SYSTEM_ARGS $KUBELET_EXTRA_ARGS

[root@k8s-master01 bootstrap]# vim /etc/kubernetes/kubelet-conf.yml
[root@k8s-master01 bootstrap]# cat !$
cat /etc/kubernetes/kubelet-conf.yml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
address: 0.0.0.0
port: 10250
readOnlyPort: 10255
authentication:
anonymous:
enabled: false
webhook:
cacheTTL: 2m0s
enabled: true
x509:
clientCAFile: /etc/kubernetes/pki/ca.pem
authorization:
mode: Webhook
webhook:
cacheAuthorizedTTL: 5m0s
cacheUnauthorizedTTL: 30s
cgroupDriver: cgroupfs
cgroupsPerQOS: true
clusterDNS:

10.96.0.10
clusterDomain: cluster.local
containerLogMaxFiles: 5
containerLogMaxSize: 10Mi
contentType: application/vnd.kubernetes.protobuf
cpuCFSQuota: true
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
enableDebuggingHandlers: true
enforceNodeAllocatable:
pods
eventBurst: 10
eventRecordQPS: 5
evictionHard:
imagefs.available: 15%
memory.available: 100Mi
nodefs.available: 10%
nodefs.inodesFree: 5%
evictionPressureTransitionPeriod: 5m0s
failSwapOn: true
fileCheckFrequency: 20s
hairpinMode: promiscuous-bridge
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 20s
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 80
imageMinimumGCAge: 2m0s
iptablesDropBit: 15
iptablesMasqueradeBit: 14
kubeAPIBurst: 10
kubeAPIQPS: 5
makeIPTablesUtilChains: true
maxOpenFiles: 1000000
maxPods: 110
nodeStatusUpdateFrequency: 10s
oomScoreAdj: -999
podPidsLimit: -1
registryBurst: 10
registryPullQPS: 5
resolvConf: /etc/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 2m0s
serializeImagePulls: true
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
volumeStatsAggPeriod: 1m0s

　　启动所有节点kubelet

systemctl daemon-reload
systemctl enable --now kubelet

　　查看集群状态

[root@k8s-master01 bootstrap]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master01 NotReady 13m v1.16.2
k8s-master02 NotReady 11m v1.16.2
k8s-master03 NotReady 10m v1.16.2
k8s-node01 NotReady 9m16s v1.16.2
k8s-node02 NotReady 53s v1.16.2

　　Kube-Proxy配置

kubectl -n kube-system create serviceaccount kube-proxy
kubectl create clusterrolebinding system:kube-proxy --clusterrole system:node-proxier --serviceaccount kube-system:kube-proxy
SECRET=$(kubectl -n kube-system get sa/kube-proxy \
--output=jsonpath='{.secrets[0].name}')
JWT_TOKEN=$(kubectl -n kube-system get secret/$SECRET \
--output=jsonpath='{.data.token}' | base64 -d)
PKI_DIR=/etc/kubernetes/pki
K8S_DIR=/etc/kubernetes
kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://192.168.1.88:8443 --kubeconfig=${K8S_DIR}/kube-proxy.kubeconfig
kubectl config set-credentials kubernetes --token=${JWT_TOKEN} --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig
kubectl config set-context kubernetes --cluster=kubernetes --user=kubernetes --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig
kubectl config use-context kubernetes --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig

　　赋值Service文件

[root@k8s-master01 k8s-ha-install]# for NODE in k8s-master01 k8s-master02 k8s-master03; do

scp ${K8S_DIR}/kube-proxy.kubeconfig $NODE:/etc/kubernetes/kube-proxy.kubeconfig
scp kube-proxy/kube-proxy.conf $NODE:/etc/kubernetes/kube-proxy.conf
scp kube-proxy/kube-proxy.service $NODE:/usr/lib/systemd/system/kube-proxy.service
done
cp kube-proxy/kube-proxy.service $NODE:/usr/lib/systemd/system/kube-proxy.service
done
kube-proxy.kubeconfig 100% 3112 973.8KB/s 00:00
kube-proxy.conf 100% 813 196.5KB/s 00:00
kube-proxy.service 100% 288 115.5KB/s 00:00
kube-proxy.kubeconfig 100% 3112 580.5KB/s 00:00
kube-proxy.conf 100% 813 160.1KB/s 00:00
kube-proxy.service 100% 288 92.4KB/s 00:00
kube-proxy.kubeconfig 100% 3112 703.1KB/s 00:00
kube-proxy.conf 100% 813 244.1KB/s 00:00
kube-proxy.service 100% 288 54.6KB/s 00:00
[root@k8s-master01 k8s-ha-install]#
[root@k8s-master01 k8s-ha-install]# for NODE in k8s-node01 k8s-node02; do
scp /etc/kubernetes/kube-proxy.kubeconfig $NODE:/etc/kubernetes/kube-proxy.kubeconfig
scp kube-proxy/kube-proxy.conf $NODE:/etc/kubernetes/kube-proxy.conf
scp kube-proxy/kube-proxy.service $NODE:/usr/lib/systemd/system/kube-proxy.service
done
kube-proxy.kubeconfig 100% 3112 604.6KB/s 00:00
kube-proxy.conf 100% 813 197.6KB/s 00:00
kube-proxy.service 100% 288 95.5KB/s 00:00
kube-proxy.kubeconfig 100% 3112 749.3KB/s 00:00
kube-proxy.conf 100% 813 206.2KB/s 00:00
kube-proxy.service 100% 288 69.9KB/s 00:00

　　所有节点启动kube-proxy

[root@k8s-master01 k8s-ha-install]# systemctl daemon-reload
[root@k8s-master01 k8s-ha-install]# systemctl enable --now kube-proxy
Created symlink /etc/systemd/system/multi-user.target.wants/kube-proxy.service → /usr/lib/systemd/system/kube-proxy.service.


9. 安装calico

[root@k8s-master01 Calico]# cd /root/k8s-ha-install/Calico/
[root@k8s-master01 Calico]# POD_CIDR="10.244.0.0/16" \

sed -i -e "s?192.168.0.0/16?$POD_CIDR?g" calico.yaml
[root@k8s-master01 Calico]# kubectl create -f calico.yaml
configmap/calico-config created
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created
clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrole.rbac.authorization.k8s.io/calico-node created
clusterrolebinding.rbac.authorization.k8s.io/calico-node created
daemonset.apps/calico-node created
serviceaccount/calico-node created
deployment.apps/calico-kube-controllers created
serviceaccount/calico-kube-controllers created
　　查看Calico状态

[root@k8s-master01 Calico]# kubectl get po -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-6d85fdfbd8-2jtj9 1/1 Running 0 3m38s
calico-node-5t9kj 1/1 Running 0 3m38s
calico-node-9ftns 1/1 Running 0 3m38s
calico-node-b6rsl 1/1 Running 0 3m38s
calico-node-hfqrd 1/1 Running 0 3m38s
calico-node-lpcmp 1/1 Running 0 3m38s

　　查看集群状态

[root@k8s-master01 Calico]# kubectl cluster-info
Kubernetes master is running at https://192.168.1.88:8443

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
[root@k8s-master01 Calico]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master01 Ready 12h v1.16.2
k8s-master02 Ready 13h v1.16.2
k8s-master03 Ready 12h v1.16.2
k8s-node01 Ready 12h v1.16.2
k8s-node02 Ready 11h v1.16.2

　　至此集群安装完毕，其他组件可以参考本博客其他版本安装的文档。

　　参考《再也不踩坑的Kubernetes实战指南》
原文链接：https://www.cnblogs.com/dukuan/p/11780729.html

你可能感兴趣的:(k8s,二进制,高可用,1.16,Kubernetes)

react-intl——react国际化使用方案苹果酱0567 面试题汇总与解析 java 开发语言中间件 spring boot 后端
国际化介绍i18n：internationalization国家化简称，首字母+首尾字母间隔的字母个数+尾字母，类似的还有k8s(Kubernetes)React-intl是React中最受欢迎的库。使用步骤安装#usenpmnpminstallreact-intl-D#useyarn项目入口文件配置//index.tsximportReactfrom"react";importReactDOMf
3286、穿越网格图的安全路径 Lenyiin 题解 c++算法 leetcode
3286、[中等]穿越网格图的安全路径1、题目描述给你一个mxn的二进制矩形grid和一个整数health表示你的健康值。你开始于矩形的左上角(0,0)，你的目标是矩形的右下角(m-1,n-1)。你可以在矩形中往上下左右相邻格子移动，但前提是你的健康值始终是正数。对于格子(i,j)，如果grid[i][j]=1，那么这个格子视为不安全的，会使你的健康值减少1。如果你可以到达最终的格子，请你返回tr
【ShuQiHere】进制与补码的世界：从符号-大小表示法到二补码 ShuQiHere 二进制计算机组成原理
【ShuQiHere】在计算机系统中，表示正数是相对简单的，只需使用其对应的二进制形式即可。然而，如何有效地表示负数一直是计算机科学中的一个关键问题。为了解决这个问题，科学家们提出了多种表示方法，包括符号-大小表示法（Sign-MagnitudeRepresentation）、一补码（One’sComplement）和二补码（Two’sComplement）。在本文中，我们将深入探讨这些表示方法的
【从问题中去学习k8s】k8s中的常见面试题（夯实理论基础）（二十八）向往风的男子 k8s 学习 kubernetes 容器
本站以分享各种运维经验和运维所需要的技能为主《python零基础入门》：python零基础入门学习《python运维脚本》：python运维脚本实践《shell》：shell学习《terraform》持续更新中：terraform_Aws学习零基础入门到最佳实战《k8》从问题中去学习k8s《docker学习》暂未更新《ceph学习》ceph日常问题解决分享《日志收集》ELK+各种中间件《运维日常》
Kubernetes数据持久化看清所苡看轻 kubernetes(k8s)emptyDir HostPath pv pvc kubernetes
在k8s中，Volume（数据卷）存在明确的生命周期（与包含该数据卷的容器组（pod）相同）。因此Volume的生命周期比同一容器组（pod）中任意容器的生命周期要更长，不管容器重启了多少次，数据都被保留下来。当然，如果pod不存在了，数据卷自然退出了。此时，根据pod所使用的数据卷类型不同，数据可能随着数据卷的退出而删除，也可能被真正持久化，并在下次容器组重启时仍然可以使用。从根本上来说，一个数
KVM+GFS分布式存储系统构建KVM高可用 henan程序媛分布式 GFS 高可用 KVM
一、案列分析1.1案列概述本章案例主要使用之前章节所学的KVM及GlusterFs技术,结合起来从而实现KVM高可用。利用GlusterFs分布式复制卷，对KVM虚拟机文件进行分布存储和冗余。分布式复制卷主要用于需要冗余的情况下把一个文件存放在两个或两个以上的节点,当其中一个节点数据丢失或者损坏之后，KVM仍然能够通过卷组找到另一节点上存储的虚拟机文件，以保证虚拟机正常运行。当节点修复之后，Glu
Kubernetes部署MySQL数据持久化沫殇-MS Kubernetes MySQL数据库 kubernetes mysql 容器
一、安装配置NFS服务端1、安装nfs-kernel-server：sudoapt-yinstallnfs-kernel-server2、服务端创建共享目录#列出所有可用块设备的信息lsblk#格式化磁盘sudomkfs-text4/dev/sdb#创建一个目录：sudomkdir-p/data/nfs/mysql#更改目录权限：sudochown-Rnobody:nogroup/data/nfs
Kubernetes的3种数据持久化方式 Seal^_^ 【云原生】容器化与编排技术持续集成 #Kubernetes kubernetes 容器云原生 EmptyDir 面试 HostPath
Kubernetes的3种数据持久化方式1.EmptyDir2.HostPath3.PersistentVolume(PV)TheBegin点点关注，收藏不迷路Kubernetes提供了几种数据持久化方式，以满足不同场景的需求：1.EmptyDir用途：临时数据存储，Pod内容器间共享。特点：生命周期与Pod相同，Pod删除时数据也删除。2.HostPath用途：访问宿主机特定文件或目录。特点：增
【Kubernetes】常见面试题汇总（十一） summer.335 Kubernetes kubernetes 容器云原生
目录33.简述Kubernetes外部如何访问集群内的服务？34.简述Kubernetesingress？35.简述Kubernetes镜像的下载策略？33.简述Kubernetes外部如何访问集群内的服务？（1）对于Kubernetes，集群外的客户端默认情况，无法通过Pod的IP地址或者Service的虚拟IP地址：虚拟端口号进行访问。（2）通常可以通过以下方式进行访问Kubernetes集群
k8s中Service暴露的种类以及用法听说唐僧不吃肉 K8S kubernetes 容器云原生
一、说明在Kubernetes中，有几种不同的方式可以将服务（Service）暴露给外部流量。这些方式通过定义服务的spec.type字段来确定。二、详解1.ClusterIP定义：默认类型，服务只能在集群内部访问。作用：通过集群内部IP地址暴露服务。示例：spec:type:ClusterIPports:-port:80targetPo
二十四、k8s 资源管理繁华依在 k8s kubernetes 容器云原生
目录一、资源配置范围管理LimitRange介绍1、LimitRange可以做什么：2、资源限制和请求的约束3、创建LimitsRange对象4、示例：创建一个pod5、测试用例测试1：测试2：测试3：二、资源服务质量管理（RequestsQos）1、Qos级别分类：1.1、Guaranteed：1.2、BestEffort：1.3、Burstable：2、Qos的工作特点3、示例三、资源配额管理
Kubernetes 自定义控制器开发 IT回忆录 Kubenetes kubernetes
目录前言一、CRD二、创建数据库表（Mysql）二、控制器开发1.使用kubernetes的examplecontroller模板2.在controller.go中新增数据表监听方法3.修改tools工具生成资源对象结构体定义这里记录开发k8s控制器的一般方式，controller开发主要使用k8s提供的client-go库进行。前言Controller监听集群内部资源对象的变化，编辑资源对象(增
用kubedam搭建的k8s证书过期处理方法我滴鬼鬼呀wks k8s 1024程序员节
kubeadm部署的k8s证书过期1、查看证书过期时间kubeadmalphacertscheck-expiration若证书已经过期无法试用kubectl命令建议修改服务器时间到未过期的时间段2、配置kube-controller-manager.yaml文件cat/etc/kubernetes/manifests/kube-controller-manager.yamlapiVersion:v
k8s证书过期问题处理 olina_qin kubernetes 容器云原生
k8s证书过期问题处理opensslx509-in/etc/kubernetes/pki/apiserver.crt-noout-dateskubeadmcertsrenewallsystemctlrestartkubeleopensslx509-in/etc/kubernetes/pki/apiserver.crt-noout-text|grep"NotAfter"cp/etc/kubernet
Kubernetes Ingress 控制器（Nginx）安装与使用教程农优影
KubernetesIngress控制器（Nginx）安装与使用教程kubernetes-ingressNGINXandNGINXPlusIngressControllersforKubernetes项目地址:https://gitcode.com/gh_mirrors/ku/kubernetes-ingress1.项目目录结构及介绍在nginxinc/kubernetes-ingress仓库中，
【K8s】专题十一：Kubernetes 集群证书过期处理方法行者Sun1989 Kubernetes kubernetes 云原生容器
本文内容均来自个人笔记并重新梳理，如有错误欢迎指正！如果对您有帮助，烦请点赞、关注、转发、订阅专栏！专栏订阅入口Linux专栏|Docker专栏|Kubernetes专栏往期精彩文章【Docker】（全网首发）KylinV10下MySQL容器内存占用异常的解决方法【Docker】（全网首发）KylinV10下MySQL容器内存占用异常的解决方法（续）【Docker】MySQL源码构建Docker镜
Docker学习十一：Kubernetes概述爱打羽球的程序猿 Docker学习系列 docker kubernetes 学习
一、Kubernetes简介2006年，Google提出了云计算的概念，当时的云计算领域还是以虚拟机为代表的云平台。2013年，Docker横空出世，Docker提出了镜像、仓库等核心概念，规范了服务的交付标准，使得复杂服务的落地变得更加简单，之后Docker又定义了OCI标准，Docker在容器领域称为事实的标准。但是，Docker诞生只是帮助定义了开发和交付标准，如果想要在生产环境中大批量的使
Cloud Native Weekly | 华为云抢先发布Redis5.0，红帽宣布收购混合云提供商 weixin_34302561 数据库 devops 大数据
1——华为云抢先发布Redis5.02——DigitalOceanK8s服务正式上线3——红帽宣布收购混合云提供商NooBaa4——微软发布多项AzureKubernetes服务更新1华为云抢先发布Redis5.012月17日，华为云在DCS2.0的基础上，快人一步，抢先推出了新的Redis5.0产品，这是一个崭新的突破。目前国内在缓存领域的发展普遍停留在Redis4.0阶段，华为云率先发布了Re
SpringBoot整合ES搜索引擎实现网站热搜词及热度计算码踏云端 springboot Elasticsearch spring boot elasticsearch 后端热搜词热度计算 java
博主简介：历代文学网（PC端可以访问：https://literature.sinhy.com/#/literature?__c=1000，移动端可微信小程序搜索“历代文学”）总架构师，15年工作经验，精通Java编程，高并发设计，Springboot和微服务，熟悉Linux，ESXI虚拟化以及云原生Docker和K8s，热衷于探索科技的边界，并将理论知识转化为实际应用。保持对新技术的好奇心，乐于
（k8s）Kubernetes 从0到1容器编排之旅道不贱卖，法不轻传 kubernets kubernetes 容器云原生
一、引言在当今数字化的浪潮中，Kubernetes如同一艘强大的航船，引领着容器化应用的部署与管理。它以其卓越的灵活性、可扩展性和可靠性，成为众多企业和开发者的首选。然而，要真正发挥Kubernetes的强大威力，仅仅掌握基本操作是远远不够的。本文将带你深入探索Kubernetes使用过程中的奇技妙法，为你开启一段优雅的容器编排之旅。二、高级资源管理之精妙艺术1.资源配额与限制：雕琢资源之美•Ku
老生常谈：MySQL高可用架构我有一头小花驴 mysql 架构数据库
引言“高可用”是互联网一个永恒的话题，先避开MySQL不谈，为了保证各种服务的高可用有几种常用的解决方案。服务冗余：把服务部署多份，当某个节点不可用时，切换到其他节点。服务冗余对于无状态的服务是相对容易的。服务备份：有些服务是无法同时存在多个运行时的，比如说：Nginx的反向代理，一些集群的leader节点。这时可以存在一个备份服务，处于随时待命状态。自动切换：服务冗余之后，当某个节点不可用时，要
第二周内容 s好学向上p
1、描述Linux发行版的系统目录名称命名规则以及用途。Linux系统基础目录的命名法则：-严格区分大小写-目录也是文件，在同一路径下，两个文件不能同名-支持使用除/以外的任意字符-最长字符不能超过255个字符Linux根下目录及用途/bin存放二进制可执行文件(ls,cat,mkdir等)，常用命令一般都在这里/etc配置文件/home用户家目录/root超级用户（系统管理员）的主目录/sbin
MySQL MHA pokemon.. mysql 数据库
一、MySQLMHA高可用环境概述1.什么是MHAMHA（MasterHighAvailability）是一款用于MySQL数据库集群的高可用解决方案，主要解决MySQL单点故障问题。在MySQL集群的主服务器发生故障时，MHA可以自动执行故障切换，将一个从服务器提升为新的主服务器，最大限度地保证数据的一致性。MHA的切换时间通常在0-30秒之间，极大提高了MySQL集群的可用性。2.MHA的组成
ETCD 六 etcd总体架构 wanghaichao1234 etcd etcd 架构数据库
etcd源码结构etcd项目代码的目录结构：包名用途apiprotobuf定义client/v3客户端sdkcontribraftexample实现etcdctl命令行客户端实现，用于网路的操作etcdutl命令行管理工具，直接操作etcd数据文件。hack基准测试、测试集群、k8s部署、分支管理、证书等pkg实用程序包的集合raftraft实现server.auth角色身份验证server.em
解决前端导出excel文件，打开为乱码荔枝，你让我拿什么荔枝！ vue 前端 elementui vue.js
前端开发中，导入和导出文件是比较常见的业务场景，常见的情况是：后端返回一个二进制的流文件，前端将其转化为excel文件即可。但是往往会出现转化后的excel文件内容位乱码的情况，本文中提供了两个解决方案：方案一：用户自定义上传方法添加附件添加附件代码解读：上述代码采用了element-ui的el-upload文件上传的组件。改组件有两种文件上传的方式：1.组件自带的上传方法，只需要给其设置acti
golang-101-hacks(3)——包 _羊羽_
注：本文是对golang-101-hacks中文翻译,原文地址在“Go”中，包分为两种类型:(1)main包:用于生成可执行的二进制文件，main函数是程序的入口点。下面以helllo.go为例:packagemainimport"greet"funcmain(){greet.Greet()}(2)其他类型的包也可以在细分成两类:库文件包:用来生成可以被其他人重用的目标文件。如greet.go这个
【Java】如何将二进制转换成MultipartFile Mxin5 Java java 开发语言
业务场景：前端发送请求到后端进行文件上传，后端接收文件并调用第三方接口进行文件处理，响应格式为二进制，然后我们需要将二进制进行转换为MultipartFile进行文件上传。如果你想要将File转换成MultipartFile，可以参考：【Java】如何将File转换成MultipartFile_javafile转multipartfile_MXin5的博客-CSDN博客1.传递二进制respons
Mall4j商城实战 - 部署 canal 数据库增量日志解析 yueerba126 Mall4j商城实战数据库 spring cloud 微服务架构
Canal简介Canal是基于MySQL数据库增量日志解析的工具，主要用于增量数据的订阅和消费。Canal主要用途基于MySQL数据库增量日志解析详细功能：实时解析MySQL的二进制日志（Binlog）。捕获数据库中的所有增量变更，如插入、更新和删除操作。使用场景：适用于实时监控数据库变化的应用，比如数据复制、数据备份或实时数据分析等。提供增量数据订阅和消费服务
【K8S】kubernetes集群架构与组件奇奇怪怪^ 云 Linux IT 运维服务器 linux
文章目录【K8S】kubernetes集群架构与组件kubernetes组件**master组件**node组件整体流程POD终止过程【K8S】kubernetes集群架构与组件kubernetes组件K8S是属于主从设备模型(Master-slave架构)，即有Master节点负责集群的调度、管理和运维，Slave节点是集群中的运算工作负载节点在K8S中，主节点一般被称为Master节点，而从节
K8S学习笔记02——K8S组件沉淅尘 #Docker #K8S kubernetes
Kubernetes组件一、控制平面组件（ControlPlaneComponents）(1)kube-apiserver(2)etcd(3)kube-scheduler(4)kube-controller-manager(5)cloud-controller-manager二、Node组件1.kubelet2.kube-proxy3.容器运行时（ContainerRuntime）三、插件（Add
JAVA基础灵静志远位运算加载 Date 字符串池覆盖
一、类的初始化顺序 1 （静态变量，静态代码块）-->（变量，初始化块）--> 构造器同一括号里的，根据它们在程序中的顺序来决定。上面所述是同一类中。如果是继承的情况，那就在父类到子类交替初始化。二、String 1 String a = "abc"; JAVA虚拟机首先在字符串池中查找是否已经存在了值为"abc"的对象，根
keepalived实现redis主从高可用 bylijinnan redis
方案说明两台机器（称为A和B），以统一的VIP对外提供服务 1.正常情况下，A和B都启动，B会把A的数据同步过来（B is slave of A） 2.当A挂了后，VIP漂移到B；B的keepalived 通知redis 执行：slaveof no one，由B提供服务 3.当A起来后，VIP不切换，仍在B上面；而A的keepalived 通知redis 执行slaveof B，开始
java文件操作大全 0624chenhong java
最近在博客园看到一篇比较全面的文件操作文章，转过来留着。 http://www.cnblogs.com/zhuocheng/archive/2011/12/12/2285290.html 转自http://blog.sina.com.cn/s/blog_4a9f789a0100ik3p.html 一.获得控制台用户输入的信息 &nbs
android学习任务不懂事的小屁孩工作
任务完成情况搞清楚带箭头的pupupwindows和不带的使用已完成熟练使用pupupwindows和alertdialog，并搞清楚两者的区别已完成熟练使用android的线程handler,并敲示例代码进行中了解游戏2048的流程，并完成其代码工作进行中-差几个actionbar 研究一下android的动画效果，写一个实例已完成复习fragem
zoom.js 换个号韩国红果果 oom
它的基于bootstrap 的 https://raw.github.com/twbs/bootstrap/master/js/transition.js transition.js模块引用顺序 <link rel="stylesheet" href="style/zoom.css"> <script src=&q
详解Oracle云操作系统Solaris 11.2 蓝儿唯美 Solaris
当Oracle发布Solaris 11时，它将自己的操作系统称为第一个面向云的操作系统。Oracle在发布Solaris 11.2时继续它以云为中心的基调。但是，这些说法没有告诉我们为什么Solaris是配得上云的。幸好，我们不需要等太久。Solaris11.2有4个重要的技术可以在一个有效的云实现中发挥重要作用：OpenStack、内核域、统一存档（UA）和弹性虚拟交换（EVS）。
spring学习——springmvc（一） a-john springMVC
Spring MVC基于模型-视图-控制器（Model-View-Controller，MVC）实现，能够帮助我们构建像Spring框架那样灵活和松耦合的Web应用程序。 1，跟踪Spring MVC的请求请求的第一站是Spring的DispatcherServlet。与大多数基于Java的Web框架一样，Spring MVC所有的请求都会通过一个前端控制器Servlet。前
hdu4342 History repeat itself-------多校联合五 aijuans 数论
水题就不多说什么了。 #include<iostream>#include<cstdlib>#include<stdio.h>#define ll __int64using namespace std;int main(){ int t; ll n; scanf("%d",&t); while(t--)
EJB和javabean的区别 asia007 bean ejb
EJB不是一般的JavaBean,EJB是企业级JavaBean,EJB一共分为3种,实体Bean,消息Bean,会话Bean,书写EJB是需要遵循一定的规范的,具体规范你可以参考相关的资料.另外,要运行EJB,你需要相应的EJB容器,比如Weblogic,Jboss等,而JavaBean不需要,只需要安装Tomcat就可以了 1.EJB用于服务端应用开发, 而JavaBeans
Struts的action和Result总结百合不是茶 struts Action配置 Result配置
一:Action的配置详解: 下面是一个Struts中一个空的Struts.xml的配置文件 <?xml version="1.0" encoding="UTF-8" ?> <!DOCTYPE struts PUBLIC &quo
如何带好自已的团队 bijian1013 项目管理团队管理团队
在网上看到博客" 怎么才能让团队成员好好干活"的评论，觉得写的比较好。原文如下：我做团队管理有几年了吧，我和你分享一下我认为带好团队的几点： 1.诚信对团队内成员，无论是技术研究、交流、问题探讨，要尽可能的保持一种诚信的态度，用心去做好，你的团队会感觉得到。 2.努力提
Java代码混淆工具 sunjing ProGuard
Open Source Obfuscators ProGuard http://java-source.net/open-source/obfuscators/proguardProGuard is a free Java class file shrinker and obfuscator. It can detect and remove unused classes, fields, m
【Redis三】基于Redis sentinel的自动failover主从复制 bit1129 redis
在第二篇中使用2.8.17搭建了主从复制，但是它存在Master单点问题，为了解决这个问题，Redis从2.6开始引入sentinel，用于监控和管理Redis的主从复制环境，进行自动failover，即Master挂了后，sentinel自动从从服务器选出一个Master使主从复制集群仍然可以工作，如果Master醒来再次加入集群，只能以从服务器的形式工作。什么是Sentine
使用代理实现Hibernate Dao层自动事务白糖_ DAO spring AOP 框架 Hibernate
都说spring利用AOP实现自动事务处理机制非常好，但在只有hibernate这个框架情况下，我们开启session、管理事务就往往很麻烦。 public void save(Object obj){ Session session = this.getSession(); Transaction tran = session.beginTransaction(); try
maven3实战读书笔记 braveCS maven3
Maven简介是什么？ Is a software project management and comprehension tool.项目管理工具是基于POM概念(工程对象模型) [设计重复、编码重复、文档重复、构建重复，maven最大化消除了构建的重复] [与XP：简单、交流与反馈；测试驱动开发、十分钟构建、持续集成、富有信息的工作区] 功能：
编程之美-子数组的最大乘积 bylijinnan 编程之美
public class MaxProduct { /** * 编程之美子数组的最大乘积 * 题目: 给定一个长度为N的整数数组，只允许使用乘法，不能用除法，计算任意N-1个数的组合中乘积中最大的一组，并写出算法的时间复杂度。 * 以下程序对应书上两种方法，求得“乘积中最大的一组”的乘积——都是有溢出的可能的。 * 但按题目的意思，是要求得这个子数组，而不
读书笔记-2 chengxuyuancsdn 读书笔记
1、反射 2、oracle年-月-日时-分-秒 3、oracle创建有参、无参函数 4、oracle行转列 5、Struts2拦截器 6、Filter过滤器(web.xml) 1、反射 (1)检查类的结构在java.lang.reflect包里有3个类Field,Method,Constructor分别用于描述类的域、方法和构造器。 2、oracle年月日时分秒 s
[求学与房地产]慎重选择IT培训学校 comsci it
关于培训学校的教学和教师的问题,我们就不讨论了,我主要关心的是这个问题培训学校的教学楼和宿舍的环境和稳定性问题我们大家都知道，房子是一个比较昂贵的东西，特别是那种能够当教室的房子... &nb
RMAN配置中通道(CHANNEL)相关参数 PARALLELISM 、FILESPERSET的关系 daizj oracle rman filesperset PARALLELISM
RMAN配置中通道(CHANNEL)相关参数 PARALLELISM 、FILESPERSET的关系转 PARALLELISM --- 我们还可以通过parallelism参数来指定同时"自动"创建多少个通道： RMAN > configure device type disk parallelism 3 ; 表示启动三个通道，可以加快备份恢复的速度。
简单排序:冒泡排序 dieslrae 冒泡排序
public void bubbleSort(int[] array){ for(int i=1;i<array.length;i++){ for(int k=0;k<array.length-i;k++){ if(array[k] > array[k+1]){
初二上学期难记单词三 dcj3sjt126com sciet
concert 音乐会 tonight 今晚 famous 有名的；著名的 song 歌曲 thousand 千 accident 事故；灾难 careless 粗心的，大意的 break 折断；断裂；破碎 heart 心（脏） happen 偶尔发生，碰巧 tourist 旅游者；观光者 science （自然）科学 marry 结婚 subject 题目；
I.安装Memcahce 1. 安装依赖包libevent Memcache需要安装libevent,所以安装前可能需要执行 Shell代码收藏代码 dcj3sjt126com redis
wget http://download.redis.io/redis-stable.tar.gz tar xvzf redis-stable.tar.gz cd redis-stable make 前面3步应该没有问题，主要的问题是执行make的时候，出现了异常。异常一： make[2]: cc: Command not found 异常原因：没有安装g
并发容器 shuizhaosi888 并发容器
通过并发容器来改善同步容器的性能，同步容器将所有对容器状态的访问都串行化，来实现线程安全，这种方式严重降低并发性，当多个线程访问时，吞吐量严重降低。并发容器ConcurrentHashMap 替代同步基于散列的Map，通过Lock控制。 &nb
Spring Security（12）——Remember-Me功能 234390216 Spring Security Remember Me 记住我
Remember-Me功能目录 1.1 概述 1.2 基于简单加密token的方法 1.3 基于持久化token的方法 1.4 Remember-Me相关接口和实现
位运算焦志广位运算
一、位运算符Ｃ语言提供了六种位运算符： & 按位与 | 按位或 ^ 按位异或 ~ 取反 << 左移 >> 右移 1. 按位与运算按位与运算符"&"是双目运算符。其功能是参与运算的两数各对应的二进位相与。只有对应的两个二进位均为1时，结果位才为1 ，否则为0。参与运算的数以补码方式出现。例如：9&am
nodejs 数据库连接 mongodb mysql liguangsong mongodb mysql node 数据库连接
1.mysql 连接 package.json中dependencies加入 "mysql":"~2.7.0" 执行 npm install 在config 下创建文件 database.js
java动态编译 olive6615 java HotSpot jvm 动态编译
在HotSpot虚拟机中，有两个技术是至关重要的，即动态编译(Dynamic compilation)和Profiling。 HotSpot是如何动态编译Javad的bytecode呢？Java bytecode是以解释方式被load到虚拟机的。HotSpot里有一个运行监视器，即Profile Monitor,专门监视
Storm0.9.5的集群部署配置优化 roadrunners 优化 storm.yaml
nimbus结点配置（storm.yaml）信息： # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional inf
101个MySQL 的调节和优化的提示 tomcat_oracle mysql
　1. 拥有足够的物理内存来把整个InnoDB文件加载到内存中——在内存中访问文件时的速度要比在硬盘中访问时快的多。　　2. 不惜一切代价避免使用Swap交换分区 – 交换时是从硬盘读取的，它的速度很慢。　　3. 使用电池供电的RAM（注：RAM即随机存储器）。　　4. 使用高级的RAID（注：Redundant Arrays of Inexpensive Disks，即磁盘阵列
zoj 3829 Known Notation(贪心) 阿尔萨斯 ZOJ
题目链接：zoj 3829 Known Notation 题目大意：给定一个不完整的后缀表达式，要求有2种不同操作，用尽量少的操作使得表达式完整。解题思路：贪心，数字的个数要要保证比∗的个数多1，不够的话优先补在开头是最优的。然后遍历一遍字符串，碰到数字+1，碰到∗-1,保证数字的个数大于等1，如果不够减的话，可以和最后面的一个数字交换位置（用栈维护十分方便），因为添加和交换代价都是1

CentOS 8 二进制 高可用 安装 k8s 1.16.x

你可能感兴趣的:(k8s,二进制,高可用,1.16,Kubernetes)

CentOS 8 二进制高可用安装 k8s 1.16.x