kubeadm部署kubernetes

kubeadm 部署方式介绍

1、使用kubeadm工具快速部署kubernetes集群：

一、创建一个 master 节点 kubeadm init
二、 将 node 节点加入集群kubeadm join

2、Kubernetes集群机器要求

操作系统 CentOS7.x-86_x64；
硬件配置：2GB+ RAM，2+ CPU，硬盘30GB+；
集群中所有机器之间网络互通；
集群中所有机器可以访问外网 (需要拉取镜像)；
禁止 swap 分区；

3、最终目标

1）在所有节点上安装 docker 和 kubeadm；
2）部署 kubernetes master；
3）部署容器网络插件；
4）部署 kubernetes node，将节点加入 kubernetes 集群中；
5）部署 dashboard web 页面，可视化查看 kubernetes 资源；

单master节点部署规划

多master节点部署规划

单master节点环境准备

1、k8s集群主机角色IP

k8s-master   192.168.6.112
k8s-worker1  192.168.6.113
k8s-worker2  192.168.6.114

2、配置网络

vim /etc/sysconfig/network-scripts/ifcfg-ens32
===============================
TYPE="Ethernet"
BOOTPROTO="static"
DEVICE="ens32"
ONBOOT="yes"
# 分别配置三台服务器的ip地址
IPADDR="192.168.6.112"
# IPADDR="192.168.6.113"
# IPADDR="192.168.6.114"
GATEWAY="192.168.6.1"
NETMASK="255.255.255.0"
DNS1="8.8.8.8"
===============================

service network restart

3、关闭防火墙

systemctl stop firewalld
systemctl disable firewalld

4、关闭selinux

# 重启永久关闭selinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
# 当前环境临时关闭selinux
setenforce 0

5、关闭swap分区

# 临时开启 swapon -a
# 临时关闭swap
swapoff -a
# 永久关闭swap
sed -ri 's/.*swap.*/#&/' /etc/fstab
=====================
UUID=a6f57ca4-f414-43ee-a102-39e970fe4741 swap                    swap    defaults        0 0

6、修改主机名

# 192.168.6.112
hostnamectl set-hostname k8s-master
# 192.168.6.113
hostnamectl set-hostname k8s-worker1
# 192.168.6.114
hostnamectl set-hostname k8s-worker2

7、在/etc/hosts文件添加本地dns

# 追加写入文件内容
cat >> /etc/hosts << EOF
192.168.6.112 k8s-master
192.168.6.113 k8s-worker1
192.168.6.114 k8s-worker2
EOF

8、将桥接的IPv4流量传递到iptables的链

# 覆盖写入文件内容
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

# 以上配置立即生效
sysctl --system

9、时间同步

yum install ntpdate -y
ntpdate time.windows.com

7、所有节点安装docker/kubeadm/kubelet
Kubernetes 默认CRI（容器运行时）为docker，因此先安装docker。

（1）安装Docker

wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
yum -y install docker-ce-18.06.1.ce-3.el7
systemctl enable docker && systemctl start docker
docker --version

（2）添加阿里云YUM 软件源
设置仓库地址

cat > /etc/docker/daemon.json << EOF
{
	"registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"]
}
EOF

systemctl restart docker
docker info

添加yum 源

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

EOF

（3）安装kubeadm，kubelet 和kubectl

yum install -y kubelet-1.18.0 kubeadm-1.18.0 kubectl-1.18.0
systemctl enable kubelet

8、部署Kubernetes Master
（1）在192.168.6.112（Master）执行

kubeadm init \
--apiserver-advertise-address=192.168.6.112 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.18.0 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16

由于默认拉取镜像地址k8s.gcr.io 国内无法访问，这里指定阿里云镜像仓库地址。
成执行后的结果：

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.6.112:6443 --token 2be5c9.y7fini11lk3f7f3f \
    --discovery-token-ca-cert-hash sha256:bf1c7f2fab37bf8f6d248087e19c2ada17327bdffc95159d0ec3c2bf8d14973a 

docker images
=======================
REPOSITORY                                                        TAG                 IMAGE ID            CREATED             SIZE
registry.aliyuncs.com/google_containers/kube-proxy                v1.18.0             43940c34f24f        2 years ago         117MB
registry.aliyuncs.com/google_containers/kube-apiserver            v1.18.0             74060cea7f70        2 years ago         173MB
registry.aliyuncs.com/google_containers/kube-controller-manager   v1.18.0             d3e55153f52f        2 years ago         162MB
registry.aliyuncs.com/google_containers/kube-scheduler            v1.18.0             a31f78c7c8ce        2 years ago         95.3MB
registry.aliyuncs.com/google_containers/pause                     3.2                 80d28bedfe5d        2 years ago         683kB
registry.aliyuncs.com/google_containers/coredns                   1.6.7               67da37a9a360        2 years ago         43.8MB
registry.aliyuncs.com/google_containers/etcd                      3.4.3-0             303ce5db0e90        2 years ago         288MB

（2）使用kubectl 工具：

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

查看

kubectl get nodes
NAME         STATUS     ROLES    AGE     VERSION
k8s-master   NotReady   master   4m39s   v1.18.0

9、加入Kubernetes Node
（1）在其他节点192.168.31.62/63（Node）执行
向集群添加新节点，执行在kubeadm init 输出的kubeadm join 命令：

kubeadm join 192.168.6.112:6443 --token 2be5c9.y7fini11lk3f7f3f \
    --discovery-token-ca-cert-hash sha256:bf1c7f2fab37bf8f6d248087e19c2ada17327bdffc95159d0ec3c2bf8d14973a 

kubectl get nodes
========================
NAME          STATUS     ROLES    AGE     VERSION
k8s-master    NotReady   master   8m34s   v1.18.0
k8s-worker1   NotReady   <none>   13s     v1.18.0
k8s-worker2   NotReady   <none>   8s      v1.18.0

默认token有效期为24小时，这时需要重新创建token，操作如下：

kubeadm token create --print-join-command

10、主节点安装Pod 网络插件（CNI）
默认镜像地址无法访问，sed命令修改为docker hub镜像仓库。

wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

确保能够访问到quay.io 这个registery。如果Pod 镜像下载失败，可以改这个镜像地址

kubectl get pods -n kube-system
=====================================
NAME                                 READY   STATUS    RESTARTS   AGE
coredns-7ff77c879f-gzg2l             1/1     Running   0          16m
coredns-7ff77c879f-tg9lr             1/1     Running   0          16m
etcd-k8s-master                      1/1     Running   0          17m
kube-apiserver-k8s-master            1/1     Running   0          17m
kube-controller-manager-k8s-master   1/1     Running   0          17m
kube-flannel-ds-gh68f                1/1     Running   0          113s
kube-flannel-ds-mm7bd                1/1     Running   0          113s
kube-flannel-ds-rj254                1/1     Running   0          113s
kube-proxy-4hjwr                     1/1     Running   0          8m40s
kube-proxy-ddtdq                     1/1     Running   0          8m45s
kube-proxy-ghxxc                     1/1     Running   0          16m
kube-scheduler-k8s-master            1/1     Running   0          17m

11、测试kubernetes 集群
在Kubernetes 集群中创建一个pod，验证是否正常运行：

kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80 --type=NodePort

kubectl get pod,svc
NAME                        READY   STATUS              RESTARTS   AGE
pod/nginx-f89759699-48qvm   0/1     ContainerCreating   0          27s

NAME                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP        19m
service/nginx        NodePort    10.97.34.213   <none>        80:31499/TCP   7s

访问地址：

http://192.168.6.112:31499
http://192.168.6.113:31499
http://192.168.6.114:31499

---

卸载kubernetes

kubeadm reset -f
modprobe -r ipip
lsmod
rm -rf ~/.kube/
rm -rf /etc/kubernetes/
rm -rf /etc/systemd/system/kubelet.service.d
rm -rf /etc/systemd/system/kubelet.service
rm -rf /usr/bin/kube*
rm -rf /etc/cni
rm -rf /opt/cni
rm -rf /var/lib/etcd
rm -rf /var/etcd
yum clean all
yum remove kube*

kubeadm init命令文档

kubeadm init --help
===========================
Run this command in order to set up the Kubernetes control plane

The "init" command executes the following phases:
`
preflight                    Run pre-flight checks
kubelet-start                Write kubelet settings and (re)start the kubelet
certs                        Certificate generation
  /ca                          Generate the self-signed Kubernetes CA to provision identities for other Kubernetes components
  /apiserver                   Generate the certificate for serving the Kubernetes API
  /apiserver-kubelet-client    Generate the certificate for the API server to connect to kubelet
  /front-proxy-ca              Generate the self-signed CA to provision identities for front proxy
  /front-proxy-client          Generate the certificate for the front proxy client
  /etcd-ca                     Generate the self-signed CA to provision identities for etcd
  /etcd-server                 Generate the certificate for serving etcd
  /etcd-peer                   Generate the certificate for etcd nodes to communicate with each other
  /etcd-healthcheck-client     Generate the certificate for liveness probes to healthcheck etcd
  /apiserver-etcd-client       Generate the certificate the apiserver uses to access etcd
  /sa                          Generate a private key for signing service account tokens along with its public key
kubeconfig                   Generate all kubeconfig files necessary to establish the control plane and the admin kubeconfig file
  /admin                       Generate a kubeconfig file for the admin to use and for kubeadm itself
  /kubelet                     Generate a kubeconfig file for the kubelet to use *only* for cluster bootstrapping purposes
  /controller-manager          Generate a kubeconfig file for the controller manager to use
  /scheduler                   Generate a kubeconfig file for the scheduler to use
control-plane                Generate all static Pod manifest files necessary to establish the control plane
  /apiserver                   Generates the kube-apiserver static Pod manifest
  /controller-manager          Generates the kube-controller-manager static Pod manifest
  /scheduler                   Generates the kube-scheduler static Pod manifest
etcd                         Generate static Pod manifest file for local etcd
  /local                       Generate the static Pod manifest file for a local, single-node local etcd instance
upload-config                Upload the kubeadm and kubelet configuration to a ConfigMap
  /kubeadm                     Upload the kubeadm ClusterConfiguration to a ConfigMap
  /kubelet                     Upload the kubelet component config to a ConfigMap
upload-certs                 Upload certificates to kubeadm-certs
mark-control-plane           Mark a node as a control-plane
bootstrap-token              Generates bootstrap tokens used to join a node to a cluster
kubelet-finalize             Updates settings relevant to the kubelet after TLS bootstrap
  /experimental-cert-rotation  Enable kubelet client certificate rotation
addon                        Install required addons for passing Conformance tests
  /coredns                     Install the CoreDNS addon to a Kubernetes cluster
  /kube-proxy                  Install the kube-proxy addon to a Kubernetes cluster
`

Usage:
  kubeadm init [flags]
  kubeadm init [command]

Available Commands:
  phase       Use this command to invoke single phase of the init workflow

Flags:
      --apiserver-advertise-address string   The IP address the API Server will advertise it‘s listening on. If not set the default network interface will be used.
      --apiserver-bind-port int32            Port for the API Server to bind to. (default 6443)
      --apiserver-cert-extra-sans strings    Optional extra Subject Alternative Names (SANs) to use for the API Server serving certificate. Can be both IP addresses and DNS names.
      --cert-dir string                      The path where to save and store the certificates. (default "/etc/kubernetes/pki")
      --certificate-key string               Key used to encrypt the control-plane certificates in the kubeadm-certs Secret.
      --config string                        Path to a kubeadm configuration file.
      --control-plane-endpoint string        Specify a stable IP address or DNS name for the control plane.
      --cri-socket string                    Path to the CRI socket to connect. If empty kubeadm will try to auto-detect this value; use this option only if you have more than one CRI installed or if you have non-standard CRI socket.
      --dry-run                              Don‘t apply any changes; just output what would be done.
  -k, --experimental-kustomize string        The path where kustomize patches for static pod manifests are stored.
      --feature-gates string                 A set of key=value pairs that describe feature gates for various features. Options are:
                                             IPv6DualStack=true|false (ALPHA - default=false)
                                             PublicKeysECDSA=true|false (ALPHA - default=false)
  -h, --help                                 help for init
      --ignore-preflight-errors strings      A list of checks whose errors will be shown as warnings. Example: 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.
      --image-repository string              Choose a container registry to pull control plane images from (default "k8s.gcr.io")
      --kubernetes-version string            Choose a specific Kubernetes version for the control plane. (default "stable-1")
      --node-name string                     Specify the node name.
      --pod-network-cidr string              Specify range of IP addresses for the pod network. If set, the control plane will automatically allocate CIDRs for every node.
      --service-cidr string                  Use alternative range of IP address for service VIPs. (default "10.96.0.0/12")
      --service-dns-domain string            Use alternative domain for services, e.g. "myorg.internal". (default "cluster.local")
      --skip-certificate-key-print           Don‘t print the key used to encrypt the control-plane certificates.
      --skip-phases strings                  List of phases to be skipped
      --skip-token-print                     Skip printing of the default bootstrap token generated by 'kubeadm init'.
      --token string                         The token to use for establishing bidirectional trust between nodes and control-plane nodes. The format is [a-z0-9]{6}\.[a-z0-9]{16} - e.g. abcdef.0123456789abcdef
      --token-ttl duration                   The duration before the token is automatically deleted (e.g. 1s, 2m, 3h). If set to '0', the token will never expire (default 24h0m0s)
      --upload-certs                         Upload control-plane certificates to the kubeadm-certs Secret.

Global Flags:
      --add-dir-header           If true, adds the file directory to the header
      --log-file string          If non-empty, use this log file
      --log-file-max-size uint   Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
      --rootfs string            [EXPERIMENTAL] The path to the 'real' host root filesystem.
      --skip-headers             If true, avoid header prefixes in the log messages
      --skip-log-headers         If true, avoid headers when opening log files
  -v, --v Level                  number for the log level verbosity

Use "kubeadm init [command] --help" for more information about a command.

—— 尚硅谷k8s教程 学习笔记