Kubernetes etcd calico self-hosted deployment note
文章目录
- 序言
- calico
- Calico CNI的配置
- 附录：离线安装
- git
- gcc
- docker
- kubenetes yum源
- Kubernetes setup
- 获取Kubernetes镜像
- 获取Calico Addon镜像
序言
网络太慢了，所以这篇博客我希望记录完全离线安装kubernetes的细节。
- 版本固定
- 稳定可靠的环境安装和清除
- 形成自动化脚本
| env | version |
|---|---|
| centos | 7 |
| docker | 19.03 |
| kubernetes | v18.0 |
| calico | – |
calico
Calico 遵从Kubernetes网络模型，以CNI(Container Network Interface)插件的形式安装在Kubernetes上。
参考：install calico on kubernetes
Calico CNI的配置
IPIP模式和BGP模式。
IPIP模式就是主机IP报文包装Pod IP报文。BGP则直接路由转发，每个主机作为路由器。
要在Kubernetes上安装CNI插件，下载对应插件的yaml定义。Calico插件的YAML定义称为manifest,在calico networking manifest里面可以配置网络模式（默认是IPIP网络）：
# Enable IPIP
- name: CALICO_IPV4POOL_IPIP
value: "Always" # 这里改成off,使用BGP网络模式
关于Calico网络模式：我看到有个博客讲的挺清楚的：k8s网络之Calico网络
附录：离线安装
git
version: 1.8.3
# git
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/git-1.8.3.1-21.el7_7.x86_64.rpm
# rsync
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/rsync-3.1.2-10.el7.x86_64.rpm
# perl(Error)
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/perl-Error-0.17020-2.el7.noarch.rpm
# perl-Git
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/perl-Git-1.8.3.1-21.el7_7.noarch.rpm
# perl(Term::ReadKey)
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/perl-TermReadKey-2.30-20.el7.x86_64.rpm
安装顺序：
rpm -ivh rsync-3.1.2-10.el7.x86_64.rpm
rpm -ivh perl-Error-0.17020-2.el7.noarch.rpm
rpm -ivh perl-TermReadKey-2.30-20.el7.x86_64.rpm
rpm -ivh perl-Git-1.8.3.1-21.el7_7.noarch.rpm git-1.8.3.1-21.el7_7.x86_64.rpm
gcc
rpm离线下载地址
# gcc-c++
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/gcc-c++-4.8.5-39.el7.x86_64.rpm
# gcc 包及依赖
# gcc
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/gcc-4.8.5-39.el7.x86_64.rpm
# cpp
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/cpp-4.8.5-39.el7.x86_64.rpm
# glibc-devel >= 2.2.90-12
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/glibc-devel-2.17-307.el7.1.x86_64.rpm
# glibc = 2.17-307.el7.1
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/glibc-2.17-307.el7.1.x86_64.rpm
# glibc-common = 2.17-307.el7.1
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/glibc-common-2.17-307.el7.1.x86_64.rpm
# glibc-headers
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/glibc-headers-2.17-307.el7.1.x86_64.rpm
# kernel-headers
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/kernel-headers-3.10.0-1127.el7.x86_64.rpm
# glibc-headers = 2.17-307.el7.1
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/glibc-headers-2.17-307.el7.1.x86_64.rpm
# libgcc >= 4.8.5-39.el7
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/libgcc-4.8.5-39.el7.x86_64.rpm
# libgomp = 4.8.5-39.el7
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/libgomp-4.8.5-39.el7.x86_64.rpm
# 可独立可装包
# libmpc.so.3()(64bit)
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/libmpc-1.0.1-3.el7.x86_64.rpm
# libmpfr.so.4()(64bit）
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/mpfr-3.1.1-4.el7.x86_64.rpm
# libstdc++ = 4.8.5-39.el7
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/libstdc++-4.8.5-39.el7.x86_64.rpm
# libstdc++-devel = 4.8.5-39.el7
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/libstdc++-devel-4.8.5-39.el7.x86_64.rpm
安装顺序：
# libmpfr.so.4()(64bit）
rpm -ivh mpfr-3.1.1-4.el7.x86_64.rpm
# libmpc.so.3()(64bit)
rpm -ivh libmpc-1.0.1-3.el7.x86_64.rpm
# libstdc++ = 4.8.5-39.el7
rpm -ivhU libstdc++-4.8.5-39.el7.x86_64.rpm
# libstdc++-devel = 4.8.5-39.el7
rpm -ivhU libstdc++-devel-4.8.5-39.el7.x86_64.rpm
# cpp
rpm -ivh cpp-4.8.5-39.el7.x86_64.rpm
######### glibc-devel
rpm -iUvh glibc-2.17-307.el7.1.x86_64.rpm glibc-common-2.17-307.el7.1.x86_64.rpm
# kernel-headers
rpm -ivh kernel-headers-3.10.0-1127.el7.x86_64.rpm
# glibc-headers
rpm -ivh glibc-headers-2.17-307.el7.1.x86_64.rpm
# glibc-devel
rpm -ivh glibc-devel-2.17-307.el7.1.x86_64.rpm
########## libgcc >= 4.8.5-39.el7
rpm -iUvh libgcc-4.8.5-39.el7.x86_64.rpm
########## libgomp = 4.8.5-39.el7
rpm -iUvh libgomp-4.8.5-39.el7.x86_64.rpm
########## gcc
rpm -iUvh gcc-4.8.5-39.el7.x86_64.rpm
rpm -iUvh gcc-c++-4.8.5-39.el7.x86_64.rpm
docker
# docker
wget -c https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-19.03.9-3.el7.x86_64.rpm
# docker cli
wget -c https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-cli-19.03.9-3.el7.x86_64.rpm
# containerd.io
wget -c https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.4-3.1.el7.x86_64.rpm
# libcgroup
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/libcgroup-0.41-21.el7.x86_64.rpm
# container-selinux
wget -c http://mirror.centos.org/centos/7/extras/x86_64/Packages/container-selinux-2.107-3.el7.noarch.rpm
# policycoreutils-python
wget -c http://mirror.centos.org/centos/7/os/x86_64/Packages/policycoreutils-python-2.5-34.el7.x86_64.rpm
yum install --downloadonly --downloaddir=/root/rpm policycoreutils-python-2.5-34.el7.x86_64
安装顺序：
rpm -ivh libcgroup-0.41-21.el7.x86_64.rpm
rpm -iUvh audit-libs-2.8.5-4.el7.x86_64.rpm audit-2.8.5-4.el7.x86_64.rpm
rpm -iUvh policycoreutils-2.5-34.el7.x86_64.rpm
rpm -iUvh python-IPy-0.75-6.el7.noarch.rpm
rpm -iUvh audit-libs-python-2.8.5-4.el7.x86_64.rpm checkpolicy-2.5-8.el7.x86_64.rpm libsemanage-python-2.5-14.el7.x86_64.rpm policycoreutils-python-2.5-34.el7.x86_64.rpm setools-libs-3.3.8-4.el7.x86_64.rpm
rpm -ivh policycoreutils-2.5-34.el7.x86_64.rpm
rpm -ivh docker-ce-cli-19.03.9-3.el7.x86_64.rpm
rpm -iUvh containerd.io-1.2.4-3.1.el7.x86_64.rpm
rpm -iUvh container-selinux-2.107-3.el7.noarch.rpm
rpm -iUvh docker-ce-19.03.9-3.el7.x86_64.rpm
kubenetes yum源
这个部分是从阿里云上拉取kubernetes的镜像。
将下面内容保存至kuber.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
复制到yum repo库：
cp -b conf/kuber.repo /etc/yum.repos.d/kubernetes.repo
yum install -y --downloadonly --downloaddir=. kubelet kubeadm kubectl --disableexcludes=kubernetes
安装顺序：
rpm -ivh socat-1.7.3.2-2.el7.x86_64.rpm
rpm -ivh libnetfilter_cthelper-1.0.0-11.el7.x86_64.rpm libnetfilter_cttimeout-1.0.0-7.el7.x86_64.rpm libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm
rpm -ivh conntrack-tools-1.4.4-7.el7.x86_64.rpm
rpm -ivh 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm
rpm -iUvh 029bc0d7b2112098bdfa3f4621f2ce325d7a2c336f98fa80395a3a112ab2a713-kubernetes-cni-0.8.6-0.x86_64.rpm de422b616a367cafae90aef704625fc34b0b222353f4fb59235bb3cf2f9d0988-kubelet-1.18.8-0.x86_64.rpm efd73a4178ebf9939f86b4200dba0247a57ead65f2403d8576b241faf478ac42-kubectl-1.18.8-0.x86_64.rpm f1795288ba09abd5e6738fbf82e237e3792ee2dcda4512fa5d35388dae622cef-kubeadm-1.18.8-0.x86_64.rpm
Kubernetes setup
参考：Creating a cluster with kubeadm 使用kubeadm搭建kubernetes环境。
获取Kubernetes镜像
# 拉取镜像
kubeadm config images pull --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.18.0
输出：
W0815 07:33:13.305739 20742 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.18.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.18.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.18.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.18.0
[config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.2
[config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.4.3-0
[config/images] Pulled registry.aliyuncs.com/google_containers/coredns:1.6.7
这些镜像是要注意离线保存的。
获取Calico Addon镜像
参考：
https://docs.projectcalico.org/getting-started/kubernetes/self-managed-onprem/onpremises
选择etcd方式保存路由信息：
https://docs.projectcalico.org/manifests/calico-etcd.yaml
docker pull calico/pod2daemon-flexvol:v3.15.1
docker pull calico/node:v3.15.1
docker pull calico/kube-controllers:v3.15.1
docker pull calico/cni:v3.15.1