云计算之docker-k8s-第一篇：ETCD安装配置
- 概述
- 安装环境准备：
K8s安装包下载
https://pan.baidu.com/s/1IiJwMYpZSNXYPGtPVx_eHg 提取码: 6e3b
部署节点：
192.168.1.2 node1 4C 4G 50G 【master】
192.168.1.3 node2 4C 4G 50G 【node】
192.168.1.4 node3 4C 4G 50G 【node】
2. 架构图：
Kubernets架构图
Flannel网络架构图
Kubernetes工作流程
集群功能各模块功能描述：
Master节点：
APIServer: APIServer负责对外提供RESTful的kubernetes API的服务，它是系统管理指令的统一接口，任何对资源的增删该查都要交给APIServer处理后再交给etcd，kubectl是直接和APIServer交互的。
schedule: schedule负责调度Pod到合适的Node上，如果把scheduler看成一个黑匣子，那么它的输入是pod和由多个Node组成的列表，输出是Pod和一个Node的绑定。 kubernetes目前提供了调度算法，同样也保留了接口。用户根据自己的需求定义自己的调度算法。
controller manager: 如果APIServer做的是前台的工作的话，那么controller manager就是负责后台的。每一个资源都对应一个控制器。而control manager就是负责管理这些控制器的，比如我们通过APIServer创建了一个Pod，当这个Pod创建成功后，APIServer的任务就算完成了。
etcd：etcd是一个高可用的键值存储系统，kubernetes使用它来存储各个资源的状态，从而实现了Restful的API。
Node节点：
kube-proxy: 实现了kubernetes中的服务发现和反向代理功能。kube-proxy支持TCP和UDP连接转发，默认基RR算法将客户端流量转发到与service对应的一组后端pod。服务发现方面，kube-proxy使用etcd的watch机制监控集群中service和endpoint对象数据的动态变化，维护一个service到endpoint的映射关系，保证了后端pod的IP变化不会对访问者造成影响。
kublet：kublet是Master在每个Node节点上面的agent，负责维护和管理该Node上的所有容器，【如果容器不是通过kubernetes创建的，它并不会管理】。本质上，它负责使Pod的运行状态与期望的状态一致
- Kubernetes安装及配置
- 初始化环境
- 设置关闭防火墙及selinux
- 关闭swap
- 设置docker所需参数
vim /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
sysctl –p /etc/sysctl.d/k8s.conf
【如果报错：cannot stat /proc/sys/net/bridge/bridge-nf-call-ip6tables: 没有那个文件或目录】
解决方案：
modprobe br_netfilter
cat > /etc/rc.sysinit << EOF
#!/bin/bash
for file in /etc/sysconfig/modules/*.modules ; do
[ -x $file ] && $file
done
EOF
cat > /etc/sysconfig/modules/br_netfilter.modules << EOF
modprobe br_netfilter
EOF
chmod 755 /etc/sysconfig/modules/br_netfilter.modules
lsmod |grep br_netfilter
[注释：二层网桥在转发数据的时候也会手iptables规则过滤]
- 安装docker
下载docker-engine docker-engine-selinx两个rpm包,并自定义yum源
]# yum -y install docker-engine
]# systemctl restart docker && systemctl enable docker
- 创建安装目录
]# mkdir /k8s/etcd/{bin,cfg,ssl} –p
]# mkdir /k8s/kubernetes/{bin,cfg,ssl} –p
- 安装及配置CFSSL [证书生成工具]
]# wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
]# wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
]# wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
]# chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64
]# mv cfssl_linux-amd64 /usr/local/bin/cfssl
]# mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
]# mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo
- 创建认证证书
创建ETCD证书
]# cat < > { > "signing": { > "default": { > "expiry": "87600h" > }, > "profiles": { > "www": { > "expiry": "87600h", > "usages": [ > "signing", > "key encipherment", > "server auth", > "client auth" > ] > } > } > } > } > EOF 创建ETCD CA 配置文件 ]# cat < > { > "CN": "etcd CA", > "key": { > "algo": "rsa", > "size": 2048 > }, > "names": [ > { > "C": "CN", > "L": "Shenzhen", > "ST": "Shenzhen" > } > ] > } > EOF 创建ETCD Server证书 ]# cat << EOF | tee server-csr.json > { > "CN": "etcd", > "hosts": [ > "192.168.1.2", > "192.168.1.3", > "192.168.1.4" > ], > "key": { > "algo": "rsa", > "size": 2048 > }, > "names": [ > { > "C": "CN", > "L": "Shenzhen", > "ST": "Shenzhen" > } > ] > } > EOF 创建ETCD CA证书和私钥 ]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca – ]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server 创建kubernetes CA证书 ]# cat << EOF | tee ca-config.json > { > "signing": { > "default": { > "expiry": "87600h" > }, > "profiles": { > "kubernetes": { > "expiry": "87600h", > "usages": [ > "signing", > "key encipherment", > "server auth", > "client auth" > ] > } > } > } > } > EOF ]# cat << EOF | tee ca-csr.json > { > "CN": "kubernetes", > "key": { > "algo": "rsa", > "size": 2048 > }, > "names": [ > { > "C": "CN", > "L": "Shenzhen", > "ST": "Shenzhen", > "O": "k8s", > "OU": "System" > } > ] > } > EOF ]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca – 生成API_SERVER证书 ]# cat << EOF | tee server-csr.json > { > "CN": "kubernetes", > "hosts": [ > "10.0.0.1", > "127.0.0.1", > "192.168.1.2", > "kubernetes", > "kubernetes.default", > "kubernetes.default.svc", > "kubernetes.default.svc.cluster", > "kubernetes.default.svc.cluster.local" > ], > "key": { > "algo": "rsa", > "size": 2048 > }, > "names": [ > { > "C": "CN", > "L": "Shenzhen", > "ST": "Shenzhen", > "O": "k8s", > "OU": "System" > } > ] > } > EOF ]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server 创建kubernetes proxy证书 ]# cat << EOF | tee kube-proxy-csr.json > { > "CN": "system:kube-proxy", > "hosts": [], > "key": { > "algo": "rsa", > "size": 2048 > }, > "names": [ > { > "C": "CN", > "L": "Shenzhen", > "ST": "Shenzhen", > "O": "k8s", > "OU": "System" > } > ] > } > EOF ]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy ssh-keygen ssh-copy-id 192.168.1.3 ssh-copy-id 192.168.1.4 2、部署ETCD 解压安装文件 ]# tar -xf etcd-v3.3.10-linux-amd64.tar.gz ]# cd etcd-v3.3.10-linux-amd64/ ]# cp etcd etcdctl /k8s/etcd/bin/ ]# vim /k8s/etcd/cfg/etcd #[Member] ETCD_NAME="etcd01" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="https://192.168.1.2:2380" ETCD_LISTEN_CLIENT_URLS="https://192.168.1.2:2379" #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.1.2:2380" ETCD_ADVERTISE_CLIENT_URLS="https://192.168.1.2:2379" ETCD_INITIAL_CLUSTER="etcd01=https://192.168.1.2:2380,etcd02=https://192.168.1.3:2380,etcd03=https://192.168.1.4:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" 创建etcd的systemd unit 文件 ]# vim /usr/lib/systemd/system/etcd.service [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify EnvironmentFile=/k8s/etcd/cfg/etcd ExecStart=/k8s/etcd/bin/etcd \ --name=${ETCD_NAME} \ --data-dir=${ETCD_DATA_DIR} \ --listen-peer-urls=${ETCD_LISTEN_PEER_URLS} \ --listen-client-urls=${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \ --advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} \ --initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} \ --initial-cluster=${ETCD_INITIAL_CLUSTER} \ --initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} \ --initial-cluster-state=new \ --cert-file=/k8s/etcd/ssl/server.pem \ --key-file=/k8s/etcd/ssl/server-key.pem \ --peer-cert-file=/k8s/etcd/ssl/server.pem \ --peer-key-file=/k8s/etcd/ssl/server-key.pem \ --trusted-ca-file=/k8s/etcd/ssl/ca.pem \ --peer-trusted-ca-file=/k8s/etcd/ssl/ca.pem Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target 拷贝证书文件 ]# cp ca*pem server*pem /k8s/etcd/ssl/ 启动ETCD服务 ]# systemctl daemon-reload ]# systemctl enable etcd ]# systemctl start etcd