k8s学习二：k8s集群搭建——单master多node简易部署
服务器环境
- centos7
- mac装的pd虚拟机
| 作用 | IP | 部署服务 | 配置 |
|---|---|---|---|
| master | 10.211.55.10 | etcd、kube-apiserver、kube-controller-manager、kube-scheduler | 2C、2G |
| node1 | 10.211.55.11 | docker 、kubelet、kube-proxy | 2C、2G |
| node2 | 10.211.55.12 | docker 、kubelet、kube-proxy | 2C、2G |
- 计划采用二进制包进行部署：
所需二进制包下载地址：
1.https://dl.k8s.io/v1.10.4/kubernetes-server-linux-amd64.tar.gz
2.https://dl.k8s.io/v1.10.4/kubernetes-node-linux-amd64.tar.gz
3.https://github.com/coreos/etcd/releases/download/v3.2.22/etcd-v3.2.22-linux-amd64.tar.gz
注意所有服务器都需要关闭防火墙
Master部署
二进制安装基本都是以下几个步骤：
1、复制对应的二进制文件到/usr/bin目录下
2、创建systemd service启动服务文件
3、创建service中对应的配置参数文件
4、将该应用加入到开机自启
5、启动服务并查看服务状态
etcd部署
- 下载二进制安装包并安装：
wget https://github.com/coreos/etcd/releases/download/v3.2.22/etcd-v3.2.22-linux-amd64.tar.gz
cd etcd-v3.2.22-linux-amd64/
cp etcd /usr/bin/
cp etcdctl /usr/bin/
mkdir /var/lib/etcd
mkdir /etc/etcd- 编辑systemd管理文件
vim /usr/lib/systemd/system/etcd.service[Unit]
Description=Etcd Server
After=network.target
[Service]
Type=simple
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
ExecStart=/usr/bin/etcd
[Install]
WantedBy=multi-user.target- 启动服务,并设置开机启动
systemctl daemon-reload
systemctl start etcd
systemctl enable etcd- 查看服务状态的三种命令
systemctl status etcd.service
curl -L http://127.0.0.1:2379/version
etcdctl cluster-health
这个安装的还挺顺利，很快就ok了。继续。。。。
kube-apiserver
- 下载并安装
wget https://dl.k8s.io/v1.10.4/kubernetes-server-linux-amd64.tar.gz
tar -xzvf kubernetes-server-linux-amd64.tar.gz
cd kubernetes/server/bin
cp kube-apiserver /usr/bin/
# 一起拷贝吧，后面就直接配置了
cp kube-controller-manager /usr/bin/
cp kube-scheduler /usr/bin/- 编辑systemd的启动文件
vim /usr/lib/systemd/system/kube-apiserver.service[Unit]
Description=Kubernetes API Server
Documentation=https://kubernetes.io/docs/concepts/overview
After=network.target
After=etcd.service
[Service]
EnvironmentFile=/etc/kubernetes/apiserver
ExecStart=/usr/bin/kube-apiserver $KUBE_API_ARGS
Restart=on-failure
Type=notify
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target- 配置参数文件
mkdir /etc/kubernetes/
vim /etc/kubernetes/apiserver KUBE_API_ARGS="--storage-backend=etcd3 \
--etcd-servers=http://127.0.0.1:2379 \
--bind-address=0.0.0.0 \
--secure-port=6443 \
--service-cluster-ip-range=192.168.2.0/16 \
--service-node-port-range=1-65535 \
--client-ca-file=/etc/kubernetes/ssl/ca.crt \
--tls-private-key-file=/etc/kubernetes/ssl/server.key \
--tls-cert-file=/etc/kubernetes/ssl/server.crt \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,DefaultStorageClass,ResourceQuota \
--logtostderr=false \
--log-dir=/var/log/kubernetes \
--v=2"service-cluster-ip-range是servcies的虚拟IP的IP范围，这里可以自己定义，不能当前的宿主机网段重叠。
bind-addres 指定的apiserver监听地址，对应的监听端口是6443，使用的https的方式。(0.0.0.0 表示绑定所有地址)
client-ca-file 这是认证的相关文件，这预先定义，后面会创建证书文件，并放置到对应的路径。
- 创建日志目录和证书目录
mkdir -p /etc/kubernetes/ssl
mkdir -p /var/log/kubernetekube-controller-manager
kube-controller-manager 依赖 kube-apiserver服务
- 编辑systemd启动文件
vim /usr/lib/systemd/system/kube-controller-manager.service [Unit]
Description=Kubernetes Controller Manager
Documentation=https://kubernetes.io/docs/setup
After=kube-apiserver.service
Requires=kube-apiserver.service
[Service]
EnvironmentFile=/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target- 配置启动参数
vim /etc/kubernetes/controller-manager KUBE_CONTROLLER_MANAGER_ARGS="--master=https://10.211.55.10:6443 \
--service-account-private-key-file=/etc/kubernetes/ssl/server.key \
--root-ca-file=/etc/kubernetes/ssl/ca.crt \
--kubeconfig=/etc/kubernetes/kubeconfig \
--logtostderr=false \
--log-dir=/var/log/kubernetes \
--v=2"kube-scheduler
kube-scheduler也依赖kubu-apiserver
- 编辑systemd启动文件
vim /usr/lib/systemd/system/kube-scheduler.service [Unit]
Description=Kubernetes Controller Manager
Documentation=https://kubernetes.io/docs/setup
After=kube-apiserver.service
Requires=kube-apiserver.service
[Service]
EnvironmentFile=/etc/kubernetes/scheduler
ExecStart=/usr/bin/kube-scheduler $KUBE_SCHEDULER_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target- 配置参数文件
vim /etc/kubernetes/scheduler KUBE_SCHEDULER_ARGS="--master=https://10.211.55.10:6443 --kubeconfig=/etc/kubernetes/kubeconfig \
--logtostderr=false \
--log-dir=/var/log/kubernetes \
--v=2"创建CA证书
注意生成证书前先同步一下服务器时间：ntpdate s2m.time.edu.cn
- 创建kube-apiserver的CA证书和私钥文件
cd /etc/kubernetes/ssl/
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=10.211.55.10" -days 5000 -out ca.crt
openssl genrsa -out server.key 2048- 创建master_ssl.cnf文件
vim master_ssl.cnf[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster.local
DNS.5 = k8s_master
IP.1 = 192.168.2.1 # ClusterIP 地址
IP.2 = 10.211.55.10 # master IP地址- 生成apiserver证书
openssl req -new -key server.key -subj "/CN=10.211.55.10" -config master_ssl.cnf -out server.csr
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 5000 -extensions v3_req -extfile master_ssl.cnf -out server.crt- 设置kube-controller-manager相关证书
openssl genrsa -out cs_client.key 2048
openssl req -new -key cs_client.key -subj "/CN=10.211.55.10" -out cs_client.csr
openssl x509 -req -in cs_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out cs_client.crt -days 5000- 创建kubeconfig文件,kube-controller-manager和kube-scheduler公用的配置文件
vim /etc/kubernetes/kubeconfig apiVersion: v1
kind: Config
users:
- name: controllermanager
user:
client-certificate: /etc/kubernetes/ssl/cs_client.crt
client-key: /etc/kubernetes/ssl/cs_client.key
clusters:
- name: local
cluster:
certificate-authority: /etc/kubernetes/ssl/ca.crt
contexts:
- context:
cluster: local
user: controllermanager
name: my-context
current-context: my-context启动服务
- 启动kube-apiserver
systemctl daemon-reload
systemctl enable kube-apiserver
systemctl start kube-apiserver
- 启动kube-controller-manager
systemctl enable kube-controller-manager
systemctl start kube-controller-manager- 启动kube-scheduler
systemctl enable kube-scheduler
systemctl start kube-schedulerNode
安装docker
- 使用aliyun的yum源
curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
curl -o /etc/yum.repos.d/docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum makecache
- yum安装docker工具
yum install docker-ce
systemctl start docker
systemctl enable docker
docker -v安装kubelet服务
- 安装包下载，整理
wget https://dl.k8s.io/v1.10.4/kubernetes-node-linux-amd64.tar.gz
tar -xzvf kubernetes-node-linux-amd64.tar.gz
cd kubernetes/node/bin
cp * /usr/bin
- 添加systemctl启动配置
vim /usr/lib/systemd/system/kubelet.service
mkdir -p /var/lib/kubelet
mkdir -p /etc/kubernetes/
mkdir -p /var/log/kubernetes[Unit]
Description=Kubelet Service
After=docker.service
Requires=docker.service
[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet $KUBELET_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target- kuberlet运行参数配置
安装kube-proxy服务
- 添加systemctl启动配置
vim /usr/lib/systemd/system/kube-proxy.service[Unit]
Description=K8s kube-proxy Service
After=network.target
After=docker.service
After=network.target
After=network.service
[Service]
EnvironmentFile=/etc/kubernetes/kube-proxy
ExecStart=/usr/bin/kube-proxy $KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target生成CA证书
- 将master节点上的kube-apiserver证书ca.crt和ca.key拷贝到Node上
- 使用ca.crt和ca.key生成node证书
openssl genrsa -out kubelet_client.key 2048
openssl req -new -key kubelet_client.key -subj "/CN=10.211.55.11" -out kubelet_client.csr
openssl x509 -req -in kubelet_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out kubelet_client.crt -days 5000
mkdir /etc/kubernetes/ssl
mv kubelet_client.* /etc/kubernetes/ssl/
mv ca.crt /etc/kubernetes/ssl/
- 配置kubeconfig
vim /etc/kubernetes/kubeconfigapiVersion: v1
kind: Config
users:
- name: kubelet
user:
client-certificate: /etc/kubernetes/ssl/kubelet_client.crt
client-key: /etc/kubernetes/ssl/kubelet_client.key
clusters:
- name: local
cluster:
certificate-authority: /etc/kubernetes/ssl/ca.crt
server: https://10.211.55.10:6443
contexts:
- context:
cluster: local
user: kubelet
name: my-context
current-context: my-context- kubelet启动参数配置
vim /etc/kubernetes/kubeletKUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --hostname-override=10.211.55.11 --logtostderr=false --log-dir=/var/log/kubernetes --v=2 --fail-swap-on=false"
这里要注意–fail-swap-on=false或者禁用swap，我这里选择配置–fail-swap-on=false
- 设置kube-proxy启动参数
vim /etc/kubernetes/kube-proxyKUBE_PROXY_ARGS="--master=https://10.211.55.10:6443 --kubeconfig=/etc/kubernetes/kubeconfig --logtostderr=false --log-dir=/var/log/kubernetes --v=2"
启动服务
systemctl daemon-reload
systemctl start kubelet.service
systemctl status kubelet.service
systemctl start kube-proxy
systemctl status kube-proxynode 2就按照上面的步骤进行安装即可