Kubernetes安装系列之Master-Apiserver安装

这篇文章整理以下Master节点的apiserver的安装与设定方法，本文以脚本的方式进行固化，内容仍然放在github的easypack上。

整体操作

• https://blog.csdn.net/liumiaocn/article/details/88413428

ApiServer设定文件

KUBE_APISERVER_OPTS="--logtostderr=true \
--v=4 \
--log-dir=/var/log/kubernetes \
--etcd-servers=https://192.168.163.131:2379 \
--bind-address=192.168.163.131 \
--secure-port=6443 \
--advertise-address=192.168.163.131 \
--allow-privileged=true \
--service-cluster-ip-range=172.200.0.0/16 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--enable-bootstrap-token-auth \
--token-auth-file=/etc/k8s/token.csv \
--service-node-port-range=30000-50000 \
--tls-cert-file=/etc/ssl/k8s/cert-k8s.pem  \
--tls-private-key-file=/etc/ssl/k8s/cert-k8s-key.pem \
--client-ca-file=/etc/ssl/ca/ca.pem \
--service-account-key-file=/etc/ssl/ca/ca-key.pem \
--etcd-cafile=/etc/ssl/ca/ca.pem \
--etcd-certfile=/etc/ssl/etcd/cert-etcd.pem \
--etcd-keyfile=//etc/ssl/etcd/cert-etcd-key.pem"

Systemd服务设定文件

[root@host131 shell]# cat /usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=etcd.service
Wants=etcd.service

[Service]
EnvironmentFile=-/etc/k8s/apiserver.conf
ExecStart=/usr/local/bin/kube-apiserver $KUBE_APISERVER_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
[root@host131 shell]#

脚本示例

[root@host131 shell]# cat step3-install-apiserver.sh 
#!/bin/sh

. ./install.cfg

echo -e "\n##  kube-apiserver service"
systemctl stop kube-apiserver 2>/dev/null

mkdir -p ${ENV_KUBE_DIR_BIN} ${ENV_KUBE_DIR_ETC}
chmod 755 ${ENV_HOME_K8S}/*
cp -p ${ENV_HOME_K8S}/{kubectl,kube-apiserver} ${ENV_KUBE_DIR_BIN}
if [ $? -ne 0 ]; then
  echo "please check kubectl and kube-apiserver binary files existed in ${ENV_HOME_K8S}/ or not"
  exit 
fi

# create kube token file
cat >${ENV_KUBE_DIR_ETC}/${ENV_KUBE_API_TOKEN} <<EOF
$(head -c 16 /dev/urandom | od -An -t x | tr -d ' '),kubelet-bootstrap,10001,"system:kubelet-bootstrap"
EOF

# create kube-apiserver configuration file
cat >${ENV_KUBE_DIR_ETC}/${ENV_KUBE_API_CONF} <<EOF
KUBE_APISERVER_OPTS="--logtostderr=${ENV_KUBE_OPT_LOGTOSTDERR} \
--v=${ENV_KUBE_OPT_LOG_LEVEL} \
--log-dir=${ENV_KUBE_OPT_LOG_DIR} \\
EOF

echo ${ENV_ETCD_HOSTS} |awk -v etcd_names="${ENV_ETCD_NAMES}" \
-v port=${ENV_ETCD_CLIENT_PORT} -F" " 'BEGIN{
    printf("--etcd-servers=");
}
{
    for(cnt=1; cnt$cnt,port);
    }
    printf("https://%s:%s ",$cnt,port);
}' >>${ENV_KUBE_DIR_ETC}/${ENV_KUBE_API_CONF}

cat >>${ENV_KUBE_DIR_ETC}/${ENV_KUBE_API_CONF} <<EOF
--bind-address=${ENV_CURRENT_HOSTIP} \
--secure-port=${ENV_KUBE_OPT_API_SSL_PORT} \
--advertise-address=${ENV_CURRENT_HOSTIP} \
--allow-privileged=${ENV_KUBE_OPT_ALLOW_PRIVILEGE} \
--service-cluster-ip-range=${ENV_KUBE_OPT_CLUSTER_IP_RANGE} \
--enable-admission-plugins=${ENV_KUBE_ADM_PLUGINS} \
--authorization-mode=${ENV_KUBE_OPT_AUTH_MODE} \
--enable-bootstrap-token-auth \
--token-auth-file=${ENV_KUBE_DIR_ETC}/${ENV_KUBE_API_TOKEN} \
--service-node-port-range=${ENV_KUBE_OPT_CLUSTER_PORT_RANGE} \
--tls-cert-file=${ENV_SSL_K8S_DIR}/${ENV_SSL_K8S_CERT_PRIFIX}.pem  \
--tls-private-key-file=${ENV_SSL_K8S_DIR}/${ENV_SSL_K8S_CERT_PRIFIX}-key.pem \
--client-ca-file=${ENV_SSL_CA_DIR}/${ENV_SSL_FILE_CA_PEM} \
--service-account-key-file=${ENV_SSL_CA_DIR}/${ENV_SSL_FILE_CA_KEY} \
--etcd-cafile=${ENV_SSL_CA_DIR}/${ENV_SSL_FILE_CA_PEM} \
--etcd-certfile=${ENV_SSL_ETCD_DIR}/${ENV_SSL_ETCD_CERT_PRIFIX}.pem \
--etcd-keyfile=/${ENV_SSL_ETCD_DIR}/${ENV_SSL_ETCD_CERT_PRIFIX}-key.pem"
EOF

# Create the kube-apiserver service.
cat >${ENV_KUBE_API_SERVICE} <<EOF
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=etcd.service
Wants=etcd.service

[Service]
EnvironmentFile=-${ENV_KUBE_DIR_ETC}/${ENV_KUBE_API_CONF}
ExecStart=${ENV_KUBE_DIR_BIN}/kube-apiserver \$KUBE_APISERVER_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

echo -e "\n##  daemon reload service "
systemctl daemon-reload
echo -e "\n##  start kube-apiserver service "
systemctl start kube-apiserver
echo -e "\n##  enable kube-apiserver service " 
systemctl enable kube-apiserver
echo -e "\n##  check  kube-apiserver status"
systemctl status kube-apiserver

echo -e "\n##  kubectl version"
kubectl version

echo -e "\n##  get cs"
kubectl get cs
[root@host131 shell]#

执行示例

[root@host131 shell]# sh step3-install-apiserver.sh 

##  kube-apiserver service

##  daemon reload service 

##  start kube-apiserver service 

##  enable kube-apiserver service 

##  check  kube-apiserver status
● kube-apiserver.service - Kubernetes API Server
   Loaded: loaded (/usr/lib/systemd/system/kube-apiserver.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2019-03-23 15:33:17 CST; 287ms ago
     Docs: https://github.com/kubernetes/kubernetes
 Main PID: 10007 (kube-apiserver)
   CGroup: /system.slice/kube-apiserver.service
           └─10007 /usr/local/bin/kube-apiserver --logtostderr=true --v=4 --log-dir=/var/log/kubernetes --etcd-servers=https://192.168.163.131...

Mar 23 15:33:18 host131 kube-apiserver[10007]: I0323 15:33:18.173826   10007 flags.go:33] FLAG: --authentication-token-webhook-config-file=""
Mar 23 15:33:18 host131 kube-apiserver[10007]: I0323 15:33:18.173830   10007 flags.go:33] FLAG: --authorization-mode="[RBAC,Node]"
Mar 23 15:33:18 host131 kube-apiserver[10007]: I0323 15:33:18.173836   10007 flags.go:33] FLAG: --authorization-policy-file=""
Mar 23 15:33:18 host131 kube-apiserver[10007]: I0323 15:33:18.173841   10007 flags.go:33] FLAG: --authorization-webhook-cache-authorize...="5m0s"
Mar 23 15:33:18 host131 kube-apiserver[10007]: I0323 15:33:18.173845   10007 flags.go:33] FLAG: --authorization-webhook-cache-unauthori...l="30s"
Mar 23 15:33:18 host131 kube-apiserver[10007]: I0323 15:33:18.173849   10007 flags.go:33] FLAG: --authorization-webhook-config-file=""
Mar 23 15:33:18 host131 kube-apiserver[10007]: I0323 15:33:18.173853   10007 flags.go:33] FLAG: --basic-auth-file=""
Mar 23 15:33:18 host131 kube-apiserver[10007]: I0323 15:33:18.173857   10007 flags.go:33] FLAG: --bind-address="192.168.163.131"
Mar 23 15:33:18 host131 kube-apiserver[10007]: I0323 15:33:18.173862   10007 flags.go:33] FLAG: --cert-dir="/var/run/kubernetes"
Mar 23 15:33:18 host131 kube-apiserver[10007]: I0323 15:33:18.173867   10007 flags.go:33] FLAG: --client-ca-file="/etc/ssl/ca/ca.pem"
Hint: Some lines were ellipsized, use -l to show in full.

##  kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-02-28T13:37:52Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-02-28T13:30:26Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

##  get cs
NAME                 STATUS      MESSAGE                                                                                     ERROR
scheduler            Unhealthy   Get http://127.0.0.1:10251/healthz: dial tcp 127.0.0.1:10251: connect: connection refused   
controller-manager   Unhealthy   Get http://127.0.0.1:10252/healthz: dial tcp 127.0.0.1:10252: connect: connection refused   
etcd-0               Healthy     {"health":"true"}                                                                           
[root@host131 shell]#

可以看到，在ETCD的基础之上，apiserver正常启动，而通过kubectl get cs也能确认到etcd服务的正常。另外scheduler和controller-manager安装部署之后也会正常显示。