centos7 安装 K8S 及启动 POD问题
测试环境：
节点名称 节点IP 节点功能
K8s-master 10.10.1.10/24 Master、etcd、registry
K8s-node-1 10.10.1.20/24 node-1
K8s-node-2 10.10.1.30/24 node-2
步骤：
1、关闭 swap
#暂时关闭
swapoff -a
#永久关闭(注释掉最后带有swap的一行，前面加个#注释)
vi /etc/fstab
关闭防火墙
#暂时关闭
systemctl stop firewalld
#永久关闭
systemctl disable firewalld
关闭SELinux
sed -i ‘s/SELINUX=permissive/SELINUX=disabled/’ /etc/sysconfig/selinux
setenforce 0
2、设置 主机名
hostnamectl --static set-hostname k8s-master
3、设置host 解析地址（每台机器）
vi /etc/hosts
10.10.1.10 k8s-master
10.10.1.10 etcd
10.10.1.10 registry
10.10.1.20 k8s-node-1
10.10.1.30 k8s-node-2
4、部署etcd(注：本次只master节点安装etcd)
4.1、yum install etcd –y
4.2、vi /etc/etcd/etcd.conf
#[Member]
#ETCD_CORS=""
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_WAL_DIR=""
#ETCD_LISTEN_PEER_URLS=“http://localhost:2380”
ETCD_LISTEN_CLIENT_URLS=“http://0.0.0.0:2379,http://0.0.0.0:4001”
ETCD_NAME=“master”
#[Clustering]
#ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://localhost:2380”
ETCD_ADVERTISE_CLIENT_URLS=http://etcd:2379,http://etcd:4001
启动服务
systemctl start etcd.service
systemctl enable etcd.service
验证集群状态：
etcdctl -C http://etcd:4001 cluster-health
显示如下：
member 8e9e05c52164694d is healthy: got healthy result from http://etcd:2379
cluster is healthy
部署Master
1、安装Docker
yum install docker
vi /etc/sysconfig/docker
# /etc/sysconfig/docker
# Modify these options if you want to change the way the docker daemon runs
OPTIONS='--selinux-enabled --log-driver=journald --signature-verification=false'
if [ -z "${DOCKER_CERT_PATH}" ]; then
DOCKER_CERT_PATH=/etc/docker
fi
OPTIONS='--insecure-registry registry:5000'
2.启动docker服务并设置开机启动
systemctl start docker.service
systemctl enable docker.service
3、安装安裝kubernets
yum install kubernetes
4、 配置并且启动kubernets服务(该步骤只在master节点)
Kubernets API Server Kubernets Controller Manager Kubernets Scheduler
vi /etc/kubernetes/apiserver
###
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
#
# The address on the local server to listen to.
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
# The port on the local server to listen on.
KUBE_API_PORT="--port=8080"
# Port minions listen on
# KUBELET_PORT="--kubelet-port=10250"
# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://etcd:2379"
# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
# default admission control policies
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"
# Add your own!
KUBE_API_ARGS=""
vi /etc/kubernetes/config
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://k8s-master:8080"
- 启动服务并设置开机启动
[root@k8s-master ~]# systemctl enable kube-apiserver.service
[root@k8s-master ~]# systemctl start kube-apiserver.service
[root@k8s-master ~]# systemctl enable kube-controller-manager.service
[root@k8s-master ~]# systemctl start kube-controller-manager.service
[root@k8s-master ~]# systemctl enable kube-scheduler.service
[root@k8s-master ~]# systemctl start kube-scheduler.service
部署节点：
1、安装并启动 docker
2、安裝kubernets
配置：vi /etc/kubernetes/config
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://k8s-master:8080"
配置：vi /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=0.0.0.0"
KUBELET_HOSTNAME="--hostname-override=k8s-node-1"
KUBELET_API_SERVER="--api-servers=http://etcd:8080"
3.启动服务并设置开机启动
[root@k8s-node-1 ~]# systemctl enable kubelet.service
[root@k8s-node-1 ~]# systemctl start kubelet.service
[root@k8s-node-1 ~]# systemctl enable kube-proxy.service
[root@k8s-node-1~]# systemctl start kube-proxy.service
查看集群状态
kubectl get node
输出结果：
NAME STATUS AGE
k8s-node-1 Ready 14h
k8s-node-2 Ready 14h
安装Flannel(所有节点)(作用：使所有虚拟IP统一，互通）
yum install flannel
修改配置：vi /etc/sysconfig/flanneld
# Flanneld configuration options
# etcd url location. Point this to the server where etcd runs
FLANNEL_ETCD_ENDPOINTS="http://etcd:2379"
配置etcd中关于flannel的key (mast主机中）
etcdctl mk /atomic.io/network/config '{ "Network": "10.10.10.0/16" }'
如果内部不通，执行下面修改语句
etcdctl set /atomic.io/network/config '{"Network":"10.10.10.0/16","SubnetLen":25,"Backend":{"Type":"vxlan","VNI":1}}'
启动：
systemctl enable flanneld.service
systemctl start flanneld.service
管理节点执行：
service docker restart
systemctl restart kube-apiserver.service
systemctl restart kube-controller-manager.service
systemctl restart kube-scheduler.service
业务节点执行
service docker restart
systemctl restart kubelet.service
systemctl restart kube-proxy.service
实例运转操作方式：
1、业务节点拉取image
docker pull nginx
2、管理节点创建实例
kubectl run nginx --image=nginx --port=80
3、管理节点做svc映射
kubectl expose deployment nginx --port=80 --target-port=80 --external-ip=192.168.1.81
常用命令：
[root@node-1 ~]# kubectl get node -o wide
NAME STATUS AGE EXTERNAL-IP
k8s-node-1 Ready 14h
k8s-node-2 Ready 14h
[root@node-1 ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE
app-556711052-ps9kr 1/1 Running 3 7h 10.0.53.2 k8s-node-1
tomcat-3343039334-0z187 1/1 Running 0 2h 10.0.74.3 k8s-node-2
web-3818241055-g11q8 1/1 Running 3 8h 10.0.74.2 k8s-node-2
[root@node-1 ~]# kubectl get svc -o wide
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes 10.254.0.1 443/TCP 15h
tomcat 10.254.69.86 10.10.1.30 7777/TCP 2h run=tomcat
web 10.254.76.251 10.10.1.30 80/TCP 6h run=web
[root@node-1 ~]# kubectl get deployments
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
app 1 1 1 1 7h
tomcat 1 1 1 1 2h
web 1 1 1 1 8h
关于外网无法访问：
由于linux还有底层的iptables，所以在node上分别执行：
[root@Node-2 ~]# vi /etc/sysctl.conf
net.ipv4.ip_forward = 1
[root@Node-2 ~]# sysctl -p
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -F
iptables -L -n
iptables -P FORWARD ACCEPT 如果不通 ，这个再执行几次，因为有缓存问题
K8S启动POD问题
mast主机
使用命令 kubectl get pods输出no resources。解决方法是修改 apiserver 的配置文件
vim /etc/kubernetes/apiserver
...
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"
去掉原先的ServiceAccount。然后重启apiserver
systemctl restart kube-apiserver.service
节点
启动pod是，一直处于ContainerCreating状态，需要安装
yum install *rhsm*
查看docker日志，docker一直再拉取registry.access.redhat.com/rhel7/pod-infrastructure，且提示不存在，所以修改 kubelet 配置文件
vim /etc/kubernetes/kubelet
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=docker.io/tianyebj/pod-infrastructure"
然后重启kubelet
systemctl restart kubelet