Kubernetes 的部署记录

操作系统均为CENTOS7

1、在master机器上配置如下：

[root@c780 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
193.168.120.80 c780             #master                   
193.168.120.81 c781            #node1
193.168.120.83 c783            #node2
193.168.120.85 c785            #node3

 

[root@c780 ~]# cat /etc/yum.repos.d/virt7-docker-common-release.repo
[virt7-docker-common-release]
name=virt7-docker-common-release
baseurl=http://cbs.centos.org/repos/virt7-docker-common-release/x86_64/os/
gpgcheck=0

安装K8S相关软件：

 yum repolist

yum -y install –enablerepo=virt7-docker-common-release kubernetes --skip-broke

master也可以安装docker

yum install docker-ce-18.03.1.ce -y

配置相关的软件：

[root@c780 kubernetes]# cat /etc/kubernetes/config | grep -v '^#'
KUBE_LOGTOSTDERR="--logtostderr=true"

KUBE_LOG_LEVEL="--v=0"

KUBE_ALLOW_PRIV="--allow-privileged=false"

KUBE_MASTER="--master=http://193.168.120.80:8080"

 

etcd服务的配置，这个文件一般不用改：

[root@c780 kubernetes]# cat /etc/etcd/etcd.conf | grep -v '^#'
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_NAME="default"
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"

 

配置APISERVER

[root@c780 kubernetes]# cat /etc/kubernetes/apiserver | grep -v '^#'

KUBE_API_ADDRESS="--address=0.0.0.0"

KUBE_API_PORT="--port=8080"

KUBELET_PORT="--kubelet-port=10250"

KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:2379"

KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"

KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"

KUBE_API_ARGS=""

 

配置etcd分配给NODES的网段：

systemctl start etcd

etcdctl mkdir /kube-centos/network

etcdctl mk /kube-centos/network/config "{ \"Network\": \"172.30.0.0/16\", \"SubnetLen\": 24, \"Backend\": { \"Type\": \"vxlan\" } }"

 

配置flanneld服务

[root@c780 ~]# cat /etc/sysconfig/flanneld | grep -v '^#'

FLANNEL_ETCD_ENDPOINTS="http://193.168.120.80:2379"

FLANNEL_ETCD_PREFIX="/kube-centos/network"
 

 

最好先关闭防火墙等：

systemctl stop firewalld
systemctl disable firewalld

setenforce 0

[root@c780 ~]# cat /etc/sysconfig/selinux 

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 

 

启动群集：

[root@c780 ~]# cat /shell/k8s/1.sh 
#!/bin/bash
for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler flanneld; do
    systemctl restart $SERVICES
    systemctl enable $SERVICES
    systemctl status $SERVICES
done

[root@c780 ~]# sh 1.sh

 

为方便配置，在master机器安装ansible

yum install ansible -y

[root@c780 ~]# cat /etc/ansible/hosts | grep -v '^#'

[cli1]
193.168.120.80
193.168.120.81
193.168.120.83
193.168.120.85
[cli1:vars]
ansible_ssh_user=root 
ansible_ssh_pass=yourpassword
ansible_ssh_port=22

 

拷贝公钥到NODE节点的root宿主目录下：

ansible clic6 -m  authorized_key  -a "user=root key='{{ lookup('file', '/root/.ssh/id_rsa.pub') }}'"

拷贝hosts等文件到NODE节点：

ansible cli1 -m copy -a "src=/etc/hosts backup=yes dest=/etc/hosts"

ansible cli1 -m copy -a "src=/etc/yum.repos.d/virt7-docker-common-release.repo dest=/etc/yum.repos.d/virt7-docker-common-release.repo"

同步时间：

ansible cli1 -m command -a "ntpdate 193.168.120.60"

 

 

NODES节点上的配置：

安装：

yum repolist
yum -y install –enablerepo=virt7-docker-common-releasekubernetes flannel

 

配置 /etc/kubernetes/kubelet：

[root@c783 bin]# cat /etc/kubernetes/kubelet | grep -v '^#'

KUBELET_ADDRESS="--address=0.0.0.0"

KUBELET_HOSTNAME="--hostname-override=c783"      #此处填写主机名即可

KUBELET_API_SERVER="--api-servers=http://193.168.120.80:8080"

KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"

KUBELET_ARGS=""

 

配置/etc/sysconfig/flanneld：

[root@c783 bin]# cat /etc/sysconfig/flanneld | grep -v '^#'

FLANNEL_ETCD_ENDPOINTS="http://193.168.120.80:2379"

FLANNEL_ETCD_PREFIX="/kube-centos/network"

 

同样关闭防火墙和SELINUX

启动服务：

[root@c783 bin]# cat /shell/k8s/1.sh 
for SERVICES in kube-proxy kubelet flanneld docker; do
    systemctl restart $SERVICES
    systemctl enable $SERVICES
    systemctl status $SERVICES
done

[root@c783 bin]# sh /shell/k8s/1.sh 

 

设置群集的配置文件：

 kubectl config set-cluster default-cluster --server=http://193.168.120.80:8080
 kubectl config set-context default-context --cluster=default-cluster --user=default-admin
 kubectl config use-context default-context

所有NODES上面执行相同的操作

 

在master查看NODES状态

[root@c780 ~]# kubectl get nodes
NAME        STATUS     AGE
c781        Ready      14d
c783        Ready      14d
c785        Ready      14d

 

简单的测试：

[root@c780 app1]# cat tomcat.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  name: myweb
spec:
  replicas: 1
  selector:
    app: myweb
  template:
    metadata:
      labels:
        app: myweb
    spec:
      containers:
        - name: myweb
          image: tomcat
          ports:
          - containerPort: 8080

 

kubectl create -f tomcat.yaml

[root@c780 app1]# kubectl create -f tomcat.yaml 
replicationcontroller "myweb" created

 

 

[root@c780 app1]# kubectl describe pod

如果出现如下报错：

details: (open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory)"

 

需要先卸载subscription-manager-rhsm-certificates.x86_64.0.1.20.11-1.el7.centos

再安装：rpm -ivh http://mirror.centos.org/centos/7/os/x86_64/Packages/python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm

原因是因为这个链接文件没有，卸载安装以后文件才能被加载，NODES节点才能下载到相应的镜像，具体解释可以百度

[root@c783 registry.access.redhat.com]# pwd
/etc/docker/certs.d/registry.access.redhat.com
[root@c783 registry.access.redhat.com]# ll
总用量 0
lrwxrwxrwx. 1 root root 27 8月  13 15:52 redhat-ca.crt -> /etc/rhsm/ca/redhat-uep.pem

 

此时再验证NODES，RC，PODS等的状态：

[root@c780 app1]# kubectl get pods
NAME          READY     STATUS    RESTARTS   AGE
myweb-v9cm6   1/1       Running   0          4h

[root@c780 app1]# kubectl get rc
NAME      DESIRED   CURRENT   READY     AGE
myweb     1         1         1         4h

 

[root@c780 app1]# kubectl describe pod
Name:           myweb-v9cm6
Namespace:      default
Node:           c781/193.168.120.81
Start Time:     Tue, 28 Aug 2018 11:59:00 +0800
Labels:         app=myweb
Status:         Running
IP:             172.30.35.2
Controllers:    ReplicationController/myweb
Containers:
  myweb:
    Container ID:               docker://ad899e084804f8046093458d74b35379d7335af3c8d3825de776e7ace3f03217
    Image:                      tomcat
    Image ID:                   docker-pullable://docker.io/tomcat@sha256:eab533908d6c7760a84a5edbb4490b648484941dfa708a1c5f4286db8caab825
    Port:                       8080/TCP
    State:                      Running
      Started:                  Tue, 28 Aug 2018 12:30:53 +0800
    Ready:                      True
    Restart Count:              0
    Volume Mounts:              
    Environment Variables:      
Conditions:
  Type          Status
  Initialized   True 
  Ready         True 
  PodScheduled  True 
No volumes.
QoS Class:      BestEffort
Tolerations:    
No events.
 

再访问POD的IP，测试业务：

 

 

 

正常打开，K8S部署成功