ansible一键部署k8s（自动化）

ANSIBLE（自动化）一键部署k8s

主要使用ansible工具来批量管理我们需要的主机，这些主机通过接收我们编写的playbook文件来部署k8s集群，以此来实现我们的自动化部署。

1. ansible是新出现的自动化运维工具，基于Python开发，集合了众多运维工具（puppet、cfengine、chef、func、fabric）的优点，实现了批量系统配置、批量程序部署、批量运行命令等功能。ansible是基于模块工作的，本身没有批量部署的能力。真正具有批量部署的是ansible所运行的模块，ansible只是提供一种框架。
2. k8s是容器集群管理系统，是一个开源的平台，可以实现容器集群的自动化部署、自动扩缩容、维护等功能。
3. 实验图：
   

一.准备服务器节点

ansible：192.168.176.140
master：192.168.176.137
node1： 192.168.176.138
node2： 192.168.176.139

二. 配置Ansible

1.安装ansible

//配置yum源为阿里云
# cd /etc/yum.repos.d/
//备份自带的yum源
# mv CentOS-Base.repo CentOS-Base.repo.backup
//下载阿里云yum源
# curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
# yum clean all
# yum -y install ansible-2.9.7-1.el7.ans.noarch.rpm / yum -y install ansible

2.设置免密连接

[root@ansible ~]# ssh-keygen
[root@ansible ~]# ssh-copy-id root@192.168.176.137
[root@ansible ~]# ssh-copy-id root@192.168.176.138
[root@ansible ~]# ssh-copy-id root@192.168.176.139

2.在Ansible服务器上的/etc/hosts文件中添加k8s服务器节点信息

[root@ansible ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.176.137 master
192.168.176.138 node1
192.168.176.139 node2

[root@ansible ~]# cat /etc/redhat-release 
CentOS Linux release 7.7.1908 (Core)

3.在Ansible服务器上的/etc/ansible/hosts文件中添加k8s服务器节点

[root@ansible ~]# vim /etc/ansible/hosts 
[k8s-all]
192.168.176.137
192.168.176.138
192.168.176.139
[master]
192.168.176.137
[nodes]
192.168.176.138
192.168.176.139
//测试主机组长成员主机是否在线
[root@ansible ~]# ansible k8s-all -m ping

三. 修改k8s集群各节点/etc/hosts

1.创建playbook文件并执行

[root@ansible ~]# cat hosts_playbook.yml 
---
- hosts: nodes
  remote_user: root

  tasks:
    - name: backup /etc/hosts
      shell: mv /etc/hosts /etc/host_bak
    - name: copy localhosts file to remote
      copy: src=/etc/hosts dest=/etc/ owner=root group=root mode=0644
[root@ansible ~]# ansible-playbook hosts_playbook.yml

四. 安装Docker

1.创建playbook文件并执行

· 所有节点安装docker
[root@ansible ~]# cat install_docker_playbook.yml 
- hosts: k8s-all
  remote_user: root
  vars: 
    docker_version: 18.09.2

  tasks:
    - name: install dependencies
      shell:  yum install -y yum-utils device-mapper-persistent-data lvm2
    - name: docker-repo
      shell: yum-config-manager --add-repo https://mirrors.ustc.edu.cn/docker-ce/linux/centos/docker-ce.repo
    - name: install docker
      yum: name=docker-ce-{{docker_version}} state=present
    - name: start docker
      shell: systemctl start docker && systemctl enable docker
      
[root@ansible ~]# vim  /etc/ansible/ansible.cfg
deprecation_warnings = false  ## 179默认是true，并且不生效
[root@ansible ~]# ansible-playbook install_docker_playbook.yml 

五. 部署k8s master

1.开始部署之前，需要做一些初始化处理：关闭防火墙、关闭selinux、禁用swap、配置k8s阿里云yum源等，所有操作放在脚本 before.sh 中，并在2中playbook中通过script模块执行

[root@ansible ~]# cat before.sh 
#!/bin/bash
#防火墙
systemctl disable firewalld
systemctl stop firewalld
setenforce 0

#禁用swap
swapoff -a

#修改内核参数
cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

#重新加载配置文件
sysctl --system

#配置阿里k8s yum源
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

#更新缓存
yum clean all -y && yum makecache -y && yum repolist -y

2.创建playbook文件，只针对master节点，安装kubectl，kubeadm，kubelet，以及flannel

· wget -c https://gitee.com/lm_py/kube-flannel.yaml?_from=gitee_search

[root@ansible ~]# cat deloy_master_playbook.yml 
- hosts: master
  remote_user: root
  vars:
    kube_version: 1.16.0-0
    k8s_version: v1.16.0
    k8s_master: 192.168.176.137
  tasks: 
    - name: before
      script: ./before.sh
    - name: install kube***
      yum: name={{item}} state=present
      with_items:
        - kubectl-{{kube_version}}
        - kubeadm-{{kube_version}}
        - kubelet-{{kube_version}}
    - name: init k8s
      shell: kubeadm init --image-repository registry.aliyuncs.com/google_containers --kubernetes-version {{k8s_version}} --apiserver-advertise-address {{k8s_master}}  --pod-network-cidr=10.244.0.0/16 --token-ttl 0
    - name: config kube
      shell: mkdir -p $HOME/.kube && cp -i /etc/kubernetes/admin.conf $HOME/.kube/config && chown $(id -u):$(id -g) $HOME/.kube/config
    - name: copy flannel yaml file
      copy: src=/root/kube-flannel.yml dest=/tmp/kube-flannel.yml
    - name: install flannel
      shell: kubectl apply -f /tmp/kube-flannel.yml
    - name: get join command
      shell: kubeadm token create --print-join-command 
      register: join_command
    - name: show join command
      debug: var=join_command verbosity=0
      
[root@ansible ~]# vim  /etc/ansible/ansible.cfg 
command_warnings = False   ### 187行
[root@ansible ~]# ansible-playbook deploy_master_playbook.yml


[root@master ~]# docker  images
REPOSITORY                                                        TAG                 IMAGE ID            CREATED             SIZE
quay-mirror.qiniu.com/coreos/flannel                              v0.12.0-amd64       4e9f801d2217        7 weeks ago         52.8MB
registry.aliyuncs.com/google_containers/kube-apiserver            v1.16.0             b305571ca60a        7 months ago        217MB
registry.aliyuncs.com/google_containers/kube-proxy                v1.16.0             c21b0c7400f9        7 months ago        86.1MB
registry.aliyuncs.com/google_containers/kube-controller-manager   v1.16.0             06a629a7e51c        7 months ago        163MB
registry.aliyuncs.com/google_containers/kube-scheduler            v1.16.0             301ddc62b80b        7 months ago        87.3MB
registry.aliyuncs.com/google_containers/etcd                      3.3.15-0            b2756210eeab        8 months ago        247MB
registry.aliyuncs.com/google_containers/coredns                   1.6.2               bf261d157914        8 months ago        44.1MB
registry.aliyuncs.com/google_containers/pause                     3.1                 da86e6ba6ca1        2 years ago         742kB
[root@master ~]# kubectl get pods --all-namespaces
NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE
kube-system   coredns-58cc8c89f4-bt86l             1/1     Running   0          17h
kube-system   coredns-58cc8c89f4-tdqsg             1/1     Running   0          17h
kube-system   etcd-k8s-master                      1/1     Running   0          17h
kube-system   kube-apiserver-k8s-master            1/1     Running   0          17h
kube-system   kube-controller-manager-k8s-master   1/1     Running   0          17h
kube-system   kube-flannel-ds-amd64-v57pg          1/1     Running   0          17h
kube-system   kube-proxy-xtjkl                     1/1     Running   0          17h
kube-system   kube-scheduler-k8s-master            1/1     Running   0          17h
[root@k8s-master ~]# kubectl get nodes
NAME     STATUS   ROLES    AGE   VERSION
master   Ready    master   17h   v1.16.0

六. 部署k8s node

1.同master一样，开始部署之前，需要做一些初始化处理,所有操作放在脚本 before.sh 中，并在playbook中通过script模块执行

[root@ansible ~]# cat deloy_nodes_playbook.yml 
- hosts: nodes
  remote_user: root
  vars:
    kube_version: 1.16.0-0
  tasks:
    - name: before
      script: ./before.sh
    - name: install kube***
      yum: name={{item}} state=present
      with_items:
        - kubeadm-{{kube_version}}
        - kubelet-{{kube_version}}
    - name: start kubelet
      shell: systemctl enable kubelet && systemctl start kubelet
    - name: join cluster
      shell: kubeadm join 192.168.176.137:6443 --token hbjg13.hx2hsjibb9a02wl9     --discovery-token-ca-cert-hash sha256:ecb6cab37a34d2513700d637c27eb2868d6b35a568afd1cb331e03ab6a839e33

[root@ansible ~]# ansible-playbook deloy_nodes_playbook.yml 

2.在master节点上通过kubectl get nodes看到加入到集群中的节点，并且status为Ready状态

[root@master ~]# kubectl get nodes
NAME    STATUS   ROLES    AGE     VERSION
master  Ready    master   33m     v1.16.0
node1   Ready    <none>   8m39s   v1.16.0
node2   Ready    <none>   2m7s    v1.16.0
node3   Ready    <none>   2m11s   v1.16.0