Kubernets(二)部署非高可用Kubernetes集群的环境准备

Kubernetes v1.20 系列文章目录

Kubernetes(一)认识 kubernetes

Kubernets(二)部署非高可用Kubernetes集群的环境准备

Kubernets(三)部署非高可用Kubernetes集群-通过阿里云源安装 kubeadm、kubelet 和 kubectl

Kubernets(四)创建集群

Kubernetes(五)揭开 kubeadm 的神秘面纱

Kubernetes(六)第一个kubernetes 对象

前言

一、开始部署之前

请一定一定要参考官方文档

1 编译 /etc/hosts 文件

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.122.140 k8s-master
192.168.122.212 k8s-node1
192.168.122.251 k8s-node2
192.168.122.156 k8s-node3

2 编译 Ansible 的主机清单 hosts.ini

[k8s_master]
k8s-master

[k8s_nodes]
k8s-node1
k8s-node2
k8s-node3

[k8s]

[k8s:children]
k8s_master
k8s_nodes

3 发送公钥

前题条件：
需要打开不检查远程服务器的公钥，就是直接信任对方的公钥

编译 /etc/ansible/ansible.cfg 文件，并打开如下配置项

host_key_checking = False

YML 文件send-pubkey.yml 内容如下:

---
- hosts: all
  gather_facts: no
  remote_user: root
  vars:
    ansible_ssh_pass: upsa
  tasks:
  - name: Set authorized key taken from file
    authorized_key:
      user: root
      state: present
      key: "{{ lookup('file', '/root/.ssh/id_rsa.pub') }}"
...

4 发送 /etc/hosts

---
- name: 同步所有节点的 /etc/hosts 文件 并且设置主机名
  hosts: k8s
  gather_facts: no
  tasks:
    - name: 同步 hosts 文件
      copy: src=/etc/hosts dest=/etc/hosts
    - name: 设置各自的主机名
      shell:
        cmd: hostnamectl set-hostname "{{ inventory_hostname }}"
      register: sethostname
    - name: 验证是否成功设置了主机名
      debug: var=sethostname.rc
...

二、部署之前检查并设置环境

1 检查并设置环境

before-you-begin.yml

---
- name: 开始部署集群之前的检查和设置
  hosts: k8s
  gather_facts: no
  tasks:
    - name: 配置禁用 SELinux
      shell: |
        setenforce 0;
        sed -ri '/^SELINUX=/ c SELINUX=disabled' /etc/selinux/config
      tags:
        - swap

    - name: 关闭交互分区
      shell:
        cmd: swapoff -a; sed -ri 's/.*swap.*/#&/g' /etc/fstab
        warn: no
      tags:
        - swap

    - name: 创建模块配置文件 /etc/modules-load.d/k8s.conf
      blockinfile:
        path: /etc/modules-load.d/k8s.conf
        create: yes
        block: |
          br_netfilter
          ip_vs
          ip_vs_rr
          ip_vs_wrr
          ip_vs_sh
          nf_conntrack_ipv4


    - name: 确保节点上的 iptables 能够正确地查看桥接流量
      blockinfile:
        path: /etc/sysctl.d/k8s.conf
        create: yes
        block: |
          net.bridge.bridge-nf-call-ip6tables = 1
          net.bridge.bridge-nf-call-iptables = 1
          net.ipv4.ip_forward = 1

    - name: 执行加载模块的命令
      shell: modprobe br_netfilter

    - name: 检查 SELinux and Swap
      shell: |
        hostname > /tmp/host-info;
        getenforce >> /tmp/host-info;
        free -m |grep 'Swap' >> /tmp/host-info;
        lsmod | grep br_netfilter >> /tmp/host-info;
        sysctl --system |grep 'k8s.conf' -A 2 >> /tmp/host-info;

    - name: 获取 mac 信息并写入信息文件
      shell: |
        host=$(hostname);
        ip link |
        awk -v host=$host '/link\/ether/ {print $2, host}' >> /tmp/host-info ;
        echo "---------------------------" >> /tmp/host-info

    - name: 获取比对报告
      fetch:
        src: /tmp/host-info
        dest: ./
...

2 检查端口

check-port.yml

- hosts: k8s
  gather_facts: no
  tasks:
    - name: hello
      script: ./check-port.py
      register: ret
    - debug: var=item
      loop: "{{ ret.stdout_lines }}"

check-port.py

基于 Python2.7

#!/bin/env python
#coding:utf-8
import re
import subprocess
import socket

hostname = socket.gethostname()

ports_set = set()
if 'master' in hostname:
    check_ports = {"6443", "10250", "10251", "102502", "2379", "2380"}
else:
    check_ports = {str(i) for i in xrange(30000, 32768) }
    check_ports.add("10250")

r = subprocess.Popen("ss -nta", stdout=subprocess.PIPE,shell=True)
result = r.stdout.read()

for line in result.splitlines():
	if re.match('^(ESTAB|LISTEN|SYN-SENT)', line):
	    line = line.split()[3]
	    port = line.split(':')[-1]
	    ports_set.add(port)

used_ports = check_ports & ports_set
used_ports = ' '.join(used_ports)
if used_ports:
	print("这些端口已使用: %s" % used_ports)
else:
	print("端口未占用")

三、部署 docker

每个节点均部署 docker

docker/deploy-docker.yml

这里是先在某个节点上下载 docker 所需要的所有 rpm 包,
之后再把这些包传输到 Ansible 机器上的某个位置。

接着把 rpm 包，从 Ansible 机器上分发到每个节点（除了刚才已经下载 rpm 包的节点）

最后每个节点使用 yum localinstall 命令从本地安装 docker

---
- name: deploy docker
  hosts: k8s
  gather_facts: no
  vars:
    pkgs_dir: /docker-pkg
    pkgs:
      - device-mapper-persistent-data
      - lvm2
      - docker-ce
      - docker-ce-cli
      - containerd.io


    # 变量 download_host 需要手动设置
    # 且值需要是此 playbook 目标主机中的一个
    # 需要写在 inventory 文件中的名称
    download_host: "k8s-master"

  tasks:
    - name: "只需要给 {{ download_host }} 的主机安装仓库文件"
      when: inventory_hostname == download_host
      get_url:
        url: https://download.docker.com/linux/centos/docker-ce.repo
        dest: /etc/yum.repos.d/docker-ce.repo
      tags:
        - deploy

    - name: 创建存放 rmp 包的目录
      when: inventory_hostname == download_host
      file:
        path: "{{ pkgs_dir }}"
        state: directory
      tags:
        - deploy

    - name:  开始下载软件包
      when: inventory_hostname == download_host
      yum:
        name: "{{ pkgs }}"
        download_only: yes
        download_dir: "{{ pkgs_dir }}"
      tags:
        - deploy

    - name: 传输 rpm 包到远程节点
      when: inventory_hostname != download_host
      copy:
        src: "{{ pkgs_dir }}"
        dest: "/"
      tags:
        - deploy

    - name: 正在执行从本地安装软件包
      shell:
        cmd: yum -y localinstall *
        chdir: "{{ pkgs_dir }}"
        warn: no
      async: 600
      poll: 0
      tags:
        - deploy

    - name: 设置 /etc/docker/daemon.json
      copy: src=files/daemon.json dest=/etc/docker/daemon.json
      notify: restart docker
      tags:
        - start
        - update

    - name: 启动 docker
      systemd:
        name: docker
        enabled: yes
        state: started
      tags:
        - start
  handlers:
    - name: restart docker
      systemd:
        name: docker
        state: restarted
...

docker daemon 配置文件

docker/files/daemon.json

{
   "registry-mirrors": ["https://自己的阿里云加速器.mirror.aliyuncs.com"],
   "exec-opts": ["native.cgroupdriver=systemd"],
   "log-driver": "json-file",
   "log-opts": {
      "max-size": "100m"
   },
   "storage-driver": "overlay2"
}

部署的时候执行：

ansible-playbook -i hosts.ini docker/deploy-docker.yml  -t deploy

使用自定义变量部署的时候执行：

ansible-playbook -i hosts.ini docker/deploy-docker.yml  -t deploy  -e "download_host=master"

master 是你指定的需要下载软件包的主机在 inventory 文件中的名称

启动 docker 服务 执行：

ansible-playbook -i hosts.ini docker/deploy-docker.yml  -t start