K8S学习

安装虚拟机

采用VirtrulBox + vagrant来安装三台centOS7

链接：https://pan.baidu.com/s/1vhQibS-nw_aMTz37-2Y66A 
提取码：1dp2 
复制这段内容后打开百度网盘手机App，操作更方便哦

配置vagrant

创建一个Vagrantfile，内容如下
根据该文件可以创建三台centos虚拟机
以k8s-node{i}来命名

Vagrant.configure("2") do |config|
  # 设定中科院CentOS镜像源
   config.vm.box_url = "https://mirrors.ustc.edu.cn/centos-cloud/centos/7/vagrant/x86_64/images/CentOS-7.box"
   (1..3).each do |i|
        config.vm.define "k8s-node#{i}" do |node|
            # 设置虚拟机的Box
            node.vm.box = "centos/7"

            # 设置虚拟机的主机名
            node.vm.hostname="k8s-node#{i}"

            # 设置虚拟机的IP
            node.vm.network "private_network", ip: "192.168.1.#{99+i}", netmask: "255.255.255.0"

            # 设置主机与虚拟机的共享目录
            # node.vm.synced_folder "~/Documents/vagrant/share", "/home/vagrant/share"

            # VirtaulBox相关配置
            node.vm.provider "virtualbox" do |v|
                # 设置虚拟机的名称
                v.name = "k8s-node#{i}"
                # 设置虚拟机的内存大小
                v.memory = 4096
                # 设置虚拟机的CPU个数
                v.cpus = 4
            end
        end
   end
end

在该文件目录下执行命令 vagrant up即可自动下载安装

开启虚拟机ssh访问

输入命令

# 进入命令行
vagrant ssh k8s-node1
# 切换root用户
su root
# 密码默认 vagrant
# 编辑文本
vi /etc/ssh/sshd_config
# 修改密码登录开
PasswordAuthentication yes
# 重启ssh服务
service sshd restart

修改网络

进入虚拟机输入命令

[root@k8s-node1 ~]# ip addr

可以看到三个虚拟机的eth0网卡ip是一样的，这是因为VirtualBox默认设置了网络地址转换，通过配置端口转发进行通信，放入k8s集群中是不好用的

修改VirtualBox配置

开始主机网络
点击管理-主机网络管理器

主机网络管理器

添加一个NAT网络，默认即可
配置虚拟机网络
关闭所有虚拟机
点击设置- 网络-网卡1
连接方式选择NAT 网络
点击刷新mac地址

image.png

对所有的虚拟机做同样操作

虚拟机系统配置

关闭防火墙

systemctl stop firewalld
systemctl disable firewalld

关闭selinux

[root@k8s-node1 ~]# sed -i 's/enforcing/disable/' /etc/selinux/config
[root@k8s-node1 ~]# setenforce 0

关闭Linux 的Swap
内存swap开启时会严重影响k8s性能，某些节点可能无法使用

[root@k8s-node1 ~]# swapoff -a
[root@k8s-node1 ~]# sed -ri 's/.*swap.*/#&/' /etc/fstab

添加主机名与IP的对应关系

vi /etc/hosts
# 以下是我的ip
10.0.2.5 k8s-node1
10.0.2.4 k8s-node2
10.0.2.15 k8s-node3

将桥接IPv4流量传递到iptables 的链：

cat> /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sysctl --system

安装K8S环境

卸载之前的docker

sudo yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-engine

安装docker-ce
安装必要依赖

sudo yum install -y yum-utils \
device-mapper-persistent-data \
lvm2

更新docker yum源

sudo yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo

安装docker-ce

sudo yum install -y docker-ce docker-ce-cli containerd.io

配置docker加速

sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://*********.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

安装k8s

添加yum源

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

yum install kubelet-1.17.3 kubeadm-1.17.3 kubectl-1.17.3
systemctl enable kubelet
systemctl start kubelet

部署 k8s-master

使用如下shell命令来下载镜像

#!/bin/bash

images=(
    kube-apiserver:v1.17.3
    kube-proxy:v1.17.3
    kube-controller-manager:v1.17.3
    kube-scheduler:v1.17.3
    coredns:1.6.5
    etcd:3.4.3-0
    pause:3.1
)

for imageName in ${images[@]} ; do
    docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
#   docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName  k8s.gcr.io/$imageName
done

master 节点初始化

kubeadm init \
--apiserver-advertise-address=10.0.2.5 \
--image-repository registry.cn-hangzhou.aliyuncs.com/google_containers \
--kubernetes-version v1.17.3 \
--service-cidr=10.96.0.0/16 \
--pod-network-cidr=10.244.0.0/16

无类别域间路由(CIDR)是用于给用户分配IP地址以及在互联网上有效地路由IP数据报的对IP地址进行归类的方法

执行完成的提示信息很有用，指导后续如何操作

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.0.2.5:6443 --token 57639i.hxy0yvqsorn55o16 \
    --discovery-token-ca-cert-hash sha256:9bdb57d74eb2d64b677ae3156e0b5519c514ad581b98e03b034bd7d789804fe1

控制面板已经初始化成功，如果想要在集群中使用，需要执行如下命令

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

需要部署一个pod网络
查看文档
- 安装pod网络插件

kubectl apply -f \
https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

可能被墙，临时解决方案
进入如下链接查询ip
将查询出来的结果加入host
cmd刷新hosts

ipconfig /flushdns

linux刷新host

/etc/init.d/network restart

输入命令能看到如下图,表示正常

kubectl get pods --all-namespaces

image.png

如果并不是全部runnning状态，一般是kube-flannel-ds-amd64-***下载不成功，可以去dockerHub上查找其他镜像
拉取镜像,并重启,再查看namespace是否成功全部启动

image.png

docker pull jmgao1983/flannel:v0.11.0-amd64
kubectl apply -f kube-flannel.yml

systemctl restart kubelet
systemctl restart docker

将其他节点join到主节点上，用上面提示的语句，如果token已经超时(2h)，则自己刷新token

kubeadm join 10.0.2.5:6443 --token 57639i.hxy0yvqsorn55o16 \
    --discovery-token-ca-cert-hash sha256:9bdb57d74eb2d64b677ae3156e0b5519c514ad581b98e03b034bd7d789804fe1

刷新token

kubeadm token create --print-join-command
# 创建不过期的token
kubeadm token create --ttl 0 --print-join-command

如果发现报错/proc/sys/net/ipv4/ip_forward contents are not set to 1

W0117 11:34:07.478113   14693 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.2. Latest validated version: 19.03
error execution phase preflight: [preflight] Some fatal errors occurred:
    [ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

输入命令修改

echo 1 > /proc/sys/net/ipv4/ip_forward

查看所有节点

kubectl get nodes

可以通过如下命令监控

watch kubectl get pod -n kube-system -o wide

另：
删除节点，在主节点上操作

kubectl drain k8s-node3 --delete-local-data --force --ignore-daemonsets
kubectl delete node k8s-node3

重新添加节点

# 重置kubeadm
kubeadm reset
# 重新join
kubeadm join 10.0.2.5:6443 --token 57639i.hxy0yvqsorn55o16 \
    --discovery-token-ca-cert-hash sha256:9bdb57d74eb2d64b677ae3156e0b5519c514ad581b98e03b034bd7d789804fe1

k8s学习(1)