24.1配置宿主机网络
查看是否支持虚拟化
grep -E '(vmx|svm)' /proc/cpuinfo
安装KVM需要的软件包
yum install qemu virt kvm -y
Question:
Solution:已安装的跳过
yum install qemu virt kvm -y --skip-broken
开机自启
systemctl start libvirtd
systemctl enable libvirtd
验证是否启动成功
virsh list
桥接工具包
yum install -y bridge-utils
#配置桥接模式
cd /etc/sysconfig/network-scripts
cp ifcfg-em2 ifcfg-br0
两个文件都要改 只保留桥接的IP
[root@localhost network-scripts]# vim ifcfg-em2
TYPE=Ethernet
BRIDGE=br0
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=em2
UUID=74c8085f-4c0d-4743-b0a0-70e51e3eb877
DEVICE=em2
ONBOOT=yes
IPADDR=172.16.10.5
PREFIX=24
GATEWAY=172.16.10.254
DNS1=114.114.114.114
#注意****IPADDR**** 要改为自己的
[root@localhost network-scripts]# vim ifcfg-br0
TYPE=Bridge
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=br0
DEVICE=br0
ONBOOT=yes
IPADDR=172.16.10.3
PREFIX=24
GATEWAY=172.16.10.254
DNS1=114.114.114.114
重启网络
systemctl restart network
#验证
brctl show
24.2 安装虚拟机
cd /home/kvm
#创建****master****虚拟机的存储盘**** 10.4
qemu-img create -f qcow2 -o cluster_size=2M k8s-master01.qcow2 200G
安装虚拟机壳子 --name k8s-master01.qcow2 --memory 8192 --vcpus 4 可改
virt-install --virt-type kvm --os-type=linux --os-variant rhel7 --name k8s-master01.qcow2 --memory 8192 --vcpus 4 --disk /home/kvm/k8s-master01.qcow2,format=qcow2 --cdrom /home/kvm/CentOS-7-x86_64-DVD-2009.iso --network bridge=br0 --graphics vnc,listen=0.0.0.0 --noautoconsole
#创建****worker****虚拟机的存储盘**** 10.5
qemu-img create -f qcow2 -o cluster_size=2M k8s-worker01.qcow2 200G
安装虚拟机壳子 --name k8s-worker01.qcow2 --memory 8192 --vcpus 4 可改
virt-install --virt-type kvm --os-type=linux --os-variant rhel7 --name k8s-worker01.qcow2 --memory 8192 --vcpus 4 --disk /home/kvm/k8s-worker01.qcow2,format=qcow2 --cdrom /home/kvm/CentOS-7-x86_64-DVD-2009.iso --network bridge=br0 --graphics vnc,listen=0.0.0.0 --noautoconsole
#创建****worker****虚拟机的存储盘**** 10.3
qemu-img create -f qcow2 -o cluster_size=2M k8s-worker02.qcow2 200G
安装虚拟机壳子--name k8s-worker02.qcow2 --memory 8192 --vcpus 4 可改
virt-install --virt-type kvm --os-type=linux --os-variant rhel7 --name k8s-worker02.qcow2 --memory 32768 --vcpus 32 --disk /home/kvm/k8s-worker02.qcow2,format=qcow2 --cdrom /home/kvm/CentOS-7-x86_64-DVD-2009.iso --network bridge=br0 --graphics vnc,listen=0.0.0.0 --noautoconsole
迅速通过vnc viewer 去安装配置操作系统
查看虚拟机启动
netstat -ntlp | grep 5900
virsh list --all
virsh shutdown k8s-master01.qcow2
virsh start k8s-master01.qcow2
虚拟机密码
ssh 172.16.10.50 root@starQuest2022
Question****:系统启动卡住
Solution:
virsh destroy k8s-master01.qcow2
virsh undefine k8s-master01.qcow2
Question****:更改桥接模式失败引发的问题
Solution:
更正桥接模式 只保留桥接ip其他网卡配置均清空ip
-
重启虚拟机 验证
image.png
24.3 配置虚拟机****网络
配置网卡 修改网段
vi /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eth0
UUID=c510f2f9-9820-45e8-9c70-65674bd35258
DEVICE=eth0
ONBOOT=yes
IPADDR=172.16.10.50
PREFIX=24
GATEWAY=172.16.10.254
DNS1=114.114.114.114
重启网络
systemctl restart network
Question:
Solution:
vi /root/.ssh/known_hosts 删除有问题IP对应行
24.4 更新系统 升级内核
#设置hostname
hostnamectl set-hostname k8s-master01
hostnamectl set-hostname k8s-worker01
hostnamectl set-hostname k8s-worker02
yum update
yum install wget
yum install vim
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm
yum --disablerepo="*" --enablerepo="elrepo-kernel" list available
yum --enablerepo=elrepo-kernel install kernel-lt -y
让开机内核生效
vi /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=",,g' /etc/system-release)"
改为0
GRUB_DEFAULT=0
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto spectre_v2=retpoline rhgb quiet"
GRUB_DISABLE_RECOVERY="true"
让配置生效
grub2-mkconfig -o /boot/grub2/grub.cfg
重启并查看
reboot
uname -a
24.5 安装docker
方式一 配置yum源
cd /etc/yum.repos.d
传文件
scp -r docker-ce.repo 172.16.10.51:/etc/yum.repos.d/
安装启动
yum install docker-ce
方式二 yum 安装rpm包
yum install -y docker-ce-18.03.1.ce-1.el7.centos.x86_64.rpm
#开机自启动
systemctl start docker
systemctl enable docker
配置守护文件
vi /etc/docker/daemon.json
{
"exec-opts": [
"native.cgroupdriver=systemd"
],
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "1000m",
"max-file": "3"
},
"registry-mirrors": [
"https://zydiol88.mirror.aliyuncs.com"
],
"insecure-registries": ["harbor.bicisims.com"],
"selinux-enabled": false
}
让K8识别,配置镜像仓库地址
重启
systemctl restart docker
验证
docker ps
24.6 安装k8s准备工作
修改时区,同步时间
yum install ntpdate -y
ntpdate time2.aliyun.com
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
echo 'Asia/Shanghai' > /etc/timezone
crontab -e
0 12 * * * /usr/sbin/ntpdate time2.aliyun.com
关闭防火墙,selinux,swap
systemctl stop firewalld
systemctl disable firewalld
systemctl status firewalld
关闭内置selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0
swapoff -a
sed -ri 's/.swap./#&/' /etc/fstab
系统优化 net.ipv4.ip_forward=1 让docker互相通信
cat > /etc/sysctl.d/k8s_better.conf << EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recycle=0
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720
EOF
验证
cat /etc/sysctl.d/k8s_better.conf
sysctl -p /etc/sysctl.d/k8s_better.conf
上面这两个错忽略
配置ipvs k8s需要的网络支持
cat > /etc/sysconfig/modules/ipvs.modules < modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack EOF chmod 755 /etc/sysconfig/modules/ipvs.modules && bash vi /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg #必须确认是否更改hostname hostnamectl set-hostname k8s-master01 yum install -y kubelet-1.23.5 kubeadm-1.23.5 kubectl-1.23.5 systemctl enable kubelet kubeadm init --apiserver-advertise-address=172.16.10.50 --kubernetes-version=1.23.5 --image-repository registry.aliyuncs.com/google_containers --service-cidr=10.96.0.0/16 --pod-network-cidr=10.244.0.0/16 [ Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown (id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 172.16.10.50:6443 --token dak7im.w25v1sjl0kcm4y3c \ --discovery-token-ca-cert-hash sha256:afb2a0b22a3e563671103f93965f71a915f65054db74b7ffa97a84932a098f42 ] vi /etc/hosts 172.16.10.50 k8s-master01 172.16.10.51 k8s-worker01 172.16.10.52 k8s-worker02 kubectl get nodes kubeadm token create --print-join-command kubectl version mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown (id -g) $HOME/.kube/config scp -r conf/ 172.16.10.50:/home/software/ wget https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml kubectl apply -f kube-flannel.yml kubectl get pod -A Question: Solution: kubectl explain DaemonSet Question:k8s Node 一直 pending****Solution: $ vim /etc/kubernetes/manifests/kube-apiserver.yaml spec: containers: command: kube-apiserver --feature-gates=RemoveSelfLink=false $ kubectl apply -f /etc/kubernetes/manifests/kube-apiserver.yaml $ kubectl apply -f /etc/kubernetes/manifests/kube-apiserver.yaml kubectl get pod yum install -y nfs-utils systemctl enable nfs systemctl start nfs 1)安装NFS $ yum install -y nfs-utils rpcbind $ yum install -y nfs-utils 2)启动服务 服务器端 systemctl enable rpcbind systemctl start rpcbind systemctl restart rpcbind 客户端 systemctl enable nfs systemctl start nfs systemctl restart nfs 3 )创建共享目录 服务器端 mkdir -p /home/data vi /etc/exports 编写NFS的共享配置 /home/data *(rw,sync,no_root_squash) 4)查看NFS共享目录 服务器端 showmount -e 172.16.10.5 5)如果要把其他服务器的磁盘加进来 就在对应的服务器安装 nfs服务端 然后建立共享文件夹 kubectl get pod -A cd /root/tools/storageclass/ vim nfs-provisioner.yaml kubectl apply -f rbac.yaml kubectl apply -f nfs-provisioner.yaml kubectl apply -f nfs-StorageClass.yaml kubectl patch storageclass huaweinfs -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}' wget https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/kubesphere-installer.yaml wget https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/cluster-configuration.yaml kubectl apply -f kubesphere-installer.yaml kubectl apply -f cluster-configuration.yaml kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f Console: http://172.16.10.50:30880 Account: admin Password: P@88w0rd starQuest2022配置yum源
24.7 安装k8s
安装kubelet
启动kubelet
用kubeadm 安装k8s 只在master上初始化
安装成功后信息
增加host
master 验证
重新生成token
worker节点 加入master kubeadm join 172.16.10.50:6443 --token 1agi0a.bsluty1ad11px2j0 --discovery-token-ca-cert-hash sha256:a630b36df4a96c76a76ced3d4ac82373ec55549464478818cb1c1c361b606835
验证
配置kubectl环境
上传配置文件
运用flannel网络插件
验证
修改apiserver的yaml文件
执行apiserver文件(twice)
验证
安装nfs客户端
24.8 重要文件
24.9安装nfs
服务端(安装在磁盘空间健康的机器 )
客户端(在所有K8S的所有节点都需要安装)
*代表对所有IP都开放此目录,rw是读写
验证
24.10 安装storageclass
修改
应用yml配置文件做storageclass
配置默认的storageclass
24.11安装****kubesphere
实时查看安装进度
Welcome to KubeSphere!