kubeasz下载:kubeasz
kubeasz:kubeasz安装方式
网站上定义了kubeasz和kubernetes的版本对应,按照要求对应安装
部署需要3台设备以上,CPU最好双核以上
角色 | IP |
---|---|
ansible | 192.168.116.130 |
k8s-master | 192.168.116.132 |
k8s-node | 192.168.116.133 |
cd /etc/yum.repos.d/
yum install -y wget
wget http://mirrors.aliyun.com/repo/Centos-7.repo
wget http://mirrors.aliyun.com/repo/epel-7.repo
mv CentOS-Base.repo CentOS-Base.repo.bak
yum clean all
yum makecache
systemctl stop firewalld
systemctl disable firewalld
sed -i 's/SELINUX=enforcing$/SELINUX=disabled/g' /etc/selinux/config
setenforce 0
hostnamectl set-hostname ansible-130
vi /etc/hosts
192.168.116.130 ansible-130
192.168.116.132 k8s-master
192.168.116.133 k8s-node1
设置秘钥连接和复制host文件
ssh-keygen
ssh-copy-id [email protected]
ssh-copy-id [email protected]
scp /etc/hosts [email protected]:/etc/hosts
scp /etc/hosts [email protected]:/etc/hosts
下载脚本,自动下载k8s需要的安装包
yum install -y python git python-pip ansible python-netaddr -y
curl -C- -fLO --retry 3 https://github.com/easzlab/kubeasz/releases/download/2.2.0/easzup
在脚本定义了下载的安装包版本,可以自己手动修改,但是需要注意kubeasz和kubernetes的版本对应关系
vi easzup
export DOCKER_VER=19.03.5
export KUBEASZ_VER=2.2.0
export K8S_BIN_VER=v1.17.2
export EXT_BIN_VER=0.4.0
export SYS_PKG_VER=0.3.3
chmod +x ./easzup
./easzup -D
配置安装的版本
cd /etc/ansible/
cp example/hosts.multi-node ./hosts
设置master和node的IP和软件配置
vi hosts 修改配置
#设置etcd的IP
[etcd]
192.168.116.132 NODE_NAME=etcd1
# master node(s) #设置master节点的IP
[kube-master]
192.168.116.132
# work node(s) #设置node节点的IP
[kube-node]
192.168.116.133
# K8S Service CIDR, not overlap with node(host) networking #k8s集群管理IP段
SERVICE_CIDR="10.0.0.0/16"
# Cluster CIDR (Pod CIDR), not overlap with node(host) networking #k8s集群容器IP段
CLUSTER_CIDR="20.0.0.0/16"
# NodePort Rang #设置node对外开放端口
NODE_PORT_RANGE="20000-40000"
# Cluster DNS Domain #设置DNS域名
CLUSTER_DNS_DOMAIN="cluster.local."
ansible all -m ping 测试ansible到节点的网络是否正常
开始安装k8s
ansible-playbook 01.prepare.yml
ansible-playbook 02.etcd.yml
ansible-playbook 03.docker.yml
ansible-playbook 04.kube-master.yml
ansible-playbook 05.kube-node.yml
ansible-playbook 06.network.yml
hostnamectl set-hostname k8s-master
kubectl get pod -A
kubectl get nodes
运行容器测试k8s能否正常工作
kubectl run net-test --image=alpine --replicas=1 sleep 360000
kubectl get pod -A
可以看到pod的的IP和node的IP
kubectl get pod -A -o wide
wget https://storage.googleapis.com/kubernetes-release/release/v1.17.2/kubernetes.tar.gz
cd /root/kubernetes/cluster/addons/dns/coredns/
cp coredns.yaml.base coredns.yml
vi coredns.yml 修改配置
#原本这行 kubernetes __PILLAR__DNS__DOMAIN__ in-addr.arpa ip6.arpa,修改为下面,这里的cluster.local.对应上面ansible的hosts的DNS配置
kubernetes cluster.local. in-addr.arpa ip6.arpa
#修改forward . /etc/resolv.conf,指定外网的DNS的IP地址
forward . 223.5.5.5
# image: k8s.gcr.io/coredns:1.6.5 修改镜像源
image: coredns/coredns:1.6.7
#修改容器使用内存,最好1G以上 memory: __PILLAR__DNS__MEMORY__LIMIT__
memory: 1Gi
#设置DNS容器的IP clusterIP: __PILLAR__DNS__SERVER__,对应上面ansible的hosts的集群管理service网段的IP
clusterIP: 10.0.0.2
kubectl apply -f coredns.yml
测试容器能否连接外网解析域名
kubectl run net-test1 --image=alpine --replicas=1 sleep 360000
kubectl get pod -A
kubectl exec -it net-test1-5fcc69db59-s8r2z /bin/sh
按照上面搭建k8s集群后,查看k8s版本为1.17.2
K8S版本下载:K8S源码包
选择这个,查看详细的源码包
选择server,注意CPU型号架构,根据设备的cpu型号下载
master查看版本
kubectl version
systemctl stop kube-apiserver.service kube-controller-manager.service kubelet.service kube-proxy.service kube-scheduler.service
node关闭k8s服务
systemctl stop kubelet.service kube-proxy.service
ansible配置
mkdir /backup 创建老文件备份目录
cd /etc/ansible/bin/
mv kube-apiserver kube-controller-manager kubectl kubelet kube-proxy kube-scheduler /backup/
cd /tmp 下载新的文件到目录
wget https://storage.googleapis.com/kubernetes-release/release/v1.19.7/kubernetes-server-linux-amd64.tar.gz
tar xf kubernetes-server-linux-amd64.tar.gz
cd kubernetes/server/bin
默认新的文件都有执行权限,没有手动加上就行
cp kube-apiserver kube-controller-manager kube-proxy kubectl kubelet kube-scheduler /etc/ansible/bin/
默认kubeasz会自动安装easzctl,这个专门用来升级版本
easzctl upgrade 开始升级master和node
在master和node查看新的版本是否安装成功
kubectl version
kubectl run net-test1 --image=alpine --replicas=1 sleep 360000
kubectl get pod -A -o wide
实现方式:
将静态文件保留在http://192.168.116.133/目录下,容器为nginx容器
将动态文件保留在http://192.168.116.133/tomcat目录下,容器为tomcat容器
nginx和tomcat容器实际都是共用同一个NFS服务器
NFS服务器配置
yum install -y nfs-utils
mkdir /data 创建共享目录
mkdir /data/k8s/tomcat -p
mkdir /data/k8s/nginx -p
vi /etc/exports
/data *(rw,no_root_squash)
systemctl start nfs
systemctl enable nfs
showmount -e 192.168.116.130 查看是否共享成功
k8s-master 和全部node节点测试能否连接
yum install nfs-utils -y
showmount -e 192.168.116.130 查看是否共享成功
mkdir /dockerfile/nginx -p
cd /dockerfile/nginx/
下载nginx源码包
wget http://nginx.org/download/nginx-1.18.0.tar.gz
vi nginx.conf 配置nginx的配置文件
user nginx;
daemon off;
worker_processes auto;
error_log /app/nginx/logs/nginx-error.log;
pid /app/nginx/run/nginx.pid;
events {
worker_connections 10240;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /app/nginx/logs/nginx-access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 4096;
server {
listen 80;
server_name k8s-nginx;
location / {
root html;
index index.html index.htm;
}
location /tomcat {
proxy_pass http://k8s-tomcat-1-service:8080/;
}
}
}
vi nginx-dockerfile
FROM centos:7.7.1908
RUN useradd -s /sbin/nologin nginx
#nginx
RUN mkdir /app
RUN yum install -y psmisc net-tools wget gcc pcre-devel zlib-devel make
ADD nginx-1.18.0.tar.gz /app/
RUN cd /app/nginx-1.18.0/ && ./configure --prefix=/app/nginx --user=nginx\
--user=nginx\
--group=nginx\
--with-http_ssl_module\
--with-http_v2_module\
--with-http_realip_module\
--with-http_stub_status_module\
--with-http_gzip_static_module\
--with-pcre\
--with-stream\
--with-stream_ssl_module\
--with-stream_realip_module && make && make install
RUN mkdir /app/nginx/run
ENV PATH=$PATH:/app/nginx/sbin
RUN chown nginx.nginx -R /app/
COPY nginx.conf /app/nginx/conf/
EXPOSE 80
CMD ["nginx"]
制作nginx镜像
docker build -t test/nginx:v1.0 -f nginx-dockerfile .
mkdir /dockerfile/tomcat
cd /dockerfile/tomcat/
下载jdk包
wget https://repo.huaweicloud.com/java/jdk/8u191-b12/jdk-8u191-linux-x64.tar.gz
下载tomcat包
wget https://mirrors.bfsu.edu.cn/apache/tomcat/tomcat-8/v8.5.77/bin/apache-tomcat-8.5.77.tar.gz --no-check-certificate
设置jdk路径
vi tomcat.conf
JAVA_HOME=/usr/local/jdk
设置tomcat主页
vi index.jsp
<%@ page language="java" contentType="text/html; charset=UTF-8"
pageEncoding="UTF-8"%>
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>tomcat-test</title>
</head>
<body>
<h1>tomcat-test</h1>
<%
out.println("test");
%>
<br>
<%=request.getRequestURL()%>
</body>
</html>
配置镜像dockerfile
vi tomcat-dockerfile
FROM centos:7.7.1908
RUN yum install -y wget curl psmisc net-tools
#jdk
COPY jdk-8u191-linux-x64.tar.gz /
ADD jdk-8u191-linux-x64.tar.gz /usr/local/
RUN ln -s /usr/local/jdk1.8.0_191 /usr/local/jdk
ENV JAVA_HOME=/usr/local/jdk
ENV PATH=$PATH:$JAVA_HOME/bin
ENV JRE_HOME=$JAVA_HOME/jre
ENV CLASSPATH=$JAVA_HOME/lib/:$JRE_HOME/lib
#tomcat
COPY apache-tomcat-8.5.77.tar.gz /
ADD apache-tomcat-8.5.77.tar.gz /usr/local/
RUN ln -s /usr/local/apache-tomcat-8.5.77 /usr/local/tomcat
ENV PATH=$PATH:/usr/local/tomcat/bin
COPY tomcat.conf /usr/local/tomcat/conf/
COPY index.jsp /usr/local/tomcat/webapps/ROOT/
EXPOSE 8080
CMD ["/usr/local/tomcat/bin/catalina.sh","run" ]
制作tomcat镜像
docker build -t test/tomcat:v1.0 -f tomcat-dockerfile .
前台运行测试镜像能否使用
docker run --rm -it -p 8080:8080 test/tomcat:v1.0
curl 192.168.116.132:8080
将k8s-master制作好的镜像保存传送到node节点,要不然容器运行不了
docker save test/nginx:v1.0 -o /root/k8s-nginx.tar
docker save test/tomcat:v1.0 -o /root/k8s-tomcat.tar
scp /root/k8s-nginx.tar [email protected]:/root/
scp /root/k8s-tomcat.tar [email protected]:/root/
node节点导入镜像到docker
注意:因为是用本地镜像,所以有多个node的话,每个node都要手动导入镜像
docker load -i k8s-nginx.tar
docker load -i k8s-tomcat.tar
在k8s-master制作tomcat.yml容器文件
vi tomcat.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: k8s-tomcat-1
name: k8s-tomcat-1-deployment
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: k8s-tomcat-1
template:
metadata:
labels:
app: k8s-tomcat-1
spec:
containers:
- name: k8s-tomcat-1-container
image: test/tomcat:v1.0
imagePullPolicy: Never
ports:
- containerPort: 8080
protocol: TCP
name: k8s-tomcat-1
volumeMounts:
- mountPath: /usr/local/tomcat/webapps/ROOT
name: tomcat-nfs
volumes:
- name: tomcat-nfs
nfs:
server: 192.168.116.130
path: /data/k8s/tomcat
---
apiVersion: v1
kind: Service
metadata:
labels:
app: k8s-tomcat-1
name: k8s-tomcat-1-service
namespace: default
spec:
ports:
- name: k8s-tomcat-1
port: 8080
selector:
app: k8s-tomcat-1
在k8s-master制作nginx.yml容器文件
vi nginx.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
name: nginx-deployment
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx-container
image: test/nginx:v1.0
imagePullPolicy: Never
ports:
- containerPort: 80
protocol: TCP
name: httpd
volumeMounts:
- mountPath: /app/nginx/html
name: nginx-nfs
volumes:
- name: nginx-nfs
nfs:
server: 192.168.116.130
path: /data/k8s/nginx
---
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx
name: k8s-nginx-service
namespace: default
spec:
type: NodePort
ports:
- name: httpd
port: 80
protocol: TCP
targetPort: 80
nodePort: 30080
selector:
app: nginx
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 10800
master运行配置文件
kubectl apply -f tomcat.yml
kubectl apply -f nginx.yml
kubectl get pod -A -o wide 查看容器状态和node节点IP
kubectl get service -o wide 查看容器映射端口
在node上查看挂载情况,可以看到容器的挂载目录和NFS的共享目录
df -h 查看挂载情况
ss -tnl 查看端口是否打开
在NFS服务器上查看目录默认都是空的
ls /data/k8s
ls /data/k8s/nginx/
ls /data/k8s/tomcat/
在master修改tomcat容器内容,测试tomcat容器的目录是否映射到了NFS服务器
kubectl get pod -A
kubectl exec -it k8s-tomcat-1-deployment-78c649d77b-dstrz /bin/bash
跟NFS服务器一样时空目录
ls /usr/local/tomcat/webapps/ROOT/
vi /usr/local/tomcat/webapps/ROOT/index.jsp
<%@ page language="java" contentType="text/html; charset=UTF-8"
pageEncoding="UTF-8"%>
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>tomcat-test</title>
</head>
<body>
<h1>tomcat-test</h1>
<%
out.println("test");
%>
<br>
<%=request.getRequestURL()%>
</body>
</html>
cat /usr/local/tomcat/webapps/ROOT/index.jsp
在master修改nginx容器内容,测试nginx容器的目录是否映射到了NFS服务器
kubectl get pod -A
kubectl exec -it nginx-deployment-64d8d89bd4-ggf75 /bin/bash
可以看到nginx容器和NFS服务器共享的目录内容是一致的
ls /app/nginx/html/
echo nginx >/app/nginx/html/index.html
cat /app/nginx/html/index.html
ls /data/k8s/tomcat/
ls /data/k8s/nginx/
curl http://192.168.116.133:30080/index.html
curl http://192.168.116.133:30080/tomcat/index.jsp
删除容器,可以看到容器已经删除
kubectl delete -f tomcat.yml
kubectl delete -f nginx.yml
kubectl get pod -A
ls /data/k8s/tomcat/
ls /data/k8s/nginx/
因为nginx和tomcat容器删除了,所以访问不了
curl http://192.168.116.133:30080/index.html
curl http://192.168.116.133:30080/tomcat/index.jsp
在master重新生成容器,测试数据是否丢失
kubectl apply -f tomcat.yml
kubectl apply -f nginx.yml
kubectl get pod -A
在客户端重新访问,可以看到能访问,证明容器重建数据不会丢失
curl http://192.168.116.133:30080/index.html
curl http://192.168.116.133:30080/tomcat/index.jsp