k8s内网安装相关记录
修改网卡配置DNS
/etc/sysconfig/network-scripts/ifcfg-ens192
sudo service network restart
在有网的服务器执行
1, Yum服务器安装
mkdir /apps/localrpm/
sudo yum install yum-utils
如果只想通过yum下载软件的安装包,但是不需要进行安装的话,可以使用 yumdownloader 命令
yumdownloader --resolve --destdir=/apps/localrpm/ telnet yum-utils createrepo2, 安装createrepo，并生成库
sudo yum install createrepo -y
sudo createrepo /apps/localrpm/
3, yumdownloader 这个插件是专门用来下载安装包的
下面准备docker安装包, 先添加稳定仓库
添加稳定仓库
sudo yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo开始下载依赖及安装包
yumdownloader --resolve --destdir=/apps/localrpm/ device-mapper-persistent-data lvm2 docker-ce docker-ce-cli containerd.io
缺少的包在主控机准备好
yumdownloader --resolve --destdir=/apps/localrpm/ libselinux-python libsepol libselinux-utils libselinux
yumdownloader --resolve --destdir=/apps/localrpm/ device-mapper-libs
yumdownloader --resolve --destdir=/apps/localrpm/ ebtables
yumdownloader --resolve --destdir=/apps/localrpm/ nss
yumdownloader --resolve --destdir=/apps/localrpm/ python-httplib2
yumdownloader --resolve --destdir=/apps/localrpm/ openssl
yumdownloader --resolve --destdir=/apps/localrpm/ curl
yumdownloader --resolve --destdir=/apps/localrpm/ rsync
yumdownloader --resolve --destdir=/apps/localrpm/ bash-completion
yumdownloader --resolve --destdir=/apps/localrpm/ socat
yumdownloader --resolve --destdir=/apps/localrpm/ unzip
yumdownloader --resolve --destdir=/apps/localrpm/ screen
yumdownloader --resolve --destdir=/apps/localrpm/ wget
yumdownloader --resolve --destdir=/apps/localrpm/ nfs-utils
yumdownloader --resolve --destdir=/apps/localrpm/ nc
yumdownloader --resolve --destdir=/apps/localrpm/ nss-softokn
yumdownloader --resolve --destdir=/apps/localrpm/ nss-util
yumdownloader --resolve --destdir=/apps/localrpm/ NetworkManager-libnm
yumdownloader --resolve --destdir=/apps/localrpm/ nspr
yumdownloader --resolve --destdir=/apps/localrpm/ mozjs17
yumdownloader --resolve --destdir=/apps/localrpm/ nss-tools
yumdownloader --resolve --destdir=/apps/localrpm/ nss-pem
yumdownloader --resolve --destdir=/apps/localrpm/ NetworkManager
yumdownloader --resolve --destdir=/apps/localrpm/ systemtap-client
yumdownloader --resolve --destdir=/apps/localrpm/ openldap
yumdownloader --resolve --destdir=/apps/localrpm/ nss-softokn-freebl
yumdownloader --resolve --destdir=/apps/localrpm/ nfs-utils
yumdownloader --resolve --destdir=/apps/localrpm/ bind-utils
yumdownloader --resolve --destdir=/apps/localrpm/ bind-libs-lite
yumdownloader --resolve --destdir=/apps/localrpm/ bind-libs
yumdownloader --resolve --destdir=/apps/localrpm/ bind-license
yumdownloader --resolve --destdir=/apps/localrpm/ dhclient
yumdownloader --resolve --destdir=/apps/localrpm/ dhcp-libs
yumdownloader --resolve --destdir=/apps/localrpm/ dhcp-common
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
yumdownloader --resolve --destdir=/apps/localrpm/
下载完成后,要安装一下repo
sudo createrepo /apps/localrpm/
使用源的地方更新一下cache
sudo yum makecache
下载ngnix安装包, 本地配置nginx yum 源
sudo vi /etc/yum.repos.d/nginx.repo , 目前用的redhat7,用如下配置
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/rhel/7/$basearch/
gpgcheck=0
enabled=1然后开始下载nginx包
yumdownloader --resolve --destdir=/apps/localrpm/ nginx
要想安装,首先解决yum本地源的问题, 下面开始配置本地yum源, 上面的步骤我们把rpm包下载到目录 /apps/localrpm
下面开始,安装一个ngnix服务器
第一台主机 模拟本地安装
配置yum仓库
sudo vi /etc/yum.repos.d/localRPM.repo
[local]
name=localRMP
baseurl=file:///apps/localrpm
gpgcheck=0
gpckey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release #注意实际文件差异
enabled=1
加载测试
先把其它源全部去掉
sudo mkdir /etc/yum.repos.d/bak
sudo mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/bak
sudo mv /etc/yum.repos.d/bak/localRPM.repo /etc/yum.repos.d/
sudo yum clean all #清空yum缓存
sudo rm -rf /var/cache/yum/*
sudo yum makecache #加载可用yum元数据
sudo yum repolist #查看yum源rpm包的具体情况
首先安装 nginx
sudo yum install nginx
sudo cp /etc/nginx/conf.d/default.conf /etc/nginx/conf.d/sharefile.conf
sudo vi /etc/nginx/conf.d/sharefile.conf
写入内容
server {
autoindex on;
autoindex_exact_size on;
autoindex_localtime on;
listen 8080 default_server;
listen [::]:8080 default_server;
server_name share;
root /apps/localrpm;
}
启动
sudo /sbin/nginx
查看 8080端口
netstat -an|grep 8080
访问网址
http://192.168.106.55:8080/
修改远程机器yum源为ngnix源
远程yum源
sudo vi /etc/yum.repos.d/localRPM.repo
[local]
name=localRMP
baseurl=http://192.168.106.55:8080
gpgcheck=0
gpckey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release #注意实际文件差异
enabled=1
主控机上安装 docker
sudo yum install docker-ce docker-ce-cli containerd.io
启动
sudo systemctl start docker
修改docker image存储位置
首先停止doker
sudo systemctl stop docker
创建目录 sudo mkdir /apps/docker
sudo mkdir /etc/systemd/system/docker.service.d
sudo vi /etc/systemd/system/docker.service.d/devicemapper.conf
写入内容
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --graph=/apps/docker
重启docker
sudo systemctl daemon-reload
sudo systemctl restart docker
sudo systemctl enable docker
sudo docker info
确定没问题后,删除原目录 rm -rf /var/lib/docker安装 docker-compose
curl -L https://github.com/docker/compose/releases/download/1.24.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
复制一份到/apps/soft目录方便后面copy到离线机器
sudo cp /usr/local/bin/docker-compose /apps/soft
sudo chmod +x /usr/local/bin/docker-compose
如果想用sudo docker-compose --version
必须移动到 /usr/bin/目录
sudo mv /usr/local/bin/docker-compose /usr/bin/docker-compose
否则报错
sudo: docker-compose: command not found
#查看版本
docker-compose version安装Harbor
下载离线安装包
sudo cd /apps/soft
wget https://storage.googleapis.com/harbor-releases/release-1.8.0/harbor-offline-installer-v1.8.2.tgz
sudo tar xvf harbor-offline-installer-v1.8.2.tgz
修改解压后目录里面的文件 harbor.yml
sudo vi harbor.yml
修改
hostname: 192.168.106.55
port: 8081
data_volume: /data/harbor
log
location: /data/harbor/log
开始安装
cd /apps/soft/harbor
sudo ./prepare
sudo ./install.sh --with-chartmuseum
安装好后访问网址
http://192.168.106.55:8081/
登陆账号
admin
密码 Harbor12345
停止harbor
cd /apps/soft/harbor
sudo docker-compose down -v
重启
sudo docker-compose up -d
启动好后,登陆harbor, 然后创建目标仓库进行数据同步
仓库管理 -> 新建目录 -> Harbor类型仓库 http://reg.k8s.rf.io
同步管理 -> 新建规则 -> Pull-based -> 手动同步
Bind安装
sudo docker pull sameersbn/bind
sudo docker run -d --name=bind --dns=192.168.120.2 --publish=192.168.106.55:53:53/udp --publish=192.168.106.55:10000:10000 --volume=/srv/docker/bind:/data/bind --env='ROOT_PASSWORD=1qaz@WSX' sameersbn/bind:latest
访问网址
https://192.168.106.55:10000/
账号:root
密码:1qaz@WSX
配置Bind
Servers - BIND DNS Server - Create master zone - 填写 Domain name / Network 和 Email address - Create - Return to zone list
进入新创建的Zone - Address - 填写name 和 Address - Return to zone list - Apply configuration
测试连通性
dig @192.168.106.55 a.k8s.rf.io
安装k8s,使用 kubespray进行安装
https://github.com/kubernetes-sigs/kubespray
修改 D:\k8s\railway\group_vars\all\all.yml , 在此之前先把harbor仓库需要的以下依赖同步过来
docker_rh_repo_base_url: 'http://192.168.106.55:8080/x86_64/7Server'
docker_rh_repo_gpgkey: 'http://192.168.106.55:8080/gpg'
dockerproject_rh_repo_base_url: 'http://192.168.106.55:8080/x86_64/7Server'
docker_rh_repo_gpgkey: 'http://192.168.106.55:8080/gpg'
etcd_image_repo: "192.168.106.55:8081/coreos/etcd"
修改 D:\k8s\railway\group_vars\k8s-cluster\k8s-cluster.yml , 相关文件放至nginx目录或harbor
# Download URLs
kubeadm_download_url: "http://192.168.106.55:8080/downloads/kubeadm"
hyperkube_download_url: "http://192.168.106.55:8080/downloads/hyperkube"
cni_download_url: "http://192.168.106.55:8080/downloads/cni-plugins-{{ image_arch }}-{{ cni_version }}.tgz"
nginx_image_repo: 192.168.106.55:8081/common/nginx
dnsmasqautoscaler_image_repo: "{{ kube_image_repo }}/cluster-proportional-autoscaler-{{ image_arch }}"
dnsautoscaler_image_repo: "{{ kube_image_repo }}/cluster-proportional-autoscaler-{{ image_arch }}"
pod_infra_image_repo: "{{ kube_image_repo }}/pause-{{ image_arch }}"
dashboard_image_repo: "{{ kube_image_repo }}/kubernetes-dashboard-{{ image_arch }}"
flannel_image_repo: "192.168.106.55:8081/coreos/flannel"
etcd_image_repo: "192.168.106.55:8081/coreos/etcd"
calicoctl_image_repo: "192.168.106.55:8081/calico/ctl"
calico_node_image_repo: "192.168.106.55:8081/calico/node"
calico_cni_image_repo: "192.168.106.55:8081/calico/cni"
calico_policy_image_repo: "192.168.106.55:8081/calico/kube-controllers"
calico_rr_image_repo: "192.168.106.55:8081/calico/routereflector"
coredns_image_repo: "192.168.106.55:8081/coredns/coredns"修改 host.ini
[all]
tck8s-02 ansible_host=192.168.106.56 etcd_member_name=etcd1
tck8s-03 ansible_host=192.168.106.57 etcd_member_name=etcd2
tck8s-04 ansible_host=192.168.106.58 etcd_member_name=etcd3
tck8s-05 ansible_host=192.168.106.59
[kube-master]
tck8s-02
tck8s-03
tck8s-04
[etcd]
tck8s-02
tck8s-03
tck8s-04
[kube-node]
tck8s-02
tck8s-03
tck8s-04
tck8s-05
[k8s-cluster:children]
kube-master
kube-node
修改成需要的用户及互信的文件绝对路径
remote_user = adaas
private_key_file = /home/adaas/.ssh/id_rsa
配置互信
ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.106.56
ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.106.57
ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.106.58
ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.106.59
开始安装
cd /apps/soft/kubespray-2.8.3
ansible-playbook -i inventory/railway/hosts.ini cluster.yml
关联集群参考
https://blog.csdn.net/gs80140/article/details/94599398
安装NFS 参考 https://blog.csdn.net/gs80140/article/details/93199622
部署 nfs-disk , 使用wayne平台部署 , 填写下面的参数
image.repository
192.168.106.55:8081/external_storage/nfs-client-provisioner
nfs.server
192.168.106.56
nfs.path
/data/nfs