ansible-playbook部署K8S集群
通过ansible-playbook，以Kubeadm方式部署K8S集群（一主多从）。
kubernetes安装目录： /etc/kubernetes/
KubeConfig： ~/.kube/config
Version： v1.18.3
主机说明：
| 系统 | ip | 角色 | cpu | 内存 | hostname |
|---|---|---|---|---|---|
| CentOS 7.8 | 192.168.30.128 | master | >=2 | >=2G | master |
| CentOS 7.8 | 192.168.30.129 | node | >=2 | >=2G | node1 |
| CentOS 7.8 | 192.168.30.130 | node | >=2 | >=2G | node2 |
| CentOS 7.8 | 192.168.30.131 | node | >=2 | >=2G | node3 |
准备
- 将所有部署k8s集群的主机分组：
# vim /etc/ansible/hosts
[master]
192.168.30.128 hostname=master
[node]
192.168.30.129 hostname=node1
192.168.30.130 hostname=node2
192.168.30.131 hostname=node3
- 创建管理目录：
mkdir -p k8s/roles/{docker_install,master_install,node_install,addons_install}/{files,handlers,meta,tasks,templates,vars}
cd k8s/
说明：
files：存放需要同步到异地服务器的源码文件及配置文件；
handlers：当资源发生变化时需要进行的操作，若没有此目录可以不建或为空；
meta：存放说明信息、说明角色依赖等信息，可留空；
tasks：K8S 安装过程中需要进行执行的任务；
templates：用于执行 K8S 安装的模板文件，一般为脚本；
vars：本次安装定义的变量
tree .
.
├── k8s.yml
└── roles
├── addons_install
│ ├── files
│ ├── handlers
│ ├── meta
│ ├── tasks
│ │ ├── calico.yml
│ │ ├── ingress.yml
│ │ └── main.yml
│ ├── templates
│ │ ├── calico-rbac.yaml
│ │ ├── calico.yaml
│ │ └── ingress-nginx.yaml
│ └── vars
│ └── main.yml
├── docker_install
│ ├── files
│ ├── handlers
│ ├── meta
│ ├── tasks
│ │ ├── install.yml
│ │ ├── main.yml
│ │ └── prepare.yml
│ ├── templates
│ │ ├── daemon.json
│ │ ├── install.sh
│ │ ├── kubernetes.conf
│ │ └── kubernetes.repo
│ └── vars
│ └── main.yml
├── master_install
│ ├── files
│ ├── handlers
│ ├── meta
│ ├── tasks
│ │ ├── install.yml
│ │ └── main.yml
│ ├── templates
│ │ └── kubeadm-config.yaml
│ └── vars
│ └── main.yml
└── node_install
├── files
├── handlers
├── meta
├── tasks
│ ├── install.yml
│ └── main.yml
├── templates
└── vars
└── main.yml
29 directories, 23 files
- 创建安装入口文件，用来调用roles：
vim k8s.yml
---
- hosts: all
remote_user: root
gather_facts: True
roles:
- docker_install
- hosts: master
remote_user: root
gather_facts: True
roles:
- master_install
- hosts: node
remote_user: root
gather_facts: True
roles:
- node_install
- hosts: master
remote_user: root
gather_facts: True
roles:
- addons_install
docker部分
- 创建docker入口文件，用来调用docker_install：
vim docker.yml
- hosts: all
remote_user: root
gather_facts: True
roles:
- docker_install
- 创建变量：
vim roles/docker_install/vars/main.yml
SOURCE_DIR: /software
VERSION: 1.18.3
- 创建模板文件：
docker配置daemon.json
vim roles/docker_install/templates/daemon.json
{
"registry-mirrors": ["http://f1361db2.m.daocloud.io"],
"exec-opts":["native.cgroupdriver=systemd"]
}
系统环境kubernetes.conf
vim roles/docker_install/templates/kubernetes.conf
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_watches=89100
repo文件kubernetes.repo
vim roles/docker_install/templates/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
docker-py安装脚本install.sh
vim roles/docker_install/templates/install.sh
#!/bin/bash
loop_exec() {
CMD=$1
while :; do
${CMD}
if [ $? -eq 0 ] ; then
break;
fi
done
}
main() {
loop_exec "yum install -y python python-pip"
loop_exec "pip install --upgrade pip"
loop_exec "pip install docker-py"
}
main
- 环境准备prepare.yml：
vim roles/docker_install/tasks/prepare.yml
- name: 关闭firewalld
service: name=firewalld state=stopped enabled=no
- name: 临时关闭 selinux
shell: "setenforce 0"
failed_when: false
- name: 永久关闭 selinux
lineinfile:
dest: /etc/selinux/config
regexp: "^SELINUX="
line: "SELINUX=disabled"
- name: 添加EPEL仓库
yum: name=epel-release state=latest
- name: 安装常用软件包
yum:
name:
- vim
- lrzsz
- net-tools
- wget
- curl
- bash-completion
- rsync
- gcc
- unzip
- git
- iptables
- conntrack
- ipvsadm
- ipset
- jq
- sysstat
- libseccomp
state: latest
- name: 更新系统
shell: "yum update -y --exclude kubeadm,kubelet,kubectl"
ignore_errors: yes
args:
warn: False
- name: 配置iptables
shell: "iptables -F && iptables -X && iptables -F -t nat && iptables -X -t nat && iptables -P FORWARD ACCEPT"
- name: 关闭swap
shell: "swapoff -a && sed -i '/swap/s/^\\(.*\\)$/#\\1/g' /etc/fstab"
- name: 系统配置
template: src=kubernetes.conf dest=/etc/sysctl.d/kubernetes.conf
- name: 加载br_netfilter
shell: "modprobe br_netfilter"
- name: 生效配置
shell: "sysctl -p /etc/sysctl.d/kubernetes.conf"
- docker安装install.yml：
vim roles/docker_install/tasks/install.yml
- name: 创建software目录
file: name={{ SOURCE_DIR }} state=directory
- name: 更改hostname
raw: "echo {{ hostname }} > /etc/hostname"
- name: 更改生效
shell: "hostname {{ hostname }}"
- name: 设置本地dns
shell: "if [ `grep '{{ ansible_ssh_host }} {{ hostname }}' /etc/hosts |wc -l` -eq 0 ]; then echo {{ ansible_ssh_host }} {{ hostname }} >> /etc/hosts; fi"
- name: 下载repo文件
shell: "if [ ! -f /etc/yum.repos.d/docker.repo ]; then curl http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -o /etc/yum.repos.d/docker.repo; fi"
- name: 生成缓存
shell: "yum makecache fast"
args:
warn: False
- name: 安装docker-ce
yum:
name: docker-ce
state: latest
- name: 启动docker并开机启动
service:
name: docker
state: started
enabled: yes
- name: 配置docker
template: src=daemon.json dest=/etc/docker/daemon.json
- name: 重启docker
service:
name: docker
state: restarted
- name: 配置kubernetes源
template: src=kubernetes.repo dest=/etc/yum.repos.d/kubernetes.repo
- name: 安装kubernetes-cni
yum:
name: kubernetes-cni
state: latest
- name: 安装kubeadm、kubelet、kubectl
shell: "yum install -y kubeadm-{{ VERSION }} kubelet-{{ VERSION }} kubectl-{{ VERSION }} --disableexcludes=kubernetes"
args:
warn: False
- name: 启动kubelet并开机启动
service:
name: kubelet
state: started
enabled: yes
- name: 拷贝脚本
template: src=install.sh dest={{ SOURCE_DIR }} mode=0755
- name: 安装docker-py
script: "{{ SOURCE_DIR }}/install.sh"
- 引用文件main.yml：
vim roles/docker_install/tasks/main.yml
- include: prepare.yml
- include: install.yml
master部分
- 创建master入口文件，用来调用master_install：
vim master.yml
- hosts: master
remote_user: root
gather_facts: True
roles:
- master_install
- 创建变量：
vim roles/master_install/vars/main.yml
SOURCE_DIR: /software
VERSION: v1.18.3
POD_CIDR: 172.10.0.0/16
MASTER_IP: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}"
- 创建模板文件：
kubeadm配置文件 kubeadm-config.yaml
vim roles/master_install/templates/kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: "{{ VERSION }}"
controlPlaneEndpoint: "{{ MASTER_IP }}:6443"
networking:
podSubnet: "{{ POD_CIDR }}"
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
- 集群初始化install.yml：
vim roles/master_install/tasks/install.yml
- name: 拷贝kubeadm配置文件
template: src=kubeadm-config.yaml dest={{ SOURCE_DIR }}
- name: 集群初始化准备1
shell: "swapoff -a && kubeadm reset -f"
- name: 集群初始化准备2
shell: "systemctl daemon-reload && systemctl restart kubelet"
- name: 集群初始化准备3
shell: "iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X"
- name: 拉取镜像
shell: "kubeadm config images pull --kubernetes-version={{ VERSION }} --image-repository=registry.aliyuncs.com/google_containers"
- name: 集群初始化
shell: "kubeadm init --config={{ SOURCE_DIR }}/kubeadm-config.yaml --upload-certs &>{{ SOURCE_DIR }}/token"
- name: 获取master的token
shell: "grep -B2 'control-plane --certificate-key' {{ SOURCE_DIR }}/token > {{ SOURCE_DIR }}/master.sh"
- name: 获取node的token
shell: "grep -A1 'kubeadm join' {{ SOURCE_DIR }}/token |tail -2 > {{ SOURCE_DIR }}/node.sh"
- name: 分发master.sh
shell: "ansible master -m copy -a 'src={{ SOURCE_DIR }}/master.sh dest={{ SOURCE_DIR }} mode=0755'"
args:
warn: False
- name: 分发node.sh
shell: "ansible node -m copy -a 'src={{ SOURCE_DIR }}/node.sh dest={{ SOURCE_DIR }} mode=0755'"
args:
warn: False
- name: 创建 $HOME/.kube 目录
file: name=$HOME/.kube state=directory
- name: 拷贝KubeConfig
copy: src=/etc/kubernetes/admin.conf dest=$HOME/.kube/config owner=root group=root
- name: kubectl命令补全1
shell: "kubectl completion bash > $HOME/.kube/completion.bash.inc"
- name: kubectl命令补全2
shell: "if [ `grep 'source $HOME/.kube/completion.bash.inc' $HOME/.bash_profile |wc -l` -eq 0 ]; then echo 'source $HOME/.kube/completion.bash.inc' >> $HOME/.bash_profile; fi"
- name: 生效配置
shell: "source $HOME/.bash_profile"
ignore_errors: yes
- 引用文件main.yml：
vim roles/master_install/tasks/main.yml
- include: install.yml
node部分
- 创建node入口文件，用来调用node_install：
vim node.yml
- hosts: node
remote_user: root
gather_facts: True
roles:
- node_install
- 创建变量：
vim roles/node_install/vars/main.yml
SOURCE_DIR: /software
- 添加node到集群install.yml：
vim roles/node_install/tasks/install.yml
- name: 集群初始化准备1
shell: "swapoff -a && kubeadm reset -f"
- name: 集群初始化准备2
shell: "systemctl daemon-reload && systemctl restart kubelet"
- name: 集群初始化准备3
shell: "iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X"
- name: 集群增加node
script: "{{ SOURCE_DIR }}/node.sh"
- name: 删除node的token
file: name={{ SOURCE_DIR }}/node.sh state=absent
- 引用文件main.yml：
vim roles/node_install/tasks/main.yml
- include: install.yml
addons部分
- 创建addons入口文件，用来调用addons_install：
vim addons.yml
- hosts: master
remote_user: root
gather_facts: True
roles:
- addons_install
- 创建变量：
vim roles/addons_install/vars/main.yml
SOURCE_DIR: /software
POD_CIDR: 172.10.0.0/16
CALICO_VER: v3.15.1
BACKEND_VER: 1.5
INGRESS_VER: 0.19.0
- 创建模板文件：
calico rbac配置文件 calico-rbac.yaml
vim roles/addons_install/templates/calico-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: calico-kube-controllers
namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-kube-controllers
rules:
- apiGroups: [""]
resources:
- nodes
verbs:
- watch
- list
- get
- apiGroups: [""]
resources:
- pods
verbs:
- get
- apiGroups: ["crd.projectcalico.org"]
resources:
- ippools
verbs:
- list
- apiGroups: ["crd.projectcalico.org"]
resources:
- blockaffinities
- ipamblocks
- ipamhandles
verbs:
- get
- list
- create
- update
- delete
- apiGroups: ["crd.projectcalico.org"]
resources:
- hostendpoints
verbs:
- get
- list
- create
- update
- delete
- apiGroups: ["crd.projectcalico.org"]
resources:
- clusterinformations
verbs:
- get
- create
- update
- apiGroups: ["crd.projectcalico.org"]
resources:
- kubecontrollersconfigurations
verbs:
- get
- create
- update
- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-kube-controllers
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: calico-kube-controllers
subjects:
- kind: ServiceAccount
name: calico-kube-controllers
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: calico-node
namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-node
rules:
- apiGroups: [""]
resources:
- pods
- nodes
- namespaces
verbs:
- get
- apiGroups: [""]
resources:
- endpoints
- services
verbs:
- watch
- list
- get
- apiGroups: [""]
resources:
- configmaps
verbs:
- get
- apiGroups: [""]
resources:
- nodes/status
verbs:
- patch
- update
- apiGroups: ["networking.k8s.io"]
resources:
- networkpolicies
verbs:
- watch
- list
- apiGroups: [""]
resources:
- pods
- namespaces
- serviceaccounts
verbs:
- list
- watch
- apiGroups: [""]
resources:
- pods/status
verbs:
- patch
- apiGroups: ["crd.projectcalico.org"]
resources:
- globalfelixconfigs
- felixconfigurations
- bgppeers
- globalbgpconfigs
- bgpconfigurations
- ippools
- ipamblocks
- globalnetworkpolicies
- globalnetworksets
- networkpolicies
- networksets
- clusterinformations
- hostendpoints
- blockaffinities
verbs:
- get
- list
- watch
- apiGroups: ["crd.projectcalico.org"]
resources:
- ippools
- felixconfigurations
- clusterinformations
verbs:
- create
- update
- apiGroups: [""]
resources:
- nodes
verbs:
- get
- list
- watch
- apiGroups: ["crd.projectcalico.org"]
resources:
- bgpconfigurations
- bgppeers
verbs:
- create
- update
- apiGroups: ["crd.projectcalico.org"]
resources:
- blockaffinities
- ipamblocks
- ipamhandles
verbs:
- get
- list
- create
- update
- delete
- apiGroups: ["crd.projectcalico.org"]
resources:
- ipamconfigs
verbs:
- get
- apiGroups: ["crd.projectcalico.org"]
resources:
- blockaffinities
verbs:
- watch
- apiGroups: ["apps"]
resources:
- daemonsets
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: calico-node
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: calico-node
subjects:
- kind: ServiceAccount
name: calico-node
namespace: kube-system
calico配置文件 calico.yaml
vim roles/addons_install/templates/calico.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: calico-config
namespace: kube-system
data:
typha_service_name: "none"
calico_backend: "bird"
veth_mtu: "1440"
cni_network_config: |-
{
"name": "k8s-pod-network",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "calico",
"log_level": "info",
"datastore_type": "kubernetes",
"nodename": "__KUBERNETES_NODE_NAME__",
"mtu": __CNI_MTU__,
"ipam": {
"type": "calico-ipam"
},
"policy": {
"type": "k8s"
},
"kubernetes": {
"kubeconfig": "__KUBECONFIG_FILEPATH__"
}
},
{
"type": "portmap",
"snat": true,
"capabilities": {"portMappings": true}
},
{
"type": "bandwidth",
"capabilities": {"bandwidth": true}
}
]
}
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: bgpconfigurations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BGPConfiguration
listKind: BGPConfigurationList
plural: bgpconfigurations
singular: bgpconfiguration
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
asNumber:
format: int32
type: integer
logSeverityScreen:
type: string
nodeToNodeMeshEnabled:
type: boolean
serviceClusterIPs:
items:
properties:
cidr:
type: string
type: object
type: array
serviceExternalIPs:
items:
properties:
cidr:
type: string
type: object
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: bgppeers.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BGPPeer
listKind: BGPPeerList
plural: bgppeers
singular: bgppeer
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
asNumber:
format: int32
type: integer
node:
type: string
nodeSelector:
type: string
peerIP:
type: string
peerSelector:
type: string
required:
- asNumber
- peerIP
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: blockaffinities.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BlockAffinity
listKind: BlockAffinityList
plural: blockaffinities
singular: blockaffinity
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
cidr:
type: string
deleted:
type: string
node:
type: string
state:
type: string
required:
- cidr
- deleted
- node
- state
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: clusterinformations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: ClusterInformation
listKind: ClusterInformationList
plural: clusterinformations
singular: clusterinformation
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
calicoVersion:
type: string
clusterGUID:
type: string
clusterType:
type: string
datastoreReady:
type: boolean
variant:
type: string
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: felixconfigurations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: FelixConfiguration
listKind: FelixConfigurationList
plural: felixconfigurations
singular: felixconfiguration
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
bpfConnectTimeLoadBalancingEnabled:
type: boolean
bpfDataIfacePattern:
type: string
bpfDisableUnprivileged:
type: boolean
bpfEnabled:
type: boolean
bpfExternalServiceMode:
type: string
bpfKubeProxyEndpointSlicesEnabled:
type: boolean
bpfKubeProxyIptablesCleanupEnabled:
type: boolean
bpfKubeProxyMinSyncPeriod:
type: string
bpfLogLevel:
type: string
chainInsertMode:
type: string
dataplaneDriver:
type: string
debugDisableLogDropping:
type: boolean
debugMemoryProfilePath:
type: string
debugSimulateCalcGraphHangAfter:
type: string
debugSimulateDataplaneHangAfter:
type: string
defaultEndpointToHostAction:
type: string
deviceRouteProtocol:
type: integer
deviceRouteSourceAddress:
type: string
disableConntrackInvalidCheck:
type: boolean
endpointReportingDelay:
type: string
endpointReportingEnabled:
type: boolean
externalNodesList:
items:
type: string
type: array
failsafeInboundHostPorts:
items:
properties:
port:
type: integer
protocol:
type: string
required:
- port
- protocol
type: object
type: array
failsafeOutboundHostPorts:
items:
properties:
port:
type: integer
protocol:
type: string
required:
- port
- protocol
type: object
type: array
genericXDPEnabled:
type: boolean
healthEnabled:
type: boolean
healthHost:
type: string
healthPort:
type: integer
interfaceExclude:
type: string
interfacePrefix:
type: string
ipipEnabled:
type: boolean
ipipMTU:
type: integer
ipsetsRefreshInterval:
type: string
iptablesBackend:
type: string
iptablesFilterAllowAction:
type: string
iptablesLockFilePath:
type: string
iptablesLockProbeInterval:
type: string
iptablesLockTimeout:
type: string
iptablesMangleAllowAction:
type: string
iptablesMarkMask:
format: int32
type: integer
iptablesNATOutgoingInterfaceFilter:
type: string
iptablesPostWriteCheckInterval:
type: string
iptablesRefreshInterval:
type: string
ipv6Support:
type: boolean
kubeNodePortRanges:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
logFilePath:
type: string
logPrefix:
type: string
logSeverityFile:
type: string
logSeverityScreen:
type: string
logSeveritySys:
type: string
maxIpsetSize:
type: integer
metadataAddr:
type: string
metadataPort:
type: integer
natOutgoingAddress:
type: string
natPortRange:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
netlinkTimeout:
type: string
openstackRegion:
type: string
policySyncPathPrefix:
type: string
prometheusGoMetricsEnabled:
type: boolean
prometheusMetricsEnabled:
type: boolean
prometheusMetricsHost:
type: string
prometheusMetricsPort:
type: integer
prometheusProcessMetricsEnabled:
type: boolean
removeExternalRoutes:
type: boolean
reportingInterval:
type: string
reportingTTL:
type: string
routeRefreshInterval:
type: string
routeSource:
type: string
routeTableRange:
properties:
max:
type: integer
min:
type: integer
required:
- max
- min
type: object
sidecarAccelerationEnabled:
type: boolean
usageReportingEnabled:
type: boolean
usageReportingInitialDelay:
type: string
usageReportingInterval:
type: string
useInternalDataplaneDriver:
type: boolean
vxlanEnabled:
type: boolean
vxlanMTU:
type: integer
vxlanPort:
type: integer
vxlanVNI:
type: integer
wireguardEnabled:
type: boolean
wireguardInterfaceName:
type: string
wireguardListeningPort:
type: integer
wireguardMTU:
type: integer
wireguardRoutingRulePriority:
type: integer
xdpEnabled:
type: boolean
xdpRefreshInterval:
type: string
required:
- bpfLogLevel
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: globalnetworkpolicies.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: GlobalNetworkPolicy
listKind: GlobalNetworkPolicyList
plural: globalnetworkpolicies
singular: globalnetworkpolicy
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
applyOnForward:
type: boolean
doNotTrack:
type: boolean
egress:
items:
properties:
action:
type: string
destination:
properties:
namespaceSelector:
type: string
nets:
items:
type: string
type: array
notNets:
items:
type: string
type: array
notPorts:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
type: string
ports:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
type: string
serviceAccounts:
properties:
names:
items:
type: string
type: array
selector:
type: string
type: object
type: object
http:
properties:
methods:
items:
type: string
type: array
paths:
items:
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
properties:
code:
type: integer
type:
type: integer
type: object
ipVersion:
type: integer
metadata:
properties:
annotations:
additionalProperties:
type: string
type: object
type: object
notICMP:
properties:
code:
type: integer
type:
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
source:
properties:
namespaceSelector:
type: string
nets:
items:
type: string
type: array
notNets:
items:
type: string
type: array
notPorts:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
type: string
ports:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
type: string
serviceAccounts:
properties:
names:
items:
type: string
type: array
selector:
type: string
type: object
type: object
required:
- action
type: object
type: array
ingress:
items:
properties:
action:
type: string
destination:
properties:
namespaceSelector:
type: string
nets:
items:
type: string
type: array
notNets:
items:
type: string
type: array
notPorts:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
type: string
ports:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
type: string
serviceAccounts:
properties:
names:
items:
type: string
type: array
selector:
type: string
type: object
type: object
http:
properties:
methods:
items:
type: string
type: array
paths:
items:
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
properties:
code:
type: integer
type:
type: integer
type: object
ipVersion:
type: integer
metadata:
properties:
annotations:
additionalProperties:
type: string
type: object
type: object
notICMP:
properties:
code:
type: integer
type:
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
source:
properties:
namespaceSelector:
type: string
nets:
items:
type: string
type: array
notNets:
items:
type: string
type: array
notPorts:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
type: string
ports:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
type: string
serviceAccounts:
properties:
names:
items:
type: string
type: array
selector:
type: string
type: object
type: object
required:
- action
type: object
type: array
namespaceSelector:
type: string
order:
type: number
preDNAT:
type: boolean
selector:
type: string
serviceAccountSelector:
type: string
types:
items:
type: string
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: globalnetworksets.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: GlobalNetworkSet
listKind: GlobalNetworkSetList
plural: globalnetworksets
singular: globalnetworkset
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
nets:
items:
type: string
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: hostendpoints.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: HostEndpoint
listKind: HostEndpointList
plural: hostendpoints
singular: hostendpoint
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
expectedIPs:
items:
type: string
type: array
interfaceName:
type: string
node:
type: string
ports:
items:
properties:
name:
type: string
port:
type: integer
protocol:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
required:
- name
- port
- protocol
type: object
type: array
profiles:
items:
type: string
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: ipamblocks.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPAMBlock
listKind: IPAMBlockList
plural: ipamblocks
singular: ipamblock
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
affinity:
type: string
allocations:
items:
type: integer
nullable: true
type: array
attributes:
items:
properties:
handle_id:
type: string
secondary:
additionalProperties:
type: string
type: object
type: object
type: array
cidr:
type: string
deleted:
type: boolean
strictAffinity:
type: boolean
unallocated:
items:
type: integer
type: array
required:
- allocations
- attributes
- cidr
- deleted
- strictAffinity
- unallocated
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: ipamconfigs.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPAMConfig
listKind: IPAMConfigList
plural: ipamconfigs
singular: ipamconfig
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
autoAllocateBlocks:
type: boolean
strictAffinity:
type: boolean
required:
- autoAllocateBlocks
- strictAffinity
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: ipamhandles.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPAMHandle
listKind: IPAMHandleList
plural: ipamhandles
singular: ipamhandle
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
block:
additionalProperties:
type: integer
type: object
handleID:
type: string
required:
- block
- handleID
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: ippools.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPPool
listKind: IPPoolList
plural: ippools
singular: ippool
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
blockSize:
type: integer
cidr:
type: string
disabled:
type: boolean
ipip:
properties:
enabled:
type: boolean
mode:
type: string
type: object
ipipMode:
type: string
nat-outgoing:
type: boolean
natOutgoing:
type: boolean
nodeSelector:
type: string
vxlanMode:
type: string
required:
- cidr
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: kubecontrollersconfigurations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: KubeControllersConfiguration
listKind: KubeControllersConfigurationList
plural: kubecontrollersconfigurations
singular: kubecontrollersconfiguration
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
controllers:
properties:
namespace:
properties:
reconcilerPeriod:
type: string
type: object
node:
properties:
hostEndpoint:
properties:
autoCreate:
type: string
type: object
reconcilerPeriod:
type: string
syncLabels:
type: string
type: object
policy:
properties:
reconcilerPeriod:
type: string
type: object
serviceAccount:
properties:
reconcilerPeriod:
type: string
type: object
workloadEndpoint:
properties:
reconcilerPeriod:
type: string
type: object
type: object
etcdV3CompactionPeriod:
type: string
healthChecks:
type: string
logSeverityScreen:
type: string
required:
- controllers
type: object
status:
properties:
environmentVars:
additionalProperties:
type: string
type: object
runningConfig:
properties:
controllers:
properties:
namespace:
properties:
reconcilerPeriod:
type: string
type: object
node:
properties:
hostEndpoint:
properties:
autoCreate:
type: string
type: object
reconcilerPeriod:
type: string
syncLabels:
type: string
type: object
policy:
properties:
reconcilerPeriod:
type: string
type: object
serviceAccount:
properties:
reconcilerPeriod:
type: string
type: object
workloadEndpoint:
properties:
reconcilerPeriod:
type: string
type: object
type: object
etcdV3CompactionPeriod:
type: string
healthChecks:
type: string
logSeverityScreen:
type: string
required:
- controllers
type: object
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: networkpolicies.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: NetworkPolicy
listKind: NetworkPolicyList
plural: networkpolicies
singular: networkpolicy
scope: Namespaced
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
egress:
items:
properties:
action:
type: string
destination:
properties:
namespaceSelector:
type: string
nets:
items:
type: string
type: array
notNets:
items:
type: string
type: array
notPorts:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
type: string
ports:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
type: string
serviceAccounts:
properties:
names:
items:
type: string
type: array
selector:
type: string
type: object
type: object
http:
properties:
methods:
items:
type: string
type: array
paths:
items:
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
properties:
code:
type: integer
type:
type: integer
type: object
ipVersion:
type: integer
metadata:
properties:
annotations:
additionalProperties:
type: string
type: object
type: object
notICMP:
properties:
code:
type: integer
type:
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
source:
properties:
namespaceSelector:
type: string
nets:
items:
type: string
type: array
notNets:
items:
type: string
type: array
notPorts:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
type: string
ports:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
type: string
serviceAccounts:
properties:
names:
items:
type: string
type: array
selector:
type: string
type: object
type: object
required:
- action
type: object
type: array
ingress:
items:
properties:
action:
type: string
destination:
properties:
namespaceSelector:
type: string
nets:
items:
type: string
type: array
notNets:
items:
type: string
type: array
notPorts:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
type: string
ports:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
type: string
serviceAccounts:
properties:
names:
items:
type: string
type: array
selector:
type: string
type: object
type: object
http:
properties:
methods:
items:
type: string
type: array
paths:
items:
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
properties:
code:
type: integer
type:
type: integer
type: object
ipVersion:
type: integer
metadata:
properties:
annotations:
additionalProperties:
type: string
type: object
type: object
notICMP:
properties:
code:
type: integer
type:
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
source:
properties:
namespaceSelector:
type: string
nets:
items:
type: string
type: array
notNets:
items:
type: string
type: array
notPorts:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
type: string
ports:
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
type: string
serviceAccounts:
properties:
names:
items:
type: string
type: array
selector:
type: string
type: object
type: object
required:
- action
type: object
type: array
order:
type: number
selector:
type: string
serviceAccountSelector:
type: string
types:
items:
type: string
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: networksets.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: NetworkSet
listKind: NetworkSetList
plural: networksets
singular: networkset
scope: Namespaced
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
nets:
items:
type: string
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: calico-node
namespace: kube-system
labels:
k8s-app: calico-node
spec:
selector:
matchLabels:
k8s-app: calico-node
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
template:
metadata:
labels:
k8s-app: calico-node
spec:
nodeSelector:
kubernetes.io/os: linux
hostNetwork: true
tolerations:
- effect: NoSchedule
operator: Exists
- key: CriticalAddonsOnly
operator: Exists
- effect: NoExecute
operator: Exists
serviceAccountName: calico-node
terminationGracePeriodSeconds: 0
priorityClassName: system-node-critical
initContainers:
- name: upgrade-ipam
image: calico/cni:{{ CALICO_VER }}
command: ["/opt/cni/bin/calico-ipam", "-upgrade"]
env:
- name: KUBERNETES_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: CALICO_NETWORKING_BACKEND
valueFrom:
configMapKeyRef:
name: calico-config
key: calico_backend
volumeMounts:
- mountPath: /var/lib/cni/networks
name: host-local-net-dir
- mountPath: /host/opt/cni/bin
name: cni-bin-dir
securityContext:
privileged: true
- name: install-cni
image: calico/cni:{{ CALICO_VER }}
command: ["/install-cni.sh"]
env:
- name: CNI_CONF_NAME
value: "10-calico.conflist"
- name: CNI_NETWORK_CONFIG
valueFrom:
configMapKeyRef:
name: calico-config
key: cni_network_config
- name: KUBERNETES_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: CNI_MTU
valueFrom:
configMapKeyRef:
name: calico-config
key: veth_mtu
- name: SLEEP
value: "false"
volumeMounts:
- mountPath: /host/opt/cni/bin
name: cni-bin-dir
- mountPath: /host/etc/cni/net.d
name: cni-net-dir
securityContext:
privileged: true
- name: flexvol-driver
image: calico/pod2daemon-flexvol:{{ CALICO_VER }}
volumeMounts:
- name: flexvol-driver-host
mountPath: /host/driver
securityContext:
privileged: true
containers:
- name: calico-node
image: calico/node:{{ CALICO_VER }}
env:
- name: DATASTORE_TYPE
value: "kubernetes"
- name: WAIT_FOR_DATASTORE
value: "true"
- name: NODENAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: CALICO_NETWORKING_BACKEND
valueFrom:
configMapKeyRef:
name: calico-config
key: calico_backend
- name: CLUSTER_TYPE
value: "k8s,bgp"
- name: IP
value: "autodetect"
- name: IP_AUTODETECTION_METHOD
value: "interface=eth.*" #匹配本地有效网卡
- name: CALICO_IPV4POOL_IPIP
value: "Always"
- name: CALICO_IPV4POOL_VXLAN
value: "Never"
- name: FELIX_IPINIPMTU
valueFrom:
configMapKeyRef:
name: calico-config
key: veth_mtu
- name: FELIX_VXLANMTU
valueFrom:
configMapKeyRef:
name: calico-config
key: veth_mtu
- name: FELIX_WIREGUARDMTU
valueFrom:
configMapKeyRef:
name: calico-config
key: veth_mtu
- name: CALICO_IPV4POOL_CIDR
value: "{{ POD_CIDR }}" #与前面定义的pod的CIDR保持一致
- name: CALICO_DISABLE_FILE_LOGGING
value: "true"
- name: FELIX_DEFAULTENDPOINTTOHOSTACTION
value: "ACCEPT"
- name: FELIX_IPV6SUPPORT
value: "false"
- name: FELIX_LOGSEVERITYSCREEN
value: "info"
- name: FELIX_HEALTHENABLED
value: "true"
securityContext:
privileged: true
resources:
requests:
cpu: 250m
livenessProbe:
exec:
command:
- /bin/calico-node
- -felix-live
- -bird-live
periodSeconds: 10
initialDelaySeconds: 10
failureThreshold: 6
readinessProbe:
exec:
command:
- /bin/calico-node
- -felix-ready
- -bird-ready
periodSeconds: 10
volumeMounts:
- mountPath: /lib/modules
name: lib-modules
readOnly: true
- mountPath: /run/xtables.lock
name: xtables-lock
readOnly: false
- mountPath: /var/run/calico
name: var-run-calico
readOnly: false
- mountPath: /var/lib/calico
name: var-lib-calico
readOnly: false
- name: policysync
mountPath: /var/run/nodeagent
volumes:
- name: lib-modules
hostPath:
path: /lib/modules
- name: var-run-calico
hostPath:
path: /var/run/calico
- name: var-lib-calico
hostPath:
path: /var/lib/calico
- name: xtables-lock
hostPath:
path: /run/xtables.lock
type: FileOrCreate
- name: cni-bin-dir
hostPath:
path: /opt/cni/bin
- name: cni-net-dir
hostPath:
path: /etc/cni/net.d
- name: host-local-net-dir
hostPath:
path: /var/lib/cni/networks
- name: policysync
hostPath:
type: DirectoryOrCreate
path: /var/run/nodeagent
- name: flexvol-driver-host
hostPath:
type: DirectoryOrCreate
path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: calico-kube-controllers
namespace: kube-system
labels:
k8s-app: calico-kube-controllers
spec:
replicas: 1
selector:
matchLabels:
k8s-app: calico-kube-controllers
strategy:
type: Recreate
template:
metadata:
name: calico-kube-controllers
namespace: kube-system
labels:
k8s-app: calico-kube-controllers
spec:
nodeSelector:
kubernetes.io/os: linux
tolerations:
- key: CriticalAddonsOnly
operator: Exists
- key: node-role.kubernetes.io/master
effect: NoSchedule
serviceAccountName: calico-kube-controllers
priorityClassName: system-cluster-critical
containers:
- name: calico-kube-controllers
image: calico/kube-controllers:{{ CALICO_VER }}
env:
- name: ENABLED_CONTROLLERS
value: node
- name: DATASTORE_TYPE
value: kubernetes
readinessProbe:
exec:
command:
- /usr/bin/check-status
- -r
ingress配置文件 ingress-nginx.yaml
vim roles/addons_install/templates/ingress-nginx.yaml
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
name: default-http-backend
namespace: ingress-nginx
labels:
app.kubernetes.io/name: default-http-backend
app.kubernetes.io/part-of: ingress-nginx
spec:
ports:
- port: 80
targetPort: 8080
selector:
app.kubernetes.io/name: default-http-backend
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: default-http-backend
namespace: ingress-nginx
labels:
app.kubernetes.io/name: default-http-backend
app.kubernetes.io/part-of: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: default-http-backend
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: default-http-backend
app.kubernetes.io/part-of: ingress-nginx
spec:
terminationGracePeriodSeconds: 60
containers:
- name: default-http-backend
image: k8s.gcr.io/defaultbackend-amd64:{{ BACKEND_VER }}
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ConfigMap
metadata:
name: tcp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ConfigMap
metadata:
name: udp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrole-binding
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-role-binding
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
revisionHistoryLimit: 2147483647
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
updateStrategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
spec:
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:{{ INGRESS_VER }}
imagePullPolicy: IfNotPresent
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
ports:
- containerPort: 80
hostPort: 80
name: http
protocol: TCP
- containerPort: 443
hostPort: 443
name: https
protocol: TCP
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources: {}
securityContext:
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
procMount: Default
runAsUser: 33
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
hostNetwork: true
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: nginx-ingress-serviceaccount
serviceAccountName: nginx-ingress-serviceaccount
terminationGracePeriodSeconds: 30
- coredns安装calico.yml：
vim roles/addons_install/tasks/calico.yml
- name: 创建addons目录
file: name=/etc/kubernetes/addons state=directory
- name: 拷贝calico-rbac.yaml
template: src=calico-rbac.yaml dest=/etc/kubernetes/addons
- name: 拷贝calico.yaml
template: src=calico.yaml dest=/etc/kubernetes/addons
- name: 拉取kube-controllers镜像
shell: "ansible all -m docker_image -a 'name=calico/kube-controllers tag={{ CALICO_VER }} source=pull timeout=3600'"
- name: 拉取cni镜像
shell: "ansible all -m docker_image -a 'name=calico/cni tag={{ CALICO_VER }} source=pull timeout=3600'"
- name: 拉取pod2daemon-flexvol镜像
shell: "ansible all -m docker_image -a 'name=calico/pod2daemon-flexvol tag={{ CALICO_VER }} source=pull timeout=3600'"
- name: 拉取node镜像
shell: "ansible all -m docker_image -a 'name=calico/node tag={{ CALICO_VER }} source=pull timeout=3600'"
- name: 部署calico-rbac
shell: "kubectl apply -f /etc/kubernetes/addons/calico-rbac.yaml"
- name: 部署calico
shell: "kubectl apply -f /etc/kubernetes/addons/calico.yaml"
- ingress安装ingress.yml：
vim roles/addons_install/tasks/ingress.yml
- name: 拷贝ingress-nginx.yaml
template: src=ingress-nginx.yaml dest=/etc/kubernetes/addons
- name: 拉取defaultbackend-amd64镜像
shell: "ansible node -m docker_image -a 'name=huqian123/nginx-ingress-default-backend tag={{ BACKEND_VER }} source=pull timeout=3600'"
- name: tag defaultbackend-amd64镜像
shell: "ansible node -m shell -a 'docker tag huqian123/nginx-ingress-default-backend:{{ BACKEND_VER }} k8s.gcr.io/defaultbackend-amd64:{{ BACKEND_VER }}'"
args:
warn: False
- name: 拉取nginx-ingress-controller镜像
shell: "ansible node -m docker_image -a 'name=quay.io/kubernetes-ingress-controller/nginx-ingress-controller tag={{ INGRESS_VER }} source=pull timeout=3600'"
- name: 部署ingress-nginx
shell: "kubectl apply -f /etc/kubernetes/addons/ingress-nginx.yaml"
- 引用文件main.yml：
vim roles/addons_install/tasks/main.yml
- include: calico.yml
- include: ingress.yml
安装测试
- 执行安装：
ansible-playbook k8s.yml
kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready master 5m30s v1.18.6
node1 Ready <none> 4m27s v1.18.6
node2 Ready <none> 4m29s v1.18.6
node3 Ready <none> 4m27s v1.18.6
kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-578894d4cd-m47sg 1/1 Running 0 4m31s
calico-node-89vkf 1/1 Running 0 4m31s
calico-node-g2lsr 1/1 Running 0 4m31s
calico-node-vdfgq 1/1 Running 0 4m31s
calico-node-x8jmd 1/1 Running 0 4m31s
coredns-546565776c-5gbmm 1/1 Running 0 5m58s
coredns-546565776c-kvb6c 1/1 Running 0 5m58s
etcd-master 1/1 Running 0 6m13s
kube-apiserver-master 1/1 Running 0 6m13s
kube-controller-manager-master 1/1 Running 0 6m13s
kube-proxy-j8pc2 1/1 Running 0 5m15s
kube-proxy-jn9wg 1/1 Running 0 5m15s
kube-proxy-m5hx4 1/1 Running 0 5m58s
kube-proxy-rhnbh 1/1 Running 0 5m17s
kube-scheduler-master 1/1 Running 0 6m13s
kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
default-http-backend-6bf4c44778-fk7rp 1/1 Running 0 4m41s
ingress-nginx-ng7mh 1/1 Running 0 4m1s
ingress-nginx-r4hxw 1/1 Running 0 4m38s
ingress-nginx-xw7f6 1/1 Running 0 4m35s
source ~/.bash_profile #kubectl命令补全生效
kubectl edit cm kube-proxy -n kube-system #修改mode为ipvs
kubectl delete pod -n kube-system `kubectl get pod -n kube-system |grep 'kube-proxy' | awk '{print $1}'` #重启kube-proxy
ps：建议提前准备好镜像；不建议部署至生产环境。
测试安装没有问题，注意kubernetes组件版本尽量一致。已存放至个人gitgub：ansible-playbook