首先说一下我的环境， 我是在windows 10 上面建了一个ubuntu18的虚拟机，同时由于某些原因 不受网络限制， 所以安装比较顺利。
Install
1.安装并启用 Docker
sudo apt install docker.io
sudo systemctl enable docker
docker --version
2.添加 Kubernetes signing key 和Repository
sudo apt install curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
3.安装Kubeadm
sudo apt install kubeadm kubeadm version
#常用命令 重启kubelet服务： systemctl daemon-reload systemctl restart kubelet sudo systemctl restart kubelet.service sudo systemctl daemon-reload sudo systemctl stop kubelet sudo systemctl enable kubelet sudo systemctl start kubelet
4.禁用 swapoff
sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab
以上的指令我只在一台Ubuntu上执行的（如果你有多台计算机，需要在所有的计算机上执行以上指令，我这里是通过拷贝虚拟机来实现的）
5.准备2台虚拟机k8s-master和k8s-node(我这里把上面的计算机命名为 k8s_master ，copy它并命名为k8s_node)
sudo hostnamectl set-hostname k8s-master #在k8s-master 上执行 IP：192.168.255.229
sudo hostnamectl set-hostname k8s-node #k8s-node 上执行 IP：192.168.255.230
Deploy
1.在master上初始化 Kubernetes
使用kubeadm config print init-defaults可以打印集群初始化默认的使用的配置,使用kubeadm默认配置初始化的集群，会在master节点打上node-role.kubernetes.io/master:NoSchedule的污点，阻止master节点接受调度运行工作负载。这里测试环境只有两个节点，所以将这个taint的effect从NoSchedule改为PreferNoSchedule 还有就是修订kubernet版本1.15.2
apiVersion: kubeadm.k8s.io/v1beta2 kind: InitConfiguration localAPIEndpoint: advertiseAddress: 192.168.100.7 bindPort: 6443 nodeRegistration: taints: - effect: PreferNoSchedule key: node-role.kubernetes.io/master --- apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration kubernetesVersion: v1.15.2 networking: podSubnet: 10.244.0.0/16
#sudo kubeadm init --pod-network-cidr=192.168.255.229/2 kubeadm init --config kubeadm.yaml #--ignore-preflight-errors=Swap
#sudo kubeadm init --pod-network-cidr=192.168.100.0/2 这里不应该用现有计算机的ip 这里有解决方案 Kubernetes-dashboard pod is crashing again and again
如果执行中遇到detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd"
请参考Container runtimes执行
# Setup daemon. cat > /etc/docker/daemon.json <<EOF { "exec-opts": ["native.cgroupdriver=systemd"], "insecure-registries":["192.168.100.20:80"] } EOF mkdir -p /etc/systemd/system/docker.service.d #我顺便吧docker的私有仓库也加在里面 # Restart docker. systemctl daemon-reload systemctl restart docker
如果遇到port 10251 and 10252 are in use 错误请执行 netstat -lnp | grep 1025 然后kill 进程ID
2在master节点上执行如下：
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config #检查 master kubectl get nodes
3.Deploy a Pod Network & view the status
sudo kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl get pods --all-namespaces
sudo kubectl get nodes
4.add slave node
在k8s-node上执行
kubeadm join 192.168.254.229:6443 --token ewlb93.v0ohocpvncaxgl16 --discovery-token-ca-cert-hash sha256:2522834081168fbe4b5b05854b964e76f1ea8bac6f8fc5e2be21c93c6a27c427
在k8s-master上检查节点信息：
5.安装 Dashboard 插件 &检查状态
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml kubectl get deployment kubernetes-dashboard -n kube-system kubectl get svc kubernetes-dashboard -n kube-system
#kubectl delete -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
为了简单，我这里直接为 Dashboard 赋予 Admin 的权限(否者会禁止访问)，参考https://github.com/kubernetes/dashboard/wiki/Access-control#admin-privileges
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: kubernetes-dashboard labels: k8s-app: kubernetes-dashboard roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kube-system
启动代理
kubectl proxy #或者 如果需要暴露给其他机器，需要指定address: kubectl proxy --address='0.0.0.0' --port=8001 --accept-hosts='^*$'
访问http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/
不过我在访问这个地址 遇到错误如下：
root@k8s-master:~# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-5c98db65d4-97xrf 1/1 Running 0 83m kube-system coredns-5c98db65d4-vvtfc 1/1 Running 0 83m kube-system etcd-k8s-master 1/1 Running 0 83m kube-system kube-apiserver-k8s-master 1/1 Running 0 83m kube-system kube-controller-manager-k8s-master 1/1 Running 0 83m kube-system kube-flannel-ds-amd64-gbg49 1/1 Running 1 80m kube-system kube-flannel-ds-amd64-hmrcp 1/1 Running 0 82m kube-system kube-proxy-lbp5k 1/1 Running 0 80m kube-system kube-proxy-szkb8 1/1 Running 0 83m kube-system kube-scheduler-k8s-master 1/1 Running 0 83m kube-system kubernetes-dashboard-7d75c474bb-p5nz5 0/1 CrashLoopBackOff 6 10m root@k8s-master:~# kubectl logs kubernetes-dashboard-7d75c474bb-p5nz5 -n kube-system 2019/08/07 11:34:33 Using in-cluster config to connect to apiserver 2019/08/07 11:34:33 Using service account token for csrf signing 2019/08/07 11:34:33 Starting overwatch 2019/08/07 11:35:03 Error while initializing connection to Kubernetes apiserver. This most likely means that the cluster is misconfigured (e.g., it has invalid apiserver certificates or service account's configuration) or the --apiserver-host param points to a server that does not exist. Reason: Get https://10.96.0.1:443/version: dial tcp 10.96.0.1:443: i/o timeout Refer to our FAQ and wiki pages for more information: https://github.com/kubernetes/dashboard/wiki/FAQ root@k8s-master:~#
后来在https://stackoverflow.com/questions/52273029/kubernetes-dashboard-pod-is-crashing-again-and-again 找到解决方案，
Make sure you understand the difference between Host network, Pod network and Service network. These 3 networks can not overlap. For example --pod-network-cidr=192.168.0.0/16 must notinclude the IP address of your host, change it to 10.0.0.0/16 or something smaller if necessary.
所以 初始化该为 sudo kubeadm init --pod-network-cidr=192.168.100.0/2 就okay 了
创建账号
kubectl create serviceaccount dashboard -n default kubectl create clusterrolebinding dashboard-admin -n default --clusterrole=cluster-admin --serviceaccount=default:dashboard kubectl get secret $(kubectl get serviceaccount dashboard -o jsonpath="{.secrets[0].name}") -o jsonpath="{.data.token}" | base64 --decode
Copy the decoded password and login to dashboard.
在Kubernetes的安装过程中会默认安装很多Service账户，分别有不同的访问权限，要找到对应的token，你可以使用下面的方式：
kubectl -n kube-system get secret
kubectl -n kube-system describe secret certificate-controller-token-4xr9x
上面我们使用token来登录， 这里我们还是用上面的dashboard账号来生成kubeconfig文件，以后用生成kubeconfig文件来登录
#配置集群信息 kubectl config set-cluster kubernetes --server=192.168.100.11:6443 --kubeconfig=/root/dashbord.conf #使用token写入集群验证 #kubectl get secret $(kubectl get serviceaccount dashboard -o jsonpath="{.secrets[0].name}") -o jsonpath="{.data.token}" | base64 --decode #查看token kubectl config set-credentials dashboard --token=eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImRhc2hib2FyZC10b2tlbi12dDl4OSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJkYXNoYm9hcmQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIxZmM1ZDc1ZS0xMjE2LTQwMDgtYThhOS03ZjEwZGQ1NWJjNWEiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpkYXNoYm9hcmQifQ.Neb5_blRSig6IU5oPtRIQlQhcELaeI8uu7jeiVEdiR3CLiCZYyiI7X6uNsrpGKAkR-OkGM1gOp09-pmxFI6m4lKHYu9S7R1MNigmQrxfZB4RJ-iYZCNp3Rra7mFrluwY_yMbzuZ__XeYShSOiO1VAS2ezWFGk9adgtbiWZkef_NxmYdwEmTAGkmazhatK9SGDWBea-1seoJx-SGFyA9j0gNcWqNrX93ozFmuNtYrPZSwhYkul-q4NHOz4Dp4Ux1C7gZzTIgBySaYZd5tiJIAmZ-6CV-ukmPtFn7tVlNaDkK4K5N6jzyDttlvHZJtWqBR7iWTyamAKAbycm_BmaQR4Q --kubeconfig=/root/dashboard.conf 配置上下文和当前上下文 kubectl config set-context dashboard@kubernetes --cluster=kubernetes --user=dashboard --kubeconfig=/root/dashboard.conf kubectl config use-context dashboard@kubernetes --kubeconfig=/root/dashboard.conf kubectl config view --kubeconfig=/root/dashboard.conf
将/root/dashboard.conf 文件发送到宿主机，浏览器访问时选择Kubeconfig认证，载入该配置文件，点击登陆，即可实现访问，如图：
卸载集群
想要撤销kubeadm做的事，首先要排除节点，并确保在关闭节点之前要清空节点。
在主节点上运行：
kubectl drain
kubectl delete node
然后在需要移除的节点上，重置kubeadm的安装状态：
kubeadm reset
重启master节点 如果遇到如下错误
The connection to the server 10.2.0.165:6443 was refused - did you specify the right host or port? 解决方案The connection to the server
sudo -i swapoff -a exit strace -eopenat kubectl version
#如果以上方法没有搞定可以执行以下命令
systemctl status docker #查看docker状态
systemctl status kubelet #查看kubelet的状态
netstat -pnlt | grep 6443 #查看6443的端口应该启动起来了
journalctl -xeu kubelet #查看kubelet的日志解决问题
systemctl restart kubelet #最后重启
-------------------------2019-08-20--------------------------
Kubernetes常用组件部署
Helm的安装
Helm由客户端命helm令行工具和服务端tiller组成，Helm的安装十分简单。 下载helm命令行工具到master节点node1的/usr/local/bin下，这里下载的2.14.1版本：
为了安装服务端tiller，还需要在这台机器上配置好kubectl工具和kubeconfig文件，确保kubectl工具可以在这台机器上访问apiserver且正常使用。 这里的node1节点已经配置好了kubectl。
因为Kubernetes APIServer开启了RBAC访问控制，所以需要创建tiller使用的service account: tiller并分配合适的角色给它。 详细内容可以查看helm文档中的Role-based Access Control。 这里简单起见直接分配cluster-admin这个集群内置的ClusterRole给它。创建helm-rbac.yaml文件
接下来使用helm部署tiller，tiller默认被部署在k8s集群中的kube-system这个namespace下
最后在k8s-master上修改helm chart仓库的地址为azure提供的镜像地址：
helm repo add stable http://mirror.azure.cn/kubernetes/charts
使用Helm部署Nginx Ingress
为了便于将集群中的服务暴露到集群外部，需要使用Ingress。接下来使用Helm将Nginx Ingress部署到Kubernetes上。 Nginx Ingress Controller被部署在Kubernetes的边缘节点上，关于Kubernetes边缘节点的高可用相关的内容可以查看之前整理的Bare metal环境下Kubernetes Ingress边缘节点的高可用,Ingress Controller使用hostNetwork。
我们将node1(192.168.100.7)做为边缘节点，打上Label：
root@k8s-master:~/linux-amd64# kubectl label node k8s-master node-role.kubernetes.io/edge=
node/k8s-master labeled
root@k8s-master:~/linux-amd64# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master Ready edge,master 34m v1.15.2
k8s-node Ready 28m v1.15.2
root@k8s-master:~/linux-amd64#
stable/nginx-ingress chart的值文件ingress-nginx.yaml如下：
controller:
replicaCount: 1
hostNetwork: true
nodeSelector:
node-role.kubernetes.io/edge: ''
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app operator: In values: - nginx-ingress - key: component operator: In values: - controller topologyKey: kubernetes.io/hostname tolerations: - key: node-role.kubernetes.io/master operator: Exists effect: NoSchedule - key: node-role.kubernetes.io/master operator: Exists effect: PreferNoSchedule defaultBackend: nodeSelector: node-role.kubernetes.io/edge: '' tolerations: - key: node-role.kubernetes.io/master operator: Exists effect: NoSchedule - key: node-role.kubernetes.io/master operator: Exists effect: PreferNoSchedule
nginx ingress controller的副本数replicaCount为1，将被调度到k8s-master这个边缘节点上。这里并没有指定nginx ingress controller service的externalIPs，而是通过hostNetwork: true设置nginx ingress controller使用宿主机网络。
helm repo update
helm install stable/nginx-ingress -n nginx-ingress --namespace ingress-nginx -f ingress-nginx.yaml
helm repo add stable http://mirror.azure.cn/kubernetes/charts
如果访问http://192.168.100.7返回default backend，则部署完成。
kubectl get pod -n ingress-nginx -o wide
kubectl -n ingress-nginx exec nginx-ingress-controller-xxxxx -- cat /etc/nginx/nginx.conf使用Helm部署dashboard
kubernetes-dashboard.yaml
image:
repository: k8s.gcr.io/kubernetes-dashboard-amd64
tag: v1.10.1
ingress:
enabled: true
hosts:
- k8s.frognew.com
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" tls: - secretName: frognew-com-tls-secret hosts: - k8s.frognew.com nodeSelector: node-role.kubernetes.io/edge: '' tolerations: - key: node-role.kubernetes.io/master operator: Exists effect: NoSchedule - key: node-role.kubernetes.io/master operator: Exists effect: PreferNoSchedule rbac: clusterAdminRole: true
安装：
helm install stable/kubernetes-dashboard -n kubernetes-dashboard --namespace kube-system -f kubernetes-dashboard.yaml
kubectl -n kube-system get secret | grep kubernetes-dashboard-token
kubectl describe -n kube-system secret/kubernetes-dashboard-token-xxx
helm search kubernetes-dashboard
kubectl get pods -n kube-system -o wide
修改本机的hosts文件：
192.168.100.11 k8s.frognew.com
参考：
kubernetes 安装遇到的一些问题
K8s折腾日记(零) -- 基于 Ubuntu 18.04 安装部署K8s集群
Install and Deploy Kubernetes on Ubuntu 18.04 LTS
Install and Configure Kubernetes (k8s) 1.13 on Ubuntu 18.04 LTS / Ubuntu 18.10
Deploy Kubernetes cluster using kubeadmin on Ubuntu Server
Kubernetes on Ubuntu 18.04 - Master and Dashboard setup
使用kubeadm安装Kubernetes 1.11
kubeadm 部署的k8s集群(1.11.1) Dashboard遇到的问题
10分钟看懂Docker和K8S
18张儿童插画让你秒懂Kubernetes
kubernetes外部访问的几种方式
深入玩转K8S之外网如何访问业务应用
k8s外网如何访问业务应用
Troubleshooting kubectl Error: The connection to the server x.x.x.x:6443 was refused – did you specify the right host or port?
Kubernetes学习之路（十九）之Kubernetes dashboard认证访问