kubectl 基本命令
检查节点状态
kubectl get nodes
检查namespace
kubectl get namespace
删除节点
etcdctl ls /registry/minions
或者
kubectl delete node xxxx
更改 rc - pods 数量
kubectl scale rc rc_name --replicas=number
nginx-ingress
原理
一般来说，svc和pod拥有的IP只能在集群内部使用。集群外部请求需要通过负载均衡转发到svc在node上暴露的NodePort,然后再由kube-proxy 将其转发给相关的pod。
[图片上传失败...(image-162def-1516283321671)]
而Ingress就是为进入集群的请求提供路由规则的集合，如图所示。
[图片上传失败...(image-8fd5c7-1516283321671)]
Ingress可以给service提供集群外部访问的URL、负载均衡、SSL终止、HTTP路由等。为了配置这些Ingress规则，集群管理员需要部署一个Ingress controller，它监听Ingress和service的变化，并根据规则配置负载均衡并提供访问入口。
配置规则
每个Ingress都需要配置rules，目前Kubernetes仅支持http规则。上面的示例表示请求/testpath时转发到服务test的80端口。
根据Ingress Spec配置的不同，Ingress可以分为以下几种类型：
单服务Ingress
单服务Ingress即该Ingress仅指定一个没有任何规则的后端服务。
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress
spec:
backend:
serviceName: testsvc
servicePort: 80
路由到多服务的Ingress
路由到多服务的Ingress即根据请求路径的不同转发到不同的后端服务上
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /foo
backend:
serviceName: s1
servicePort: 80
- path: /bar
backend:
serviceName: s2
servicePort: 80
虚拟主机Ingress
虚拟主机Ingress即根据名字的不同转发到不同的后端服务上，而他们共用同一个的IP地址，如下所示
foo.bar.com --| |-> foo.bar.com s1:80
| 178.91.123.132 |
bar.foo.com --| |-> bar.foo.com s2:80
下面是一个基于Host header路由请求的Ingress：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
spec:
rules:
- host: foo.bar.com
http:
paths:
- backend:
serviceName: s1
servicePort: 80
- host: bar.foo.com
http:
paths:
- backend:
serviceName: s2
servicePort: 80
总结
对于我来说，最常用的还是虚拟主机模式。就像nginx中conf.d多个主机似的一回事。但是对于一个大型网站的话，或许使用路由到多服务模式会更好一些。
前面是对模式的整理，但还有一个组件。其中default-http-backend作为 Nginx Ingress Controller默认的后端，处理所有404请求。
安装配置
default-http-backend
首先配置 default-http-backend
当前pod作为Nginx Ingress Controller默认的后端，处理所有404请求。当前还没有做任何配置，此时请求的时候将由此pod响应。
镜像准备
如果有加速器
docker pull gcr.io/google_containers/defaultbackend:1.4
如果没有
docker pull registry.cn-beijing.aliyuncs.com/cloudexp/defaultbackend:latest
yml 文件
curl -o default-http-backend.yml https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/default-backend.yaml
或者
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: default-http-backend
labels:
k8s-app: default-http-backend
namespace: kube-system
spec:
replicas: 1
template:
metadata:
labels:
k8s-app: default-http-backend
spec:
terminationGracePeriodSeconds: 60
containers:
- name: default-http-backend
# Any image is permissable as long as:
# 1. It serves a 404 page at /
# 2. It serves 200 on a /healthz endpoint
image: registry.cn-beijing.aliyuncs.com/cloudexp/defaultbackend:latest
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
ports:
- containerPort: 8080
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
部署
kubectl create -f default-http-backend.yml
#返回结果
deployment "default-http-backend" created
service "default-http-backend" created
nginx-ingress-controller
镜像准备
docker pull registry.cn-hangzhou.aliyuncs.com/google-containers/nginx-ingress-controller:0.9.0
yml文件
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: kube-system
labels:
k8s-app: nginx-ingress-controller
spec:
replicas: 1
template:
metadata:
labels:
k8s-app: nginx-ingress-controller
spec:
# hostNetwork makes it possible to use ipv6 and to preserve the source IP correctly regardless of docker configuration
# however, it is not a hard dependency of the nginx-ingress-controller itself and it may cause issues if port 10254 already is taken on the host
# that said, since hostPort is broken on CNI (https://github.com/kubernetes/kubernetes/issues/31307) we have to use hostNetwork where CNI is used
# like with kubeadm
# hostNetwork: true
terminationGracePeriodSeconds: 60
containers:
- image: registry.cn-hangzhou.aliyuncs.com/google-containers/nginx-ingress-controller:0.9.0
name: nginx-ingress-controller
readinessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
livenessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
timeoutSeconds: 1
ports:
- containerPort: 80
hostPort: 80
- containerPort: 443
hostPort: 443
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
args:
- /nginx-ingress-controller
- --default-backend-service=kube-system/default-http-backend
#- --publish-service=kube-system/nginx-ingress-lb
- --apiserver-host=http://10.10.30.102:8080
注意如果启动报错找不到APIServer，就加上这一行。
如果github上的镜像无法pull，可以更换为阿里云。
部署
部署成功后创建以下两个pod：
[root@localhost kubernetes]# kubectl get pods --namespace=kube-system -l k8s-app=nginx-ingress-controller -o wide
NAME READY STATUS RESTARTS AGE IP NODE
nginx-ingress-controller-2867543418-f4t1t 1/1 Running 0 46s 172.30.14.4 centos-minion-1
[root@localhost kubernetes]# kubectl get pods --namespace=kube-system -l k8s-app=nginx-ingress-controller -o wide
NAME READY STATUS RESTARTS AGE IP NODE
nginx-ingress-controller-2867543418-f4t1t 1/1 Running 0 49s 172.30.14.4 centos-minion-1
Ingress
创建服务
部署两个版本的nginx。
nginx1
apiVersion: v1
kind: Service
metadata:
name: nginx1-8
spec:
ports:
- port: 80
targetPort: 80
selector:
app: nginx1-8
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx1-8-deployment
spec:
replicas: 2
template:
metadata:
labels:
app: nginx1-8
spec:
containers:
- name: nginx
image: docker.io/nginx:latest
ports:
- containerPort: 80
nginx2
为了更好区分，还了个别的示例
apiVersion: v1
kind: Service
metadata:
name: nginx2-8
spec:
ports:
- port: 80
targetPort: 80
selector:
app: nginx2-8
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx2-8-deployment
spec:
replicas: 2
template:
metadata:
labels:
app: nginx2-8
spec:
containers:
- name: nginx
image: docker.io/kubeguide/guestbook-php-frontend
ports:
- containerPort: 80
创建完成后，检查
kubectl get pods -o wide
配置ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
spec:
rules:
- host: test.xxx1.com
http:
paths:
- backend:
serviceName: nginx1-8
servicePort: 80
- host: test.xxx2.com
http:
paths:
- backend:
serviceName: nginx2-8
servicePort: 80
配置hosts文件和host对应，访问即可
#主机 域名
10.10.30.102 test.xxx1.com
10.10.30.102 test.xxx2.com
测试
curl -I test.xxx1.com
curl -I test.xxx2.com
自定义上传文件大小
nginx ingress controller默认定义的上传大小为1M，因此需要更改nginx配置中client_max_body_size的大小，具体修改ingress文件，如下
cat nginx-ingress-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configuration
namespace: kube-system
labels:
k8s-app: nginx-ingress-controller
data:
proxy-body-size: "50m"
nginx-ingress-controller.yml 需要匹配官方文件
https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/without-rbac.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: ingress-nginx
template:
metadata:
labels:
app: ingress-nginx
annotations:
prometheus.io/port: '10254'
prometheus.io/scrape: 'true'
spec:
nodeName: centos-minion-1
#hostNetwork: true
containers:
- name: nginx-ingress-controller
#image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.10.0
image: registry.cn-hangzhou.aliyuncs.com/google-containers/nginx-ingress-controller:0.9.0
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --annotations-prefix=nginx.ingress.kubernetes.io
- --apiserver-host=http://10.10.30.102:8080
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: logs
mountPath: /var/log/nginx/
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
volumes:
- name: logs
hostPath:
path: /data/logs/nginx
更多参考：
Kubernetes + Dashboard + Heapster (一) 安装配置
Kubernetes + Dashboard + Heapster (二) 监控部署
Kubernetes + Dashboard + Heapster (三) ingress负载均衡
Kubernetes + Dashboard + Heapster (四) 慢慢填坑