kubernetes学习例子
翻译自kubernetesbyexample网站，对于快速了解kubernetes的基本概念及其用法有一定的快速入门作用。
可以说是一篇了解k8s的属于科普文。
操作环境：基于Red Hat的Openshift在线云平台。
目录
- 1 pods
- 2 labels
- 3 deployments
- 4 services
- 5 service discovery
- 6 port forward
- 7 health checks
- 8 environment variables
- 9 namespaces
- 10 volumes
- 11 persistent volumes
- 12 secrets
- 13 logging
- 14 jobs
- 15 stateful sets
- 16 init containers
- 17 nodes
- 18 API server
1 pods
pod作为共享network和mount命令空间的容器集合，是k8s中基本的部署和调度单元。pod中的容器(如果有多个)会被调度到同一node(宿主机).
使用mhausenblas/simpleservice:0.5.0作为pod中容器的镜像，同时在9876端口导出HTTP API：
$ kubectl run sise --image=mhausenblas/simpleservice:0.5.0 --port=9876
可以看到pod已经运行：
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
sise-3210265840-k705b 1/1 Running 0 1m
$ kubectl describe pod sise-3210265840-k705b | grep IP:
IP: 172.17.0.3
在集群内部可以通过pod的IP 172.17.0.3访问。通过minishift ssh命令或者上文中的 kubectl describe命令获取IP。
[cluster] $ curl 172.17.0.3:9876/info
{"host": "172.17.0.3:9876", "version": "0.5.0", "from": "172.17.0.1"}
注意： kubectl run 会创建一个deployment对象，所以删除该pod时，必须执行: kubectl delete deployment sise 删除相应的deployment.
使用配置文件
可以使用配置文件创建pod。下面的例子使用一个包含了simpleservice和CentOS两个容器的名为twocontainers的pod:
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/pods/pod.yaml
pod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: twocontainers
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
- name: shell
image: centos:7
command:
- "bin/bash"
- "-c"
- "sleep 10000"
查看pod:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
twocontainers 2/2 Running 0 7s
可以登录CentOS 容器，然后通过localhost访问simpleservice容器:
$ kubectl exec twocontainers -c shell -i -t -- bash
[root@twocontainers /]# curl -s localhost:9876/info
{"host": "localhost:9876", "version": "0.5.0", "from": "127.0.0.1"}
下面的例子说明如何限制容器中CPU和内存的配额:
$ kubectl create -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/pods/constraint-pod.yaml
constraint-pod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: constraintpod
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
resources:
limits:
memory: "64Mi"
cpu: "500m"
$ kubectl describe pod constraintpod
...
Containers:
sise:
...
Limits:
cpu: 500m
memory: 64Mi
Requests:
cpu: 500m
memory: 64Mi
...
删除刚才创建的pods :
$ kubectl delete pod twocontainers
$ kubectl delete pod constraintpod
小结：在k8s环境中，直接启动一个或多个容器的pod是非常简单的，但是这种方式有一定的局限性：在失败后必须自行重启来保证运行。一个更好的解决方法是使用deployment来部署pod.
2 labels
Labels 是一种组织kubernetes对象的机制。一个label一一个满足长度和值约束的键值对。例如：可以标识在生产环境运行的pod，或者部门X拥有的pod.
下面创建一个带有 env=development标签的pod.
:
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/labels/pod.yaml
pod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: labelex
labels:
env: development
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
$ kubectl get pods --show-labels
NAME READY STATUS RESTARTS AGE LABELS
labelex 1/1 Running 0 10m env=development
使用–show-labels选项可以输出额外的列。
可以通过label命令增加标签：
$ kubectl label pods labelex owner=michael
$ kubectl get pods --show-labels
NAME READY STATUS RESTARTS AGE LABELS
labelex 1/1 Running 0 16m env=development,owner=michael
可以使用–selector选项，将label作为过滤器，例如只列表owner等于michael的pods:
$ kubectl get pods --selector owner=michael
NAME READY STATUS RESTARTS AGE
labelex 1/1 Running 0 27m
–selector选项可以使用简短的-l代替：
$ kubectl get pods -l env=development
NAME READY STATUS RESTARTS AGE
labelex 1/1 Running 0 27m
Kubernetes对象支持集合选择器，我们启动一个具有env=production和owner=michael标签的pod:
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/labels/anotherpod.yaml
anotherpod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: labelexother
labels:
env: production
owner: michael
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
现在，列出所有满足env=development或者env=production标签的pods:
$ kubectl get pods -l 'env in (production, development)'
NAME READY STATUS RESTARTS AGE
labelex 1/1 Running 0 43m
labelexother 1/1 Running 0 3m
其他的操作也支持label选择器, 例如移除pods:
$ kubectl delete pods -l 'env in (production, development)'
当然也可以使用pod名字直接删除：
$ kubectl delete pods labelex
$ kubectl delete pods labelexother
注意：labels不仅能用于pods,也可以用于其他的对象，例如nodes或者services.
3 deployments
deployment是pod的监视器。可以更细粒度地控制何时以及如何创建、升级、回滚pod。
创建一个名为sise-deploy的deployment,它通过replica set监视两个pod副本：
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/deployments/d09.yaml
d09.yaml:
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: sise-deploy
spec:
replicas: 2
template:
metadata:
labels:
app: sise
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
env:
- name: SIMPLE_SERVICE_VERSION
value: "0.9"
可以查看deployment以及replica set和 pods:
$ kubectl get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
sise-deploy 2 2 2 2 10s
$ kubectl get rs
NAME DESIRED CURRENT READY AGE
sise-deploy-3513442901 2 2 2 19s
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
sise-deploy-3513442901-cndsx 1/1 Running 0 25s
sise-deploy-3513442901-sn74v 1/1 Running 0 25s
我们注意到，pods和replica set的名字是从deployment派生出来的。
此时pods中运行的容器的版本为0.9. 我们通过kubectl describe获取到pod的ip，调用其提供的/info API接口：
[cluster] $ curl 172.17.0.3:9876/info
{"host": "172.17.0.3:9876", "version": "0.9", "from": "172.17.0.1"}
他们通过更新deployment，升级到1.0版本：
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/deployments/d10.yaml
deployment "sise-deploy" configured
d10.yaml:
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: sise-deploy
spec:
replicas: 2
template:
metadata:
labels:
app: sise
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
env:
- name: SIMPLE_SERVICE_VERSION
value: "1.0"
可以使用下面的命令：
kubectl edit deploy/sise-deploy
手动编辑之前的配置文件。
我们现在可以看到滚动升级到1.0版本，同时之前0.9版本的正在终止：
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
sise-deploy-2958877261-nfv28 1/1 Running 0 25s
sise-deploy-2958877261-w024b 1/1 Running 0 25s
sise-deploy-3513442901-cndsx 1/1 Terminating 0 16m
sise-deploy-3513442901-sn74v 1/1 Terminating 0 16m
同时，一个新的replica set已经被deployment创建出来：
$ kubectl get rs
NAME DESIRED CURRENT READY AGE
sise-deploy-2958877261 2 2 2 4s
sise-deploy-3513442901 0 0 0 24m
可以使用下面的命令查看升级的过程：
kubectl rollout status deploy/sise-deploy
为了验证1.0版本已经生效，还是调用之前的API接口验证：
[cluster] $ curl 172.17.0.5:9876/info
{"host": "172.17.0.5:9876", "version": "1.0", "from": "172.17.0.1"}
通过下面的命令可以查看所有的deployment的历史信息：
$ kubectl rollout history deploy/sise-deploy
deployments "sise-deploy"
REVISION CHANGE-CAUSE
1
2
假如部署之后发现有问题，可以通过roll back回滚到之前的版本，但是必须指定回滚的修订记录，例如下面的，回滚到之前revision=1的修订记录：
$ kubectl rollout undo deploy/sise-deploy --to-revision=1
deployment "sise-deploy" rolled back
$ kubectl rollout history deploy/sise-deploy
deployments "sise-deploy"
REVISION CHANGE-CAUSE
2
3
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
sise-deploy-3513442901-ng8fz 1/1 Running 0 1m
sise-deploy-3513442901-s8q4s 1/1 Running 0 1m
此时我们已经回滚到之前0.9版本。
最后，我们通过下面的命令删除deployment以及对应的replica sets和pods：
$ kubectl delete deploy sise-deploy
deployment "sise-deploy" deleted
4 services
service是pod的抽象，提供稳定的名为VIP的地址。通过VIP可以访问到pod中的容器。VIP部署不是绑定在网络接口的真实的IP地址，它的主要功能是用于转发流量到pod中。由kube-proxy组件负责维护VIP和pod的映射关系，kube-proxy是在每个node上运行的独立进程，它会从API Server来获取service和pod信息。
下面分别创建ReplicationController和service对象，注意：推荐用Replication set替换Replication controller.
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/services/rc.yaml
rc.yaml:
apiVersion: v1
kind: ReplicationController
metadata:
name: rcsise
spec:
replicas: 1
selector:
app: sise
template:
metadata:
name: somename
labels:
app: sise
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/services/svc.yaml
svc.yaml:
apiVersion: v1
kind: Service
metadata:
name: simpleservice
spec:
ports:
- port: 80
targetPort: 9876
selector:
app: linhz
可以看到pod已经运行：
$ kubectl get pods -l app=linhz
NAME READY STATUS RESTARTS AGE
rcsise-6nq3k 1/1 Running 0 57s
$ kubectl describe pod rcsise-6nq3k
Name: rcsise-6nq3k
Namespace: default
Security Policy: restricted
Node: localhost/192.168.99.100
Start Time: Tue, 25 Apr 2017 14:47:45 +0100
Labels: app=linhz
Status: Running
IP: 172.17.0.3
Controllers: ReplicationController/rcsise
Containers:
...
在集群内部，可以通过分配的IP172.17.0.3直接访问：
[cluster] $ curl 172.17.0.3:9876/info
{"host": "172.17.0.3:9876", "version": "0.5.0", "from": "172.17.0.1"}
但是我们不推荐这种直接通过pod ip访问的方式，因为ip地址可能会发生改变。我们看下创建的simpleservice service:
$ kubectl get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
simpleservice 172.30.228.255 80/TCP 5m
$ kubectl describe svc simpleservice
Name: simpleservice
Namespace: default
Labels:
Selector: app=sise
Type: ClusterIP
IP: 172.30.228.255
Port: 80/TCP
Endpoints: 172.17.0.3:9876
Session Affinity: None
No events.
service通过label保持流量转发到pod，我们例子的label是: app=sise.
直接通过ClusterIP访问：
[cluster] $ curl 172.30.228.255:80/info
{"host": "172.30.228.255", "version": "0.5.0", "from": "10.0.2.15"}
是什么导致VIP 172.30.228.255将流量转发到pod中？答案是IPtables,它在本质上是一系列规则，用于告诉Linux内核如何处理一个IP包。
在集群的node上可以通过下面的命令查看这些规则：
[cluster] $ sudo iptables-save | grep simpleservice
-A KUBE-SEP-4SQFZS32ZVMTQEZV -s 172.17.0.3/32 -m comment --comment "default/simpleservice:" -j KUBE-MARK-MASQ
-A KUBE-SEP-4SQFZS32ZVMTQEZV -p tcp -m comment --comment "default/simpleservice:" -m tcp -j DNAT --to-destination 172.17.0.3:9876
-A KUBE-SERVICES -d 172.30.228.255/32 -p tcp -m comment --comment "default/simpleservice: cluster IP" -m tcp --dport 80 -j KUBE-SVC-EZC6WLOVQADP4IAW
-A KUBE-SVC-EZC6WLOVQADP4IAW -m comment --comment "default/simpleservice:" -j KUBE-SEP-4SQFZS32ZVMTQEZV
我们可以看到上面有4条规则，这4条规则由kube-proxy增加到路由表中，用于表明从172.30.228.255:80的TCP流量应该转发中pod的 172.17.0.3:9876
我们现在扩容到2个pod:
$ kubectl scale --replicas=2 rc/rcsise
replicationcontroller "rcsise" scaled
$ kubectl get pods -l app=sise
NAME READY STATUS RESTARTS AGE
rcsise-6nq3k 1/1 Running 0 15m
rcsise-nv8zm 1/1 Running 0 5s
当我们再次检查路由表是，会发现增加了新的IPtables规则：
[cluster] $ sudo iptables-save | grep simpleservice
-A KUBE-SEP-4SQFZS32ZVMTQEZV -s 172.17.0.3/32 -m comment --comment "default/simpleservice:" -j KUBE-MARK-MASQ
-A KUBE-SEP-4SQFZS32ZVMTQEZV -p tcp -m comment --comment "default/simpleservice:" -m tcp -j DNAT --to-destination 172.17.0.3:9876
-A KUBE-SEP-PXYYII6AHMUWKLYX -s 172.17.0.4/32 -m comment --comment "default/simpleservice:" -j KUBE-MARK-MASQ
-A KUBE-SEP-PXYYII6AHMUWKLYX -p tcp -m comment --comment "default/simpleservice:" -m tcp -j DNAT --to-destination 172.17.0.4:9876
-A KUBE-SERVICES -d 172.30.228.255/32 -p tcp -m comment --comment "default/simpleservice: cluster IP" -m tcp --dport 80 -j KUBE-SVC-EZC6WLOVQADP4IAW
-A KUBE-SVC-EZC6WLOVQADP4IAW -m comment --comment "default/simpleservice:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-4SQFZS32ZVMTQEZV
-A KUBE-SVC-EZC6WLOVQADP4IAW -m comment --comment "default/simpleservice:" -j KUBE-SEP-PXYYII6AHMUWKLYX
上面我可以发现有些规则是新创建的172.17.0.4:9876 pod 相关的,例如下面这条：
-A KUBE-SVC-EZC6WLOVQADP4IAW -m comment --comment "default/simpleservice:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-4SQFZS32ZVMTQEZV
这会是的流量均匀地转发到两个pod中。
清理资源：
$ kubectl delete svc simpleservice
$ kubectl delete rc rcsise
5 service发现
Service发现是关于在集群内部如何连接service的一个方式。主要包含环境变量和DNS两种方式，推荐使用DNS,因为环境变量存在先启动的服务发现不了后启动的服务的问题。DNS是需要我们在kubernetes集群中额外安装的组件。
我们先创建svc和RC来管理pods:
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/sd/rc.yaml
rc.yaml:
apiVersion: v1
kind: ReplicationController
metadata:
name: rcsise
spec:
replicas: 2
selector:
app: sise
template:
metadata:
name: somename
labels:
app: sise
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/sd/svc.yaml
svc.yaml:
apiVersion: v1
kind: Service
metadata:
name: thesvc
spec:
ports:
- port: 80
targetPort: 9876
selector:
app: sise
现在我们通过创建一个名为jumpod的pod来莫宁从另外一个服务中连接thesvc服务：
jumpod.yaml：
apiVersion: v1
kind: Pod
metadata:
name: jumpod
spec:
containers:
- name: shell
image: centos:7
command:
- "bin/bash"
- "-c"
- "sleep 10000"
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/sd/jumpod.yaml
DNS插件确保集群中的其他pod能够通过FQDN=thesvc.default.svc.cluster.local访问到thesvc服务:
$ kubectl exec -it jumpod -c shell -- ping thesvc.default.svc.cluster.local
PING thesvc.default.svc.cluster.local (172.30.251.137) 56(84) bytes of data.
...
可以看懂pingt的返回结果告诉我们thesvc的集群ip为172.30.251.137。
在同一命名空间中，也可以直接通过服务名访问:
$ kubectl exec -it jumpod -c shell -- curl http://thesvc/info
{"host": "thesvc", "version": "0.5.0", "from": "172.17.0.5"}
注意172.17.0.5IP是我们jump pod的集群内部IP.
对于不同命名空间的服务之间的访问，可以通过FQDN= S V C . SVC. SVC.NAMESPACE.svc.cluster.local的方式.
我们使用下面的流程实验下：
- 一个名为other的命名空间；
- 在other命名空间中的thesvc服务；
- 在other命名空间中创建一个RC
首先创建命名空间：
other-ns.yaml：
apiVersion: v1
kind: Namespace
metadata:
name: other
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/sd/other-ns.yaml
other-rc.yaml：
apiVersion: v1
kind: ReplicationController
metadata:
name: rcsise
namespace: other
spec:
replicas: 2
selector:
app: sise
template:
metadata:
name: somename
labels:
app: sise
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/sd/other-rc.yaml
other-svc.yaml：
apiVersion: v1
kind: Service
metadata:
name: thesvc
namespace: other
spec:
ports:
- port: 80
targetPort: 9876
selector:
app: sise
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/sd/other-svc.yaml
我们通过default命名空间中的jump pod访问other命名空间中的thesvc服务：
$ kubectl exec -it jumpod -c shell -- curl http://thesvc.other/info
{"host": "thesvc.other", "version": "0.5.0", "from": "172.17.0.5"}
DNS方式通过了更加灵活和通用的跨集群访问服务的方式。
删除所有的资源：
$ kubectl delete pods jumpod
$ kubectl delete svc thesvc
$ kubectl delete rc rcsise
$ kubectl delete ns other
注意：删除命名空间将删除其包含的所有资源。
6 端口转发
在开发部署到Kubernetes的应用时，我们经常需要在本地环境快速访问服务，而不想创建所谓的load balancer和ingress资源。此时端口转发就派上用场了。
我们先创建一个名为simpleservice的deployment和servcie对象。对外服务的端口80:
app.yaml:
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: sise-deploy
spec:
replicas: 1
template:
metadata:
labels:
app: sise
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
---
apiVersion: v1
kind: Service
metadata:
name: simpleservice
spec:
ports:
- port: 80
targetPort: 9876
selector:
app: sise
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/pf/app.yaml
为了在本地开发环境通过8080端口访问simpleservice服务，我们使用下面的命令来转发流量：
$ kubectl port-forward service/simpleservice 8080:80
Forwarding from 127.0.0.1:8080 -> 9876
Forwarding from [::1]:8080 -> 9876
我们可以看到服务的流量已经路由到了pod的9876端口。
此时就可以使用下面的方式访问API接口：
$ curl localhost:8080/info
{"host": "localhost:8080", "version": "0.5.0", "from": "127.0.0.1"}
注意：端口转发只适用于生产和实验环境，不能用于生产环境。
7 健康检查
为了验证pod中的容器是否已经处于接收流量的健康状态，kubernetes提供了健康检查机制。健康检查的存活探针livenessProbe用于告诉kubelet进程是否要重启容器，而就绪探针readinessProbe用于觉得services和deployments能否接收请求流量。
我们下面聚焦于HTTP的健康检查，需要开发者在应用到导出用于检测的URL，用于kubelet检查容器是否健康。
我们创建一个pod并导出一个/health端口，该端口会返回表明健康的200状态码。
pod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: hc
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
livenessProbe:
initialDelaySeconds: 2
periodSeconds: 5
httpGet:
path: /health
port: 9876
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/healthz/pod.yaml
在pod的清单规范中，我们定义了如下的存活探针：
livenessProbe:
initialDelaySeconds: 2
periodSeconds: 5
httpGet:
path: /health
port: 9876
Kubernetes将在初始化2秒后，每隔5是检查一次/health 端口是正常。
我们可以看到现在它是健康的：
$ kubectl describe pod hc
Name: hc
Namespace: default
Security Policy: anyuid
Node: 192.168.99.100/192.168.99.100
Start Time: Tue, 25 Apr 2017 16:21:11 +0100
Labels:
Status: Running
...
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
3s 3s 1 {default-scheduler } Normal Scheduled Successfully assigned hc to 192.168.99.100
3s 3s 1 {kubelet 192.168.99.100} spec.containers{sise} Normal Pulled Container image "mhausenblas/simpleservice:0.5.0"
already present on machine
3s 3s 1 {kubelet 192.168.99.100} spec.containers{sise} Normal Created Created container with docker id 8a628578d6ad; Security:[seccomp=unconfined]
2s 2s 1 {kubelet 192.168.99.100} spec.containers{sise} Normal Started Started container with docker id 8a628578d6ad
我们启动一个失败的pod,其中的容器会随机返回非200的状态码:
badpod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: badpod
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
env:
- name: HEALTH_MIN
value: "1000"
- name: HEALTH_MAX
value: "4000"
livenessProbe:
initialDelaySeconds: 2
periodSeconds: 5
httpGet:
path: /health
port: 9876
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/healthz/badpod.yaml
我们可以看到健康检查失败：
$ kubectl describe pod badpod
...
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
1m 1m 1 {default-scheduler } Normal Scheduled Successfully assigned badpod to 192.168.99.100
1m 1m 1 {kubelet 192.168.99.100} spec.containers{sise} Normal Created Created container with docker id 7dd660f04945; Security:[seccomp=unconfined]
1m 1m 1 {kubelet 192.168.99.100} spec.containers{sise} Normal Started Started container with docker id 7dd660f04945
1m 23s 2 {kubelet 192.168.99.100} spec.containers{sise} Normal Pulled Container image "mhausenblas/simpleservice:0.5.0" already present on machine
23s 23s 1 {kubelet 192.168.99.100} spec.containers{sise} Normal Killing Killing container with docker id 7dd660f04945: pod "badpod_default(53e5c06a-29cb-11e7-b44f-be3e8f4350ff)" container "sise" is unhealthy, it will be killed and re-created.
23s 23s 1 {kubelet 192.168.99.100} spec.containers{sise} Normal Created Created container with docker id ec63dc3edfaa; Security:[seccomp=unconfined]
23s 23s 1 {kubelet 192.168.99.100} spec.containers{sise} Normal Started Started container with docker id ec63dc3edfaa
1m 18s 4 {kubelet 192.168.99.100} spec.containers{sise} Warning Unhealthy Liveness probe failed: Get http://172.17.0.4:9876/health: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
我们同样可以通过下面的方式验证：
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
badpod 1/1 Running 4 2m
hc 1/1 Running 0 6m
从上面输出我们可以看到badpod因为健康检查失败，已经重启了4次。
除了存活探针livenessProbe，还可以设置就绪探针readinessProbe。可以通过相同的方式配置，但它有不同的用途和语义：它主要用于检测pod中容器的启动阶段。假设一个容器从外部存储例如S3或者数据库中加载数据用于初始化某些表。此时你希望容器就绪时，能获取到信号。
我们创建一个包含就绪探针的pod:
ready.yaml:
apiVersion: v1
kind: Pod
metadata:
name: ready
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
readinessProbe:
initialDelaySeconds: 10
httpGet:
path: /health
port: 9876
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/healthz/ready.yaml
通过观察pod的事件查看容器已经就绪：
$ kubectl describe pod ready
...
Conditions: [0/1888]
Type Status
Initialized True
Ready True
PodScheduled True
...
删除资源：
$ kubectl delete pod/hc pod/ready pod/badpod
8 环境变量
可以在pod的容器中设置环境变量,kubernetes会自动导出某些运行时的环境变量。
我们启动一个pod，传递环境变量SIMPLE_SERVICE_VERSION=1.0到容器中:
pod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: envs
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
env:
- name: SIMPLE_SERVICE_VERSION
value: "1.0"
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/envs/pod.yaml
$ kubectl describe pod envs | grep IP:
IP: 172.17.0.3
我们查看环境变量SIMPLE_SERVICE_VERSION是否生效:
[cluster] $ curl 172.17.0.3:9876/info
{"host": "172.17.0.3:9876", "version": "1.0", "from": "172.17.0.1"}
默认的版本为0.5.0,因为传入了环境变量，所以变为1.0.
可以看到kubernetes自动导出的环境变量：
[cluster] $ curl 172.17.0.3:9876/env
{"version": "1.0", "env": "{'HOSTNAME': 'envs', 'DOCKER_REGISTRY_SERVICE_PORT': '5000', 'KUBERNETES_PORT_443_TCP_ADDR': '172.30.0.1', 'ROUTER_PORT_80_TCP_PROTO': 'tcp', 'KUBERNETES_PORT_53_UDP_PROTO': 'udp', 'ROUTER_SERVICE_HOST': '172.30.246.127', 'ROUTER_PORT_1936_TCP_PROTO': 'tcp', 'KUBERNETES_SERVICE_PORT_DNS': '53', 'DOCKER_REGISTRY_PORT_5000_TCP_PORT': '5000', 'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'ROUTER_SERVICE_PORT_443_TCP': '443', 'KUBERNETES_PORT_53_TCP': 'tcp://172.30.0.1:53', 'KUBERNETES_SERVICE_PORT': '443', 'ROUTER_PORT_80_TCP_ADDR': '172.30.246.127', 'LANG': 'C.UTF-8', 'KUBERNETES_PORT_53_TCP_ADDR': '172.30.0.1', 'PYTHON_VERSION': '2.7.13', 'KUBERNETES_SERVICE_HOST': '172.30.0.1', 'PYTHON_PIP_VERSION': '9.0.1', 'DOCKER_REGISTRY_PORT_5000_TCP_PROTO': 'tcp', 'REFRESHED_AT': '2017-04-24T13:50', 'ROUTER_PORT_1936_TCP': 'tcp://172.30.246.127:1936', 'KUBERNETES_PORT_53_TCP_PROTO': 'tcp', 'KUBERNETES_PORT_53_TCP_PORT': '53', 'HOME': '/root', 'DOCKER_REGISTRY_SERVICE_HOST': '172.30.1.1', 'GPG_KEY': 'C01E1CAD5EA2C4F0B8E3571504C367C218ADD4FF', 'ROUTER_SERVICE_PORT_80_TCP': '80', 'ROUTER_PORT_443_TCP_ADDR': '172.30.246.127', 'ROUTER_PORT_1936_TCP_ADDR': '172.30.246.127', 'ROUTER_SERVICE_PORT': '80', 'ROUTER_PORT_443_TCP_PORT': '443', 'KUBERNETES_SERVICE_PORT_DNS_TCP': '53', 'KUBERNETES_PORT_53_UDP_ADDR': '172.30.0.1', 'KUBERNETES_PORT_53_UDP': 'udp://172.30.0.1:53', 'KUBERNETES_PORT': 'tcp://172.30.0.1:443', 'ROUTER_PORT_1936_TCP_PORT': '1936', 'ROUTER_PORT_80_TCP': 'tcp://172.30.246.127:80', 'KUBERNETES_SERVICE_PORT_HTTPS': '443', 'KUBERNETES_PORT_53_UDP_PORT': '53', 'ROUTER_PORT_80_TCP_PORT': '80', 'ROUTER_PORT': 'tcp://172.30.246.127:80', 'ROUTER_PORT_443_TCP': 'tcp://172.30.246.127:443', 'SIMPLE_SERVICE_VERSION': '1.0', 'ROUTER_PORT_443_TCP_PROTO': 'tcp', 'KUBERNETES_PORT_443_TCP': 'tcp://172.30.0.1:443', 'DOCKER_REGISTRY_PORT_5000_TCP': 'tcp://172.30.1.1:5000', 'DOCKER_REGISTRY_PORT': 'tcp://172.30.1.1:5000', 'KUBERNETES_PORT_443_TCP_PORT': '443', 'ROUTER_SERVICE_PORT_1936_TCP': '1936', 'DOCKER_REGISTRY_PORT_5000_TCP_ADDR': '172.30.1.1', 'DOCKER_REGISTRY_SERVICE_PORT_5000_TCP': '5000', 'KUBERNETES_PORT_443_TCP_PROTO': 'tcp'}"}
或者使用kubectl exec命名连上容器直接输出环境变量：
$ kubectl exec envs -- printenv
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=envs
SIMPLE_SERVICE_VERSION=1.0
KUBERNETES_PORT_53_UDP_ADDR=172.30.0.1
KUBERNETES_PORT_53_TCP_PORT=53
ROUTER_PORT_443_TCP_PROTO=tcp
DOCKER_REGISTRY_PORT_5000_TCP_ADDR=172.30.1.1
KUBERNETES_SERVICE_PORT_DNS_TCP=53
ROUTER_PORT=tcp://172.30.246.127:80
...
删除资源：
$ kubectl delete pod/envs
9 命名空间
命名空间提供了一个将集群资源划分为更小单元的机制。 我们可以把它想象成和别人共享的工作空间。有些资源，例如pod和service是命名空间范围的。而有些资源，例如node是集群范围的。 开发者经常使用的是特定的命名空间，而管理员则负责管理这些命名空间，例如控制范围资源的配额。
列出所有的命名空间：
$ kubectl get ns
NAME STATUS AGE
default Active 13d
kube-system Active 13d
namingthings Active 12d
openshift Active 13d
openshift-infra Active 13d
使用describe命令查看命名空间详情：
$ kubectl describe ns default
Name: default
Labels:
Status: Active
No resource quota.
No resource limits.
创建一个test命名空间：
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/ns/ns.yaml
namespace "test" created
$ kubectl get ns
NAME STATUS AGE
default Active 13d
kube-system Active 13d
namingthings Active 12d
openshift Active 13d
openshift-infra Active 13d
test Active 3s
也可以使用命名：kubectl create namespace test 创建。
在新的命名空间启动pod:
pod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: podintest
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
$ kubectl apply --namespace=test -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/ns/pod.yaml
我们使用命名空间作为命令行的运行属性，同样可以用来部署相同的pod或者service到不同的命名空间。硬编码的方式如下，但缺乏灵活性：
apiVersion: v1
kind: Pod
metadata:
name: podintest
namespace: test
列出命名空间中的资源，例如我们的podintest pod:
$ kubectl get pods --namespace=test
NAME READY STATUS RESTARTS AGE
podintest 1/1 Running 0 16s
删除命名空间：
$ kubectl delete ns test
10 volumes
A Kubernetes volume is essentially a directory accessible to all containers running in a pod. In contrast to the container-local filesystem, the data in volumes is preserved across container restarts. The medium backing a volume and its contents are determined by the volume type:
- node-local types such as emptyDir or hostPath
- file-sharing types such as nfs
- cloud provider-specific types like awsElasticBlockStore, azureDisk, or gcePersistentDisk
- distributed file system types, for example glusterfs or cephfs
- special-purpose types like secret, gitRepo
A special type of volume is PersistentVolume, which we will cover elsewhere.
我们创建一个包含两个容器的pod，容器之间通过emtpyDir 卷交换数据：
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/volumes/pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: sharevol
spec:
containers:
- name: c1
image: centos:7
command:
- "bin/bash"
- "-c"
- "sleep 10000"
volumeMounts:
- name: xchange
mountPath: "/tmp/xchange"
- name: c2
image: centos:7
command:
- "bin/bash"
- "-c"
- "sleep 10000"
volumeMounts:
- name: xchange
mountPath: "/tmp/data"
volumes:
- name: xchange
emptyDir: {}
$ kubectl describe pod sharevol
Name: sharevol
Namespace: default
...
Volumes:
xchange:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
We first exec into one of the containers in the pod, c1, check the volume mount and generate some data:
$ kubectl exec -it sharevol -c c1 -- bash
[root@sharevol /]# mount | grep xchange
/dev/sda1 on /tmp/xchange type ext4 (rw,relatime,data=ordered)
[root@sharevol /]# echo 'some data' > /tmp/xchange/data
When we now exec into c2, the second container running in the pod, we can see the volume mounted at /tmp/data and are able to read the data created in the previous step:
$ kubectl exec -it sharevol -c c2 -- bash
[root@sharevol /]# mount | grep /tmp/data
/dev/sda1 on /tmp/data type ext4 (rw,relatime,data=ordered)
[root@sharevol /]# cat /tmp/data/data
some data
Note that in each container you need to decide where to mount the volume and that for emptyDir you currently can not specify resource consumption limits.
删除pod会自动删除相关的共享volume。
$ kubectl delete pod/sharevol
11 persistent volumes
A persistent volume (PV) is a cluster-wide resource that you can use to store data in a way that it persists beyond the lifetime of a pod. The PV is not backed by locally-attached storage on a worker node but by networked storage system such as EBS or NFS or a distributed filesystem like Ceph.
In order to use a PV you need to claim it first, using a persistent volume claim (PVC). The PVC requests a PV with your desired specification (size, speed, etc.) from Kubernetes and binds it then to a pod where you can mount it as a volume. So let’s create such a PVC, asking Kubernetes for 1 GB of storage using the default storage class:
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/pv/pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
myclaim Bound pvc-27fed6b6-3047-11e9-84bb-12b5519f9b58 1Gi RWO gp2-encrypted 18m
To understand how the persistency plays out, let’s create a deployment that uses above PVC to mount it as a volume into /tmp/persistent:
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/pv/deploy.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: pv-deploy
spec:
replicas: 1
template:
metadata:
labels:
app: mypv
spec:
containers:
- name: shell
image: centos:7
command:
- "bin/bash"
- "-c"
- "sleep 10000"
volumeMounts:
- name: mypd
mountPath: "/tmp/persistent"
volumes:
- name: mypd
persistentVolumeClaim:
claimName: myclaim
Now we want to test if data in the volume actually persists. For this we find the pod managed by above deployment, exec into its main container and create a file called data in the /tmp/persistent directory (where we decided to mount the PV):
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
pv-deploy-69959dccb5-jhxx 1/1 Running 0 16m
$ kubectl exec -it pv-deploy-69959dccb5-jhxxw -- bash
bash-4.2$ touch /tmp/persistent/data
bash-4.2$ ls /tmp/persistent/
data lost+found
It’s time to destroy the pod and let the deployment launch a new pod. The expectation is that the PV is available again in the new pod and the data in /tmp/persistent is still present. Let’s check that:
$ kubectl delete po pv-deploy-69959dccb5-jhxxw
pod pv-deploy-69959dccb5-jhxxw deleted
$ kubectl get po
NAME READY STATUS RESTARTS AGE
pv-deploy-69959dccb5-kwrrv 1/1 Running 0 16m
$ kubectl exec -it pv-deploy-69959dccb5-kwrrv -- bash
bash-4.2$ ls /tmp/persistent/
data lost+found
And indeed, the data file and its content is still where it is expected to be.
Note that the default behavior is that even when the deployment is deleted, the PVC (and the PV) continues to exist. This storage protection feature helps avoiding data loss. Once you’re sure you don’t need the data anymore, you can go ahead and delete the PVC and with it eventually destroy the PV:
$ kubectl delete pvc myclaim
persistentvolumeclaim "myclaim" deleted
The types of PV available in your Kubernetes cluster depend on the environment (on-prem or public cloud). Check out the Stateful Kubernetes reference site if you want to learn more about this topic.
12 secrets
You don’t want sensitive information such as a database password or an API key kept around in clear text. Secrets provide you with a mechanism to use such information in a safe and reliable way with the following properties:
- Secrets are namespaced objects, that is, exist in the context of a namespace
- You can access them via a volume or an environment variable from a container running in a pod
- The secret data on nodes is stored in tmpfs volumes
- A per-secret size limit of 1MB exists
- The API server stores secrets as plaintext in etcd
Let’s create a secret apikey that holds a (made-up) API key:
$ echo -n "A19fh68B001j" > ./apikey.txt
$ kubectl create secret generic apikey --from-file=./apikey.txt
secret "apikey" created
$ kubectl describe secrets/apikey
Name: apikey
Namespace: default
Labels:
Annotations:
Type: Opaque
Data
====
apikey.txt: 12 bytes
Now let’s use the secret in a pod via a volume:
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/secrets/pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: consumesec
spec:
containers:
- name: shell
image: centos:7
command:
- "bin/bash"
- "-c"
- "sleep 10000"
volumeMounts:
- name: apikeyvol
mountPath: "/tmp/apikey"
readOnly: true
volumes:
- name: apikeyvol
secret:
secretName: apikey
If we now exec into the container we see the secret mounted at /tmp/apikey:
$ kubectl exec -it consumesec -c shell -- bash
[root@consumesec /]# mount | grep apikey
tmpfs on /tmp/apikey type tmpfs (ro,relatime)
[root@consumesec /]# cat /tmp/apikey/apikey.txt
A19fh68B001j
Note that for service accounts Kubernetes automatically creates secrets containing credentials for accessing the API and modifies your pods to use this type of secret.
You can remove both the pod and the secret with:
$ kubectl delete pod/consumesec secret/apikey
13 logging
Logging is one option to understand what is going on inside your applications and the cluster at large. Basic logging in Kubernetes makes the output a container produces available, which is a good use case for debugging. More advanced setups consider logs across nodes and store them in a central place, either within the cluster or via a dedicated (cloud-based) service.
Let’s create a pod called logme that runs a container writing to stdout and stderr:
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/logging/pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: logme
spec:
containers:
- name: gen
image: centos:7
command:
- "bin/bash"
- "-c"
- "while true; do echo $(date) | tee /dev/stderr; sleep 1; done"
To view the five most recent log lines of the gen container in the logme pod, execute:
$ kubectl logs --tail=5 logme -c gen
Thu Apr 27 11:34:40 UTC 2017
Thu Apr 27 11:34:41 UTC 2017
Thu Apr 27 11:34:41 UTC 2017
Thu Apr 27 11:34:42 UTC 2017
Thu Apr 27 11:34:42 UTC 2017
To stream the log of the gen container in the logme pod (like tail -f), do:
$ kubectl logs -f --since=10s logme -c gen
Thu Apr 27 11:43:11 UTC 2017
Thu Apr 27 11:43:11 UTC 2017
Thu Apr 27 11:43:12 UTC 2017
Thu Apr 27 11:43:12 UTC 2017
Thu Apr 27 11:43:13 UTC 2017
...
Note that if you wouldn’t have specified --since=10s in the above command, you would have gotten all log lines from the start of the container.
You can also view logs of pods that have already completed their lifecycle. For this we create a pod called oneshot that counts down from 9 to 1 and then exits. Using the -p option you can print the logs for previous instances of the container in a pod:
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/logging/oneshotpod.yaml
apiVersion: v1
kind: Pod
metadata:
name: oneshot
spec:
containers:
- name: gen
image: centos:7
command:
- "bin/bash"
- "-c"
- "for i in 9 8 7 6 5 4 3 2 1 ; do echo $i ; done"
$ kubectl logs -p oneshot -c gen
9
8
7
6
5
4
3
2
1
You can remove the created pods with:
$ kubectl delete pod/logme pod/oneshot
14 jobs
A job in Kubernetes is a supervisor for pods carrying out batch processes, that is, a process that runs for a certain time to completion, for example a calculation or a backup operation.
Let’s create a job called countdown that supervises a pod counting from 9 down to 1:
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/jobs/job.yaml
You can see the job and the pod it looks after like so:
$ kubectl get jobs
NAME DESIRED SUCCESSFUL AGE
countdown 1 1 5s
$ kubectl get pods --show-all
NAME READY STATUS RESTARTS AGE
countdown-lc80g 0/1 Completed 0 16s
To learn more about the status of the job, do:
$ kubectl describe jobs/countdown
Name: countdown
Namespace: default
Image(s): centos:7
Selector: controller-uid=ff585b92-2b43-11e7-b44f-be3e8f4350ff
Parallelism: 1
Completions: 1
Start Time: Thu, 27 Apr 2017 13:21:10 +0100
Labels: controller-uid=ff585b92-2b43-11e7-b44f-be3e8f4350ff
job-name=countdown
Pods Statuses: 0 Running / 1 Succeeded / 0 Failed
No volumes.
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
2m 2m 1 {job-controller } Normal SuccessfulCreate Created pod: countdown-lc80g
And to see the output of the job via the pod it supervised, execute:
kubectl logs countdown-lc80g
9
8
7
6
5
4
3
2
1
To clean up, use the delete verb on the job object which will remove all the supervised pods:
$ kubectl delete job countdown
job "countdown" deleted
15 stateful sets
If you have a stateless app you want to use a deployment. However, for a stateful app you might want to use a StatefulSet. Unlike a deployment, the StatefulSet provides certain guarantees about the identity of the pods it is managing (that is, predictable names) and about the startup order. Two more things that are different compared to a deployment: for network communication you need to create a headless services and for persistency the StatefulSet manages a persistent volume per pod.
In order to see how this all plays together, we will be using an educational Kubernetes-native NoSQL datastore.
Let’s start with creating the stateful app, that is, the StatefulSet along with the persistent volumes and the headless service:
$ kubectl apply -f https://raw.githubusercontent.com/mhausenblas/mehdb/master/app.yaml
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mehdb
spec:
selector:
matchLabels:
app: mehdb
serviceName: "mehdb"
replicas: 2
template:
metadata:
labels:
app: mehdb
spec:
containers:
- name: shard
image: quay.io/mhausenblas/mehdb:0.6
ports:
- containerPort: 9876
env:
- name: MEHDB_DATADIR
value: "/mehdbdata"
livenessProbe:
initialDelaySeconds: 2
periodSeconds: 10
httpGet:
path: /status
port: 9876
readinessProbe:
initialDelaySeconds: 15
periodSeconds: 30
httpGet:
path: /status?level=full
port: 9876
volumeMounts:
- name: data
mountPath: /mehdbdata
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "ebs"
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
name: mehdb
labels:
app: mehdb
spec:
ports:
- port: 9876
clusterIP: None
selector:
app: mehdb
一分钟以后，可以看到对象已经创建完成：
$ kubectl get sts,po,pvc,svc
NAME DESIRED CURRENT AGE
statefulset.apps/mehdb 2 2 1m
NAME READY STATUS RESTARTS AGE
pod/mehdb-0 1/1 Running 0 1m
pod/mehdb-1 1/1 Running 0 56s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/data-mehdb-0 Bound pvc-bc2d9b3b-310d-11e9-aeff-123713f594ec 1Gi RWO ebs 1m
persistentvolumeclaim/data-mehdb-1 Bound pvc-d4b7620f-310d-11e9-aeff-123713f594ec 1Gi RWO ebs 56s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/mehdb ClusterIP None 9876/TCP 1m
Now we can check if the stateful app is working properly. To do this, we use the /status endpoint of the headless service mehdb:9876 and since we haven’t put any data yet into the datastore, we’d expect that 0 keys are reported:
$ kubectl run -it --rm jumpod --restart=Never --image=quay.io/mhausenblas/jump:0.2 -- curl mehdb:9876/status?level=full
If you don't see a command prompt, try pressing enter.
0
pod "jumpod" deleted
And indeed we see 0 keys being available, reported above.
Note that sometimes a StatefulSet is not the best fit for your stateful app. You might be better off defining a custom resource along with writing a custom controller to have finer-grained control over your workload.
16 init container
It’s sometimes necessary to prepare a container running in a pod. For example, you might want to wait for a service being available, want to configure things at runtime, or init some data in a database. In all of these cases, init containers are useful. Note that Kubernetes will execute all init containers (and they must all exit successfully) before the main container(s) are executed.
So let’s create an deployment consisting of an init container that writes a message into a file at /ic/this and the main (long-running) container reading out this file, then:
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/ic/deploy.yaml
Now we can check the output of the main container:
$ kubectl get deploy,po
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.extensions/ic-deploy 1 1 1 1 11m
NAME READY STATUS RESTARTS AGE
pod/ic-deploy-bf75cbf87-8zmrb 1/1 Running 0 59s
$ kubectl logs ic-deploy-bf75cbf87-8zmrb -f
INIT_DONE
INIT_DONE
INIT_DONE
INIT_DONE
INIT_DONE
^C
If you want to learn more about init containers and related topics, check out the blog post Kubernetes: A Pod’s Life.
17 nodes
In Kubernetes, nodes are the (virtual) machines where your workloads in shape of pods run. As a developer you typically don’t deal with nodes directly, however as an admin you might want to familiarize yourself with node operations.
To list available nodes in your cluster (note that the output will depend on the environment you’re using, I’m using Minishift):
$ kubectl get nodes
NAME STATUS AGE
192.168.99.100 Ready 14d
One interesting task, from a developer point of view, is to make Kubernetes schedule a pod on a certain node. For this, we first need to label the node we want to target:
$ kubectl label nodes 192.168.99.100 shouldrun=here
node "192.168.99.100" labeled
Now we can create a pod that gets scheduled on the node with the label shouldrun=here:
pod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: onspecificnode
spec:
containers:
- name: sise
image: mhausenblas/simpleservice:0.5.0
ports:
- containerPort: 9876
nodeSelector:
shouldrun: here
$ kubectl apply -f https://raw.githubusercontent.com/openshift-evangelists/kbe/master/specs/nodes/pod.yaml
$ kubectl get pods --output=wide
NAME READY STATUS RESTARTS AGE IP NODE
onspecificnode 1/1 Running 0 8s 172.17.0.3 192.168.99.100
获取特定节点的详情，例如192.168.99.100:
$ kubectl describe node 192.168.99.100
Name: 192.168.99.100
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/hostname=192.168.99.100
shouldrun=here
Taints:
CreationTimestamp: Wed, 12 Apr 2017 17:17:13 +0100
Phase:
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
OutOfDisk False Thu, 27 Apr 2017 14:55:49 +0100 Thu, 27 Apr 2017 09:18:13 +0100 KubeletHasSufficientDisk kubelet has sufficient disk space available
MemoryPressure False Thu, 27 Apr 2017 14:55:49 +0100 Wed, 12 Apr 2017 17:17:13 +0100 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Thu, 27 Apr 2017 14:55:49 +0100 Wed, 12 Apr 2017 17:17:13 +0100 KubeletHasNoDiskPressure kubelet has no disk pressure
Ready True Thu, 27 Apr 2017 14:55:49 +0100 Thu, 27 Apr 2017 09:18:24 +0100 KubeletReady kubelet is posting ready status
Addresses: 192.168.99.100,192.168.99.100,192.168.99.100
Capacity:
alpha.kubernetes.io/nvidia-gpu: 0
cpu: 2
memory: 2050168Ki
pods: 20
Allocatable:
alpha.kubernetes.io/nvidia-gpu: 0
cpu: 2
memory: 2050168Ki
pods: 20
System Info:
Machine ID: 896b6d970cd14d158be1fd1c31ff1a8a
System UUID: F7771C31-30B0-44EC-8364-B3517DBC8767
Boot ID: 1d589b36-3413-4e82-af80-b2756342eed4
Kernel Version: 4.4.27-boot2docker
OS Image: CentOS Linux 7 (Core)
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://1.12.3
Kubelet Version: v1.5.2+43a9be4
Kube-Proxy Version: v1.5.2+43a9be4
ExternalID: 192.168.99.100
Non-terminated Pods: (3 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits
--------- ---- ------------ ---------- --------------- -------------
default docker-registry-1-hfpzp 100m (5%) 0 (0%) 256Mi (12%) 0 (0%)
default onspecificnode 0 (0%) 0 (0%) 0 (0%) 0 (0%)
default router-1-cdglk 100m (5%) 0 (0%) 256Mi (12%) 0 (0%)
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.
CPU Requests CPU Limits Memory Requests Memory Limits
------------ ---------- --------------- -------------
200m (10%) 0 (0%) 512Mi (25%) 0 (0%)
No events.
Note that there are more sophisticated methods than shown above, such as using affinity, to assign pods to nodes and depending on your use case, you might want to check those out as well.
18 API server
有时直接访问kubernetes的API server是有用的，例如导出或者测试目的。
可以在本地环境中开启代理来访问：
$ kubectl proxy --port=8080
Starting to serve on 127.0.0.1:8080
在不同的终端会话中查看：
$ curl http://localhost:8080/api/v1
{
"kind": "APIResourceList",
"groupVersion": "v1",
"resources": [
{
...
{
"name": "services/status",
"singularName": "",
"namespaced": true,
"kind": "Service",
"verbs": [
"get",
"patch",
"update"
]
}
]
}
或者不用代理，采用kubectl也可以获的相同的效果：
$ kubectl get --raw=/api/v1
可以使用下面的命令导出API的版本和资源:
$ kubectl api-versions
admissionregistration.k8s.io/v1beta1
...
v1
$ kubectl api-resources
NAME SHORTNAMES APIGROUP NAMESPACED KIND
bindings true Binding
...
storageclasses sc storage.k8s.io