一、Pod简介
二、标签
三、Pod控制器ReplicaController
四、Pod控制器Deployment
五、Pod控制器DaemonSet
六、Service
七、Ingress
八、Job/CronJob
九、ServiceAccount/UserAccount
十、Secret&Configmap
一、Pod简介
Deployment为Pod和Replica Set（升级版的 Replication Controller）提供声明式更新。
k8s的常见资源：
- workload（对外提供服务的）：Pod、ReplicaSet、Deployment、StaefulSet、Job、CronJob...
- Service发现及均衡：service、ingress
- 配置与存储：Volume、CSI(存储接口)
ConfigMap、Secret、DownwardAPI- 集群级资源：
Namespace、Node、Role、ClusterRole、RoleBinding、ClusterRoleBinding- 元数据资源：
HPA、PodTemplate、LimitRange、
一个Pod（就像一群鲸鱼，或者一个豌豆夹）相当于一个共享context的配置组，在同一个context下，应用可能还会有独立的cgroup隔离机制，一个Pod是一个容器环境下的“逻辑主机”，它可能包含一个或者多个紧密相连的应用，这些应用可能是在同一个物理主机或虚拟机上。
Pod 的context可以理解成多个linux命名空间的联合
PID 命名空间（同一个Pod中应用可以看到其它进程）
网络 命名空间（同一个Pod的中的应用对相同的IP地址和端口有权限）
IPC 命名空间（同一个Pod中的应用可以通过VPC或者POSIX进行通信）
UTS 命名空间（同一个Pod中的应用共享一个主机名称）1、Pod种类：
自主式pod。不接受pod控制器管理，删除后不会自动创建
Pod控制器管理的Pod，删除后会自动创建，如果确实要修改数量，可以使用scale调整
2、资源清单格式：
[root@registry metrics-server]# kubectl get pods $pod_name -o yaml #可以查看当前pod的yaml格式定义
YAML格式的基本格式：apiVersion:(group/version)、metadata、kind、spec、status //amkss
[root@registry metrics-server]# kubectl explain pod #可以查pod的yaml定义方法
apiVersion:定义使用的api版本信息
kind: 资源类型，比如pod
metadatab:元数据信息
name: 名称
annotations ：注解
labels ：标签，字符串映射格式
namespace ：namespace名称
status:当前pod的状态，动态生成
sepc:
containers:pod内容器定义
restartPolicy:重启策略OnFailure,Never,Default to Always
nodeSelector:根据node标签调度pod
nodeName:指定node
hostname:pod的名称
hostPID:使用host PID
hostNetwork:使用hostNetwork
affinity:亲和性调度 //nodeAffinity,podAffinity,podAntiAffinity
serviceAccountName:pod使用的sa，在serviceAccount重点说明
volumes:要创建的volume,在pv详细说明
tolerations:容忍度Taints与tolerations一起工作确保pod不会被调度到不合适的节点上 3、containers
[root@registry metrics-server]# kubectl explain pods.spec.containers
args <[]string> #Arguments to the entrypoint
command <[]string> #Entrypoint array， The docker image's ENTRYPOINT is used if this is not provided
env <[]Object> #环境变量
envFrom <[]Object> #
image
imagePullPolicy #镜像拉取策略Always, Never(本地没有就不下载，需要用户手动pull), IfNotPresent. 如果镜像tag是latest，默认策略是Defaults to Always(因为latest可能会变，latest会指向一个新的标签)，其他的标签则默认是IfNotPresent;并且这个策略不能修改
lifecycle dockerfile中如果只有cmd，就运行cmd，如果cmd和entrypoint都有，cmd的内容将作为参数传递给entrypoint
https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/
============================================
Description Docker field name Kubernetes field name
The command run by the container Entrypoint command
The arguments passed to the command Cmd args
============================================
If you do not supply command or args for a Container, the defaults defined in the Docker image are used.
If you supply a command but no args for a Container, only the supplied command is used. The default EntryPoint and the default Cmd defined in the Docker image are ignored.
If you supply only args for a Container, the default Entrypoint defined in the Docker image is run with the args that you supplied.
If you supply a command and args, the default Entrypoint and the default Cmd defined in the Docker image are ignored. Your command is run with your args.
Image Entrypoint Image Cmd Container command Container args Command run
[/ep-1] [foo bar] [ep-1 foo bar]
[/ep-1] [foo bar] [/ep-2] [ep-2]
[/ep-1] [foo bar] [zoo boo] [ep-1 zoo boo]
[/ep-1] [foo bar] [/ep-2] [zoo boo] [ep-2 zoo boo] ====================================
示例：
[root@registry work]# cat broker.yaml
apiVersion: v1
kind: Pod
metadata:
name: appv1
namespace: default
labels:
name: broker
version: latest
spec:
containers:
- name: broker
args:
env:
- name: verbase
value: testv1
- name: MY_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
command:
- "ping"
- "-i 2"
- "127.0.0.1"
image: "192.168.192.234:888/broker:latest"
imagePullPolicy: Always
- name: nginx
image: 192.168.192.234:888/nginx:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
hostPort: 8888
protocol: TCP
dnsPolicy: Default
restartPolicy: Always4、lifecycle
容器生命周期钩子（Container Lifecycle Hooks）监听容器生命周期的特定事件，并在事件发生时执行已注册的回调函数
支持2种钩子：
postStart：容器启动后执行，注意由于是异步执行，它无法保证一定在ENTRYPOINT之后运行。如果失败，容器会被杀死，并根据RestartPolicy决定是否重启
preStop：容器停止前执行，常用于资源清理。如果失败，容器同样也会被杀死
回调函数支持两种方式
exec：在容器内执行命令
httpGet：向指定URL发起GET请求
示例：
[root@registry work]# cat v2.yaml
apiVersion: v1
kind: Pod
metadata:
name: lifecycle-demo
spec:
containers:
- name: lifecycle-demo-container
image: nginx
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]
preStop:
exec:
command: ["/usr/sbin/nginx","-s","quit"]5、容器生命周期探测
k8s支持2中类型的Pod生命周期检测：
1)liveness Probe存活，容器状态
2)readiness Probe就绪型检测，是否服务Ready
Pod的状态：pending、running、failed、succeed、Unknown
探针类型三种：Exec、TCPSocketAction、HTTPGetAction
[root@registry work]# kubectl explain pods.spec.containers.readinessProbe
[root@registry work]# kubectl explain pods.spec.containers.livenessProbe
exec 为什么要做readnessProbe和livenessProbe:
比如service根据label关联容器，一个新建的容器服务启动可能要10s，在刚创建成功后就被service关联
收到的请求就会失败，最好是在服务运行就绪(ready)后再提供服务
[root@master1 ~]# kubectl explain pods.spec.containers.readinessProbe
[root@master1 yaml]# cat readness.yaml
apiVersion: v1
kind: Pod
metadata:
name: readiness-exec-pod
namespace: default
spec:
containers:
- name: readiness-container
image: 192.168.192.234:888/nginx
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
livenessProbe:
httpGet:
port: http
path: /index.html
initialDelaySeconds: 1
periodSeconds: 36、资源限制
1)resources
[root@registry work]# kubectl explain pods.spec.containers.resources
spec.containers[].resources.limits.cpu
spec.containers[].resources.limits.memory
spec.containers[].resources.requests.cpu
spec.containers[].resources.requests.memory
[root@registry work]# cat v3.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- image: nginx
name: nginx
resources:
requests:
cpu: "300m"
memory: "56Mi"
limits:
cpu: "500m"
memory: "128Mi"mem支持单位：Ki | Mi | Gi | Ti | Pi 等
2)限制网络带宽
可以通过给Pod增加kubernetes.io/ingress-bandwidth和kubernetes.io/egress-bandwidth这两个annotation来限制Pod的网络带宽
目前只有kubenet网络插件支持限制网络带宽，其他CNI网络插件暂不支持这个功能。
apiVersion: v1
kind: Pod
metadata:
name: qos
annotations:
kubernetes.io/ingress-bandwidth: 3M
kubernetes.io/egress-bandwidth: 4M
spec:
containers:
- name: iperf3
image: networkstatic/iperf3
command:
- iperf3
- -s7、Init Container
Init Container在所有容器运行之前执行（run-to-completion），常用来初始化配置。
[root@registry ~]# cat v3.yaml
apiVersion: v1
kind: Pod
metadata:
name: init-demo
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
volumeMounts:
- name: workdir
mountPath: /usr/share/nginx/html
initContainers:
- name: install
image: busybox
command:
- wget
- "-O"
- "/work-dir/index.html"
- http://kubernetes.io
volumeMounts:
- name: workdir
mountPath: "/work-dir"
dnsPolicy: Default
volumes:
- name: workdir
emptyDir: {}二、标签
1、标签
一个资源resouces可存在多个label，一个label也可以应用于多个resources
每个标签都可以被label selector进行匹配
label可以在资源创建时yaml或者命令创建资源时定义，也可以在创建后命令添加
key=value
key：字母、数字、_、-、.
value：可以为空、只能以字母或者数字开头或者结尾，中间可使用
[root@master1 ~]# kubectl get pods --show-labels
NAME READY STATUS RESTARTS AGE LABELS
client 1/1 Running 0 5d23h run=client
nginx-deloy-67f8c9dc5c-bgx9k 1/1 Running 0 5d22h pod-template-hash=67f8c9dc5c,run=nginx-deloy
[root@master1 ~]# kubectl get pods -L=run,nginx //输出增加run和nginx两个列、
NAME READY STATUS RESTARTS AGE RUN NGINX
client 1/1 Running 0 5d23h client
nginx-deloy-67f8c9dc5c-bgx9k 1/1 Running 0 5d22h nginx-deloy
[root@master1 ~]# kubectl get pods -l pod-template-hash //具有标签 pod-template-hash 的pod
[root@master1 ~]# kubectl get pods -l run=client //标签run为client的
NAME READY STATUS RESTARTS AGE
client 1/1 Running 0 5d23h添加标签：
[root@master1 ~]# kubectl label pods client name=label1,run
[root@master1 ~]# kubectl label pods client name=label2 --overwrite //强制修改标签删除标签：
[root@master1 ~]# kubectl label client client name- //key- 即可删除标签2、标签选择器：
等值关系：=,==,!=
集合关系：
KEY in (value1,value2,...) # kubectl get pods --show-labels -l "name notin (label1,label2)
Key not in (value1,value2...)# kubectl get pods --show-labels -l "name in (label1,label2)"
Key 存在
!key 不存在 # kubectl get pods --show-labels -l "! name"
许多资源支持内嵌字段定义其使用的标签选择器：
[root@registry ~]# kubectl explain deployment.spec.selector
matchLabels:直接给定键值
mathExpressions:{key:"KEY",operator:"OPERATOR",value:[VAL1,VAL2,VAL3..]}
操作符：
In,NotIn:value字段的值必须为非空列表
Exists,NotExists:value字段的值必须为空列表
标签的对象可以是：pods、node...
annotations:与label不同的地方在于、他不能用于挑选资源对象，仅用于对象提供“元数据”
可以在yaml文件中定义，也可以在
[root@registry ~]# kubectl describe pods $pod_name 查看 三、Pod控制器ReplicaSet
注：Replication Controller被下一代ReplicaSet副本控制器替代
ReplicaSet是下一代复本控制器。ReplicaSet和 Replication Controller之间的唯一区别是现在的选择器支持。
Replication Controller 保证了在所有时间内，都有特定数量的Pod副本正在运行，如果太多了，Replication Controller就杀死几个，如果太少了，Replication Controller会新建几个
Deployments是一个更高层次的概念，它管理ReplicaSets，并提供对pod的声明性更新以及许多其他的功能，因此一般建议使用Deployment
Pod控制器：
ReplicationController:
ReplicaSet：确保副本处于用户期待状态,新的RC，支持动态扩容，（无状态pod资源） #核心概念：标签选择器、用户期望的副本数、Pod资源模板
Deployment工作在ReplicaSet之上，Deploement支持滚动更新等等
DaemonSet：集群的所有同一个label的node都运行一个pod副本
Job：只操作一次的Pod，Job负责保证任务正常运行结束，而不是异常任务
CronJob:周期性job，
StatefuleSet：有状态Pod，需要自定义操作内容
TPR:Third party resource，1.2+ 1.7废弃
CDR：custom defined resource，1.8+
1、控制器ReplicationSet
[root@master1 ~]# kubectl explain ReplicaSet //kubectl explain rs
[root@master1 yaml]# kubectl create -f rs.yaml
replicaset.apps/myrs created
[root@master1 yaml]# cat rs.yaml
apiVersion: apps/v1
kind: ReplicaSet
spec:
replicas: 2
selector:
matchLabels:
app: myapp
release: beta
template:
metadata:
name: myapp-pod
labels:
app: myapp
release: beta
environment: dev
spec:
containers:
- name: myrs-container
image: 192.168.192.234:888/nginx:latest
ports:
- name: http
containerPort: 80
metadata:
name: myrs
namespace: default
[root@master1 yaml]#
[root@master1 yaml]# kubectl get pods -l app
NAME READY STATUS RESTARTS AGE
myrs-fnhwd 1/1 Running 0 61s
myrs-x5zgh 1/1 Running 0 61s如果为已有的pod添加上pod标签 app: myapp release: beta ,pod控制器会选取其中一个进行干掉
[root@master1 yaml]# kubectl get pods --show-labels
NAME READY STATUS RESTARTS AGE LABELS
client 1/1 Running 0 10d run=client
liveness-httpget-pod 1/1 Running 1 4d
myrs-fnhwd 1/1 Running 0 5m5s app=myapp,environment=dev,release=beta
myrs-x5zgh 1/1 Running 0 5m5s app=myapp,environment=dev,release=beta
nginx-deloy-67f8c9dc5c-bgx9k 1/1 Running 0 10d pod-template-hash=67f8c9dc5c,run=nginx-deloy
readiness-exec-pod 1/1 Running 0 3d23h
[root@master1 yaml]# kubectl label pods liveness-httpget-pod app=myapp release=beta --overwrite
pod/liveness-httpget-pod labeled
[root@master1 ~]# kubectl get pods --show-labels -l app,release //发现myrs被干掉一个
NAME READY STATUS RESTARTS AGE LABELS
liveness-httpget-pod 1/1 Running 1 4d app=myapp,release=beta
myrs-x5zgh 1/1 Running 0 7m28s app=myapp,environment=dev,release=beta 2、Pod数量调整
1)kubectl scale命令
2)修改yaml方式 ,然后kubectl apply -f yaml文件
3)Kubectl edit ReplicaSet $rs名称
更新升级：更改容器iamge版本
kubectl edit rs $rs名称也可以，但是只修改了控制器(rs)的版本，只有在pod被删除后才会使用新image
Deployment是建立在RS之上的，支持滚动升级，支持控制更新逻辑和更新策略(最多/最少)多个pods
3、ReplicaSet作为Horizontal Pod Autoscalers(HPA)
[root@registry ~]# cat v3.yaml
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
name: frontend-scaler
spec:
scaleTargetRef:
kind: ReplicaSet
name: frontend
minReplicas: 3
maxReplicas: 10
targetCPUUtilizationPercentage: 50四、Pod控制器Deployment
1、deployment的镜像更新
[root@registry ~]# kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1
[root@registry ~]# kubectl edit deployment/nginx-deployment
[root@registry ~]# kubectl rollout status deployment/nginx-deployment #查看 rollout 的状态，只要执行
Deployment 可以保证在升级时只有一定数量的 Pod 是 down 的。默认的，它会确保至少有比期望的Pod数量少一个是up状态（最多一个不可用）。
Deployment 同时也可以确保只创建出超过期望数量的一定数量的 Pod。默认的，它会确保最多比期望的Pod数量多一个的 Pod 是 up 的（最多1个 surge ）。
[root@registry work]# kubectl describe deployment $deployment_名称
RollingUpdateStrategy: 1 max unavailable, 25% max surge #查看更新策略2、更新策略
[root@registry work]# kubectl explain deployment.spec.strategy #镜像更新策略
rollingUpdate #仅到type位RollingUpdate的时候有效
maxSurge #最大超出pod个数，百分比或者更个数
maxUnavailable #最大不可用
type #"Recreate" or "RollingUpdate"默认RollingUpdate
[root@master1 yaml]# cat deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: dpv1
namespace: default
spec:
replicas: 2
selector:
matchLabels:
name: mydp
version: dpv1
template:
metadata:
labels:
name: mydp
version: dpv1
spec:
containers:
- name: mydp
image: 192.168.192.234:888/nginx:latest
ports:
- name: dpport
containerPort: 80
strategy:
rollingUpdate:
maxSurge: 1
[root@master1 yaml]# kubeclt apploy -f mydp.yaml //apply可以执行多次，自动更新
label：pod、replicaSet、deployment都可以有label
[root@master1 yaml]# kubectl explain deployment.metadata.labels //deployment可以简写为deploy(限kubectl命令行)
[root@master1 yaml]# kubectl explain replicaSet.metadata.labels
[root@master1 yaml]# kubectl explain pods.metadata.labels
[root@master1 yaml]# kubectl get deployment --show-labels
NAME READY UP-TO-DATE AVAILABLE AGE LABELS
dpv1 2/2 2 2 6m2s
[root@master1 yaml]# kubectl get replicaSet --show-labels
NAME DESIRED CURRENT READY AGE LABELS
dpv1-7b7df4f86c 2 2 2 6m5s name=mydp,pod-template-hash=7b7df4f86c,version=dpv1
[root@master1 yaml]# kubectl get pods --show-labels -l version=dpv1
NAME READY STATUS RESTARTS AGE LABELS
dpv1-7b7df4f86c-9sdk8 1/1 Running 0 6m11s name=mydp,pod-template-hash=7b7df4f86c,version=dpv1
dpv1-7b7df4f86c-nv6xv 1/1 Running 0 6m11s name=mydp,pod-template-hash=7b7df4f86c,version=dpv1 Pod-template-hash label:
当 Deployment 创建或者接管 ReplicaSet 时，Deployment controller 会自动为 Pod 添加 pod-template-hash label。这样做的目的是防止 Deployment 的子ReplicaSet 的 pod 名字重复
通过将 ReplicaSet 的 PodTemplate 进行哈希散列，使用生成的哈希值作为 label 的值，并添加到 ReplicaSet selector 里、 pod template label 和 ReplicaSet 管理中的 Pod 上。
3、修改Pod
修改yaml的image的版本号会看到整个更新过程
[root@master1 ~]# kubectl get pods --show-labels -l version -w
[root@master1 ~]# kubectl apply -f deployment.yaml
deployment.apps/dpv1 configured
查看image更新生效
[root@master1 ~]# kubectl describe pods $pods
[root@master1 ~]# kubectl get rs -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
dpv1-5778f9d958 2 2 2 11m mydp 192.168.192.234:888/nginx:v2 name=mydp,pod-template-hash=5778f9d958,version=dpv1
dpv1-7b7df4f86c 0 0 0 26m mydp 192.168.192.234:888/nginx:latest name=mydp,pod-template-hash=7b7df4f86c,version=dpv1
可以看到保存了2个模板，一个为0，一个为2
[root@master1 yaml]# kubectl rollout history deployment dpv1 //查看版本历史
deployment.extensions/dpv1
REVISION CHANGE-CAUSE
1
2 修改副本数量
方法1：scale
方法2：kubectl edit
方法3：kubectl apply -f *.yaml
方法4：kubectl patch 以打补丁方式操作
[root@master1 yaml]# kubectl patch deployment dpv1 -p '{"spec":{"replicas":5}}'
deployment.extensions/dpv1 patched
如果只修改镜像还可以使用：kubectl set image deployment dbv1 $image名称
4、pause
暂停容器，用户批量修改操作，容器再次启动会应用所有更新策略
[root@master1 yaml]# kubectl rollout pause $资源名称
[root@master1 yaml]# kubectl set image deployment dpv1 mydp=192.168.192.234:888/nginx:v3 && kubectl rollout pause deployment dpv1
[root@master1 ~]# kubectl get pods -o wide -l version -w //会在创建一个新的pod后停止
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
dpv1-5778f9d958-2jn9b 1/1 Running 0 44m 172.30.200.5 master3
dpv1-5778f9d958-bsctb 1/1 Running 0 22m 172.30.200.3 master3
dpv1-5778f9d958-tq64f 1/1 Running 0 44m 172.30.72.5 master1
dpv1-6fb8f5799f-286bg 0/1 Pending 0 0s
dpv1-6fb8f5799f-286bg 0/1 Pending 0 0s master2
dpv1-6fb8f5799f-286bg 0/1 ContainerCreating 0 0s master2
dpv1-6fb8f5799f-286bg 1/1 Running 0 1s 172.30.56.4 master2 新开一个终端：//会发现有4个，desired为 3
[root@master1 yaml]# kubectl get pods -l version
NAME READY STATUS RESTARTS AGE
dpv1-5778f9d958-2jn9b 1/1 Running 0 46m
dpv1-5778f9d958-bsctb 1/1 Running 0 24m
dpv1-5778f9d958-tq64f 1/1 Running 0 46m
dpv1-6fb8f5799f-286bg 1/1 Running 0 78s
[root@master1 yaml]# kubectl rollout status deployment dpv1 //查看当前rollout状态
Waiting for deployment "dpv1" rollout to finish: 1 out of 3 new replicas have been updated...
重新resume deployment：
[root@master1 ~]# kubectl rollout resume deployment dpv1
deployment.extensions/dpv1 resumed
回滚：undo ，默认回滚到上一个版本
[root@master1 ~]# kubectl rollout undo deployment dpv1 --to-revision=1 #第一个版本，不加参数--to-revision默认回滚到上一个版本
[root@registry work]# kubectl explain deploy.spec.revisionHistoryLimit #deployment 最多保留多少 revision 历史记录五、Pod控制器DaemonSet
DaemonSet保证在每个Node上都运行一个容器副本，常用来部署一些集群的日志、监控或者其他系统管理应用。典型的应用包括：
日志收集，比如fluentd，logstash等
系统监控，比如Prometheus Node Exporter，collectd，New Relic agent，Ganglia gmond等
系统程序，比如kube-proxy, kube-dns, glusterd, ceph等
[root@master1 yaml]# cat daemon.yaml
[root@registry prometheus]# cat node-exporter-ds.yml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: node-exporter
namespace: kube-system
labels:
k8s-app: node-exporter
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
version: v0.15.2
spec:
selector:
matchLabels:
k8s-app: node-exporter
version: v0.15.2
updateStrategy:
type: OnDelete
template:
metadata:
labels:
k8s-app: node-exporter
version: v0.15.2
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ''
spec:
priorityClassName: system-node-critical
containers:
- name: prometheus-node-exporter
image: "prom/node-exporter:v0.15.2"
imagePullPolicy: "IfNotPresent"
args:
- --path.procfs=/host/proc
- --path.sysfs=/host/sys
ports:
- name: metrics
containerPort: 9100
hostPort: 9100
volumeMounts:
- name: proc
mountPath: /host/proc
readOnly: true
- name: sys
mountPath: /host/sys
readOnly: true
resources:
limits:
cpu: 10m
memory: 50Mi
requests:
cpu: 10m
memory: 50Mi
hostNetwork: true
hostPID: true
volumes:
- name: proc
hostPath:
path: /proc
- name: sys
hostPath:
path: /sys
[root@registry prometheus]# kubectl explain daemonset.spec
updateStrategy #Pod更新策略
template #Pod模板
selector #node标签选择器，根据matchExpressions,matchLabels
revisionHistoryLimit #历史保留个数
minReadySeconds #最少多少秒必须保证可用六、Service
为了给客户端提供一个固定的访问地址：service; k8s提供的三种类型的ip:(node,pod,clusterIP),service的域名解析，强依赖与CoreDNS
kube-proxy始终监听着api-server 获取任何一个与service相关的资源变动，并在本地添加规则 ;api server---[watch]-----kube-proxy
service实现的三种模型：
1、user namespace
Client Pod[user空间]-->service(iptables kernel空间)-->kube-proxy-->转发到服务所在节点的kube-proxy->对应的服务pod
由kube-proxy负责调度
2、iptables
Client Pod-->servie(iptables) ->服务端
不再依赖kube-proxy调度
3、ipvs //version 1.11+之后 ipvs模块：ip_vs_rr,ip_vs_wrr,ip_vs_sh,nf_contrack_ipv4(连接追踪)需要添加专门的选项 KUBE_RPXOY_MODE=ipvs,ipvs
Client Pod-->servie(ipvs) ->服务端
service类型：
clusterip
nodeport :Client->node_ip:node_port->cluster_ip:cluster_port-->pod_ip:contaier_port
loadbalancer //云产品的lb,
ExtraName 集群外部的域名
FQDN:
CNAME->FQDN //集群外部的域名解析为集群内使用的域名
注：这几个都是需要clusterip的
无头服务：么有clusterip
headless service :
ServiceNmae-->PodIP
1、yaml示例service
[root@registry work]# cat v3.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
name: nginx
department: dev
replicas: 2
template:
metadata:
name: nginx
labels:
name: nginx
department: dev
spec:
containers:
- name: nginx
image: 192.168.192.234:888/nginx:latest
imagePullPolicy: IfNotPresent
restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
name: test-nginx
spec:
ports:
- name: http
port: 8080
targetPort: 80
protocol: TCP
- name: ssh
port: 2222
targetPort: 22
protocol: TCP
selector:
name: nginx
department: dev
type: ClusterIP[root@registry work]# kubectl explain svc.spec
clusterIP #服务ip，集群内有效
externalIPs #引入外部的IP
healthCheckNodePort #健康检查端口
ports #nodePort使用node的端口,port服务端口,targetPort pod端口
selector #标签选择器
type #服务类型 ExternalName, ClusterIP, NodePort, and LoadBalancer
ExternalName:集群外部的服务引用到集群内部使用
ClusterIP:集群内使用
NodePort:物理机网段2、命令行方式
[root@master1 ~]# kubectl run nginx-deloy --image=192.168.192.234:888/nginx:latest --port=80 --replicas=1
[root@master1 ~]# kubectl get pods -o wide //该pod在Replicas和Deployment中都可以看到，describe 的controlled by Replicas
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deloy-67f8c9dc5c-4nk68 1/1 Running 0 3m5s 172.30.72.3 master1 可以看到pod运行在master1上
直接curl $pod_ip:80 是可以的
问题1：Kubectl delete pods $podname 后新建的pod名称会发生改变
解决方法：kubectl expose service_ip:service_port dnat到 pod_ip:pod_port
Usage:
kubectl expose (-f FILENAME | TYPE NAME) [--port=port] [--protocol=TCP|UDP|SCTP] [--target-port=number-or-name]
[--name=name] [--external-ip=external-ip-of-service] [--type=type] [options]
--type='': Type for this service: ClusterIP, NodePort, LoadBalancer, or ExternalName. Default is 'ClusterIP'.
(-f FILENAME | TYPE NAME) //TYPE:控制器类型 NAME:控制器名称，expose之后 service提供一个固定的ip，但是仅限集群内部pod客户端使用确定pod控制器：kubectel describe pods $pod名称
[root@master1 ~]# kubectl expose deployment nginx-deloy --name nginx1 --port=80 --target-port=80 --protocol=TCP
service/nginx1 exposed
[root@master1 ~]# kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dnsutils-ds NodePort 10.254.98.197 80:31707/TCP 2d5h
kubernetes ClusterIP 10.254.0.1 443/TCP 2d9h
my-nginx ClusterIP 10.254.1.216 80/TCP 2d5h
nginx1 ClusterIP 10.254.63.79 80/TCP 11s pod网段内：直接curl $service_ip:80 也可以访问 //不在集群内的节点是无法访问的
3、域名解析
kubectl get svc -n kube-system //可以使用kubedns提供的域名解析功能，直接解析 service
创建的pod，默认容器内/etc/resolv.conf配置中会有nameserver为 $kubedns的ip 记录
配置search 域 //域名不全的情况下，会自动补全
[root@client /home/admin]
#cat /etc/resolv.conf
nameserver 10.244.0.2
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
//client机器内ping nginx1会自动补全 nginx1.default.svc.cluster.local (10.254.63.79)，解析的地址为clusterip
[root@master1 ~]# dig -t A nginx1.default.svc.cluster.local @10.254.0.2 这种是可以的
kubectl describe service nginx //可以看到实际的情况
kubectl get pods --show-labels
kubectl edit service $服务名 //可以直接修改服务信息
[root@master1 ~]# kubectl describe svc nginx1 //删除pod后ip会发生变化，但是使用cluster ip人仍然可以访问，
//svc和pod 通过label selector 关联
Name: nginx1
Namespace: default
Labels: run=nginx-deloy
Annotations:
Selector: run=nginx-deloy //关联 pod的标签为nginx-deploy
Type: ClusterIP
IP: 10.254.63.79
Port: 80/TCP
TargetPort: 80/TCP
Endpoints: 172.30.200.3:80
Session Affinity: None
Events:
[root@master1 ~]# kubectl get pods --show-labels
NAME READY STATUS RESTARTS AGE LABELS
client 1/1 Running 0 24m run=client
nginx-deloy-67f8c9dc5c-zpv4q 1/1 Running 0 38m pod-template-hash=67f8c9dc5c,run=nginx-deloy service-->endpoints-->pod //k8s有endpoints资源的概念
使用curl $cluster_ip:80的方式就可以访问了
资源记录：
SVC_NAME.NS_NAME.DOMAIN.LTD. #举例:mysvc.default.svc.cluter.local.
4、Headless service
没有clusterIP，客户端根据service可以获取 Label selector后的po列表，由客户端自行决定如何处理这个Pod列表
定义：headless service
[root@master1 yaml]# kubectl apply -f heaness.yaml
service/headness created
[root@master1 yaml]# cat heaness.yaml
apiVersion: v1
kind: Service
metadata:
name: headness
namespace: default
spec:
clusterIP: ""
ports:
- name: mysrvport
port: 80
targetPort: 80
selector:
name: mydp
version: dpv1
[root@master1 yaml]#
[root@master1 yaml]# cat heaness.yaml
apiVersion: v1
kind: Service
metadata:
name: headness
namespace: default
spec:
selector:
name: mydp
version: dpv1
clusterIP: "None"
ports:
- name: mysrvport
port: 80
targetPort: 80
[root@master1 yaml]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dnsutils-ds NodePort 10.254.98.197 80:31707/TCP 12d
headness ClusterIP None 80/TCP 9s //这里一定要是"None"
kubernetes ClusterIP 10.254.0.1 443/TCP 13d
mysvc ClusterIP 10.254.0.88 80/TCP 38m
[root@master1 yaml]# dig -t A headness.default.svc.cluster.local @172.30.200.2 //可以获取三个地址
headness.default.svc.cluster.local. 5 IN A 172.30.72.6
headness.default.svc.cluster.local. 5 IN A 172.30.56.4
headness.default.svc.cluster.local. 5 IN A 172.30.200.3
[root@master1 yaml]# dig -t A mysvc.default.svc.cluster.local @172.30.200.2 //有clusterip的只会被解析会clusterip
mysvc.default.svc.cluster.local. 5 IN A 10.254.0.88 七、Ingress
1、ingress
K8s暴露服务的方式：LoadBlancer Service、ExternalName、NodePort Service、Ingress
Ingress Controller通过与Kubernetes API交互，动态的去感知集群中Ingress 规则变化，然后读取他，按照他自己模板生成一段 Nginx 配置，再写到 Nginx Pod 里，最后 reload 一下
四层 调度器 不负责建立会话(看工作模型nat,dr,fullnat,tunn) client需要与后端建立会话
七层的调度器： client 只需要和调度器建立连接调度器管理会话
Ingress的资源类型有以下几种:
1、单Service资源型Ingress #只设置spec.backend，不设置其他的规则
2、基于URL路径进行流量转发 #根据spec.rules.http.paths 区分对同一个站点的不同的url的请求，并转发到不同主机
3、基于主机名称的虚拟主机 #spec.rules.host 设置不同的host来区分不同的站点
4、TLS类型的Ingress资源 #通过Secret获取TLS私钥和证书 (名为 tls.crt 和 tls.key)
Ingress controller #HAproxy/nginx/Traefik/Envoy (服务网格)
要调度的肯定不止一个服务，url 区分不同的虚拟主机，server，一个server定向不同的一组pod
service使用label selector始终关心 watch自己的pod，一旦pod发生变化，自己也理解作出相应的改变
ingress controller 借助于service(headless)关注pod的状态变化，service会把状态变化及时反馈给ingress
service对后端pod进行分类(headless),ingress在发现service分类的pod资源发生改变的时候，及时作出反应
ingress基于service对pod的分类，获取分类的pod ip列表，并注入ip列表信息到ingress中
创建ingress需要的步骤：
1、ingress controller
2、配置前端，server虚拟主机
3、根据service收集到的pod 信息，生成upstream server，反映在ingress并注册到ingress controller中
2、安装
安装步骤 #https://github.com/kubernetes/ingress-nginx/blob/master/docs/deploy/index.md
介绍：https://github.com/kubernetes/ingress-nginx: ingress-nginx
默认会监听所有的namespace，如果想要特定的监听--watch-namespace
如果单个host定义了不同路径，ingress会 合并配置
[root@master1 yaml]# kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
内容包含：
1)创建Namespace[ingress-nginx]
2)创建ConfigMap[nginx-configuration]、ConfigMap[tcp-services]、ConfigMap[udp-services]
3)创建RoleBinding[nginx-ingress-role-nisa-binding]=Role[nginx-ingress-role]+ServiceAccount[nginx-ingress-serviceaccount]
4)创建ClusterRoleBinding[nginx-ingress-clusterrole-nisa-binding]=ClusterRole[nginx-ingress-clusterrole]+ServiceAccount[nginx-ingress-serviceaccount]
5)Deployment[nginx-ingress-controller]应用ConfigMap[nginx-configuration]、ConfigMap[tcp-services]、ConfigMap[udp-services]作为配置，
[root@master1 yaml]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml
[root@master1 yaml]# cat service-nodeport.yaml #修改后
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx
namespace: ingress-nginx
spec:
type: NodePort
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
nodePort: 30080
- name: https
port: 443
targetPort: 443
protocol: TCP
nodePort: 30443
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
[root@master1 yaml]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx NodePort 10.254.90.250 80:30080/TCP,443:30443/TCP 10m
验证：
[root@master1 ingress]# kubectl describe svc ingress-nginx -n ingress-nginx
Name: ingress-nginx
Namespace: ingress-nginx
Labels: app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/part-of=ingress-nginx
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/par...
Selector: app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
Type: NodePort
IP: 10.254.90.250
Port: http 80/TCP
TargetPort: 80/TCP
NodePort: http 30080/TCP
Endpoints: 172.30.72.7:80
Port: https 443/TCP
TargetPort: 443/TCP
NodePort: https 30443/TCP
Endpoints: 172.30.72.7:443 //这个endpoint一定有内容
Session Affinity: None
External Traffic Policy: Cluster
Events:
[root@master1 ingress]# curl $ingress_nginx_srv_Ip:80 #会提示404这就代表已经正常解析
如果要在集群内所有节点或者集群内部分节点上部署，可以修改该yaml的 deployment部分为DaemonSet，设置为共享物理机的network namespace
[root@master1 yaml]# kubectl explain DaemonSet.spec.template.spec.hostNetwork
[root@master1 yaml]# kubectl get pods -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ingress-controller-6c777cd89c-65xr4 1/1 Running 0 6m53s 172.30.72.7 master1
[root@master1 yaml]# curl -I 172.30.72.7/healthz #返回200 3、验证安装
[root@registry ~]# kubectl get pods --all-namespaces -l app.kubernetes.io/name=ingress-nginx --watch
[root@registry ~]# POD_NAMESPACE=ingress-nginx
[root@registry ~]# POD_NAME=$(kubectl get pods -n $POD_NAMESPACE -l app.kubernetes.io/name=ingress-nginx -o jsonpath='{.items[0].metadata.name}')
[root@registry ~]# kubectl exec -it $POD_NAME -n $POD_NAMESPACE -- /nginx-ingress-controller --version4、创建ingress
1)创建service和deployment
[root@master1 ingress]# kubectl apply -f ingress.yaml
apiVersion: v1
kind: Service
metadata:
name: mysrv-v2
namespace: default
spec:
selector:
name: mydpv2
version: dpv2
ports:
- name: http
port: 80
targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: dpv2
namespace: default
spec:
replicas: 2
selector:
matchLabels:
name: mydpv2
version: dpv2
template:
metadata:
labels:
name: mydpv2
version: dpv2
spec:
containers:
- name: mydpv2
image: 192.168.192.234:888/nginx:v2
ports:
- name: dpport
containerPort: 80
strategy:
rollingUpdate:
maxSurge: 1
[root@master1 ingress]# kubectl get pods -l name=mydpv2 --show-labels
NAME READY STATUS RESTARTS AGE LABELS
dpv2-697556c88f-pbjm9 1/1 Running 0 66s name=mydpv2,pod-template-hash=697556c88f,version=dpv2
dpv2-697556c88f-sxr84 1/1 Running 0 66s name=mydpv2,pod-template-hash=697556c88f,version=dpv22)发布服务为ingress
[root@master1 ingress]# kubectl apply -f ingress-myapp.yaml
[root@master1 ingress]# cat ingress-myapp.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default #要和发布的deployment在同一个名称空间中
annotations:
kubernetes.io/ingress.class: "nginx" #说明自己使用的ingress controller是哪一个
spec:
rules:
- host: www.mt.com
http:
paths:
- path:
backend:
serviceName: mysrv-v2
servicePort: 80 查看nginx配置文件
[root@master1 ingress]# kubectl describe ingress
Name: ingress-myapp
Namespace: default
Address:
Default backend: default-http-backend:80 ()
Rules:
Host Path Backends
---- ---- --------
www.mt.com
mysrv-v2:80 (172.30.56.6:80,172.30.72.8:80)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-myapp","namespace":"default"},"spec":{"rules":[{"host":"www.mt.com","http":{"paths":[{"backend":{"serviceName":"mysrv-v2","servicePort":80},"path":null}]}}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 21s nginx-ingress-controller Ingress default/ingress-myapp
[root@master1 ingress]# curl www.mt.com:30080 //返回正常 3)https
[root@master1 ingress]# kubectl apply -f tomcat.yaml
service/tomcat created
deployment.apps/tomcat-deploy created
[root@master1 ingress]# cat tomcat.yaml
apiVersion: v1
kind: Service
metadata:
name: tomcat
namespace: default
spec:
selector:
app: tomcat
release: canary
ports:
- name: http
targetPort: 8080
port: 8080
- name: ajp
targetPort: 8009
port: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deploy
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: tomcat
release: canary
template:
metadata:
labels:
app: tomcat
release: canary
spec:
containers:
- name: myapp
image: 192.168.192.234:888/tomcat:latest
ports:
- name: http
containerPort: 8080
- name: ajp
containerPort: 8009
[root@master1 ingress]# kubectl describe svc tomcat
[root@master1 ingress]# cat ingress-tomcat.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-tomcat
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: www.mt.com
http:
paths:
- path:
backend:
serviceName: tomcat
servicePort: 8080
[root@master1 ingress]# curl www.mt.com:30080 验证，30080是nginx暴露出去的https:(使用secret注入到pod中)
[root@master1 ssl]# openssl genrsa -out tls.key 2048
[root@master1 ssl]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=HangZhou/L=HangZhou/O=DevOps/CN=www.mt.com
[root@master1 ssl]# kubectl create secret tls tomcat-ingress-secret --cert=tls.crt --key=tls.key
[root@master1 ingress]# kubectl describe secret tomcat-ingress-secret 修改nginx为https模式:修改
[root@master1 ingress]# kubectl apply -f ingress-tomcat-tls.yaml
[root@master1 ingress]# cat ingress-tomcat-tls.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-tomcat
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- www.mt.com
secretName: tomcat-ingress-secret
rules:
- host: www.mt.com
http:
paths:
- path:
backend:
serviceName: tomcat
servicePort: 8080 登陆ingress的pod查看是否正常生成ssl配置
八、Job/CronJob
Job负责批量处理短暂的一次性任务 (short lived one-off tasks)，即仅执行一次的任务，它保证批处理任务的一个或多个Pod成功结束。
Kubernetes支持以下几种Job：
1)非并行Job：通常一个Pod对应一个Job,除非Pod异常才会重启Pod，一旦此Pod正常结束，Job将结束
2)固定结束次数的Job：启动多个Pod，设置.spec.parallelism控制并行度，直到.spec.completions个Pod成功结束，Job结束
3)带有工作队列的并行Job：设置.spec.Parallelism但不设置.spec.completions，当所有Pod结束并且至少一个成功时，Job就认为是成功
Job Controller负责根据Job Spec创建Pod，并持续监控Pod的状态，直至其成功结束。如果失败，则根据restartPolicy（只支持OnFailure和Never，不支持Always）决定是否创建新的Pod再次重试任务。
1、Job yaml定义
[root@registry work]# cat job.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: pi
spec:
completions: 2
parallelism: 3
template:
metadata:
name: nginx
spec:
containers:
- name: nginx
image: "192.168.192.234:888/nginx:latest"
command: ["sh","-c","echo test for nginx && sleep 5"]
restartPolicy: Never2、CronJob
[root@registry work]# cat cronjob.yaml
apiVersion: batch/v2alpha1
kind: CronJob
metadata:
name: hello
spec:
schedule: "*/1 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: hello
image: 192.168.192.234:888/nginx:latest
args:
- /bin/sh
- -c
- date; echo Hello from the Kubernetes cluster && sleep 5
restartPolicy: OnFailure支持的时间格式；分 时 天 月 周 //支持的字符: "*"匹配该域的任意值,"/"每隔多久,
spec.schedule指定任务运行周期，格式同Cron
spec.jobTemplate指定需要运行的任务，格式同Job
spec.startingDeadlineSeconds指定任务开始的截止期限
spec.concurrencyPolicy指定任务的并发策略，支持Allow、Forbid和Replace三个选项
spec.suspend 设置为true，后续所有执行被挂起
九、ServiceAccount/UserAccount
K8s 2套独立的账号系统：User Account账号是给别人用的，Service Account是给Pod中的进程使用的，面对的对象不通
User Account是全局性的，Service Account有namespace的限制
每个namespace都会自动创建一个default service account
Token controller检测service account的创建，并为它们创建secret
Pod客户端访问API Server的https安全端口：
1)controller-manager使用api-server的私钥为Pod创建token
2)pod访问api server的时候，传递Token到HTTP Header中
3)api server使用自己的私钥验证该Token是否合法
[root@master1 merged]# kubectl exec -it nginx-587764dd97-29n2g -- ls -l /run/secrets/kubernetes.io/serviceaccount
total 0
lrwxrwxrwx 1 root root 13 Aug 9 10:02 ca.crt -> ..data/ca.crt
lrwxrwxrwx 1 root root 16 Aug 9 10:02 namespace -> ..data/namespace
lrwxrwxrwx 1 root root 12 Aug 9 10:02 token -> ..data/token1、创建serviceaccount
[root@master1 merged]# kubectl create serviceaccount nginx
serviceaccount/nginx created
[root@master1 merged]# kubectl get serviceaccount nginx -o yaml #自动创建sercrets
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: "2019-08-10T13:37:38Z"
name: nginx
namespace: default
resourceVersion: "1066770"
selfLink: /api/v1/namespaces/default/serviceaccounts/nginx
uid: 04af83c1-bb74-11e9-9c2a-00163e000999
secrets:
- name: nginx-token-pthgt
[root@master1 merged]# kubectl get secret nginx-token-pthgt -o yaml
apiVersion: v1
data:
ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR4akNDQXE2Z0F3SUJBZ0lVSXJ4Q2diY2lGZXNUVGxuVU1heDlla3JDSmxjd0RRWUpLb1pJaHZjTkFRRUwKQlFBd2FURUxNQWtHQTFVRUJoTUNRMDR4RVRBUEJnTlZCQWdUQ0VoaGJtZGFhRzkxTVJFd0R3WURWUVFIRXdoSQpZVzVuV21odmRURU1NQW9HQTFVRUNoTURhemh6TVJFd0R3WURWUVFMRXdoR2FYSnpkRTl1WlRFVE1CRUdBMVVFCkF4TUthM1ZpWlhKdVpYUmxjekFlRncweE9UQTRNREl4TVRNNU1EQmFGdzB5TWpBNE1ERXhNVE01TURCYU1Ha3gKQ3pBSkJnTlZCQVlUQWtOT01SRXdEd1lEVlFRSUV3aElZVzVuV21odmRURVJNQThHQTFVRUJ4TUlTR0Z1WjFwbwpiM1V4RERBS0JnTlZCQW9UQTJzNGN6RVJNQThHQTFVRUN4TUlSbWx5YzNSUGJtVXhFekFSQmdOVkJBTVRDbXQxClltVnlibVYwWlhNd2dnRWlNQTBHQ1NxR1NJYjNEUUVCQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUURKN1hueEVjWFAKS1hqZFpVY3VETVhQdFE5NzkxQjlvNDFFV05FWGNJbFZqM0NueVVuVmIxaHk1WittQzNGMGRtcGxGbTE3eTI2UQp6bThSV2RNZlh4MnlJZS9DSHJlV2o5Y3hMc0JLSm9xM0JvRmNLdmZqc1Z2dzIwaUZ1ZEEwektHREZEc01US3JYClQ3UUdGTEtveWM1N3BTS08yUGt5aG9OeDc2cHdDYllCMndpam4vRnRmMEYvTXpiTXBlR3E0WXkzK3VEUm9ubTIKeUI5TG1uOFRRT2NLb1ZWZHBPWDRIb1E5MGpkdy9EcUlzMUk4OXg3ZjNIZTBISDBGWkpzN3JJQ01TbG50QlR5RApWZkpaZ3N6YldYeHdzb25IeitzaVh3cnBTUDN6RGlmaWJLWHlmdUM1KzBreEJxWDNOQUZUVVFwVDErbTFmNEJwCnRCNTI3ZUpDdXdPakFnTUJBQUdqWmpCa01BNEdBMVVkRHdFQi93UUVBd0lCQmpBU0JnTlZIUk1CQWY4RUNEQUcKQVFIL0FnRUNNQjBHQTFVZERnUVdCQlQ3dmtPOUZOUGl2T0hhMjl2ODAzZFlrZGcvMERBZkJnTlZIU01FR0RBVwpnQlQ3dmtPOUZOUGl2T0hhMjl2ODAzZFlrZGcvMERBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQVJFdTlSZmZMCmdDYS9kS3kvTU1sNmd5dHZMMzJPMlRnNko4RGw5L0ZJanJtaXpVeC9xV256b29wTGNzY0hRcC9rcEQ1QURLTDUKcmJkTXZCcFNicUVncTlaT2hYeHg5SCtVYnZiY3IwV3h3N2xEbTdKR0lUb2FhckVrckk2ZExXcVpqczl4bTBPdAowb3RETjcwaFkxaG0rWTNqQXQzanV3WVByUFcxb2RJM013L3ArWG9DWTR3bkcycDMyc1grNzdCVUl0eHhkeGY1CkJJWHoyWURMdzNLbXNScCtXdW1DaTNwSUV3bXFHbnJDNFhQUzhXbDdleGFZZkxDRWgwRjVQU2NYb1MxdjZGYjYKYlNrM0h2ZVRLNzduNDgwUXZ0blJFOVZVSGFMRkdBUm5SRkh0NDVRbS9teUc1dXdKbW9zVnlRajVZdVRSVjdTbwpSZE1DVEVXOHFESGp1UT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
namespace: ZGVmYXVsdA==
token: ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNklpSjkuZXlKcGMzTWlPaUpyZFdKbGNtNWxkR1Z6TDNObGNuWnBZMlZoWTJOdmRXNTBJaXdpYTNWaVpYSnVaWFJsY3k1cGJ5OXpaWEoyYVdObFlXTmpiM1Z1ZEM5dVlXMWxjM0JoWTJVaU9pSmtaV1poZFd4MElpd2lhM1ZpWlhKdVpYUmxjeTVwYnk5elpYSjJhV05sWVdOamIzVnVkQzl6WldOeVpYUXVibUZ0WlNJNkltNW5hVzU0TFhSdmEyVnVMWEIwYUdkMElpd2lhM1ZpWlhKdVpYUmxjeTVwYnk5elpYSjJhV05sWVdOamIzVnVkQzl6WlhKMmFXTmxMV0ZqWTI5MWJuUXVibUZ0WlNJNkltNW5hVzU0SWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXpaWEoyYVdObExXRmpZMjkxYm5RdWRXbGtJam9pTURSaFpqZ3pZekV0WW1JM05DMHhNV1U1TFRsak1tRXRNREF4TmpObE1EQXdPVGs1SWl3aWMzVmlJam9pYzNsemRHVnRPbk5sY25acFkyVmhZMk52ZFc1ME9tUmxabUYxYkhRNmJtZHBibmdpZlEuVUpzV3pHTEloVHBkVWc4LTVweXlya0pkWnBJWEhpbFB6Nzk3Xy1qLW5nQVNxNFRjYkVrSFFkXzR1d1J6X2lkaGZvc0NUY2NxYW1oR1pzcU9PWldWWXBzeGdZbHY4aTRuQjJWNTJtaXN2OTZHTVFBb3FUVlE5aXBMNWN6VFY0cXlicGVyZ2w4WHpXU3B1SDhVTUVfQk5YRFZuZ0d4UGU2RUNTcU1qaEpWUDZjczVWTVFVZlp2dEl1dHRuRHhvWHlwdFN4RXNyaWJEUVNxNDQyeVYwU3hHRGpOMDZYZThtblNxamlpc0R2MmQ0c1NYNmpaczU0R3hlMlBQdUtWZjFoLUtRRzNKdDVwRTZic3FBcnJkcm83TDlJZHJRbkl6dUQ2QlJ5ZnRqTWdGcXVXVk9CeWNJcVkzaGY3Q0pmeThmNmJ0QXZrVi1XNUZFdE5TY3pKZXQ4SWZ3
kind: Secret
metadata:
annotations:
kubernetes.io/service-account.name: nginx
kubernetes.io/service-account.uid: 04af83c1-bb74-11e9-9c2a-00163e000999
creationTimestamp: "2019-08-10T13:37:38Z"
name: nginx-token-pthgt
namespace: default
resourceVersion: "1066769"
selfLink: /api/v1/namespaces/default/secrets/nginx-token-pthgt
uid: 04b16e50-bb74-11e9-a28c-00163e000318
type: kubernetes.io/service-account-token
[root@master1 merged]# kubectl describe secret nginx-token-pthgt
Name: nginx-token-pthgt
Namespace: default
Labels:
Annotations: kubernetes.io/service-account.name: nginx
kubernetes.io/service-account.uid: 04af83c1-bb74-11e9-9c2a-00163e000999
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1371 bytes
namespace: 7 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6Im5naW54LXRva2VuLXB0aGd0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6Im5naW54Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMDRhZjgzYzEtYmI3NC0xMWU5LTljMmEtMDAxNjNlMDAwOTk5Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6bmdpbngifQ.UJsWzGLIhTpdUg8-5pyyrkJdZpIXHilPz797_-j-ngASq4TcbEkHQd_4uwRz_idhfosCTccqamhGZsqOOZWVYpsxgYlv8i4nB2V52misv96GMQAoqTVQ9ipL5czTV4qybpergl8XzWSpuH8UME_BNXDVngGxPe6ECSqMjhJVP6cs5VMQUfZvtIuttnDxoXyptSxEsribDQSq442yV0SxGDjN06Xe8mnSqjiisDv2d4sSX6jZs54Gxe2PPuKVf1h-KQG3Jt5pE6bsqArrdro7L9IdrQnIzuD6BRyftjMgFquWVOBycIqY3hf7CJfy8f6btAvkV-W5FEtNSczJet8Ifw Service Account 为服务提供了一种方便的认证机制，但它不关心授权的问题。可以配合 RBAC 来为 Service Account 鉴权：
Secret从属于Service Account资源对象，属于Service Account的一部分，一个Service Account可以包含多个不同的Secret对象
Secret说明:
1)名为Token的secret用于访问API Server的Secret,也被称作Service Account Secret
2)名为imagePullSecrets的Secret用于下载容器镜像时的认证过程
3)用户自定义的其他Secret，用于用户的进程
2、创建userAccount
大体步骤如下：
1)生成个人私钥，和证书签署请求
2)k8s集群ca对csr进行认证
3)认证后的内容写入kubeconfig方便使用
参考：https://blog.51cto.com/hmtk520/2423253
十、Secret&Configmap
ConfigMap用于保存配置数据的键值对，可以用来保存单个属性，也可以用来保存配置文件。ConfigMap跟secret很类似，但它可以更方便地处理不包含敏感信息的字符串。
配置容器化应用的方式：
1、自定义命令行参数
command
args
2、把配置文件直接放在镜像中
3、环境变量
1)Cloud Native的应用程序一般可直接通过env var加载配置
2)通过entrypoint 脚本来预处理配置文件中的信息
4、存储卷
configmap
sercret
[root@master1 yaml]# kubectl explain pods.spec.containers
[root@master1 yaml]# kubectl explain pods.spec.containers.envFrom 可以是configMap
configMapRef
prefix
secretRef
[root@master1 yaml]# kubectl explain pods.spec.containers.env.valueFrom
configMapKeyRef 使用configmap
fieldRef 字段，可以是pod自身的字段，比如:metadata.name,metadata.namespace....
resourceFieldRef
secretKeyRef secret 1、创建 configmap
[root@master1 yaml]# kubectl create configmap nginx-configmap --from-literal=nginx_port=80 --from-literal=server_name=www.mt.com
[root@master1 configmap]# kubectl create configmap configmap-1 --from-file=www=./www.conf
configmap/configmap-1 created
[root@master1 configmap]# cat www.conf
server {
server_name www.mt.com;
listen 80;
root /data/web/html;
}
[root@master1 configmap]# kubectl create configmap special-config --from-file=config/ #从目录创建
[root@master1 configmap]# kubectl describe configmap configmap-1 #查看2、configmap使用
可用于：设置环境变量、设置容器命令行参数、在Volume中创建配置文件等 #configmap要在Pod创建前创建
[root@registry work]# cat configmap.yaml
apiVersion: v1
kind: Pod
metadata:
name: test-nginx
spec:
containers:
- name: test-container
image: 192.168.192.234:888/nginx:latest
env:
- name: pod-port #这个name为pod内环境变量，引用的是nginx-configmap.nginx_port这个变量
valueFrom:
configMapKeyRef:
name: nginx-configmap #引用的nginx-configmap这个configmap的nginx_port这个变量
key: nginx_port
optional: True #是否为可选
- name: pod_name
valueFrom:
configMapKeyRef:
name: nginx-configmap
key: server_name
envFrom: #引用的configmap，可设置多个，将引用nginx-configmap的所有内容
- configMapRef:
name: nginx-configmap
restartPolicy: Never修改configmap：
[root@master1 configmap]# kubectl edit configmap nginx-configmap
[root@master1 configmap]# kubectl describe configmap nginx-configmap 会发现已经修改
但是pod内的仍然没有改变，环境变量方式只在系统启动时生效。存储卷方式3、Volume方式使用configmap
[root@master1 configmap]# kubectl apply -f conf2.yaml
[root@master1 configmap]# cat conf2.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-cm1
namespace: default
labels:
app: cm1
release: centos
annotations:
www.mt.com/created-by: "mt"
spec:
containers:
- name: pod-cm1
image: 192.168.192.225:80/csb-broker:latest
ports:
- name:
containerPort: 80
volumeMounts:
- name: nginxconf
mountPath: /etc/nginx/config.d/
readOnly: True
volumes:
- name: nginxconf
configMap:
name: configmap-1
[root@pod-cm2 /home/admin]
#cat /etc/nginx/config.d/nginx_port
8080
[root@pod-cm2 /home/admin]
#cat /etc/nginx/config.d/server_name
www.mt.com使用kubectl edit configmap 容器内会发生改变
如果只需要挂载configmap中的部分key:value (一个configmap 可能有多个key/value)
[root@master1 configmap]# kubectl explain pods.spec.volumes.configMap4、secret
Secret和configMap类型，用于保存敏感配置信息
Secret 有三种类型：
Opaque：base64 编码格式的 Secret，用来存储密码、密钥等；但数据也通过 base64 --decode 解码得到原始数据，所有加密性很弱。
kubernetes.io/dockerconfigjson：用来存储私有 docker registry 的认证信息。
kubernetes.io/service-account-token： 用于被 serviceaccount 引用。serviceaccout 创建时Kubernetes 会默认创建对应的 secret。Pod 如果使用了 serviceaccount，对应的 secret 会自动挂载到 Pod 的 /run/secrets/kubernetes.io/serviceaccount 目录中。
[root@master1 configmap]# kubectl create secret
Create a secret using specified subcommand.
Available Commands:
docker-registry Create a secret for use with a Docker registry //连接私有仓库需要的认证信息
generic Create a secret from a local file, directory or literal value
tls Create a TLS secret //秘钥信息
[root@master1 configmap]# kubectl create secret generic mysql-root-password --from-literal=password=password123
secret/mysql-root-password created
[root@master1 configmap]# kubectl get secret mysql-root-password
NAME TYPE DATA AGE
mysql-root-password Opaque 1 8s
[root@master1 configmap]# kubectl describe secret mysql-root-password
[root@master1 configmap]# kubectl get secret mysql-root-password -o yaml
apiVersion: v1
data:
password: cGFzc3dvcmQxMjM= //base64编码;echo cGFzc3dvcmQxMjM= | base64 -d 解码
kind: Secret
metadata:
creationTimestamp: "2019-07-05T08:55:48Z"
name: mysql-root-password
namespace: default
resourceVersion: "2794776"
selfLink: /api/v1/namespaces/default/secrets/mysql-root-password
uid: af3180b4-9f02-11e9-8691-00163e000bdd
type: OpaqueSecret引用:以 Volume 方式 或者 以环境变量方式，参考ConfigMap
Docker_registry
[root@registry ~]# kubectl create secret docker-registry myregistrykey --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD 参考博客：
https://www.kubernetes.org.cn/kubernetes-pod
http://docs.kubernetes.org.cn/317.html#Pod-template-hash_label
https://kubernetes.io/docs/concepts/configuration/secret/
https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
https://kubernetes.io/docs/concepts/services-networking/ingress/