https://github.com/zq2599/blog_demos
内容:所有原创文章分类汇总及配套源码,涉及Java、Docker、Kubernetes、DevOPS等;
这是篇万字长文,所以一开始就要明确本文的核心内容:开发一个SpringBoot应用并部署在kubernetes环境,这个应用通过kubernetes的java客户端向API Server发请求,请求内容包括:创建名为test123的deployment、对这个deployment进行patch操作,如下图:
接下来先了解一些kubernetes的patch相关的基本知识;
kubernetes的patch一共有四种:
json patch:在请求中指定操作类型,例如:add、replace,再指定json内容进行操作z,请参考:https://tools.ietf.org/html/rfc6902
merge patch:合并操作,可以提交整个资源的信息,与现有信息进行合并后生效,也可以提交部分信息用于替换,请参考:https://tools.ietf.org/html/rfc7386
strategic merge patch:json patch和merge patch都遵守rfc标准,但是strategic merge patch却是kubernetes独有的,官方中文文档中称为策略性合并,也是merge的一种,但是真正执行时kubernetes会做合并还是替换是和具体的资源定义相关的(具体策略由 Kubernetes 源代码中字段标记中的 patchStrategy 键的值指定),以Pod的Container为例,下面是其源码,红框中显示其Container节点的patchStrategy属性是merge,也就是说如果您提交了一份strategic merge patch,里面的内容是关于Pod的Container的,那么原有的Container不会被替换,而是合并(例如以前只有nginx,提交的strategic merge patch是redis,那么最终pod下会有两个container:nginx和redis):
通过源码查看资源的patchStrategy属性是很麻烦的事情,因此也可以通过Kubernetes API 文档来查看,如下图,地址是:https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#podspec-v1-core
第四种是apply patch:主要是指kubernetes 1.14版本开始的server-side apply,由APIServer 做 diff 和 merge 操作,很多原本易碎的现象都得到了解决(例如controller和kubectl都在更新),另外要格外注意的是:1.14版本默认是不开启server-side apply特性的,具体的开启操作在下面会详细讲解;
名称 | 链接 | 备注 |
---|---|---|
项目主页 | https://github.com/zq2599/blog_demos | 该项目在GitHub上的主页 |
git仓库地址(https) | https://github.com/zq2599/blog_demos.git | 该项目源码的仓库地址,https协议 |
git仓库地址(ssh) | [email protected]:zq2599/blog_demos.git | 该项目源码的仓库地址,ssh协议 |
准备工作包括创建工程、编写辅助功能代码、初始化代码等:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0modelVersion>
<parent>
<groupId>com.bolingcavalrygroupId>
<artifactId>kubernetesclientartifactId>
<version>1.0-SNAPSHOTversion>
<relativePath>../pom.xmlrelativePath>
parent>
<groupId>com.bolingcavalrygroupId>
<artifactId>patchartifactId>
<version>0.0.1-SNAPSHOTversion>
<name>patchname>
<description>patch demodescription>
<packaging>jarpackaging>
<dependencies>
<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-webartifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-jsonartifactId>
exclusion>
exclusions>
dependency>
<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-actuatorartifactId>
dependency>
<dependency>
<groupId>org.projectlombokgroupId>
<artifactId>lombokartifactId>
<optional>trueoptional>
dependency>
<dependency>
<groupId>io.kubernetesgroupId>
<artifactId>client-javaartifactId>
dependency>
dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-maven-pluginartifactId>
<version>2.3.0.RELEASEversion>
<configuration>
<layers>
<enabled>trueenabled>
layers>
configuration>
plugin>
plugins>
build>
project>
package com.bolingcavalry.patch;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.stream.Collectors;
import org.springframework.core.io.ClassPathResource;
public class ClassPathResourceReader {
private final String path;
private String content;
public ClassPathResourceReader(String path) {
this.path = path;
}
public String getContent() {
if (content == null) {
try {
ClassPathResource resource = new ClassPathResource(path);
BufferedReader reader = new BufferedReader(new InputStreamReader(resource.getInputStream()));
content = reader.lines().collect(Collectors.joining("\n"));
reader.close();
} catch (IOException ex) {
throw new RuntimeException(ex);
}
}
return content;
}
}
public static void main(String[] args) {
SpringApplication.run(PatchExample.class, args);
}
static String DEPLOYMENT_NAME = "test123";
static String NAMESPACE = "default";
static String deployStr, jsonStr, mergeStr, strategicStr, applyYamlStr;
在resources文件夹中放入json文件,稍后的初始化代码会将这些文件读取到字符串变量中,如下图,这些json文件的内容稍后会详细说明:
编写初始化代码(通过PostConstruct注解实现),主要是客户端配置,还有将json文件的内容读出来,保存到刚刚准备的字符串变量中:
@PostConstruct
private void init() throws IOException {
// 设置api配置
ApiClient client = Config.defaultClient();
Configuration.setDefaultApiClient(client);
// 设置超时时间
Configuration.getDefaultApiClient().setConnectTimeout(30000);
// 部署用的JSON字符串
deployStr = new ClassPathResourceReader("deploy.json").getContent();
// json patch用的JSON字符串
jsonStr = new ClassPathResourceReader("json.json").getContent();
// merge patch用的JSON字符串,和部署的JSON相比:replicas从1变成2,增加一个名为from的label,值为merge
mergeStr = new ClassPathResourceReader("merge.json").getContent();
// strategic merge patch用的JSON字符串
strategicStr = new ClassPathResourceReader("strategic.json").getContent();
// server side apply用的JSON字符串
applyYamlStr = new ClassPathResourceReader("applyYaml.json").getContent();
}
/**
* 通用patch方法
* @param patchFormat patch类型,一共有四种
* @param deploymentName deployment的名称
* @param namespace namespace名称
* @param jsonStr patch的json内容
* @param fieldManager server side apply用到
* @param force server side apply要设置为true
* @return patch结果对象转成的字符串
* @throws Exception
*/
private String patch(String patchFormat, String deploymentName, String namespace, String jsonStr, String fieldManager, Boolean force) throws Exception {
// 创建api对象,指定格式是patchFormat
ApiClient patchClient = ClientBuilder
.standard()
.setOverridePatchFormat(patchFormat)
.build();
log.info("start deploy : " + patchFormat);
// 开启debug便于调试,生产环境慎用!!!
patchClient.setDebugging(true);
// 创建deployment
ExtensionsV1beta1Deployment deployment = new ExtensionsV1beta1Api(patchClient)
.patchNamespacedDeployment(
deploymentName,
namespace,
new V1Patch(jsonStr),
null,
null,
fieldManager,
force
);
log.info("end deploy : " + patchFormat);
return new GsonBuilder().setPrettyPrinting().create().toJson(deployment);
}
{
"kind":"Deployment",
"apiVersion":"extensions/v1beta1",
"metadata":{
"name":"test123",
"labels":{
"run":"test123"
}
},
"spec":{
"replicas":1,
"selector":{
"matchLabels":{
"run":"test123"
}
},
"template":{
"metadata":{
"creationTimestamp":null,
"labels":{
"run":"test123"
}
},
"spec":{
"terminationGracePeriodSeconds":30,
"containers":[
{
"name":"test123",
"image":"nginx:1.18.0",
"ports":[
{
"containerPort":80
}
],
"resources":{
}
}
]
}
},
"strategy":{
}
},
"status":{
}
}
/**
* 通用patch方法
* @param patchFormat patch类型,一共有四种
* @param deploymentName deployment的名称
* @param namespace namespace名称
* @param jsonStr patch的json内容
* @param fieldManager server side apply用到
* @param force server side apply要设置为true
* @return patch结果对象转成的字符串
* @throws Exception
*/
private String patch(String patchFormat, String deploymentName, String namespace, String jsonStr, String fieldManager, Boolean force) throws Exception {
// 创建api对象,指定格式是patchFormat
ApiClient patchClient = ClientBuilder
.standard()
.setOverridePatchFormat(patchFormat)
.build();
log.info("start deploy : " + patchFormat);
// 开启debug便于调试,生产环境慎用!!!
patchClient.setDebugging(true);
// 创建deployment
ExtensionsV1beta1Deployment deployment = new ExtensionsV1beta1Api(patchClient)
.patchNamespacedDeployment(
deploymentName,
namespace,
new V1Patch(jsonStr),
null,
null,
fieldManager,
force
);
log.info("end deploy : " + patchFormat);
return new GsonBuilder().setPrettyPrinting().create().toJson(deployment);
}
/**
* 通用patch方法,fieldManager和force都默认为空
* @param patchFormat patch类型,一共有四种
* @param jsonStr patch的json内容
* @return patch结果对象转成的字符串
* @throws Exception
*/
private String patch(String patchFormat, String jsonStr) throws Exception {
return patch(patchFormat, DEPLOYMENT_NAME, NAMESPACE, jsonStr, null, null);
}
[
{
"op":"replace",
"path":"/spec/template/spec/terminationGracePeriodSeconds",
"value":27
}
]
/**
* JSON patch格式的关系
*
* @return
* @throws Exception
*/
@RequestMapping(value = "/patch/json", method = RequestMethod.GET)
public String json() throws Exception {
return patch(V1Patch.PATCH_FORMAT_JSON_PATCH, jsonStr);
}
先尝试全量的merge patch,也就是准备好完整的deployment内容,修改其中一部分后进行提交,下图是json文件merge.json的内容,其内容前面的deploy.json相比,仅增加了红框处的内容,即增加了一个label:
代码依然很简单:
@RequestMapping(value = "/patch/fullmerge", method = RequestMethod.GET)
public String fullmerge() throws Exception {
return patch(V1Patch.PATCH_FORMAT_JSON_MERGE_PATCH, mergeStr);
}
{
"spec":{
"template":{
"spec":{
"containers":[
{
"name":"test456",
"image":"tomcat:7.0.105-jdk8"
}
]
}
}
}
}
@RequestMapping(value = "/patch/partmerge", method = RequestMethod.GET)
public String partmerge() throws Exception {
return patch(V1Patch.PATCH_FORMAT_JSON_MERGE_PATCH, strategicStr);
}
@RequestMapping(value = "/patch/strategic", method = RequestMethod.GET)
public String strategic() throws Exception {
return patch(V1Patch.PATCH_FORMAT_STRATEGIC_MERGE_PATCH, strategicStr);
}
@RequestMapping(value = "/patch/apply", method = RequestMethod.GET)
public String apply() throws Exception {
return patch(V1Patch.PATCH_FORMAT_APPLY_YAML, DEPLOYMENT_NAME, NAMESPACE, applyYamlStr, "example-field-manager", true);
}
apply yaml patch的json字符串来自文件applyYaml.json,其内容是从deploy.json直接复制的,然后改了下图两个红框中的内容,红框1修改了nginx的版本号,用来验证patch是否生效(原有版本是1.18),红框2是kubernetes1.16之前的一个问题,protocol字段必填,否则会报错,问题详情请参考:https://github.com/kubernetes-sigs/structured-merge-diff/issues/130
以上就是所有代码和patch的内容了,接下来部署到kubernetes环境实战吧
mvn clean package -U -DskipTests
# 指定基础镜像,这是分阶段构建的前期阶段
FROM openjdk:8u212-jdk-stretch as builder
# 执行工作目录
WORKDIR application
# 配置参数
ARG JAR_FILE=target/*.jar
# 将编译构建得到的jar文件复制到镜像空间中
COPY ${JAR_FILE} application.jar
# 通过工具spring-boot-jarmode-layertools从application.jar中提取拆分后的构建结果
RUN java -Djarmode=layertools -jar application.jar extract
# 正式构建镜像
FROM openjdk:8u212-jdk-stretch
WORKDIR application
# 前一阶段从jar中提取除了多个文件,这里分别执行COPY命令复制到镜像空间中,每次COPY都是一个layer
COPY --from=builder application/dependencies/ ./
COPY --from=builder application/spring-boot-loader/ ./
COPY --from=builder application/snapshot-dependencies/ ./
COPY --from=builder application/application/ ./
ENTRYPOINT ["java", "org.springframework.boot.loader.JarLauncher"]
docker build -t 192.168.50.43:5888/common/patch:1.0-SNAPSHOT .
docker push 192.168.50.43:5888/common/patch:1.0-SNAPSHOT
apiVersion: v1
kind: Service
metadata:
name: patch
namespace : kubernetesclient
spec:
type: NodePort
ports:
- port: 8080
nodePort: 30102
selector:
name: patch
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
namespace : kubernetesclient
name: patch
spec:
replicas: 1
template:
metadata:
labels:
name: patch
spec:
serviceAccountName: kubernates-client-service-account
containers:
- name: patch
image: 192.168.50.43:5888/common/patch:1.0-SNAPSHOT
tty: true
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: 8080
initialDelaySeconds: 5
failureThreshold: 10
timeoutSeconds: 10
periodSeconds: 5
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: 8080
initialDelaySeconds: 5
timeoutSeconds: 10
periodSeconds: 5
ports:
- containerPort: 8080
resources:
requests:
memory: "512Mi"
cpu: "100m"
limits:
memory: "1Gi"
cpu: "1000m"
kubectl apply -f patch.yaml
apiVersion: v1
kind: Service
metadata:
name: test123
namespace : default
spec:
type: NodePort
ports:
- port: 80
nodePort: 30101
selector:
run: test123
kubectl apply -f nginx-service.yaml
先说一下验证的步骤:
假设kubernetes宿主机IP地址是192.168.50.135,所以通过浏览器访问:http://192.168.50.135:30102/patch/deploy,即可创建名为test123的deployment,如下图,创建成功后deployment的详细信息会展示在浏览器上:
浏览器访问test123这个deployment里面的nginx容器,地址是http://192.168.50.135:30101/ ,如下图红框所示,返回的header中显示,nginx的版本是1.18.0:
接下来开始提交patch;
json patch修改的是原deployment的terminationGracePeriodSeconds属性,所以咱们先来看看修改前是啥样的,执行命令kubectl get deployment test123 -o yaml,如下图红框,terminationGracePeriodSeconds等于30:
浏览器访问http://192.168.50.135:30102/patch/json,即可发起json patch请求,并将deployment的结果返回,如下图所示,terminationGracePeriodSeconds属性值已经改变:
再次用命令kubectl get deployment test123 -o yaml查看,如下图红框,json patch已经生效:
执行kubectl logs -f patch-56cd7f7f87-bpl5r -n kubernetesclient可以看到咱们应用通过java客户端与kubernetes 的API Server之前的详细日志(patch-56cd7f7f87-bpl5r是patch工程对应的pod),如下图:
json patch验证已经完成,接着验证下一个;
merge patch(全量)给原有的deployment增加了一个label,先看看执行patch之前是啥样,如下图红框:
浏览器访问http://192.168.50.135:30102/patch/fullmerge ,就发起了merge请求,操作成功后再次查看,如下图红框,新增了一个label:
浏览器访问http://192.168.50.135:30102/patch/partmerge
执行命令kubectl describe pod test123-5ff477967-tv729查看新pod的详情,发现已经部署nginx了,而是patch中提交的tomcat,如下图所示,看来增量merge patch实际上做是替换操作:
上述操作完成后,test123就恢复到最初状态了,在浏览器执行http://192.168.50.135:30102/patch/strategic ,即可提交strategic merge patch
再去执行kubectl get pod命令查看,会发现pod会被删除,然后创建新pod,这个新pod的容器数量是2,如下图红框:
执行命令kubectl describe pod test123-59db4854f5-bstz5查看新pod的详情,下图两个红框,可见原有的strategic merge patch并没有替换container,而是新增了一个:
此时您应该对merge patch和strategic merge patch的区别应该有了清晰的认识;
最后一个实战是apply yaml patch,此类型patch主要是用于Server-side Apply特性的,
该特性自kubernetes的1.14版本就有了,但是默认并未开启,直到1.16版本才默认开启,因此,如果您的kubernetes低于1.16版本,需要开启这个特性;
kubernetes的官方文档中,提到此特性在低版本可以通过开关来开启,文档地址:https://kubernetes.cn/docs/reference/command-line-tools-reference/feature-gates/
我这里kubernetes版本是1.14,因此需要手动开启Server-side Apply,首先SSH登录kubernetes环境;
打开文件/etc/kubernetes/manifests/kube-apiserver.yaml ,给spec.containers.command增加一个参数–feature-gates=ServerSideApply=true,如下图红框所示:
修改完毕后请保存,由于kubernetes一直在监听此文件,因此会立即自动生效;
至此,通过kubernetes的java客户端执行patch操作的实战就全部完成了,从理论到环境准备,再到实际操作,涉及到太多内容,感谢您的耐心,希望本文能助您用好java客户端这个利器,更高效的操作kubernetes环境;