cgroup version jdk version k8s
bug info:
in centos, linux 3.10, cgroup:v1, service works well.
in ubuntu 22.04 LTS, linux 5, systemd:v2, service is always crash on boot, or running some minutes then killed by OOM.
deploy.yaml文件相关内容:
specify limits:memory 1000M in yaml, killed by OOM.
die into pod, use jmap -heap pid, the max heap size is more bigger than 2000M.
so, the memory limit does not in effect.
what is cgroup
cgroup is: control group(控制组)，专为子系统分配资源(cpu, memory, disk, net...)
cgroups(7) - Linux manual page
cgroup v1/v2 - linux version
cgroups(7) - Linux manual page
cgroup v1 implementation was in linux 2.6.24.
cgroup v2 starting in linux 3.10 and official with linux 4.5
cgroup in docker
Runtime metrics | Docker Docs
cgroup in k8s
About cgroup v2 | Kubernetes
Configuring a cgroup driver | Kubernetes
Container Runtimes | Kubernetes
k8s - ??? - containerd
k8s - docker shim - docker - containerd
k8s - ctr/crictl containerd
【K8S】ctr和crictl的区别_u010157986的博客-CSDN博客
https://www.cnblogs.com/aozhejin/p/16193359.html
k8s都开始抛弃dockerd了，ctr需要搞起了_ctr pause_柳清风09的博客-CSDN博客
UseContainerSupport最小支持版本
当时只有cgroup:v1
最小支持版本, oraclejdk-8u191/openjdk-8u191
Java™ SE Development Kit 8, Update 191 Release Notes
Bug Database
验证方法：
https://medium.com/ci-cd-devops/effect-on-container-mem-usage-with-usecontainersupport-f0071a6a577e
docker run -m 1gb openjdk:8u131 java -XshowSettings:vm -version
docker run -m 1gb openjdk:8u191 java -XshowSettings:vm -version
cgroup:v2/systemd:v2
openjdk-8u372， (未发布), hub.docker上最新是8u342
openjdk-11.0.16，(已发布), hub.docker上最新是11.0.16
Bug Database
[JDK-8230305] Cgroups v2: Container awareness - Java Bug System
how max heap size calculate?
jmap -heap pid | grep heap #jdk11相关命令稍有不同
free -h
max heap size = available * 25%
+UseContainerSupport to the Rescue
how container know memory info of pod?
服务启动后，kubectl exec进入pod，下图2个位置文件和 xx.yaml文件里配置的limits:memory应该是一样的才对。
cgroup:v1，对应 /sys/fs/cgroup/memory/memory.limit_in_bytes
cgroup:v2，对应 /sys/fs/cgroup/memory.max
Support cgroups memory limit detection for cgroups V2 · Issue #6894 · dask/distributed · GitHub
other links：
+UseContainerSupport to the Rescue
https://medium.com/ci-cd-devops/effect-on-container-mem-usage-with-usecontainersupport-f0071a6a577e
运行Debian11的Linux服务器中 docker run -m 命令 不生效 不能能限制住内存使用 cgroups_memory cgroup out of memory-CSDN博客
https://hostadvice.com/how-to/how-to-limit-a-docker-containers-resources-on-ubuntu-18-04/
How to Limit Memory and CPU for Docker Containers - Serverlab
修改 docker cgroup 版本的方法 （changing cgroup version）_shida_csdn的博客-CSDN博客
Runtime metrics | Docker Docs
Docker: Placing limits on container memory using cgroups | Fabian Lee : Software Engineer
Support cgroups memory limit detection for cgroups V2 · Issue #6894 · dask/distributed · GitHub
/sys/fs/cgroup/memory/memory.limit_in_bytes is missing in the container in version > 4.2.0 · Issue #6118 · docker/for-mac · GitHub
[JDK-8230305] Cgroups v2: Container awareness - Java Bug System
https://medium.com/ci-cd-devops/effect-on-container-mem-usage-with-usecontainersupport-f0071a6a577e
jvm参数配置看这里（dockerfile方式和k8s的yaml方式）：JVM参数
the command line:
kubectl edit 编辑yaml，设置limits:memory
kubectl exec 进入pod容器
top 查看内存
free -h 查看内存
top -Hp pid 查看cpu
jmap -heap pid 查看jvm 内存
jinfo -flags pid 查看java启动参数
jstat -gc pid 2000 查看gc状态
java -version 查看jdk版本
java -XshowSettings:vm -XX:+PrintFlagsFinal -version | grep -Ei "maxheapsize|maxram|UseContainerSupport" 查看内存和UseContainerSupport是否生效
conclusion:
方案1： 保持linux kernel使用cgroup:v2或systemd:v2：
linux 4.5+(含)
使用opendjk-8u372+(含)， 未发布，当前8u342， 统计时间为2022-12-07。
使用openjdk-11.0.16+(含)，已发布，当前11.0.16，统计时间为2022-12-07。
方案2： 修改linux kernel后，使用cgroup:v1：
linux 2.6.24(含)-4.5(不含)
使用openjdk-8u191(含)+，已发布，统计时间为2022-12-07。
方案3：或直接使用openjdk:11.0.16，兼容cgroup:v1和cgroup:v2