cpu就那么多,吞吐量和延时只能依靠调整jvm取最优。想让马儿跑,还想马儿不吃草?——加cpu。
前言
系统服务的核心指标即cpu,mem,disk,net。在稳定的滴滴容器云上,以及结合一般的web服务属性,mem,disk,net往往不会成为我们的瓶颈。最难搞的就是cpu。先抛出来一个大图,大家感受下通过jvm调优对cpu的影响
从图中可看,大致有三个阶段:
生不如死阶段:经常听到内存泄露,内存溢出,没错,今天见到了cpu泄露。这个阶段,cpu随着时间,会慢慢跌到0点。
苟延残喘阶段:调小jvm年轻代之后,就进入了我们的第二阶段,锯齿型cpu使用率,伴随着fgc的发生都会恢复cpu的使用
假药停阶段:因为考虑我们是web型高并发服务,所有极大部分对象都是朝夕生死,所以我调大了年轻代的内存,调小了老年代的内存,观察cpu发现很稳定。。其实不然,此处发现fgc频率和ygc频率一致了。。。
理论分析
对于web服务,高并发往往是我们追求的目标,而高的并发量,自然代表会有更多的对象创建和销毁,这个阶段基本上发生在年轻代。所以年轻代需要尽可能的大。而上文说的cpu掉底,正是发生在年轻代的垃圾收集导致的。
ygc步骤
此时,对于使用标记-复制算法的ygc,我们一般有以下几个步骤:
1.从GC Roots开始进行可达性分析,分析当前处于年轻代对象是否存活(极有可能出现问题)
2.将eden区和survivor from区的对象拷贝到to区(耗时稳定,和内存成正比)
3.清理eden区和from区(耗时稳定,和内存成正比)
而针对于ygc越来越慢的问题的,我们其实需要去分析的就是上边的几步骤的耗时。
GC-ROOT
能够成为gc-root的对象有以下这些:
所有JNI Handles JNIHandles
所有线程的栈
所有被Synchronize锁持有的对象
VM内实现的MBean所持有的对象
JVMTI所持有的对象
(可选)所有已加载的类 或 所有已加载的系统类
(可选)所有驻留字符串(StringTable)
(可选)代码缓存(CodeCache)
(可选)PermGen的remember set所记录的存在跨代引用的区域
实操过程
而如果使用parnew和cms垃圾回收器,官方并没有提供ygc在各阶段耗时的方法,,万幸发现AliJdk中,提供了该功能,通过 PrintGCRootsTraceTime
能打印出 ParNew
和 CMS
的详细耗时。
- 从Alibaba Dragonwell的Github页面下载二进制tar包,链接 https://github.com/alibaba/dragonwell8/releases
- 将下载下来的tar包解压到目标安装目录即可
- 将jdk启动路径替换为新安装jdk路径即可,记得添加参数 -XX:+PrintGCRootsTraceTime.
ygc对比:
采样1:
2020-05-22T10:08:41.685+0800: 44736.891: [GC (Allocation Failure) 2020-05-22T10:08:41.685+0800: 44736.891: [ParNew [GC Worker Start (ms): 44736891.4 44736891.5 44736891.5 44736891.5 44736891.6 44736891.6 44736891.7 44736891.7
Min: 44736891.4, Avg: 44736891.6, Max: 44736891.7, Diff: 0.2]
[Root Processing (ms): 75.5 75.5 75.4 74.8 75.4 75.5 74.9 75.3
Min: 74.8, Avg: 75.3, Max: 75.5, Diff: 0.7, Sum: 602.2]
[Thread Roots (ms): 0.0 0.6 0.6 0.6 0.5 0.5 0.5 0.4
Min: 0.0, Avg: 0.5, Max: 0.6, Diff: 0.6, Sum: 3.7]
[StringTable Roots (ms): 59.3 67.2 67.4 67.1 67.3 67.4 67.2 67.3
Min: 59.3, Avg: 66.3, Max: 67.4, Diff: 8.0, Sum: 530.2]
[Universe Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[JNI Handles Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[ObjectSynchronizer Roots (ms): 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.3, Diff: 0.3, Sum: 0.3]
[FlatProfiler Roots (ms): 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.1]
[Management Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[SystemDictionary Roots (ms): 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.1]
[CLDG Roots (ms): 8.7 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 1.1, Max: 8.7, Diff: 8.7, Sum: 8.7]
[CodeCache Roots (ms): 0.0 0.2 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.2, Diff: 0.2, Sum: 0.2]
[JVMTI Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[older-gen scanning (ms): 7.5 7.5 7.4 6.8 7.5 7.6 7.2 7.6
Min: 6.8, Avg: 7.4, Max: 7.6, Diff: 0.8, Sum: 59.0]
[GC Worker Other (ms): 0.1 0.1 0.1 0.8 0.1 0.0 0.5 0.1
Min: 0.0, Avg: 0.2, Max: 0.8, Diff: 0.8, Sum: 1.9]
[GC Worker Total (ms): 75.6 75.6 75.6 75.5 75.5 75.5 75.4 75.4
Min: 75.4, Avg: 75.5, Max: 75.6, Diff: 0.2, Sum: 604.1]
[GC Worker End (ms): 44736967.1 44736967.1 44736967.1 44736967.1 44736967.1 44736967.1 44736967.1 44736967.1
Min: 44736967.1, Avg: 44736967.1, Max: 44736967.1, Diff: 0.0]
: 1743479K->94992K(1887488K), 0.0804062 secs] 2133152K->490104K(3984640K), 0.0806634 secs] [Times: user=0.54 sys=0.07, real=0.08 secs]
采样2:
2020-05-22T10:13:29.342+0800: 45024.548: [GC (Allocation Failure) 2020-05-22T10:13:29.342+0800: 45024.548: [ParNew [GC Worker Start (ms): 45024548.5 45024548.6 45024548.6 45024548.6 45024549.7 45024552.0 45024552.0 45024554.2
Min: 45024548.5, Avg: 45024550.3, Max: 45024554.2, Diff: 5.6]
[Root Processing (ms): 113.7 113.6 113.4 113.2 112.3 110.0 109.8 107.7
Min: 107.7, Avg: 111.7, Max: 113.7, Diff: 6.0, Sum: 893.8]
[Thread Roots (ms): 0.0 2.9 1.3 2.8 0.2 0.0 0.0 0.0
Min: 0.0, Avg: 0.9, Max: 2.9, Diff: 2.9, Sum: 7.2]
[StringTable Roots (ms): 100.5 104.0 105.5 104.3 105.6 103.6 103.4 101.3
Min: 100.5, Avg: 103.5, Max: 105.6, Diff: 5.1, Sum: 828.1]
[Universe Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[JNI Handles Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[ObjectSynchronizer Roots (ms): 0.0 0.0 0.2 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.2, Diff: 0.2, Sum: 0.2]
[FlatProfiler Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[Management Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[SystemDictionary Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[CLDG Roots (ms): 6.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.8, Max: 6.5, Diff: 6.5, Sum: 6.5]
[CodeCache Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[JVMTI Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[older-gen scanning (ms): 6.7 6.8 6.4 6.2 6.4 6.4 6.4 6.4
Min: 6.2, Avg: 6.5, Max: 6.8, Diff: 0.6, Sum: 51.7]
[GC Worker Other (ms): 0.0 0.1 0.3 0.4 0.3 0.3 0.5 0.4
Min: 0.0, Avg: 0.3, Max: 0.5, Diff: 0.4, Sum: 2.3]
[GC Worker Total (ms): 113.7 113.7 113.7 113.6 112.6 110.3 110.3 108.1
Min: 108.1, Avg: 112.0, Max: 113.7, Diff: 5.6, Sum: 896.1]
[GC Worker End (ms): 45024662.3 45024662.3 45024662.3 45024662.3 45024662.3 45024662.3 45024662.3 45024662.3
Min: 45024662.3, Avg: 45024662.3, Max: 45024662.3, Diff: 0.0]
: 1802020K->124247K(1887488K), 0.1179329 secs] 2406433K->734108K(3984640K), 0.1181915 secs] [Times: user=0.79 sys=0.10, real=0.12 secs]
如上对比可知,ygc在做root processing阶段时,耗时出现了明显的增高:
[Root Processing (ms): 75.5 75.5 75.4 74.8 75.4 75.5 74.9 75.3
[Root Processing (ms): 113.7 113.6 113.4 113.2 112.3 110.0 109.8 107.7
而其中正是在扫描 StringTable Roots 耗时出现了明显的增高
[StringTable Roots (ms): 59.3 67.2 67.4 67.1 67.3 67.4 67.2 67.3
[StringTable Roots (ms): 100.5 104.0 105.5 104.3 105.6 103.6 103.4 101.3
以上信息我们只能观察到ygc耗时是越来越高的,只能间接推断出来影响到我们的cpu占用,所以我在另外一个f服务又复现了ygc耗时高和gc线程占用cpu比例高的现象:
服务启动1分钟后:其中一个gc占用的cpu比例为0.7%,StringTable扫描平均耗时在4.4ms,看下图
随着时间的推移,服务启动10分钟后:其中一个gc占用的cpu比例为1.0-2.3%徘徊,StringTable扫描平均耗时在26.7ms,看下图
随着时间的推移,服务启动15分钟后:其中一个gc占用的cpu比例为1.7-4.0%徘徊,StringTable扫描平均耗时在39.6ms
以上信息,我们可以得知,ygc的StringTable扫描对于CPU的占用是有影响的。
什么是StringTable
那么问题就出现在这个StringTable Roots上。那么接下来分析一波这个玩意是干啥的。
StringTable的数据结构是一个HashTable,代码调用String.intern()时,会先去该StringTable中查找是否存在相同的字符串。以此达到缓存的作用。
JVM字符串常量池在JDK8 中是将字符串对象放在堆中(jdk7是放到perm区,但是那样会经常导致永久代内存溢出,所以在jdk8中将它移到了heap区),通过StringTable来维护引用,StringTable 结构是类HashMap的实现,如果数组长度过短(String Table (JDK8)默认长度时60013 ),hash碰撞频繁就会导致链表过长,影响查找效率,耗时过长。
优化过程
寻找StringTable触发点
找到了问题的根因,那么我们只需要分析出来究竟是谁引起的StringTable越来越大即可。查阅资料后发现:
jackson内部使用了String.intern对key进行了缓存优化,故优化了两个地方
- JsonFormat工具,使用的是jackson工具
- SpringBoot web 默认使用的是jackson工具来序列化参数
优化方式如下:
private static JsonFactory factory = new JsonFactory().disable(JsonFactory.Feature.INTERN_FIELD_NAMES);
private static ObjectMapper objectMapper = new ObjectMapper(factory).configure(
DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.configure(DeserializationFeature.FAIL_ON_IGNORED_PROPERTIES, false)
.configure(DeserializationFeature.FAIL_ON_INVALID_SUBTYPE, false)
.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
但是优化之后,发现StringTable仍然是在慢慢变大(没有之前变大的那么快了),所以肯定是还有其他工具使用了String.intern方法了。
故有个思路就是运行期检测String,intern方法的调用栈,以此来发现究竟是谁调用了它。
- 使用btrace动态监测方法执行:工具是很强大。。但是奈何intern是一个native方法,该工具无法作用到native方法。
- 在寻找其他方法中(排查问题可能是个持续过程,很多隐藏的问题不是一时半会就能解决的,如果暂时没有思路,不妨跳出当前的思考范式,换个角度先把问题缓解或解决一部分)
- 目前正在尝试使用openjdk,修改StringTable的c++源码,通过这种方式来打印当前方法的调用栈,以此来确认是谁调用了string.intern方法。
StringTableSize优化
在寻找StringTable的触发点的同时,我们还是要保证线上服务的稳定运行的。所以对StringTable大小进行了优化。
调大StringTableSize=600131,参数命令: -XX:StringTableSize=600131 以此先缓解StringTable的问题,如图,在单机250qps读+250qps写的压力下,我们可以看到系统cpu idle在75-85之间徘徊,延时稳定,基本满足我们的要求。
控制变量优化大法
虽说通过调整StringTableSize能缓解这个问题,但是随着时间的增加,接口延时越来越高的问题,同样不是我们业务能接受的。所以这里,我采取了控制变量法,来排除具体是哪个地方导致的StringTable增大。
第一步,因为我之前是同时压读写两个接口,所以不能定位到具体哪个接口导致的问题,所以我分别对两个接口(特征查询,特征上报)进行了单独的压测。 最终发现,特征查询接口对StringTable的影响很大。
第二步,定位到特征查询接口对StringTable有直接的影响,那么我就分析是不是这个接口的代码逻辑里边有涉及到String.intern()相关的逻辑。所以我将这个接口的逻辑拆分为多步,然后依次减少这个接口的操作步数,这样就能够精确的定位到是哪一步的代码存在问题了。
第三步,通过上一步的多次验证,发现我的接口不加任何逻辑,就是请求过来,直接返回结果,StringTable仍然是处于增长的状态。所以答案渐渐浮出水面,我自己写的代码逻辑其实不存在问题。问题发生在代码逻辑执行之前。
-
第四步,所以这个时候,能想到的就是对于web请求,我们使用了webutil工具,对于请求数据和返回数据做了通用的处理。顺势一想,确实webutil工具肯定会有反序列化数据的逻辑,分析了一下webutil的代码,发现确实有使用jackson工具的地方
所以将该工具进行了,、都禁用了字符串缓存功能。效果不错,以下是对比:
优化webutil之前,随着时间的推移,StringTable的查询耗时在25.5ms左右,
2020-06-04T12:05:03.332+0800: 145.688: [GC (Allocation Failure) 2020-06-04T12:05:03.332+0800: 145.688: [ParNew [GC Worker Start (ms): 145688.5 145688.6 145688.7 145688.7 145688.7 145688.7 145688.7 145688.7 145688.8 145688.8 145688.8 145688.8 145688.8
Min: 145688.5, Avg: 145688.7, Max: 145688.8, Diff: 0.3]
[Root Processing (ms): 31.1 36.5 30.4 29.6 30.4 30.6 30.8 34.1 29.9 29.8 29.7 29.3 29.7
Min: 29.3, Avg: 30.9, Max: 36.5, Diff: 7.2, Sum: 401.9]
[Thread Roots (ms): 0.0 0.5 0.5 0.5 0.5 0.5 0.4 0.5 0.4 0.4 0.4 0.2 0.3
Min: 0.0, Avg: 0.4, Max: 0.5, Diff: 0.5, Sum: 5.0]
[StringTable Roots (ms): 21.3 26.1 25.9 25.8 25.9 25.7 25.8 25.7 25.8 25.7 25.8 26.0 25.8
Min: 21.3, Avg: 25.5, Max: 26.1, Diff: 4.8, Sum: 331.2]
[Universe Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[JNI Handles Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[ObjectSynchronizer Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[FlatProfiler Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[Management Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[SystemDictionary Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[CLDG Roots (ms): 5.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.4, Max: 5.2, Diff: 5.2, Sum: 5.2]
[CodeCache Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[JVMTI Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[older-gen scanning (ms): 4.6 10.0 4.1 3.3 4.1 4.4 4.6 7.9 3.7 3.6 3.5 3.1 3.6
Min: 3.1, Avg: 4.7, Max: 10.0, Diff: 6.9, Sum: 60.5]
[GC Worker Other (ms): 5.6 0.0 6.1 6.9 6.1 5.9 5.6 2.4 6.9 6.6 6.7 7.0 6.6
Min: 0.0, Avg: 5.6, Max: 7.0, Diff: 7.0, Sum: 72.4]
[GC Worker Total (ms): 36.7 36.5 36.5 36.5 36.5 36.4 36.4 36.4 36.9 36.4 36.4 36.4 36.4
Min: 36.4, Avg: 36.5, Max: 36.9, Diff: 0.5, Sum: 474.3]
[GC Worker End (ms): 145725.2 145725.2 145725.2 145725.2 145725.2 145725.2 145725.2 145725.2 145725.6 145725.2 145725.2 145725.2 145725.2
Min: 145725.2, Avg: 145725.2, Max: 145725.6, Diff: 0.5]
优化webutil之后,随着时间的推移,StringTable的查询耗时在0.4ms左右,
2020-06-04T12:00:29.907+0800: 801.182: [GC (Allocation Failure) 2020-06-04T12:00:29.907+0800: 801.183: [ParNew [GC Worker Start (ms): 801182.7 801185.0 801185.1 801185.1 801185.8 801187.4 801187.5 801187.6 801188.5 801188.5 801189.5 801189.5 801189.5
Min: 801182.7, Avg: 801187.0, Max: 801189.5, Diff: 6.8]
[Root Processing (ms): 10.9 8.2 10.0 8.6 7.3 5.6 5.6 5.8 5.5 5.6 3.9 3.5 3.5
Min: 3.5, Avg: 6.5, Max: 10.9, Diff: 7.5, Sum: 84.0]
[Thread Roots (ms): 0.0 1.3 0.6 0.6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.2, Max: 1.3, Diff: 1.3, Sum: 2.6]
[StringTable Roots (ms): 0.0 0.8 1.3 1.3 1.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.4, Max: 1.3, Diff: 1.3, Sum: 4.6]
[Universe Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[JNI Handles Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[ObjectSynchronizer Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[FlatProfiler Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[Management Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[SystemDictionary Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[CLDG Roots (ms): 9.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.7, Max: 9.2, Diff: 9.2, Sum: 9.2]
[CodeCache Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[JVMTI Roots (ms): 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[older-gen scanning (ms): 1.7 6.1 8.0 6.6 6.0 5.6 5.6 5.7 5.5 5.6 3.9 3.5 3.5
Min: 1.7, Avg: 5.2, Max: 8.0, Diff: 6.4, Sum: 67.5]
[GC Worker Other (ms): 1.5 2.0 0.0 1.4 2.0 2.1 2.0 1.8 1.1 1.0 1.7 2.1 2.1
Min: 0.0, Avg: 1.6, Max: 2.1, Diff: 2.1, Sum: 20.8]
[GC Worker Total (ms): 12.4 10.2 10.0 10.0 9.3 7.7 7.6 7.6 6.6 6.6 5.7 5.6 5.6
Min: 5.6, Avg: 8.1, Max: 12.4, Diff: 6.8, Sum: 104.9]
[GC Worker End (ms): 801195.1 801195.1 801195.1 801195.1 801195.1 801195.1 801195.1 801195.1 801195.1 801195.1 801195.1 801195.1 801195.1
Min: 801195.1, Avg: 801195.1, Max: 801195.1, Diff: 0.0]
: 840630K->2352K(943744K), 0.0164835 secs] 892655K->54377K(4089472K), 0.0166493 secs] [Times: user=0.08 sys=0.03, real=0.02 secs]
参考文献:
jvm优化
蚂蚁消息中间件 (MsgBroker) 在 YGC 优化上的探索
JVM源码分析之String.intern()导致的YGC不断变长
Jackson的一个坑
深入解析String#intern
深入 JVM 彻底剖析 ygc 越来越慢的原因(下)