My First Step to JVM
JVM Analysis Report
在我的Github Wiki阅读本文。
JVM的理论知识，实战调优与分析是面试官必问的考点，是一个增删改查程序员，Spring MVC程序员争取进阶的关键一步！
本文节选自我在公司内部的JVM分析报告。
前言
背景
在博世（中国）投资有限公司，软件创新事业部（Software Innovation）中国分部实习。
中国这边太穷了。同事很不靠谱，是外包的。头也不靠谱。
很感谢有这样的环境让我接触到相对于增删改查更高阶的领域。
问题
线上有一个服务发现请求多了以后会OOM: Java Heap Space，因为相关异常处理代码写的太，导致OOM的堆栈信息都看不到，只能看到一行冷冰冰的日志。
有一个接口需要接收前端传来的一张图片，大小为几兆到10M不等。在这个接口里需要读图片，走Zxing，NIO存文件，启线程，线程里启进程去跑 opencv-python , torch 的算法。
我们不要谈这个接口设计的问题……我也知道不太合理，太重了。
总之它现在Java Heap Space了。
结论
先说结论：没问题，就是堆内存给小了。
- 有一个地方出现了，实践证明明显地加大了开销：不使用线程池而手动启动线程。（但其实更应该使用Spring的
@Async和JDK的Future） - 由于之前团队没人懂JVM，所以没有使用任何的JVM参数！默认的资源完全不够用。
- 本地根本复现不了内存泄漏、溢出等问题。
- 加大了堆大小后也不再出现问题。
线上分析
线上分析的时候其实我被束手束脚的，因为这是生产环境，我什么都动不了。而且刚开始的一天，这方面知识还比较欠缺。
0. 获取基本信息
我懒得多问同事，就自己查吧。
查看服务进程号
$ netstat -tulpn |grep 80
tcp6 0 0 :::80 :::* LISTEN 937/java
查看服务的工作目录
$ cat /proc/937/environ
1. 获取进程中线程
$ top -H -p 937
top - 16:49:07 up 16 days, 1:19, 4 users, load average: 0.00, 0.01, 0.05
Threads: 51 total, 0 running, 51 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.1 sy, 0.0 ni, 99.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 8009808 total, 774372 free, 5664808 used, 1570628 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 1658036 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
937 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.00 java
938 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:05.14 java
939 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:03.26 java
940 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:03.22 java
941 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:03.20 java
942 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:03.26 java
943 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:03.73 java
944 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.00 java
945 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.07 java
946 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.00 java
947 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:13.41 java
948 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:11.79 java
949 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:05.10 java
950 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.00 java
951 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:33.19 java
957 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:05.71 java
958 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:05.94 java
959 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.26 java
961 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.32 java
964 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:02.04 java
965 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.00 java
966 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.01 java
967 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:04.50 java
968 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:04.50 java
969 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:04.37 java
970 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:04.31 java
971 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:04.21 java
972 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:04.18 java
976 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:04.14 java
977 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:04.16 java
978 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:04.20 java
979 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:04.19 java
980 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.31 java
982 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:06.35 java
983 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:16.27 java
984 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:08.55 java
985 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:06.40 java
986 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:06.76 java
987 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:11.04 java
988 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:08.80 java
989 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:02.42 java
990 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:10.25 java
991 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:07.22 java
992 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:05.41 java
993 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:02.97 java
994 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:02.98 java
995 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.13 java
1066 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.00 java
28842 root 20 0 5817088 2.218g 14856 S 0.0 29.0 0:00.00 java
至始至终CPU占用都很正常，没有格外显眼的线程。
2. jstack看线程JVM信息
Full thread dump OpenJDK 64-Bit Server VM (25.161-b14 mixed mode):
"Attach Listener" #114 daemon prio=9 os_prio=0 tid=0x00007fd4ec00c860 nid=0x2fcb waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"logback-2" #81 daemon prio=5 os_prio=0 tid=0x00007fd4a40609c0 nid=0x49ce waiting on condition [0x00007fd4c81b3000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ac3f3160> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"logback-1" #67 daemon prio=5 os_prio=0 tid=0x00007fd498006490 nid=0x70aa waiting on condition [0x00007fd4c82b4000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ac3f3160> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Java2D Disposer" #48 daemon prio=10 os_prio=0 tid=0x00007fd49c434df0 nid=0x42a in Object.wait() [0x00007fd4c8a15000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x00000000ac0225a8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
at sun.java2d.Disposer.run(Disposer.java:148)
at java.lang.Thread.run(Thread.java:748)
"DestroyJavaVM" #46 prio=5 os_prio=0 tid=0x00007fd52c009440 nid=0x3aa waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"http-nio-80-Acceptor-0" #44 daemon prio=5 os_prio=0 tid=0x00007fd4881e2990 nid=0x3e3 runnable [0x00007fd4c95f0000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
- locked <0x0000000093743240> (a java.lang.Object)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:448)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:70)
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:95)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-ClientPoller-1" #43 daemon prio=5 os_prio=0 tid=0x00007fd4881d5a30 nid=0x3e2 runnable [0x00007fd4c96f1000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000000931b6228> (a sun.nio.ch.Util$3)
- locked <0x00000000931b6218> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000931b6080> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:743)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-ClientPoller-0" #42 daemon prio=5 os_prio=0 tid=0x00007fd4881a2860 nid=0x3e1 runnable [0x00007fd4c97f2000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000093743a70> (a sun.nio.ch.Util$3)
- locked <0x0000000093743a80> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000093743a28> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:743)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-exec-10" #41 daemon prio=5 os_prio=0 tid=0x00007fd488195450 nid=0x3e0 waiting on condition [0x00007fd4c98f3000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000093745818> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:107)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-exec-9" #40 daemon prio=5 os_prio=0 tid=0x00007fd488193b30 nid=0x3df waiting on condition [0x00007fd4c99f4000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000093745818> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:107)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-exec-8" #39 daemon prio=5 os_prio=0 tid=0x00007fd488192210 nid=0x3de waiting on condition [0x00007fd4c9af5000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000093745818> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:107)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-exec-7" #38 daemon prio=5 os_prio=0 tid=0x00007fd4881908f0 nid=0x3dd waiting on condition [0x00007fd4c9bf6000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000093745818> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:107)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-exec-6" #37 daemon prio=5 os_prio=0 tid=0x00007fd48818efd0 nid=0x3dc waiting on condition [0x00007fd4c9cf7000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000093745818> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:107)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-exec-5" #36 daemon prio=5 os_prio=0 tid=0x00007fd48818d6b0 nid=0x3db waiting on condition [0x00007fd4c9df8000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000093745818> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:107)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-exec-4" #35 daemon prio=5 os_prio=0 tid=0x00007fd48818bd90 nid=0x3da waiting on condition [0x00007fd4c9ef9000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000093745818> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:107)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-exec-3" #34 daemon prio=5 os_prio=0 tid=0x00007fd48818a490 nid=0x3d9 waiting on condition [0x00007fd4c9ffa000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000093745818> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:107)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-exec-2" #33 daemon prio=5 os_prio=0 tid=0x00007fd488189000 nid=0x3d8 waiting on condition [0x00007fd4ca0fb000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000093745818> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:107)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-80-exec-1" #32 daemon prio=5 os_prio=0 tid=0x00007fd488164fb0 nid=0x3d7 waiting on condition [0x00007fd4ca1fc000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000093745818> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:107)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"NioBlockingSelector.BlockPoller-1" #31 daemon prio=5 os_prio=0 tid=0x00007fd48813e820 nid=0x3d6 runnable [0x00007fd4ca2fd000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000093757d88> (a sun.nio.ch.Util$3)
- locked <0x0000000093757d98> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000093757d40> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:304)
"quartzScheduler_QuartzSchedulerThread" #30 prio=5 os_prio=0 tid=0x00007fd52d300790 nid=0x3d4 in Object.wait() [0x00007fd4ca5fe000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:427)
- locked <0x00000000930d9c40> (a java.lang.Object)
"quartzScheduler_Worker-10" #29 prio=5 os_prio=0 tid=0x00007fd52d1ae370 nid=0x3d3 in Object.wait() [0x00007fd4ca6ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
- locked <0x00000000930d3098> (a java.lang.Object)
"quartzScheduler_Worker-9" #28 prio=5 os_prio=0 tid=0x00007fd52f039c30 nid=0x3d2 in Object.wait() [0x00007fd4ca800000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
- locked <0x00000000930d2e08> (a java.lang.Object)
"quartzScheduler_Worker-8" #27 prio=5 os_prio=0 tid=0x00007fd52fcead90 nid=0x3d1 in Object.wait() [0x00007fd4ca901000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
- locked <0x00000000930d2b80> (a java.lang.Object)
"quartzScheduler_Worker-7" #26 prio=5 os_prio=0 tid=0x00007fd52de92560 nid=0x3d0 in Object.wait() [0x00007fd4caa02000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
- locked <0x00000000930d28f8> (a java.lang.Object)
"quartzScheduler_Worker-6" #25 prio=5 os_prio=0 tid=0x00007fd52d45e820 nid=0x3cc in Object.wait() [0x00007fd4cab03000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
- locked <0x00000000930d2670> (a java.lang.Object)
"quartzScheduler_Worker-5" #24 prio=5 os_prio=0 tid=0x00007fd52fb61760 nid=0x3cb in Object.wait() [0x00007fd4cac04000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
- locked <0x00000000930d23e8> (a java.lang.Object)
"quartzScheduler_Worker-4" #23 prio=5 os_prio=0 tid=0x00007fd52de50a30 nid=0x3ca in Object.wait() [0x00007fd4cad05000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
- locked <0x00000000930d2160> (a java.lang.Object)
"quartzScheduler_Worker-3" #22 prio=5 os_prio=0 tid=0x00007fd52dc23f10 nid=0x3c9 in Object.wait() [0x00007fd4cae06000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
- locked <0x00000000930d1ed8> (a java.lang.Object)
"quartzScheduler_Worker-2" #21 prio=5 os_prio=0 tid=0x00007fd52dfdcb40 nid=0x3c8 in Object.wait() [0x00007fd4caf07000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
- locked <0x00000000930d1c50> (a java.lang.Object)
"quartzScheduler_Worker-1" #20 prio=5 os_prio=0 tid=0x00007fd52c98a8f0 nid=0x3c7 in Object.wait() [0x00007fd4cb008000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
- locked <0x00000000930d19c8> (a java.lang.Object)
"Druid-ConnectionPool-Destroy-1489933928" #19 daemon prio=5 os_prio=0 tid=0x00007fd52e7a4030 nid=0x3c6 waiting on condition [0x00007fd4cb109000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at com.alibaba.druid.pool.DruidDataSource$DestroyConnectionThread.run(DruidDataSource.java:2250)
"Druid-ConnectionPool-Create-1489933928" #18 daemon prio=5 os_prio=0 tid=0x00007fd52c908cc0 nid=0x3c5 waiting on condition [0x00007fd4cb20a000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000092f97278> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at com.alibaba.druid.pool.DruidDataSource$CreateConnectionThread.run(DruidDataSource.java:2157)
"Abandoned connection cleanup thread" #17 daemon prio=5 os_prio=0 tid=0x00007fd52db00440 nid=0x3c4 in Object.wait() [0x00007fd4cb50b000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x00000000934d8a70> (a java.lang.ref.ReferenceQueue$Lock)
at com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"commons-pool-evictor-thread" #16 prio=5 os_prio=0 tid=0x00007fd52e5cdc10 nid=0x3c1 waiting on condition [0x00007fd4f14d5000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000935bb4a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"container-0" #14 prio=5 os_prio=0 tid=0x00007fd52f81b370 nid=0x3bf waiting on condition [0x00007fd504851000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.catalina.core.StandardServer.await(StandardServer.java:568)
at org.springframework.boot.web.embedded.tomcat.TomcatWebServer$1.run(TomcatWebServer.java:181)
"Catalina-utility-2" #13 prio=1 os_prio=0 tid=0x00007fd4d8d86e60 nid=0x3be waiting on condition [0x00007fd50500c000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000862c4590> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"Catalina-utility-1" #12 prio=1 os_prio=0 tid=0x00007fd52f59e230 nid=0x3bd waiting on condition [0x00007fd506fee000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000862c4590> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"Service Thread" #8 daemon prio=9 os_prio=0 tid=0x00007fd52c19d330 nid=0x3b6 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread2" #7 daemon prio=9 os_prio=0 tid=0x00007fd52c190940 nid=0x3b5 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007fd52c01afe0 nid=0x3b4 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007fd52c1823f0 nid=0x3b3 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007fd52c180b70 nid=0x3b2 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007fd52c157470 nid=0x3b1 in Object.wait() [0x00007fd5076f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x0000000085c74a60> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007fd52c1531e0 nid=0x3b0 in Object.wait() [0x00007fd5077f6000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
- locked <0x0000000085c74c90> (a java.lang.ref.Reference$Lock)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
"VM Thread" os_prio=0 tid=0x00007fd52c149db0 nid=0x3af runnable
"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007fd52c01dea0 nid=0x3ab runnable
"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007fd52c01f430 nid=0x3ac runnable
"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00007fd52c0209c0 nid=0x3ad runnable
"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00007fd52c021f50 nid=0x3ae runnable
"VM Periodic Task Thread" os_prio=0 tid=0x00007fd52c19f0e0 nid=0x3b7 waiting on condition
JNI global references: 1544
java.awt的Java2D Disposer是负责awt组件的GC的线程。- tomcat的1个container线程，2个ClientPoller线程，10个exec线程都是默认的配置。
- 4个parallel GC线程。
- 10个
quartzScheduler线程，项目里用了quartz跑定时任务。（我不知道为什么不用Spring的）
总之就是没有什么异样。但懂一点GC的人就要敏感了，为什么还在用parallel GC！？
3. jmap查看堆信息
查看存活对象
$ jmap -histo:live 937 |head -n 100
num #instances #bytes class name
----------------------------------------------
1: 6926 821156136 [B
2: 88761 12008952 [C
3: 82739 1985736 java.lang.String
4: 21136 1859968 java.lang.reflect.Method
5: 47252 1512064 java.util.concurrent.ConcurrentHashMap$Node
6: 12960 1440712 java.lang.Class
7: 20423 816920 java.util.LinkedHashMap$Entry
8: 5785 816208 [I
9: 12877 771704 [Ljava.lang.Object;
10: 8287 663480 [Ljava.util.HashMap$Node;
11: 10745 601720 java.util.LinkedHashMap
12: 17836 570752 java.util.HashMap$Node
13: 29662 474592 java.lang.Object
14: 260 450736 [Ljava.util.concurrent.ConcurrentHashMap$Node;
15: 13536 295136 [Ljava.lang.Class;
16: 7427 178248 org.springframework.core.MethodClassKey
17: 4667 176104 [Ljava.lang.String;
18: 3678 147120 java.lang.ref.SoftReference
19: 4577 146464 java.util.LinkedList
20: 1487 142752 org.springframework.beans.GenericTypeAwarePropertyDescriptor
21: 5643 135432 java.util.ArrayList
22: 2798 134160 [F
23: 2620 125760 java.util.HashMap
24: 1495 119600 java.lang.reflect.Constructor
25: 3503 112096 java.lang.ref.WeakReference
26: 1456 104832 org.springframework.core.annotation.AnnotationAttributes
27: 3865 92760 java.beans.MethodRef
28: 3548 85152 java.util.LinkedList$Node
29: 2032 81280 java.util.TreeMap$Entry
30: 1538 73824 org.springframework.core.ResolvableType
31: 1130 72320 java.net.URL
32: 4515 72240 java.util.LinkedHashSet
33: 2749 65976 sun.reflect.generics.tree.SimpleClassTypeSignature
34: 567 63504 org.springframework.boot.loader.jar.JarEntry
35: 986 63104 org.springframework.core.MethodParameter
36: 470 56624 [[C
37: 54 54768 [S
38: 969 54264 java.lang.invoke.MemberName
39: 2749 51720 [Lsun.reflect.generics.tree.TypeArgument;
40: 1058 50784 org.apache.tomcat.util.modeler.AttributeInfo
41: 702 50544 java.lang.reflect.Field
42: 1567 50144 java.util.Hashtable$Entry
43: 892 49952 java.lang.Package
44: 1491 47712 java.util.concurrent.locks.ReentrantLock$NonfairSync
45: 2093 46088 [Ljava.lang.reflect.Type;
46: 1854 44496 sun.reflect.annotation.AnnotationInvocationHandler
47: 1095 43800 java.util.WeakHashMap$Entry
48: 270 43200 org.springframework.beans.factory.support.RootBeanDefinition
49: 2630 42080 sun.reflect.generics.tree.ClassTypeSignature
50: 2585 41360 org.springframework.core.annotation.AnnotationUtils$DefaultValueHolder
51: 469 41272 org.apache.catalina.webresources.CachedResource
52: 10 41120 [Ljava.nio.ByteBuffer;
53: 1706 40944 java.util.jar.Attributes$Name
54: 2451 39216 java.util.LinkedHashMap$LinkedKeySet
55: 596 38144 java.util.concurrent.ConcurrentHashMap
56: 932 37280 java.lang.invoke.MethodType
57: 218 36624 org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader$ConfigurationClassBeanDefinition
58: 1100 35200 java.lang.ref.ReferenceQueue
59: 1075 34400 sun.security.util.DerInputBuffer
60: 1075 34400 sun.security.util.DerValue
61: 607 33992 java.beans.MethodDescriptor
62: 2059 32944 java.lang.Integer
63: 675 32400 ch.qos.logback.classic.Logger
64: 1336 32064 java.lang.Class$AnnotationData
65: 442 31824 org.springframework.core.type.classreading.AnnotationMetadataReadingVisitor
66: 752 30080 org.springframework.util.ConcurrentReferenceHashMap$Segment
67: 752 29920 [Lorg.springframework.util.ConcurrentReferenceHashMap$Reference;
68: 280 29888 [Ljava.util.Hashtable$Entry;
69: 933 29856 java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
70: 603 28944 java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync
71: 1767 28272 java.util.LinkedHashMap$LinkedEntrySet
72: 13 26832 [Lorg.apache.ibatis.ognl.internal.Entry;
73: 313 26016 [Z
74: 1075 25800 sun.security.util.DerInputStream
75: 1057 25368 [Lsun.security.x509.AVA;
76: 1057 25368 sun.security.x509.AVA
77: 1057 25368 sun.security.x509.RDN
78: 525 25200 org.springframework.util.ConcurrentReferenceHashMap$SoftEntryReference
79: 749 23968 javax.management.MBeanAttributeInfo
80: 477 22896 org.apache.tomcat.util.modeler.OperationInfo
81: 1412 22592 java.util.LinkedHashMap$LinkedValues
82: 922 22128 sun.reflect.generics.factory.CoreReflectionFactory
83: 551 22040 java.math.BigInteger
84: 184 21448 [Ljava.util.WeakHashMap$Entry;
85: 444 21312 org.apache.tomcat.util.buf.ByteChunk
86: 873 20952 sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl
87: 424 20352 org.apache.tomcat.util.buf.CharChunk
88: 420 20160 org.apache.tomcat.util.buf.MessageBytes
89: 826 19824 org.springframework.http.MediaType
90: 810 19440 org.springframework.beans.BeanMetadataAttribute
91: 806 19344 java.util.concurrent.atomic.AtomicLong
92: 590 18880 sun.reflect.generics.repository.ClassRepository
93: 577 18464 sun.security.util.ObjectIdentifier
94: 572 18304 org.springframework.boot.loader.jar.AsciiBytes
95: 569 18208 java.lang.invoke.DirectMethodHandle
96: 752 18048 org.springframework.util.ConcurrentReferenceHashMap$ReferenceManager
97: 443 17720 org.thymeleaf.engine.HTMLAttributeName
byte[],char[]有些过多了吧。但不借助工具，我们并不知道这些基础类型到底是哪里用到的。
查看堆
$ jmap -heap 937
Attaching to process ID 937, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.161-b14
using thread-local object allocation.
Parallel GC with 4 thread(s)
Heap Configuration:
MinHeapFreeRatio = 0
MaxHeapFreeRatio = 100
MaxHeapSize = 2051014656 (1956.0MB)
NewSize = 42991616 (41.0MB)
MaxNewSize = 683671552 (652.0MB)
OldSize = 87031808 (83.0MB)
NewRatio = 2
SurvivorRatio = 8
MetaspaceSize = 21807104 (20.796875MB)
CompressedClassSpaceSize = 1073741824 (1024.0MB)
MaxMetaspaceSize = 17592186044415 MB
G1HeapRegionSize = 0 (0.0MB)
Heap Usage:
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.tools.jmap.JMap.runTool(JMap.java:201)
at sun.tools.jmap.JMap.main(JMap.java:130)
Caused by: java.lang.RuntimeException: unknown CollectedHeap type : class sun.jvm.hotspot.gc_interface.CollectedHeap
at sun.jvm.hotspot.tools.HeapSummary.run(HeapSummary.java:144)
at sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
at sun.jvm.hotspot.tools.HeapSummary.main(HeapSummary.java:49)
... 6 more
debugInfo插件没有装，获取不到具体的Heap Usage信息，很遗憾。
所有的参数都是默认的。老年代加最大新生代总共离2个G的堆大小很遥远，不知道为什么。
4. jstat查看GC信息
$ jstat -gccause 937 1000
# 截取部分
S0 S1 E O M CCS YGC YGCT FGC FGCT GCT LGCC GCC
0.00 98.39 74.23 77.47 96.47 94.59 187 2.284 29 2.435 4.719 Allocation Failure No GC
0.00 98.39 74.23 77.47 96.47 94.59 187 2.284 29 2.435 4.719 Allocation Failure No GC
0.00 0.00 0.00 62.65 96.47 94.59 188 2.290 30 2.510 4.800 Heap Inspection Initiated GC No GC
0.00 0.00 0.01 62.65 96.47 94.59 188 2.290 30 2.510 4.800 Heap Inspection Initiated GC No GC
这一块线上分析的时候还不太会分析，我都不知道这些数字意味着什么。后面本地复现的时候分析的比较透彻。
5. 使用MAT分析jamp -dump
MAT: https://www.eclipse.org/mat/
可以看到，/qrv这个请求非常大，大到MAT怀疑它是内存泄露的元凶。
6. 使用JProfile分析JVM
JProfile: https://www.ej-technologies.com/products/jprofiler/overview.html
JProfile和jstat -gc的信息综合显示：进程的GC并没有出故障，没有常见内存泄露时出现的异常频率GC，大量Full GC等等；CPU占用，线程资源也没有异常；使用JProfile触发GC后，内存的下降并不明显。
重启后的JVM：
本地分析
过了一两天，忽然tech leader提示我可以本地一步步地，一边执行部分代码，一边看JVM的情况。我茅塞顿开，有了一些结论。
我的朋友，在饿了么高就的IceHe.xyz提醒我，是不是JVM资源给少了。我说都是默认的，他说先给到 4G 到 8G 再看看吧，JVM没必要扣太细。
以下会有实际的代码截图，代码写的非常。
都不是我写的！！！ 都不是我写的！！！ 都不是我写的！！！
都不是我写的！！！ 都不是我写的！！！ 都不是我写的！！！
都不是我写的！！！ 都不是我写的！！！ 都不是我写的！！！
0. JVM参数
线上使用的是默认的Parallel GC，我这边默认的是G1 GC（JDK 13），需要手动调整。堆的大小也与服务端保持一致。其他都和服务端一样保持默认。（如有可能，应该使用更新的G1 GC）
# JVM Options:
-server
-XX:+UseParallelGC
-Xms2048M
-Xmx2048M
-Xlog:gc*
后来发现，/qrv接口所涉及的对象都比较大，故需要将young generation的大小初始化得大一些。如果这是一个4G的Heap，young generation就需要 1.5G 左右。
# JVM Options:
-Xmn600M
后来发现，项目在启动时必发生Full GC(Metadata GC Threshold)，所以加大了MetaSpace。
# JVM Options:
-XX:MetaspaceSize=128M
1. 阶段1
解析二维码前，读取图片后。
结果表明：手动触发Full GC后堆内存能明显下降；持续不断地调用时GC也能正常触发。综上这段代码应该没有问题。
2. 阶段2
存储图片前，解析二维码后。
结果表明：这一部分代码就是一个性能瓶颈。代码跑到这里，一次请求就会有2-3次Young GC，请求的多了以后老年代用的多了还会触发Full GC。
但截至目前，无论性能多么有瓶颈，还没有出现不释放的问题。
3. 阶段3
解析二维码后，算法校验前。
结果表明：UploadUtils.storageFile的高效NIO操作没有引发任何问题。
4. 阶段4
全部覆盖。我这里是没有opencv-python算法可以跑的，所以那一块覆盖不到。但那一块tech leader测过了。
结果表明：/qrv在没有运行Python脚本的本地测试环境没有致命的系统故障。现在3秒的运行耗时也是基本满足要求的（哪怕没有做更进一步的优化）
总结
搞JVM先要懂JVM的内存结构，超链接中是老外的一篇资料。
本文主要分析的是堆，摘一张图片出来：
GC Collector主要有 Serial, Parallel, CMS, G1和最新的 ZGC，每一种还有不同代的配置。不同的GC有不同的GC算法，有一张比较直观的图：
不同GC的详细区别，理论知识我就不在这里赘述了，网上都有。但最好我们应该看经典的书和官方文档：Java Garbage Collection Basics
另外，在未来，老GC都要被Deprecate啦！
我空下来一定开始认真读深入理解Java虚拟机（第2版）。