NGINX + TOMCAT出现大量的TIME-WAIT状态的TCP连接解决

昨天把两个tomcat服务停止后，更新项目，更新完启动，两个tomcat都报了异常，其中一个tomcat报的异常如下：

[com.alibaba.druid.proxy.DruidDriver]，但在Web应用程序停止时无法注销它。 为防止内存泄漏，JDBC驱动程序已被强制取消注册。
14-May-2020 14:38:24.188 警告 [Thread-6] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc Web应用程序 [ROOT] 注册了JDBC驱动程序 [com.mysql.jdbc.Driver]，但在Web应用程序停止时无法注销它。 为防止内存泄漏，JDBC驱动程序已被强制取消注册。
14-May-2020 14:38:24.189 警告 [Thread-6] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.ref.ReferenceQueue.remove(Unknown Source)
 com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:43)
14-May-2020 14:38:24.231 警告 [Thread-6] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc Web应用程序 [WebLogin] 注册了JDBC驱动程序 [com.alibaba.druid.proxy.DruidDriver]，但在Web应用程序停止时无法注销它。 为防止内存泄漏，JDBC驱动程序已被强制取消注册。
14-May-2020 14:38:24.231 警告 [Thread-6] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc Web应用程序 [WebLogin] 注册了JDBC驱动程序 [com.mysql.jdbc.Driver]，但在Web应用程序停止时无法注销它。 为防止内存泄漏，JDBC驱动程序已被强制取消注册。
14-May-2020 14:38:24.232 警告 [Thread-6] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [WebLogin] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.ref.ReferenceQueue.remove(Unknown Source)
 com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:43)
14-May-2020 14:38:24.245 信息 [Thread-6] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["http-nio-9026"]
14-May-2020 14:38:24.247 信息 [Thread-6] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["ajp-nio-9029"]
14-May-2020 14:38:24.250 信息 [Thread-6] org.apache.coyote.AbstractProtocol.destroy 正在摧毁协议处理器 ["http-nio-9026"]
14-May-2020 14:38:24.251 信息 [Thread-6] org.apache.coyote.AbstractProtocol.destroy 正在摧毁协议处理器 ["ajp-nio-9029"]

另一个tomcat报的异常如下：

14-May-2020 19:50:17.160 警告 [main] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [myProject] registered the JDBC driver [com.alibaba.druid.proxy.DruidDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
14-May-2020 19:50:17.162 警告 [main] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [myProject] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
14-May-2020 19:50:17.230 警告 [main] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [myProject] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.ref.ReferenceQueue.remove(Unknown Source)
 com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:43)
14-May-2020 19:50:17.310 信息 [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [D:\tomcatServer\tomcat-9.0.10\webapps\myProject.war] has finished in [7,240] ms
14-May-2020 19:50:17.314 信息 [main] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [D:\tomcatServer\tomcat-9.0.10\webapps\docs]
14-May-2020 19:50:17.361 信息 [main] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [D:\tomcatServer\tomcat-9.0.10\webapps\docs] has finished in [47] ms
14-May-2020 19:50:17.362 信息 [main] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [D:\tomcatServer\tomcat-9.0.10\webapps\manager]
14-May-2020 19:50:17.391 信息 [main] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [D:\tomcatServer\tomcat-9.0.10\webapps\manager] has finished in [28] ms
14-May-2020 19:50:17.395 信息 [main] org.apache.catalina.startup.Catalina.start Server startup in 18478 ms
14-May-2020 19:50:17.397 严重 [main] org.apache.catalina.core.StandardServer.await StandardServer.await: create[localhost:9015]: 
 java.net.BindException: Address already in use: JVM_Bind
	at java.net.DualStackPlainSocketImpl.bind0(Native Method)
	at java.net.DualStackPlainSocketImpl.socketBind(Unknown Source)
	at java.net.AbstractPlainSocketImpl.bind(Unknown Source)
	at java.net.PlainSocketImpl.bind(Unknown Source)
	at java.net.ServerSocket.bind(Unknown Source)
	at java.net.ServerSocket.(Unknown Source)
	at org.apache.catalina.core.StandardServer.await(StandardServer.java:422)
	at org.apache.catalina.startup.Catalina.await(Catalina.java:770)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:716)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:350)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:492)

当时是百思不得其解，我是正常关闭tomcat服务以后才更新的文件，怎么会报内存溢出呢？当时也是一时手足无措，把想到的关联因素redis和nginx关闭后重新启动还是老样子。今天有点头绪，会不会是关闭tomcat服务时还有残余线程存在呢？于是今天再现昨天的情景，先把两个tomcat关闭，然后通过cmd命令提示符的netstat -aon|findstr "9010"命令查看指定端口号的占用情况，

 可以看到，尽管两个tomcat服务已关闭，但依然有TIME_WAIT进程在占用。所以重新启动服务时报了内存溢出的异常。由于我通过nginx反向代理了tomcat，受到TCP协议的重传机制的限制。那么问题就出在tomcat和nginx的配置上了，查了下网上资料，主要原因是：

1.nginx开启了keepalive ，而且默认用的http1.1

2.tomcat开启了keepalive，默认也使用http1.1

3.但是nginx代理却使用的http1.0，默认不开启keepalive，所以就造成了大量的TIME-WAID状态的TCP连接

同时也给出两种解决方法，第二种我没试，感兴趣可以前往一试https://my.oschina.net/haitaohu/blog/3043113，不知效果如何，我是参照第一种方法即nginx官网的方法，修改我的nginx配置，

nginx官网提供

For HTTP, the proxy_http_version directive should be set to “1.1” and the “Connection” header field should be cleared:
参考地址:http://nginx.org/en/docs/http/ngx_http_upstream_module.html

upstream http_backend {
    server 127.0.0.1:8080;

    keepalive 16;
}

server {
    ...

    location /http/ {
        proxy_pass http://http_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        ...
    }
}

修改好后，重新加载nginx,再次使用cmd命令查看端口号占用情况，结果如下：

可以看到，原先TIME_WAIT那些进程消失了，然后去启动tomcat,，去看日志，发现异常已经消失，也可以访问项目了。