No buffer space available终极解决办法

这是个标题党。

周一到公司打开电脑准备再测试下上周写完的接口

卧X，查询数据库报错了！

这两天程序乖乖的跑在服务器上，也没人动啊！

因为上次遇到过Druid连接池关闭链接报错的问题，首先怀疑是不是数据库连不上了？

因为调其他接口也不通，又怀疑是不是网络断了？

一看日志，无法建立链接，无论是查数据库还是调其他服务都是无法建立链接。可是为啥会出现这种情况呐？

Caused by: com.dareway.framework.exception.AppException: -1java.sql.SQLException: The Network Adapter could not establish the connection
	at com.dareway.framework.dbengine.dbmetadata.DBMetaDataUtil.getConnection(DBMetaDataUtil.java:20)
	at com.dareway.framework.dbengine.dbmetadata.dbrelation.OracleDBRelation.initDBRelation(OracleDBRelation.java:88)
	at com.dareway.framework.dbengine.dbmetadata.dbrelation.DBRelation.(DBRelation.java:48)
	at com.dareway.framework.dbengine.dbmetadata.dbrelation.OracleDBRelation.(OracleDBRelation.java:30)
	at com.dareway.framework.dbengine.dbmetadata.dbuser.OracleDBUser.getDBRelation(OracleDBUser.java:84)
	at com.dareway.framework.dbengine.dbmetadata.DDSMetaData.getRelationMd5FromCache(DDSMetaData.java:169)
	at com.dareway.framework.dbengine.datasource.DDS.getRelationMd5FromCache(DDS.java:243)
	at com.dareway.framework.dbengine.datasource.DDS.getPlan(DDS.java:199)
	at com.dareway.framework.dbengine.DE.query(DE.java:272)
	... 109 common frames omitted
Caused by: java.sql.SQLException: The Network Adapter could not establish the connection
	at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:412)
	at oracle.jdbc.driver.PhysicalConnection.(PhysicalConnection.java:531)
	at oracle.jdbc.driver.T4CConnection.(T4CConnection.java:221)
	at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
	at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:503)
	at java.sql.DriverManager.getConnection(Unknown Source)
	at java.sql.DriverManager.getConnection(Unknown Source)
	at com.dareway.framework.dbengine.dbmetadata.DBMetaDataUtil.getConnection(DBMetaDataUtil.java:17)
	... 117 common frames omitted
Caused by: oracle.net.ns.NetException: The Network Adapter could not establish the connection
	at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:359)
	at oracle.net.resolver.AddrResolution.resolveAndExecute(AddrResolution.java:422)
	at oracle.net.ns.NSProtocol.establishConnection(NSProtocol.java:672)
	at oracle.net.ns.NSProtocol.connect(NSProtocol.java:237)
	at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1042)
	at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:301)
	... 124 common frames omitted
Caused by: java.net.SocketException: No buffer space available (maximum connections reached?): connect
	at java.net.DualStackPlainSocketImpl.connect0(Native Method)
	at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
	at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
	at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
	at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
	at java.net.PlainSocketImpl.connect(Unknown Source)
	at java.net.SocksSocketImpl.connect(Unknown Source)
	at java.net.Socket.connect(Unknown Source)
	at oracle.net.nt.TcpNTAdapter.connect(TcpNTAdapter.java:141)
	at oracle.net.nt.ConnOption.connect(ConnOption.java:123)
	at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:337)
	... 129 common frames omitted

看到最后一行   No buffer space available (maximum connections reached?): connect

·难倒达到最大连接数了？这台服务器虽然跑着4个服务，但是都是自己要用的，怎么会有这么多链接数呐？

通过netstat -ano命令，一看处于TIME_WAIT的链接数为0，但状态为ESTABLISHED状态的链接数是1940，且大多数都是同一个pid的，经过查询，确实是自己的一个服务产生的。

经过详细的排查，发现，那个服务运行后就会主动链接xxl-job，但是xxl_job所在的服务器的网络不通。

关闭这个模块就好了。

遇到这个问题，一般都是达到最大连接数了，那么此时就要排查下，究竟是什么地方建立了链接，是服务器最大连接数不够还是大量的链接被占用了