分析:数据包对象PcapPacket是JMemory子类,JMemory使用直接内存(java堆外分配的内存)来分配内存,分配时需使用maxDirectMemory()方法来获取最大允许的直接内存,由于获取的最大内存总是64M,所以当数据包所占空间大于64M时就会OutOfMemeory。
解决办法:可能这种并不算是bug,只是没找到合适的API去设置最大直接内存,(额,我找了一圈是没找到),所以就简单粗暴的将maxDirectMemory()方法的返回值设为虚拟机的最大直接内存(sun.misc.VM.maxDirectMemory())了。。。
---------------------------------------------------------详细过程------------------------------------------------------------------
最近在用jnetpcap开发一个实时流量监控分析系统
由于需要实时查看数据包信息,我是用ArrayList将所有数据包保存起来,使用时发现当捕获到的数据包数量达到一定量时(大约50000左右)程序就崩溃了,并抛出如下异常。
图中显示错误来自pcap的loop方法,但loop方法是本地方法,并没法看到其具体什么地方抛出异常。
几经辗转,使用jvisualvm捕捉到了异常时的线程dump
可以看到出错和org.jnetpcap.nio.JMemory有关。
查看JMemory源码有如下方法:
/**
* Returns the hard limit for the amount of memory native is allowed to
* allocate. The memory setting defaults to JVMs max memory which can be
* specified with JVM command line option '-Xmx<size>.Once the 'nio.mx'
* limit is reached, the allocating thread is blocked and a JVM GC request is
* issued. The allocating thread continues to wait, until sufficient minimum
* amount (Default: {@literal DisposableGC#MIN_MEMORY_RELEASE})of native
* memory was cleaned up or a timeout (Default:
*
* @return the limit in number of bytes
* {@literal DisposableGC#OUT_OF_MEMORY_TIMEOUT} ms) occurs.
*
* This limit can be set at startup of the application using the
* following system properties, which are checked in the order listed
* below:
*
* org.jnetsoft.nio.MaxDirectMemorySize
* nio.MaxDirectMemorySize
* org.jnetsoft.nio.mx
* nio.mx
*
* The different property names, from the most fully qualified to the
* least, are provided to property name conflict resolution. For
* convenience, it is recommended that the user choose the least
* qualified property name to use. In the unlikely event that another
* library within the same runtime application uses the same property
* name, one of the more qualified (or longer) property names can be
* used to resolve the conflict.
*
*/
public static long maxDirectMemory() {
if (directMemory != 0) {
return directMemory;
}
Properties p = System.getProperties();
String s = p.getProperty("org.jnetsoft.nio.MaxDirectMemorySize");
s = (s == null) ? p.getProperty("nio.MaxDirectMemorySize") : s;
s = (s == null) ? p.getProperty("org.jnetsoft.nio.mx") : s;
s = (s == null) ? p.getProperty("nio.mx") : s;
if (s != null) {
directMemory = parseSize(s); // process suffixes kb,mb,gb,tb
}
if (directMemory == 0) {
directMemory = maxDirectoryMemoryDefault();
}
return directMemory;
}
将此方法直接测试运行,发现返回总是64M,也就是
public static final long MAX_DIRECT_MEMORY_DEFAULT = 64 * Units.MEBIBYTE;
定义的大小。
分析:数据包对象PcapPacket是JMemory子类,JMemory使用直接内存(java堆外分配的内存)来分配内存,分配时需使用maxDirectMemory()方法来获取最大允许的直接内存,由于获取的最大内存总是64M,所以当数据包所占空间大于64M时就会OutOfMemeory。
解决办法:可能这种并不算是bug,只是没找到合适的API去设置最大直接内存,(额,我找了一圈是没找到),所以就简单粗暴的将maxDirectMemory()方法的返回值设为虚拟机的最大直接内存了。。。
public static long maxDirectMemory() {
if (directMemory != 0) {
return directMemory;
}
Properties p = System.getProperties();
String s = p.getProperty("org.jnetsoft.nio.MaxDirectMemorySize");
s = (s == null) ? p.getProperty("nio.MaxDirectMemorySize") : s;
s = (s == null) ? p.getProperty("org.jnetsoft.nio.mx") : s;
s = (s == null) ? p.getProperty("nio.mx") : s;
if (s != null) {
directMemory = parseSize(s); // process suffixes kb,mb,gb,tb
}
if (directMemory == 0) {
directMemory = maxDirectoryMemoryDefault();
}
return sun.misc.VM.maxDirectMemory();
}
只是在最后return 处返回了sun.misc.VM.maxDirectMemory(),其他代码尽量不动,然后将源码重新打成jar包放入项目(具体过程:找到Jnetpcap源码,新建一个工程,将src代码复制进去,修改源码,打成jar包)中,额算是顺利解决吧。
2020/3/19 重新回顾此项目,似乎源码并没有问题,只是我没正确配置最大直接内存(原先实在太蠢- -!)
在程序中直接加入以下代码似乎就可以了(未测试):
System.setProperty("org.jnetsoft.nio.MaxDirectMemorySize", "2gb");