1 提供堆外内存数据库
org.mapdb.DBMaker.newDirectMemoryDB()
2 提供基于B+树的磁盘数据库
org.mapdb.BTreeMap<K, V>,http://www.cs.cornell.edu/courses/cs4411/2009sp/blink.pdf
B+树变种的一个实现,实际数据存储在叶子节点。使用StoreDirect存储的时候,每个节点作为一条记录,对应索引文件的一个索引,索引存储节点所在数据文件的偏移量和size。B树的初始化会从索引文件读取根节点进行初始化。一旦根节点载入内存中,就可以对B树进行操作了。B树的操作主要是针对节点,每个节点都是以块为单位进行磁盘io操作,节点的大小尽可能和磁盘块的大小相当。
http://hill007299.iteye.com/admin/blogs/1434660
3 有几种存储引擎
org.mapdb.StoreDirect直接存储磁盘,通过索引文件和数据文件来实现。数据文件以块为单位进行存储,当一条记录大于一个块大小的时候,需要有指针指向下一个块。当记录更新时,更新后的size和原来不同时,需要重新释放块和重新分配块,在新的块中进行写入。
org.mapdb.StoreWAL,提供事务功能,以及批量写操作。
org.mapdb.StoreHeap,使用堆内存储,无序列化。
4 StoreDirect
写入新数据需要申请索引文件空间和数据文件空间,删除或者更新数据后,可能会释放索引文件空间和数据文件空间。
索引文件空间的申请和回收需要从索引文件的第15个slot中获取。(每个slot为一个64bit的long,维护偏移量和数据大小)
15 | {@link StoreDirect#IO_FREE_RECID} |Long Stack of deleted recids, those will be reused and returned by {@link Engine#put(Object, Serializer)}
每个索引项的空间是定长的,所以只要用一个slot来维护就行了,从slot中拿到偏移量和尺寸,然后到数据文件中拿到索引文件的空闲项列表。
数据文件空闲项需要从索引文件的16..4111个slot中获得。
16..4111 | |Long Stack of free physical records. This contains free space released by record update or delete. Each slots corresponds to free record size.
由于空闲数据项是不定长的,所以需要由多个slot维护,每个slot维护同样大小的空闲项列表。从slot中拿到偏移量和尺寸,然后到数据文件中拿到数据文件的空闲项列表。
数据文件的每个块最小为16byte,最大为64kb,当大于64kb之后,就要通过链接法来扩容。对应的索引项会有指针指向下一个数据块。由此也可以看出数据块有4096种尺寸,所以需要用4096个slot来维护空闲数据块,类似伙伴系统算法。
http://www.coderli.com/translate-java-collections-bigdata-mapdb
5 mapdb堆外内存性能测试
本地测试环境:win7 4核 8g jdk 7
hashmap测试,put 1000000次,get 1000000次;key为字符串,value字符串。
- 堆外hashmap,put需要3600毫秒,get需要900毫秒
- jdk原生 ConcurrentHashMap,put需要2500毫秒,get需要2800毫秒
hashmap测试,put 1000000次,get 1000000次;key为字符串,value为一个可序列化小对象。
- 堆外hashmap,put需要5500毫秒,get需要1500毫秒
- jdk原生 ConcurrentHashMap,put需要5000毫秒,get需要450毫秒
blocking queue,add 1000000次,poll 1000000次,value字符串
- 堆外queue, add 3000毫秒,poll 1300毫秒
- LinkedBlockingQueue,add 1300毫秒,poll 100毫秒
blocking queue,add 1000000次,poll 1000000次,value为一个可序列化小对象。
- 堆外queue, add 4300毫秒,poll 2100毫秒
- LinkedBlockingQueue,add 1300毫秒,poll 100毫秒
大部分情况下,堆外内存的性能都是不如堆内内存。但是当内存消耗比较大的时候,gc频繁或者full gc频繁的情况下,可以考虑使用堆外内存,可以减少gc的性能消耗。
hashmap测试,put 1500000次,get 1500000次;key为字符串,value为一个可序列化小对象。堆内、堆外内存均分配300M,gc较为频繁。
- 堆外hashmap,put需要7800毫秒。
gc状况:
2014-02-23T21:44:39.901+0800: [GC [PSYoungGen: 76800K->8667K(89600K)] 76800K->8667K(294400K), 0.0270176 secs] [Times: user=0.05 sys=0.01, real=0.03 secs]
2014-02-23T21:44:40.358+0800: [GC [PSYoungGen: 85467K->8056K(89600K)] 85467K->8056K(294400K), 0.0232078 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T21:44:40.825+0800: [GC [PSYoungGen: 84856K->8040K(89600K)] 84856K->8040K(294400K), 0.0255123 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T21:44:41.288+0800: [GC [PSYoungGen: 84840K->8024K(89600K)] 84840K->8032K(294400K), 0.0208046 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T21:44:41.769+0800: [GC [PSYoungGen: 84824K->8056K(89600K)] 84832K->8064K(294400K), 0.0250652 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T21:44:42.225+0800: [GC [PSYoungGen: 84856K->8056K(93760K)] 84864K->8064K(298560K), 0.0248708 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T21:44:42.727+0800: [GC [PSYoungGen: 93112K->7136K(92224K)] 93120K->8032K(297024K), 0.0240836 secs] [Times: user=0.02 sys=0.02, real=0.03 secs]
2014-02-23T21:44:43.231+0800: [GC [PSYoungGen: 92192K->7136K(93376K)] 93088K->8032K(298176K), 0.0298674 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T21:44:43.803+0800: [GC [PSYoungGen: 91744K->7136K(93504K)] 92640K->8040K(298304K), 0.0203759 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T21:44:44.322+0800: [GC [PSYoungGen: 91744K->7136K(93504K)] 92648K->8040K(298304K), 0.0280807 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T21:44:44.836+0800: [GC [PSYoungGen: 91744K->7136K(93504K)] 92648K->8048K(298304K), 0.0256818 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T21:44:45.344+0800: [GC [PSYoungGen: 91744K->7136K(93504K)] 92656K->8048K(298304K), 0.0259804 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T21:44:45.863+0800: [GC [PSYoungGen: 91744K->7136K(93504K)] 92656K->8048K(298304K), 0.0202933 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T21:44:46.367+0800: [GC [PSYoungGen: 91744K->7136K(93568K)] 92656K->8048K(298368K), 0.0305617 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
7824
Heap
PSYoungGen total 93568K, used 18175K [0x00000000f9c00000, 0x0000000100000000, 0x0000000100000000)
eden space 84736K, 13% used [0x00000000f9c00000,0x00000000fa6c7ef0,0x00000000feec0000)
from space 8832K, 80% used [0x00000000ff760000,0x00000000ffe58000,0x0000000100000000)
to space 8832K, 0% used [0x00000000feec0000,0x00000000feec0000,0x00000000ff760000)
ParOldGen total 204800K, used 912K [0x00000000ed400000, 0x00000000f9c00000, 0x00000000f9c00000)
object space 204800K, 0% used [0x00000000ed400000,0x00000000ed4e40b0,0x00000000f9c00000)
PSPermGen total 32768K, used 4805K [0x00000000eb400000, 0x00000000ed400000, 0x00000000ed400000)
object space 32768K, 14% used [0x00000000eb400000,0x00000000eb8b1690,0x00000000ed400000)
- jdk原生 ConcurrentHashMap,put需要9000毫秒
2014-02-23T21:42:55.082+0800: [GC [PSYoungGen: 89096K->12768K(89600K)] 144862K->142498K(294400K), 0.3413265 secs] [Times: user=0.66 sys=0.03, real=0.34 secs]
2014-02-23T21:42:55.423+0800: [Full GC [PSYoungGen: 12768K->0K(89600K)] [ParOldGen: 129730K->138298K(204800K)] 142498K->138298K(294400K) [PSPermGen: 4347K->4344K(32768K)], 2.8367373 secs] [Times: user=5.19 sys=0.02, real=2.84 secs]
2014-02-23T21:42:58.493+0800: [Full GC [PSYoungGen: 76800K->2103K(89600K)] [ParOldGen: 138298K->204798K(204800K)] 215098K->206901K(294400K) [PSPermGen: 4344K->4344K(32768K)], 4.7044996 secs] [Times: user=8.70 sys=0.08, real=4.70 secs]
9080
Heap
PSYoungGen total 89600K, used 62999K [0x00000000f9c00000, 0x0000000100000000, 0x0000000100000000)
eden space 76800K, 82% used [0x00000000f9c00000,0x00000000fd985fd0,0x00000000fe700000)
from space 12800K, 0% used [0x00000000ff380000,0x00000000ff380000,0x0000000100000000)
to space 12800K, 0% used [0x00000000fe700000,0x00000000fe700000,0x00000000ff380000)
ParOldGen total 204800K, used 204798K [0x00000000ed400000, 0x00000000f9c00000, 0x00000000f9c00000)
object space 204800K, 99% used [0x00000000ed400000,0x00000000f9bff818,0x00000000f9c00000)
PSPermGen total 32768K, used 4355K [0x00000000eb400000, 0x00000000ed400000, 0x00000000ed400000)
object space 32768K, 13% used [0x00000000eb400000,0x00000000eb840c38,0x00000000ed40000
堆内400M,原生并发hashmap,插入2000000条记录,耗时10600毫秒
2014-02-23T21:52:06.133+0800: [GC [PSYoungGen: 102464K->17016K(119488K)] 102464K->89694K(392576K), 0.3148040 secs] [Times: user=0.55 sys=0.08, real=0.32 secs]
2014-02-23T21:52:06.745+0800: [GC [PSYoungGen: 119480K->16992K(119488K)] 192158K->188490K(392576K), 0.4307298 secs] [Times: user=0.80 sys=0.08, real=0.43 secs]
2014-02-23T21:52:07.177+0800: [Full GC [PSYoungGen: 16992K->0K(119488K)] [ParOldGen: 171498K->181082K(273088K)] 188490K->181082K(392576K) [PSPermGen: 4347K->4344K(32768K)], 3.8082684 secs] [Times: user=6.99 sys=0.00, real=3.81 secs]
2014-02-23T21:52:11.311+0800: [Full GC [PSYoungGen: 102464K->7215K(119488K)] [ParOldGen: 181082K->273084K(273088K)] 283546K->280300K(392576K) [PSPermGen: 4344K->4344K(32768K)], 4.6063078 secs] [Times: user=8.39 sys=0.05, real=4.61 secs]
10486
Heap
PSYoungGen total 119488K, used 102464K [0x00000000f7ab0000, 0x0000000100000000, 0x0000000100000000)
eden space 102464K, 100% used [0x00000000f7ab0000,0x00000000fdec0000,0x00000000fdec0000)
from space 17024K, 0% used [0x00000000fef60000,0x00000000fef60000,0x0000000100000000)
to space 17024K, 0% used [0x00000000fdec0000,0x00000000fdec0000,0x00000000fef60000)
ParOldGen total 273088K, used 273084K [0x00000000e7000000, 0x00000000f7ab0000, 0x00000000f7ab0000)
object space 273088K, 99% used [0x00000000e7000000,0x00000000f7aaf2e8,0x00000000f7ab0000)
PSPermGen total 32768K, used 4355K [0x00000000e5000000, 0x00000000e7000000, 0x00000000e7000000)
object space 32768K, 13% used [0x00000000e5000000,0x00000000e5440c38,0x00000000e7000000)
堆内100M,堆外300M。读写1500000.
堆外内存
2014-02-23T22:26:05.236+0800: [GC [PSYoungGen: 25664K->4200K(29888K)] 25664K->7389K(98176K), 0.0210433 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:05.412+0800: [GC [PSYoungGen: 29864K->4200K(29888K)] 33053K->11213K(98176K), 0.0271677 secs] [Times: user=0.05 sys=0.02, real=0.03 secs]
2014-02-23T22:26:05.591+0800: [GC [PSYoungGen: 29864K->4200K(29888K)] 36877K->14741K(98176K), 0.0257126 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:05.774+0800: [GC [PSYoungGen: 29864K->4200K(29888K)] 40405K->18573K(98176K), 0.0229136 secs] [Times: user=0.05 sys=0.02, real=0.02 secs]
2014-02-23T22:26:05.944+0800: [GC [PSYoungGen: 29864K->4200K(29888K)] 44237K->22389K(98176K), 0.0216205 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T22:26:06.112+0800: [GC [PSYoungGen: 29864K->4200K(21568K)] 48053K->26213K(89856K), 0.0293965 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:06.238+0800: [GC [PSYoungGen: 21544K->6304K(23680K)] 43557K->29198K(91968K), 0.0182432 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:06.370+0800: [GC [PSYoungGen: 23648K->7136K(25472K)] 46542K->30038K(93760K), 0.0215713 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:06.490+0800: [GC [PSYoungGen: 23968K->7136K(25472K)] 46870K->30046K(93760K), 0.0191508 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:06.605+0800: [GC [PSYoungGen: 23968K->7136K(25536K)] 46878K->30054K(93824K), 0.0278604 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:06.729+0800: [GC [PSYoungGen: 24096K->5024K(22016K)] 47014K->30046K(90304K), 0.0263060 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:06.851+0800: [GC [PSYoungGen: 21984K->5056K(24832K)] 47006K->32182K(93120K), 0.0293177 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:06.970+0800: [GC [PSYoungGen: 20992K->4736K(20672K)] 48118K->34270K(88960K), 0.0270565 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:07.088+0800: [GC [PSYoungGen: 20672K->4736K(24576K)] 50206K->36686K(92864K), 0.0258594 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:07.200+0800: [GC [PSYoungGen: 20032K->4544K(19840K)] 51982K->38542K(88128K), 0.0269296 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:07.315+0800: [GC [PSYoungGen: 19840K->4576K(24640K)] 53838K->41166K(92928K), 0.0305050 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:07.436+0800: [GC [PSYoungGen: 19744K->4544K(24640K)] 56334K->43766K(92928K), 0.0257104 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:07.580+0800: [GC [PSYoungGen: 19712K->4512K(24768K)] 58934K->46366K(93056K), 0.0294646 secs] [Times: user=0.05 sys=0.02, real=0.03 secs]
2014-02-23T22:26:07.697+0800: [GC [PSYoungGen: 19872K->4576K(24704K)] 61726K->49014K(92992K), 0.0261062 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:07.810+0800: [GC [PSYoungGen: 19936K->4576K(25024K)] 64374K->51590K(93312K), 0.0247326 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:07.933+0800: [GC [PSYoungGen: 20384K->3232K(24896K)] 67398K->54198K(93184K), 0.0273302 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:08.054+0800: [GC [PSYoungGen: 19040K->4704K(25216K)] 70006K->57174K(93504K), 0.0243379 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T22:26:08.174+0800: [GC [PSYoungGen: 20768K->4800K(24960K)] 73238K->59630K(93248K), 0.0296724 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:08.299+0800: [GC [PSYoungGen: 20864K->4800K(25728K)] 75694K->62006K(94016K), 0.0260036 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:08.423+0800: [GC [PSYoungGen: 21888K->5088K(25472K)] 79094K->64358K(93760K), 0.0266596 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:08.552+0800: [GC [PSYoungGen: 22176K->5088K(26304K)] 81446K->66422K(94592K), 0.0298539 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:08.689+0800: [GC [PSYoungGen: 23328K->5440K(26048K)] 84662K->68494K(94336K), 0.0292999 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:08.828+0800: [GC [PSYoungGen: 23680K->5440K(26624K)] 86734K->70222K(94912K), 0.0277757 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:08.856+0800: [Full GC [PSYoungGen: 5440K->0K(26624K)] [ParOldGen: 64782K->7974K(68288K)] 70222K->7974K(94912K) [PSPermGen: 4794K->4791K(32768K)], 0.0958269 secs] [Times: user=0.14 sys=0.00, real=0.10 secs]
2014-02-23T22:26:09.061+0800: [GC [PSYoungGen: 18944K->5632K(26432K)] 26918K->13606K(94720K), 0.0175213 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:09.191+0800: [GC [PSYoungGen: 24576K->5632K(26752K)] 32550K->15126K(95040K), 0.0288010 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:09.331+0800: [GC [PSYoungGen: 24960K->5728K(26688K)] 34454K->16622K(94976K), 0.0305644 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:09.477+0800: [GC [PSYoungGen: 25056K->3392K(26560K)] 35950K->18046K(94848K), 0.0201778 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:09.610+0800: [GC [PSYoungGen: 22656K->5728K(26688K)] 37310K->21798K(94976K), 0.0243406 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:09.748+0800: [GC [PSYoungGen: 24992K->5728K(26752K)] 41062K->23222K(95040K), 0.0254426 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:09.886+0800: [GC [PSYoungGen: 25056K->5760K(26688K)] 42550K->24654K(94976K), 0.0257207 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:10.024+0800: [GC [PSYoungGen: 25088K->5728K(26816K)] 43982K->26014K(95104K), 0.0242931 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:10.225+0800: [GC [PSYoungGen: 25184K->5792K(26752K)] 45470K->27438K(95040K), 0.0202609 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:10.355+0800: [GC [PSYoungGen: 25248K->5760K(26880K)] 46894K->28774K(95168K), 0.0202884 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:10.488+0800: [GC [PSYoungGen: 25344K->5824K(26816K)] 48358K->30166K(95104K), 0.0307858 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:10.636+0800: [GC [PSYoungGen: 25408K->5760K(27008K)] 49750K->31502K(95296K), 0.0253001 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:10.775+0800: [GC [PSYoungGen: 25600K->5888K(26944K)] 51342K->32878K(95232K), 0.0237111 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:10.930+0800: [GC [PSYoungGen: 25728K->3584K(26624K)] 52718K->34142K(94912K), 0.0300698 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:11.075+0800: [GC [PSYoungGen: 23104K->5760K(26816K)] 53662K->37694K(95104K), 0.0231350 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T22:26:11.215+0800: [GC [PSYoungGen: 25280K->5760K(26880K)] 57214K->39086K(95168K), 0.0211000 secs] [Times: user=0.01 sys=0.01, real=0.02 secs]
2014-02-23T22:26:11.352+0800: [GC [PSYoungGen: 25408K->5824K(26880K)] 58734K->40486K(95168K), 0.0249761 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:11.494+0800: [GC [PSYoungGen: 25472K->5792K(27008K)] 60134K->41790K(95296K), 0.0299500 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:11.642+0800: [GC [PSYoungGen: 25632K->5856K(26944K)] 61630K->43150K(95232K), 0.0251581 secs] [Times: user=0.05 sys=0.00, real=0.03 secs]
2014-02-23T22:26:11.784+0800: [GC [PSYoungGen: 25696K->5856K(27072K)] 62990K->44430K(95360K), 0.0258454 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:11.926+0800: [GC [PSYoungGen: 25824K->5920K(27008K)] 64398K->45734K(95296K), 0.0230664 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:12.067+0800: [GC [PSYoungGen: 25888K->5888K(27072K)] 65702K->46950K(95360K), 0.0305990 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:12.218+0800: [GC [PSYoungGen: 25920K->5920K(27072K)] 66982K->48214K(95360K), 0.0205676 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T22:26:12.356+0800: [GC [PSYoungGen: 25952K->4544K(27008K)] 68246K->49454K(95296K), 0.0204423 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T22:26:12.495+0800: [GC [PSYoungGen: 24576K->5920K(27072K)] 69486K->52054K(95360K), 0.0248438 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:12.639+0800: [GC [PSYoungGen: 25952K->5792K(27136K)] 72086K->53262K(95424K), 0.0250760 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:12.785+0800: [GC [PSYoungGen: 25888K->5952K(27072K)] 73358K->54622K(95360K), 0.0272174 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:12.929+0800: [GC [PSYoungGen: 26048K->5952K(27136K)] 74718K->55830K(95424K), 0.0292977 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:13.075+0800: [GC [PSYoungGen: 26112K->5632K(27136K)] 75990K->57014K(95424K), 0.0277379 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:13.222+0800: [GC [PSYoungGen: 25792K->5952K(27264K)] 77174K->58526K(95552K), 0.0275775 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:13.370+0800: [GC [PSYoungGen: 26368K->6048K(27264K)] 78942K->59734K(95552K), 0.0235658 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
9210
2014-02-23T22:26:13.539+0800: [GC [PSYoungGen: 26464K->5920K(27328K)] 80150K->60814K(95616K), 0.0256710 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:13.843+0800: [GC [PSYoungGen: 26400K->6784K(27264K)] 81294K->61974K(95552K), 0.0285569 secs] [Times: user=0.05 sys=0.00, real=0.03 secs]
2014-02-23T22:26:13.921+0800: [GC [PSYoungGen: 27264K->6848K(25856K)] 82454K->62310K(94144K), 0.0308090 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:13.995+0800: [GC [PSYoungGen: 25856K->7136K(26176K)] 81318K->62598K(94464K), 0.0285591 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:14.070+0800: [GC [PSYoungGen: 26144K->7136K(25920K)] 81606K->62598K(94208K), 0.0227667 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:14.137+0800: [GC [PSYoungGen: 25248K->7072K(25216K)] 80710K->62590K(93504K), 0.0281336 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:14.207+0800: [GC [PSYoungGen: 25184K->7168K(25728K)] 80702K->62694K(94016K), 0.0288409 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:14.277+0800: [GC [PSYoungGen: 24768K->4992K(22592K)] 80294K->62662K(90880K), 0.0238574 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T22:26:14.343+0800: [GC [PSYoungGen: 22592K->7136K(25536K)] 80262K->64806K(93824K), 0.0224849 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:14.405+0800: [GC [PSYoungGen: 24352K->6944K(25664K)] 82022K->64822K(93952K), 0.0239891 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:14.469+0800: [GC [PSYoungGen: 24160K->7136K(25536K)] 82038K->65022K(93824K), 0.0233774 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:14.533+0800: [GC [PSYoungGen: 24224K->5728K(25600K)] 82110K->65046K(93888K), 0.0272266 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:14.601+0800: [GC [PSYoungGen: 22816K->7136K(25536K)] 82134K->66454K(93824K), 0.0327101 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:14.673+0800: [GC [PSYoungGen: 24224K->6816K(25600K)] 83542K->66486K(93888K), 0.0255441 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:14.738+0800: [GC [PSYoungGen: 23904K->7136K(25536K)] 83574K->66806K(93824K), 0.0259766 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:14.806+0800: [GC [PSYoungGen: 24224K->7136K(25600K)] 83894K->66806K(93888K), 0.0304316 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:14.877+0800: [GC [PSYoungGen: 24224K->6752K(25728K)] 83894K->66814K(94016K), 0.0236976 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:14.940+0800: [GC [PSYoungGen: 23968K->7136K(25600K)] 84030K->67198K(93888K), 0.0271693 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:15.006+0800: [GC [PSYoungGen: 24352K->6240K(25792K)] 84414K->67198K(94080K), 0.0256807 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:15.072+0800: [GC [PSYoungGen: 23648K->7136K(25728K)] 84606K->68094K(94016K), 0.0254815 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:15.139+0800: [GC [PSYoungGen: 24544K->7136K(25792K)] 85502K->68094K(94080K), 0.0196313 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:15.198+0800: [GC [PSYoungGen: 24608K->7136K(25792K)] 85566K->68094K(94080K), 0.0249983 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:15.263+0800: [GC [PSYoungGen: 24608K->5664K(25664K)] 85566K->68126K(93952K), 0.0216685 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:15.325+0800: [GC [PSYoungGen: 23008K->7136K(25728K)] 85470K->69598K(94016K), 0.0301654 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:15.395+0800: [GC [PSYoungGen: 24480K->7040K(25792K)] 86942K->69614K(94080K), 0.0228180 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:15.459+0800: [GC [PSYoungGen: 24448K->7136K(25728K)] 87022K->69710K(94016K), 0.0254480 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T22:26:15.523+0800: [GC [PSYoungGen: 24544K->7136K(25792K)] 87118K->69710K(94080K), 0.0315946 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:15.600+0800: [GC [PSYoungGen: 24608K->7136K(25792K)] 87182K->69710K(94080K), 0.0220243 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T22:26:15.663+0800: [GC [PSYoungGen: 24608K->7136K(25856K)] 87182K->69710K(94144K), 0.0284997 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:15.733+0800: [GC [PSYoungGen: 24736K->4416K(22016K)] 87310K->69742K(90304K), 0.0250814 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:15.801+0800: [GC [PSYoungGen: 22016K->7136K(25600K)] 87342K->72470K(93888K), 0.0300655 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:15.873+0800: [GC [PSYoungGen: 24224K->6880K(25600K)] 89558K->72494K(93888K), 0.0267039 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:15.939+0800: [GC [PSYoungGen: 23968K->7136K(25728K)] 89582K->72750K(94016K), 0.0280613 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:16.010+0800: [GC [PSYoungGen: 24480K->6400K(25728K)] 90094K->72782K(94016K), 0.0229103 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T22:26:16.033+0800: [Full GC [PSYoungGen: 6400K->0K(25728K)] [ParOldGen: 66382K->7978K(68288K)] 72782K->7978K(94016K) [PSPermGen: 4824K->4824K(32768K)], 0.0864607 secs] [Times: user=0.14 sys=0.00, real=0.09 secs]
2014-02-23T22:26:16.164+0800: [GC [PSYoungGen: 17344K->7136K(25792K)] 25322K->15114K(94080K), 0.0299953 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:16.235+0800: [GC [PSYoungGen: 24608K->7136K(25792K)] 32586K->15114K(94080K), 0.0210713 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:16.298+0800: [GC [PSYoungGen: 24608K->6464K(25920K)] 32586K->15138K(94208K), 0.0226901 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]
2014-02-23T22:26:16.361+0800: [GC [PSYoungGen: 24064K->7136K(25792K)] 32738K->15818K(94080K), 0.0319223 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:16.435+0800: [GC [PSYoungGen: 24736K->6688K(26048K)] 33418K->15818K(94336K), 0.0282897 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
2014-02-23T22:26:16.506+0800: [GC [PSYoungGen: 24544K->7136K(25920K)] 33674K->16266K(94208K), 0.0216874 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
2014-02-23T22:26:16.570+0800: [GC [PSYoungGen: 24992K->6720K(26112K)] 34122K->16274K(94400K), 0.0315422 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:16.645+0800: [GC [PSYoungGen: 24768K->7136K(26048K)] 34322K->16698K(94336K), 0.0313446 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
2014-02-23T22:26:16.719+0800: [GC [PSYoungGen: 25184K->6944K(26176K)] 34746K->16682K(94464K), 0.0282411 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]
3249
Heap
PSYoungGen total 26176K, used 12584K [0x00000000fdeb0000, 0x0000000100000000, 0x0000000100000000)
eden space 18176K, 31% used [0x00000000fdeb0000,0x00000000fe4320f0,0x00000000ff070000)
from space 8000K, 86% used [0x00000000ff830000,0x00000000ffef8000,0x0000000100000000)
to space 7936K, 0% used [0x00000000ff070000,0x00000000ff070000,0x00000000ff830000)
ParOldGen total 68288K, used 9738K [0x00000000f9c00000, 0x00000000fdeb0000, 0x00000000fdeb0000)
object space 68288K, 14% used [0x00000000f9c00000,0x00000000fa582900,0x00000000fdeb0000)
PSPermGen total 32768K, used 4834K [0x00000000f7c00000, 0x00000000f9c00000, 0x00000000f9c00000)
object space 32768K, 14% used [0x00000000f7c00000,0x00000000f80b8a00,0x00000000f9c00000)
堆内
2014-02-23T22:28:04.216+0800: [GC [PSYoungGen: 102464K->17016K(119488K)] 102464K->89695K(392576K), 0.3195424 secs] [Times: user=0.58 sys=0.05, real=0.32 secs]
2014-02-23T22:28:04.840+0800: [GC [PSYoungGen: 119480K->16992K(119488K)] 192159K->188484K(392576K), 0.4353862 secs] [Times: user=0.84 sys=0.03, real=0.44 secs]
2014-02-23T22:28:05.276+0800: [Full GC [PSYoungGen: 16992K->0K(119488K)] [ParOldGen: 171492K->181082K(273088K)] 188484K->181082K(392576K) [PSPermGen: 4347K->4345K(32768K)], 3.8585327 secs] [Times: user=7.04 sys=0.02, real=3.86 secs]
5589
2014-02-23T22:28:09.569+0800: [Full GC [PSYoungGen: 102464K->0K(119488K)] [ParOldGen: 181082K->266532K(273088K)] 283546K->266532K(392576K) [PSPermGen: 4345K->4345K(32768K)], 1.8830764 secs] [Times: user=3.56 sys=0.09, real=1.88 secs]
2480
Heap
PSYoungGen total 119488K, used 67992K [0x00000000f7ab0000, 0x0000000100000000, 0x0000000100000000)
eden space 102464K, 66% used [0x00000000f7ab0000,0x00000000fbd161e0,0x00000000fdec0000)
from space 17024K, 0% used [0x00000000fef60000,0x00000000fef60000,0x0000000100000000)
to space 17024K, 0% used [0x00000000fdec0000,0x00000000fdec0000,0x00000000fef60000)
ParOldGen total 273088K, used 266532K [0x00000000e7000000, 0x00000000f7ab0000, 0x00000000f7ab0000)
object space 273088K, 97% used [0x00000000e7000000,0x00000000f7449030,0x00000000f7ab0000)
PSPermGen total 32768K, used 4355K [0x00000000e5000000, 0x00000000e7000000, 0x00000000e7000000)
object space 32768K, 13% used [0x00000000e5000000,0x00000000e5440ff8,0x00000000e7000000)
堆内350M 堆外10M
堆内
2014-02-23T22:28:45.215+0800: [GC [PSYoungGen: 76800K->12792K(89600K)] 76800K->67198K(294400K), 0.2394863 secs] [Times: user=0.47 sys=0.00, real=0.24 secs]
2014-02-23T22:28:45.672+0800: [GC [PSYoungGen: 89592K->12768K(89600K)] 143998K->141778K(294400K), 0.3314830 secs] [Times: user=0.59 sys=0.06, real=0.33 secs]
2014-02-23T22:28:46.004+0800: [Full GC [PSYoungGen: 12768K->0K(89600K)] [ParOldGen: 129010K->137586K(204800K)] 141778K->137586K(294400K) [PSPermGen: 4398K->4396K(32768K)], 2.8380564 secs] [Times: user=5.24 sys=0.02, real=2.84 secs]
2014-02-23T22:28:49.077+0800: [Full GC [PSYoungGen: 76800K->1403K(89600K)] [ParOldGen: 137586K->204798K(204800K)] 214386K->206202K(294400K) [PSPermGen: 4396K->4396K(32768K)], 4.2016372 secs] [Times: user=7.85 sys=0.02, real=4.20 secs]
8559
2014-02-23T22:28:53.632+0800: [Full GC [PSYoungGen: 76800K->61891K(89600K)] [ParOldGen: 204798K->204798K(204800K)] 281598K->266689K(294400K) [PSPermGen: 4397K->4397K(32768K)], 1.2589946 secs] [Times: user=2.50 sys=0.00, real=1.26 secs]
2014-02-23T22:28:55.030+0800: [Full GC [PSYoungGen: 76800K->61891K(89600K)] [ParOldGen: 204798K->204798K(204800K)] 281598K->266689K(294400K) [PSPermGen: 4397K->4397K(32768K)], 1.2247279 secs] [Times: user=2.43 sys=0.00, real=1.23 secs]
2014-02-23T22:28:56.375+0800: [Full GC [PSYoungGen: 76800K->61891K(89600K)] [ParOldGen: 204798K->204798K(204800K)] 281598K->266689K(294400K) [PSPermGen: 4397K->4397K(32768K)], 1.2589935 secs] [Times: user=2.48 sys=0.00, real=1.26 secs]
2014-02-23T22:28:57.709+0800: [Full GC [PSYoungGen: 76800K->61891K(89600K)] [ParOldGen: 204798K->204798K(204800K)] 281598K->266689K(294400K) [PSPermGen: 4397K->4397K(32768K)], 1.2410986 secs] [Times: user=2.45 sys=0.00, real=1.24 secs]
2014-02-23T22:28:59.021+0800: [Full GC [PSYoungGen: 76800K->61891K(89600K)] [ParOldGen: 204798K->204798K(204800K)] 281598K->266689K(294400K) [PSPermGen: 4397K->4397K(32768K)], 2.0274452 secs] [Times: user=3.95 sys=0.00, real=2.03 secs]
7610
运行配置
-Xms300M -Xmx300M -XX:PermSize=32m -XX:MaxDirectMemorySize=10M -XX:MaxPermSize=32m -Xss1m -XX:+UseParallelGC -XX:ParallelGCThreads=2 -XX:+HeapDumpOnOutOfMemoryError -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+DisableExplicitGC
package com.mycompany.app; import java.util.Map; import java.util.Set; import java.util.concurrent.BlockingQueue; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentNavigableMap; import java.util.concurrent.LinkedBlockingQueue; import org.mapdb.DB; import org.mapdb.DBMaker; import junit.framework.TestCase; public class MapDBTest extends TestCase { public void testTemp() { ConcurrentNavigableMap treeMap = DBMaker.newTempTreeMap(); // and now use disk based Map as any other Map for (int i = 0; i < 10; i++) { treeMap.put(i, "value"+i); } treeMap.put(5, "5"); System.out.println(treeMap.get(5)); treeMap.remove(5); } public void testOffHeapSet(){ for (int i = 0; i <10000; i++) { } DB db=DBMaker.newDirectMemoryDB().asyncWriteFlushDelay(200).transactionDisable().make(); Set<String> set=db.getHashSet("longji"); //Map<String, String> map=new ConcurrentHashMap<>(); long start=System.currentTimeMillis(); for (int i = 0; i < 1000000; i++) { set.add(String.valueOf(i)); } System.out.println(System.currentTimeMillis()-start); start=System.currentTimeMillis(); for (int i = 0; i < 1000000; i++) { set.contains(String.valueOf(i)); } System.out.println(System.currentTimeMillis()-start); } public void testOffHeapMap(){ for (int i = 0; i <10000; i++) { } DB db=DBMaker.newDirectMemoryDB().asyncWriteFlushDelay(200).transactionDisable().make(); Map<String, ItemDO> map=db.getHashMap("longji"); //Map<String, ItemDO> map=new ConcurrentHashMap<>(); long start=System.currentTimeMillis(); for (int i = 0; i < 1500000; i++) { map.put(String.valueOf(i),new ItemDO(String.valueOf(i))); //System.out.println(""); } System.out.println(System.currentTimeMillis()-start); start=System.currentTimeMillis(); for (int i = 0; i < 1500000; i++) { map.get(String.valueOf(i)); } System.out.println(System.currentTimeMillis()-start); } public void testOffHeapQueue() throws InterruptedException{ for (int i = 0; i <10000; i++) { } DB db=DBMaker.newDirectMemoryDB().transactionDisable().make(); BlockingQueue<ItemDO> queue=db.getQueue("longji"); //BlockingQueue<ItemDO> queue=new LinkedBlockingQueue<ItemDO>(); long start=System.currentTimeMillis(); for (int i = 0; i < 1000000; i++) { queue.add(new ItemDO(String.valueOf(i))); } System.out.println(System.currentTimeMillis()-start); start=System.currentTimeMillis(); for (int i = 0; i < 1000000; i++) { queue.poll(); } System.out.println(System.currentTimeMillis()-start); } }