转载请注明出处哈:http://carlosfu.iteye.com/blog/2237511
更多BigMemory Go可参考官方文档：
一、BigMemory证书：
(1). 由于BigMemory是商业版，需要从官网上注册、下载证书（目前只支持90天）和对应版本的jar包：
下载地址：http://terracotta.org/downloads/bigmemorygo，完成表单填写，完成下载。
(2). 可以将证书放到classpath下，或者加入启动参数：
-Dcom.tc.productkey.path=/path/to/terracotta-license.key
(3). 证书示例：
其中:
二、依赖
1. BigMemory依赖
<bigmemory.version>4.0.5</bigmemory.version>
<ehcache-ee.version>2.7.5</ehcache-ee.version>
<dependency>
<groupId>org.terracotta.bigmemory</groupId>
<artifactId>bigmemory</artifactId>
<version>${bigmemory.version}</version>
</dependency>
<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache-ee</artifactId>
<version>${ehcache-ee.version}</version>
</dependency>
2. 序列化工具protostuff依赖
<protostuff.version>1.0.8</protostuff.version>
<dependency>
<groupId>com.dyuproject.protostuff</groupId>
<artifactId>protostuff-runtime</artifactId>
<version>${protostuff.version}</version>
</dependency>
<dependency>
<groupId>com.dyuproject.protostuff</groupId>
<artifactId>protostuff-core</artifactId>
<version>${protostuff.version}</version>
</dependency>
3. logback依赖（logback实现了slf4j-api）
<logback.version>1.0.13</logback.version>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>${logback.version}</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>${logback.version}</version>
</dependency>
4. 引入junit
<junit.version>4.11</junit.version>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
</dependency>
5.最终pom配置
<properties>
<bigmemory.version>4.0.5</bigmemory.version>
<ehcache-ee.version>2.7.5</ehcache-ee.version>
<protostuff.version>1.0.8</protostuff.version>
<logback.version>1.0.13</logback.version>
<junit.version>4.11</junit.version>
</properties>
<dependencies>
<dependency>
<groupId>org.terracotta.bigmemory</groupId>
<artifactId>bigmemory</artifactId>
<version>${bigmemory.version}</version>
</dependency>
<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache-ee</artifactId>
<version>${ehcache-ee.version}</version>
</dependency>
<dependency>
<groupId>com.dyuproject.protostuff</groupId>
<artifactId>protostuff-runtime</artifactId>
<version>${protostuff.version}</version>
</dependency>
<dependency>
<groupId>com.dyuproject.protostuff</groupId>
<artifactId>protostuff-core</artifactId>
<version>${protostuff.version}</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>${logback.version}</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>${logback.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
<encoding>UTF-8</encoding>
</configuration>
</plugin>
</plugins>
</build>
三、BigMemory配置
BigMemory的配置一般也叫ehcache.xml， 放到classpath下：我们使用极简配置
<?xml version="1.0" encoding="UTF-8" ?>
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd">
<cache name="firstOffHeapCache" maxBytesLocalHeap="32M" maxBytesLocalOffHeap="1024M"/>
</ehcache>
Bigmemory的配置相对于Ehcache添加了两个属性：
maxBytesLocalOffHeap： 最大堆外内存
maxBytesLocalHeap：堆外内存在堆内内存的热点数据最大值
maxBytesLocalHeap：堆外内存在堆内内存的热点数据最大值
四、logback配置
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="5 seconds">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="STDOUT"/>
</root>
</configuration>
五、BigMemory测试：
1. 一个实体类：
package com.sohu.tv.bigmemory.first;
import java.util.Date;
/**
* 俱乐部
*
* @author leifu
* @Date 2015年7月28日
* @Time 下午1:43:53
*/
public class Club {
/**
* 俱乐部id
*/
private int id;
/**
* 俱乐部名
*/
private String clubName;
/**
* 俱乐部描述
*/
private String clubInfo;
/**
* 创建日期
*/
private Date createDate;
/**
* 排名
*/
private int rank;
public Club(int id, String clubName, String clubInfo, Date createDate, int rank) {
super();
this.id = id;
this.clubName = clubName;
this.clubInfo = clubInfo;
this.createDate = createDate;
this.rank = rank;
}
public Club() {
super();
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getClubName() {
return clubName;
}
public void setClubName(String clubName) {
this.clubName = clubName;
}
public String getClubInfo() {
return clubInfo;
}
public void setClubInfo(String clubInfo) {
this.clubInfo = clubInfo;
}
public Date getCreateDate() {
return createDate;
}
public void setCreateDate(Date createDate) {
this.createDate = createDate;
}
public int getRank() {
return rank;
}
public void setRank(int rank) {
this.rank = rank;
}
@Override
public String toString() {
return "Club [id=" + id + ", clubName=" + clubName + ", clubInfo=" + clubInfo + ", createDate=" + createDate
+ ", rank=" + rank + "]";
}
}
2. BigMemory测试（Ehcache单元测试作为测试）
(1) 单元测试：
package com.sohu.tv.bigmemory.first;
import java.util.Date;
import org.junit.BeforeClass;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import net.sf.ehcache.Cache;
import net.sf.ehcache.CacheManager;
import net.sf.ehcache.Element;
/**
* 第一个BigMemory测试
*
* @author leifu
* @Date 2015年8月12日
* @Time 上午10:14:13
*/
public class FirstBigmemoryTest {
private static Logger logger = LoggerFactory.getLogger(FirstBigmemoryTest.class);
private static Cache cache;
private static final String BIGMEORY_CACHE_NAME = "firstOffHeapCache";
@BeforeClass
public static void setUp() {
CacheManager cacheManager = CacheManager.create(FirstBigmemoryTest.class.getClassLoader()
.getResourceAsStream("ehcache.xml"));
// 打印cacheManager管理的cache
String[] cacheNameArr = cacheManager.getCacheNames();
for (String cacheName : cacheNameArr) {
logger.info("cacheName: {}", cacheName);
}
cache = cacheManager.getCache(BIGMEORY_CACHE_NAME);
}
@Test
public void testCRUD() {
logger.info("At start, bigmemory object size: {}", cache.getSize());
// 唯一key
String key = "football:club:1";
Club club = new Club(1, "AC", "AC米兰", new Date(), 1);
// 增
Element element = new Element(key, club);
cache.put(element);
logger.info("after add bigmemory object size: {}", cache.getSize());
// 查
Element elementResult = cache.get(key);
Club clubResult = (Club) elementResult.getObjectValue();
logger.info("get key {} value is {}", key, clubResult.toString());
// 修改
club.setRank(8888);
cache.put(element);
logger.info("after set bigmemory object size: {}", cache.getSize());
// 再查
Element elementResultAgain = cache.get(key);
Club clubResultAgain = (Club) elementResultAgain.getObjectValue();
logger.info("get key {} again value is {}", key, clubResultAgain.toString());
// 删
boolean removeResult = cache.remove(key);
logger.info("remove result is {}, after remove bigmemory object size: {}", removeResult, cache.getSize());
// 增加一条，观察下次启动
cache.put(element);
logger.info("At final, bigmemory object size: " + cache.getSize());
// 生产环境不要使用，影响性能
logger.info("At final, bigmemory memory size: " + cache.calculateInMemorySize());
}
}
(2) 输出报错：Club必须序列化
(3) 修复：Club实现序列化接口
public class Club implements Serializable
(4) 输出结果正常：
输出信息包含证书的验证、BigMemory堆外内存的分配
11:04:00.965 [main] INFO o.t.license.ehcache.LicenseManager - Terracotta license loaded from resource /terracotta-license.key
Date of Issue: 2015-08-19
Expiration Date: 2015-11-17
License Type: Trial
License Number: 1009994
Licensee: deren zhang, it
Email: [email protected]
Product: BigMemory Go
Edition: DX
Capabilities: ehcache, TMC, ehcache offheap
Max Client Count: 0
ehcache.maxOffHeap: 32G
11:04:01.287 [main] INFO n.s.e.p.s.f.AnnotationSizeOfFilter - Using regular expression provided through VM argument net.sf.ehcache.pool.sizeof.ignore.pattern for IgnoreSizeOf annotation : ^.*cache\..*IgnoreSizeOf$
11:04:01.296 [main] INFO n.sf.ehcache.pool.sizeof.AgentLoader - Located valid 'tools.jar' at 'C:\Program Files\Java\jdk1.7.0_60\jre\..\lib\tools.jar'
11:04:01.358 [main] INFO n.s.e.pool.sizeof.JvmInformation - Detected JVM data model settings of: 64-Bit HotSpot JVM with Compressed OOPs
11:04:01.430 [main] INFO n.sf.ehcache.pool.sizeof.AgentLoader - Extracted agent jar to temporary file C:\Users\leifu\AppData\Local\Temp\ehcache-sizeof-agent1571240522467793322.jar
11:04:01.430 [main] INFO n.sf.ehcache.pool.sizeof.AgentLoader - Trying to load agent @ C:\Users\leifu\AppData\Local\Temp\ehcache-sizeof-agent1571240522467793322.jar
11:04:01.438 [main] INFO n.s.e.pool.impl.DefaultSizeOfEngine - using Agent sizeof engine
11:04:01.510 [main] INFO n.s.e.s.offheap.OffHeapStoreFactory - Creating Off-Heap Area Using Config
Heuristic Configuration: firstOffHeapCache
Maximum Size (specified) : 2GB
Minimum Chunk Size : 128MB
Maximum Chunk Size : 1GB
Concurrency : 64
Initial Segment Table Size : 1K slots
Segment Data Page Size : 1MB
11:04:01.515 [main] INFO c.t.o.p.UpfrontAllocatingPageSource - Allocating 2GB in chunks
11:04:02.215 [main] INFO c.t.o.p.UpfrontAllocatingPageSource - Allocated 2GB in 1GB chunks.
11:04:02.215 [main] INFO c.t.o.p.UpfrontAllocatingPageSource - Took 699 ms to create CacheManager off-heap storage of 2GB.
11:04:02.304 [main] INFO c.s.t.b.first.FirstBigmemoryTest - cacheName: firstOffHeapCache
11:04:02.307 [main] INFO c.s.t.b.first.FirstBigmemoryTest - At start, bigmemory object size: 0
11:04:02.343 [main] INFO c.s.t.b.first.FirstBigmemoryTest - after add bigmemory object size: 1
11:04:02.346 [main] INFO c.s.t.b.first.FirstBigmemoryTest - get key football:club:1 value is Club [id=1, clubName=AC, clubInfo=AC米兰, createDate=Sat Aug 22 11:04:02 CST 2015, rank=1]
11:04:02.396 [main] INFO c.s.t.b.first.FirstBigmemoryTest - after set bigmemory object size: 1
11:04:02.396 [main] INFO c.s.t.b.first.FirstBigmemoryTest - get key football:club:1 again value is Club [id=1, clubName=AC, clubInfo=AC米兰, createDate=Sat Aug 22 11:04:02 CST 2015, rank=8888]
11:04:02.397 [main] INFO c.s.t.b.first.FirstBigmemoryTest - remove result is true, after remove bigmemory object size: 0
11:04:02.398 [main] INFO c.s.t.b.first.FirstBigmemoryTest - At final, bigmemory object size: 1
11:04:02.398 [main] INFO c.s.t.b.first.FirstBigmemoryTest - At final, bigmemory memory size: 784
Date of Issue: 2015-08-19
Expiration Date: 2015-11-17
License Type: Trial
License Number: 1009994
Licensee: deren zhang, it
Email: [email protected]
Product: BigMemory Go
Edition: DX
Capabilities: ehcache, TMC, ehcache offheap
Max Client Count: 0
ehcache.maxOffHeap: 32G
11:04:01.287 [main] INFO n.s.e.p.s.f.AnnotationSizeOfFilter - Using regular expression provided through VM argument net.sf.ehcache.pool.sizeof.ignore.pattern for IgnoreSizeOf annotation : ^.*cache\..*IgnoreSizeOf$
11:04:01.296 [main] INFO n.sf.ehcache.pool.sizeof.AgentLoader - Located valid 'tools.jar' at 'C:\Program Files\Java\jdk1.7.0_60\jre\..\lib\tools.jar'
11:04:01.358 [main] INFO n.s.e.pool.sizeof.JvmInformation - Detected JVM data model settings of: 64-Bit HotSpot JVM with Compressed OOPs
11:04:01.430 [main] INFO n.sf.ehcache.pool.sizeof.AgentLoader - Extracted agent jar to temporary file C:\Users\leifu\AppData\Local\Temp\ehcache-sizeof-agent1571240522467793322.jar
11:04:01.430 [main] INFO n.sf.ehcache.pool.sizeof.AgentLoader - Trying to load agent @ C:\Users\leifu\AppData\Local\Temp\ehcache-sizeof-agent1571240522467793322.jar
11:04:01.438 [main] INFO n.s.e.pool.impl.DefaultSizeOfEngine - using Agent sizeof engine
11:04:01.510 [main] INFO n.s.e.s.offheap.OffHeapStoreFactory - Creating Off-Heap Area Using Config
Heuristic Configuration: firstOffHeapCache
Maximum Size (specified) : 2GB
Minimum Chunk Size : 128MB
Maximum Chunk Size : 1GB
Concurrency : 64
Initial Segment Table Size : 1K slots
Segment Data Page Size : 1MB
11:04:01.515 [main] INFO c.t.o.p.UpfrontAllocatingPageSource - Allocating 2GB in chunks
11:04:02.215 [main] INFO c.t.o.p.UpfrontAllocatingPageSource - Allocated 2GB in 1GB chunks.
11:04:02.215 [main] INFO c.t.o.p.UpfrontAllocatingPageSource - Took 699 ms to create CacheManager off-heap storage of 2GB.
11:04:02.304 [main] INFO c.s.t.b.first.FirstBigmemoryTest - cacheName: firstOffHeapCache
11:04:02.307 [main] INFO c.s.t.b.first.FirstBigmemoryTest - At start, bigmemory object size: 0
11:04:02.343 [main] INFO c.s.t.b.first.FirstBigmemoryTest - after add bigmemory object size: 1
11:04:02.346 [main] INFO c.s.t.b.first.FirstBigmemoryTest - get key football:club:1 value is Club [id=1, clubName=AC, clubInfo=AC米兰, createDate=Sat Aug 22 11:04:02 CST 2015, rank=1]
11:04:02.396 [main] INFO c.s.t.b.first.FirstBigmemoryTest - after set bigmemory object size: 1
11:04:02.396 [main] INFO c.s.t.b.first.FirstBigmemoryTest - get key football:club:1 again value is Club [id=1, clubName=AC, clubInfo=AC米兰, createDate=Sat Aug 22 11:04:02 CST 2015, rank=8888]
11:04:02.397 [main] INFO c.s.t.b.first.FirstBigmemoryTest - remove result is true, after remove bigmemory object size: 0
11:04:02.398 [main] INFO c.s.t.b.first.FirstBigmemoryTest - At final, bigmemory object size: 1
11:04:02.398 [main] INFO c.s.t.b.first.FirstBigmemoryTest - At final, bigmemory memory size: 784
3. BigMemory测试结论：
(1) 堆外内存存储的数据需要序列化。
(2) BigMemory使用Ehcache ee可以自动完成序列化，如果使用BigMemory存储java对象，就会存在SizeOf问题（简单对象除外），所以可以使用预序列化方式序列化要存的数据。
具体参考：BigMemory关于sizeOf引擎的测试.
(3) BigMemory启动时候，会寻找BigMemory证书。
11:04:00.965 [main] INFO o.t.license.ehcache.LicenseManager - Terracotta license loaded from resource /terracotta-license.key
(4) 以下是bigMemory启动时打印的DirectMemory分区概述，
Maximum Size (specified) : 2GB
Minimum Chunk Size : 128MB
Maximum Chunk Size : 1GB
Concurrency : 64
Initial Segment Table Size : 1K slots
Segment Data Page Size : 1MB
Minimum Chunk Size : 128MB
Maximum Chunk Size : 1GB
Concurrency : 64
Initial Segment Table Size : 1K slots
Segment Data Page Size : 1MB
根据日志,可以猜测出BigMemory预先将数据空间划分为一系列Chunk,目的为了防止内存碎片化,与Memcache内存分配策略很像.
六、BigMemory堆外内存，JVM启动参数：
-XX:MaxDirectMemorySize=4G
七、BigMemory更多使用示例：
http://www.terracotta.org/documentation/4.0/bigmemorygo/code-samples
八、使用ProtoStuff序列化：
1. ProtoStuff序列化工具：
package com.sohu.tv.serializer;
import com.dyuproject.protostuff.LinkedBuffer;
import com.dyuproject.protostuff.ProtostuffIOUtil;
import com.dyuproject.protostuff.Schema;
import com.dyuproject.protostuff.runtime.RuntimeSchema;
import java.util.concurrent.ConcurrentHashMap;
/**
* protostuff序列化工具
*
* @author leifu
* @Date 2015-8-22
* @Time 上午10:05:20
*/
public class ProtostuffSerializer {
private static ConcurrentHashMap<Class<?>, Schema<?>> cachedSchema = new ConcurrentHashMap<Class<?>, Schema<?>>();
public <T> byte[] serialize(final T source) {
VO<T> vo = new VO<T>(source);
final LinkedBuffer buffer = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
try {
final Schema<VO> schema = getSchema(VO.class);
return serializeInternal(vo, schema, buffer);
} catch (final Exception e) {
throw new IllegalStateException(e.getMessage(), e);
} finally {
buffer.clear();
}
}
public <T> T deserialize(final byte[] bytes) {
try {
Schema<VO> schema = getSchema(VO.class);
VO vo = deserializeInternal(bytes, schema.newMessage(), schema);
if (vo != null && vo.getValue() != null) {
return (T) vo.getValue();
}
} catch (final Exception e) {
throw new IllegalStateException(e.getMessage(), e);
}
return null;
}
private <T> byte[] serializeInternal(final T source, final Schema<T> schema, final LinkedBuffer buffer) {
return ProtostuffIOUtil.toByteArray(source, schema, buffer);
}
private <T> T deserializeInternal(final byte[] bytes, final T result, final Schema<T> schema) {
ProtostuffIOUtil.mergeFrom(bytes, result, schema);
return result;
}
private static <T> Schema<T> getSchema(Class<T> clazz) {
@SuppressWarnings("unchecked")
Schema<T> schema = (Schema<T>) cachedSchema.get(clazz);
if (schema == null) {
schema = RuntimeSchema.createFrom(clazz);
cachedSchema.put(clazz, schema);
}
return schema;
}
}
package com.sohu.tv.serializer;
import java.io.Serializable;
/**
* @author leifu
* @Date 2015-8-22
* @Time 上午10:05:44
* @param <T>
*/
public class VO<T> implements Serializable {
private T value;
public VO(T value) {
this.value = value;
}
public VO() {
}
public T getValue() {
return value;
}
@Override
public String toString() {
return "VO{" +
"value=" + value +
'}';
}
}
2. ProtoStuff + BigMemory单元测试：
@Test
public void testBigMemoryWithSerializable() {
ProtostuffSerializer protostuffSerializer = new ProtostuffSerializer();
// 唯一key
String key = "football:club:1";
byte[] clubBytes = protostuffSerializer.serialize(new Club(1, "AC", "AC米兰", new Date(), 1));
// 增
Element element = new Element(key, clubBytes);
cache.put(element);
// 查
Element elementResult = cache.get(key);
byte[] clubBytesResult = (byte[]) elementResult.getObjectValue();
Club clubResult = protostuffSerializer.deserialize(clubBytesResult);
logger.info("get key {} value is {}", key, clubResult.toString());
}
输出：