前言
本次分享探讨java平台的本地缓存,是指占用JVM的heap区域来缓冲存储数据的缓存组件。
一、本地缓存应用场景
localcache有着极大的性能优势:
1. 单机情况下适当使用localcache会使应用的性能得到很大的提升。
2. 集群环境下对于敏感性要求不高的数据可以使用localcache,只配置简单的失效机制来保证数据的相对一致性。
哪些数据可以存储到本地缓存?
1.访问频繁的数据;
2.静态基础数据(长时间内不变的数据);
3.相对静态数据(短时间内不变的数据)。
二、java本地缓存标准
Java缓存新标准(javax.cache),这个标准由JSR107所提出,已经被包含在Java EE 7中。
特性: 1.原子操作,跟java.util.ConcurrentMap类似 2.从缓存中读取 3.写入缓存 4.缓存事件监听器 5.数据统计 6.包含所有隔离(ioslation)级别的事务 7.缓存注解(annotations) 8.保存定义key和值类型的泛型缓存 9.引用保存(只适用于堆缓存)和值保存定义
但目前应用不是很普遍。
三、java开源缓存框架
比较有名的本地缓存开源框架有:
1.EHCache
EHCache是一个纯java的在进程中的缓存,它具有以下特性:快速,简单,为Hibernate2.1充当可插入的缓存,最小的依赖性,全面的文档和测试。
BUG: 过期失效的缓存元素无法被GC掉,时间越长缓存越多,内存占用越大,导致内存泄漏的概率越大。
2.OSCache
OSCache有以下特点:缓存任何对象,你可以不受限制的缓存部分jsp页面或HTTP请求,任何java对象都可以缓存。拥有全面的API--OSCache API给你全面的程序来控制所有的OSCache特性。永久缓存--缓存能随意的写入硬盘,因此允许昂贵的创建(expensive-to-create)数据来保持缓存,甚至能让应用重启。支持集群--集群缓存数据能被单个的进行参数配置,不需要修改代码。缓存记录的过期--你可以有最大限度的控制缓存对象的过期,包括可插入式的刷新策略(如果默认性能不需要时)。
3.JCache
Java缓存新标准(javax.cache)
4.cache4j
cache4j是一个有简单API与实现快速的Java对象缓存。它的特性包括:在内存中进行缓存,设计用于多线程环境,两种实现:同步与阻塞,多种缓存清除策略:LFU, LRU, FIFO,可使用强引用。
5.ShiftOne
ShiftOne Java Object Cache是一个执行一系列严格的对象缓存策略的Java lib,就像一个轻量级的配置缓存工作状态的框架。
6.WhirlyCache
Whirlycache是一个快速的、可配置的、存在于内存中的对象的缓存。
四、LocalCache实现
1、LocalCache简介
LocalCache是一个精简版本地缓存组件,有以下特点:
1. 有容量上限maxCapacity; 2. 缓存达到容量上限时基于LRU策略来移除缓存元素; 3. 缓存对象的生命周期(缓存失效时间)由调用方决定; 4. 缓存对象失效后,将会有定时清理线程来清理掉,不会导致内存泄漏。 5. 性能比Ehcache稍强。
2、总体设计
LocalCache总体设计:
1. 缓存元素 CacheElement; 2. 缓存容器 LRULinkedHashMap; 3. 缓存接口 Cache; 4. 缓存组件实现 LocalCache。
3、详细设计
1. CacheElement设计
/**
* 缓存元素
*
*/
public class CacheElement {
private Object key;
private Object value;
private long createTime;
private long lifeTime;
private int hitCount;
public CacheElement() {
}
public CacheElement(Object key ,Object value) {
this.key = key;
this.value = value;
this.createTime = System.currentTimeMillis();
}
public Object getKey() {
return key;
}
public void setKey(Object key) {
this.key = key;
}
public Object getValue() {
hitCount++;
return value;
}
public void setValue(Object value) {
this.value = value;
}
public long getCreateTime() {
return createTime;
}
public void setCreateTime(long createTime) {
this.createTime = createTime;
}
public int getHitCount() {
return hitCount;
}
public void setHitCount(int hitCount) {
this.hitCount = hitCount;
}
public long getLifeTime() {
return lifeTime;
}
public void setLifeTime(long lifeTime) {
this.lifeTime = lifeTime;
}
public boolean isExpired() {
boolean isExpired = System.currentTimeMillis() - getCreateTime() > getLifeTime();
return isExpired;
}
/*
* (non-Javadoc)
* @see java.lang.Object#toString()
*/
public String toString() {
StringBuffer sb = new StringBuffer();
sb.append("[ key=").append(key).append(", isExpired=").append(isExpired())
.append(", lifeTime=").append(lifeTime).append(", createTime=").append(createTime)
.append(", hitCount=").append(hitCount)
.append(", value=").append(value).append(" ]");
return sb.toString();
}
/*
* (non-Javadoc)
* @see java.lang.Object#hashCode()
*/
public final int hashCode(){
if(null == key){
return "".hashCode();
}
return this.key.hashCode();
}
/*
* (non-Javadoc)
* @see java.lang.Object#equals(java.lang.Object)
*/
public final boolean equals(Object object) {
if ((object == null) || (!(object instanceof CacheElement))) {
return false;
}
CacheElement element = (CacheElement) object;
if ((this.key == null) || (element.getKey() == null)) {
return false;
}
return this.key.equals(element.getKey());
}
}
2. LRULinkedHashMap实现
import java.util.LinkedHashMap;
import java.util.Set;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
/**
* 实现 LRU策略的 LinkedHashMap
*
* @param
* @param
*/
public class LRULinkedHashMap extends LinkedHashMap
{
protected static final long serialVersionUID = 2828675280716975892L;
protected static final int DEFAULT_MAX_ENTRIES = 100;
protected final int initialCapacity;
protected final int maxCapacity;
protected boolean enableRemoveEldestEntry = true;//是否允许自动移除比较旧的元素(添加元素时)
protected static final float DEFAULT_LOAD_FACTOR = 0.8f;
protected final Lock lock = new ReentrantLock();
public LRULinkedHashMap(int initialCapacity)
{
this(initialCapacity, DEFAULT_MAX_ENTRIES);
}
public LRULinkedHashMap(int initialCapacity ,int maxCapacity)
{
//set accessOrder=true, LRU
super(initialCapacity, DEFAULT_LOAD_FACTOR, true);
this.initialCapacity = initialCapacity;
this.maxCapacity = maxCapacity;
}
/*
* (non-Javadoc)
* @see java.util.LinkedHashMap#removeEldestEntry(java.util.Map.Entry)
*/
protected boolean removeEldestEntry(java.util.Map.Entry eldest)
{
return enableRemoveEldestEntry && ( size() > maxCapacity );
}
/*
* (non-Javadoc)
* @see java.util.LinkedHashMap#get(java.lang.Object)
*/
public V get(Object key)
{
try {
lock.lock();
return super.get(key);
}
finally {
lock.unlock();
}
}
/*
* (non-Javadoc)
* @see java.util.HashMap#put(java.lang.Object, java.lang.Object)
*/
public V put(K key, V value)
{
try {
lock.lock();
return super.put(key, value);
}
finally {
lock.unlock();
}
}
/*
* (non-Javadoc)
* @see java.util.HashMap#remove(java.lang.Object)
*/
public V remove(Object key) {
try {
lock.lock();
return super.remove(key);
}
finally {
lock.unlock();
}
}
/*
* (non-Javadoc)
* @see java.util.LinkedHashMap#clear()
*/
public void clear() {
try {
lock.lock();
super.clear();
}
finally {
lock.unlock();
}
}
/*
* (non-Javadoc)
* @see java.util.HashMap#keySet()
*/
public Set keySet() {
try {
lock.lock();
return super.keySet();
}
finally {
lock.unlock();
}
}
public boolean isEnableRemoveEldestEntry() {
return enableRemoveEldestEntry;
}
public void setEnableRemoveEldestEntry(boolean enableRemoveEldestEntry) {
this.enableRemoveEldestEntry = enableRemoveEldestEntry;
}
public int getInitialCapacity() {
return initialCapacity;
}
public int getMaxCapacity() {
return maxCapacity;
}
}
3. Cache接口设计
/**
* 缓存接口
*
*/
public interface Cache {
/**
* 获取缓存
* @param key
* @return
*/
public T getCache(Object key);
/**
* 缓存对象
* @param key
* @param value
* @param milliSecond 缓存生命周期(毫秒)
*/
public void putCache(Object key, Object value ,Long milliSecond);
/**
* 缓存容器中是否包含 key
* @param key
* @return
*/
public boolean containsKey(Object key);
/**
* 缓存列表大小
* @return
*/
public int getSize();
/**
* 是否启用缓存
*/
public boolean isEnabled();
/**
* 启用 或 停止
* @param enable
*/
public void setEnabled(boolean enabled);
/**
* 移除所有缓存
*/
public void invalidateCaches();
/**
* 移除 指定key缓存
* @param key
*/
public void invalidateCache(Object key);
}
4. LocalCache实现
import java.util.Date;
import java.util.Iterator;
import java.util.Random;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* 本地缓存组件
*/
public class LocalCache implements Cache{
private Logger logger = LoggerFactory.getLogger(this.getClass());
private LRULinkedHashMap cacheMap;
protected boolean initFlag = false;//初始化标识
protected final long defaultLifeTime = 5 * 60 * 1000;//5分钟
protected boolean warnLongerLifeTime = false;
protected final int DEFAULT_INITIAL_CAPACITY = 100;
protected final int DEFAULT_MAX_CAPACITY = 100000;
protected int initialCapacity = DEFAULT_INITIAL_CAPACITY;//初始化缓存容量
protected int maxCapacity = DEFAULT_MAX_CAPACITY;//最大缓存容量
protected int timeout = 20;//存取缓存操作响应超时时间(毫秒数)
private boolean enabled = true;
private Thread gcThread = null;
private String lastGCInfo = null;//最后一次GC清理信息{ size, removeCount, time ,nowTime}
private boolean logGCDetail = false;//记录gc清理细节
private boolean enableGC = true;//是否允许清理的缓存(添加元素时)
private int gcMode = 0;//清理过期元素模式 { 0=迭代模式 ; 1=随机模式 }
private int gcIntervalTime = 2 * 60 * 1000;//间隔时间(分钟)
private boolean iterateScanAll = true;//是否迭代扫描全部
private float gcFactor = 0.5F;//清理百分比
private int maxIterateSize = DEFAULT_MAX_CAPACITY/2;//迭代模式下一次最大迭代数量
private volatile int iterateLastIndex = 0;//最后迭代下标
private int maxRandomTimes = 100;//随机模式下最大随机次数
protected final static Random random = new Random();
private static LocalCache instance = new LocalCache();
public static LocalCache getInstance() {
return instance;
}
private LocalCache(){
this.init();
}
protected synchronized void init() {
if(initFlag){
logger.warn("init repeat.");
return ;
}
this.initCache();
this.startGCDaemonThread();
initFlag = true;
if(logger.isInfoEnabled()){
logger.info("init -- OK");
}
}
private void startGCDaemonThread(){
if(initFlag){
return ;
}
this.maxIterateSize = maxCapacity /2;
try{
this.gcThread = new Thread() {
public void run() {
logger.info("[" + (Thread.currentThread().getName()) + "]start...");
//sleep
try {
Thread.sleep(getGcIntervalTime() < 30000 ? 30000 : getGcIntervalTime());
} catch (Exception e) {
e.printStackTrace();
}
while( true ){
//gc
gc();
//sleep
try {
Thread.sleep(getGcIntervalTime() < 30000 ? 30000 : getGcIntervalTime());
} catch (Exception e) {
e.printStackTrace();
}
}
}
};
this.gcThread.setName("localCache-gcThread");
this.gcThread.setDaemon(true);
this.gcThread.start();
if(logger.isInfoEnabled()){
logger.info("startGCDaemonThread -- OK");
}
}catch(Exception e){
logger.error("[localCache gc]DaemonThread -- error: " + e.getMessage(), e);
}
}
private void initCache(){
if(initFlag){
return ;
}
initialCapacity = (initialCapacity <= 0 ? DEFAULT_INITIAL_CAPACITY : initialCapacity);
maxCapacity = (maxCapacity < initialCapacity ? DEFAULT_MAX_CAPACITY : maxCapacity);
cacheMap = new LRULinkedHashMap(initialCapacity ,maxCapacity);
if(logger.isInfoEnabled()){
logger.info("initCache -- OK");
}
}
/*
* (non-Javadoc)
*/
@SuppressWarnings("unchecked")
public T getCache(Object key) {
if(!isEnabled()){
return null;
}
long st = System.currentTimeMillis();
T objValue = null;
CacheElement cacheObj = cacheMap.get(key);
if (isExpiredCache(cacheObj)) {
cacheMap.remove(key);
}else {
objValue = (T) (cacheObj == null ? null : cacheObj.getValue());
}
long et = System.currentTimeMillis();
if((et - st)>timeout){
if(this.logger.isWarnEnabled()){
this.logger.warn("getCache_timeout_" + (et - st) + "_[" + key + "]");
}
}
if(logger.isDebugEnabled()){
String message = ("get( " + key + ") return: " + objValue);
logger.debug(message);
}
return objValue;
}
/*
* (non-Javadoc)
*/
public void putCache(Object key, Object value ,Long lifeTime) {
if(!isEnabled()){
return;
}
Long st = System.currentTimeMillis();
lifeTime = (null == lifeTime ? defaultLifeTime : lifeTime);
CacheElement cacheObj = new CacheElement();
cacheObj.setCreateTime(System.currentTimeMillis());
cacheObj.setLifeTime(lifeTime);
cacheObj.setValue(value);
cacheObj.setKey(key);
cacheMap.put(key, cacheObj);
long et = System.currentTimeMillis();
if((et - st)>timeout){
if(this.logger.isWarnEnabled()){
this.logger.warn("putCache_timeout_" + (et - st) + "_[" + key + "]");
}
}
if(logger.isDebugEnabled()){
String message = ("putCache( " + cacheObj + " ) , 耗时 " + (et - st) + "(毫秒).");
logger.debug(message);
}
if(lifeTime > defaultLifeTime && this.isWarnLongerLifeTime()){
if(logger.isWarnEnabled()){
String message = ("LifeTime[" + (lifeTime/1000) + "秒] too long for putCache(" + cacheObj + ")");
logger.warn(message);
}
}
}
/**
* key 是否过期
* @param key
* @return
*/
protected boolean isExpiredKey(Object key) {
CacheElement cacheObj = cacheMap.get(key);
return this.isExpiredCache(cacheObj);
}
/**
* cacheObj 是否过期
* @param key
* @return
*/
protected boolean isExpiredCache(CacheElement cacheObj) {
if (cacheObj == null) {
return false;
}
return cacheObj.isExpired();
}
/*
* (non-Javadoc)
*/
public void invalidateCaches(){
try{
cacheMap.clear();
}catch(Exception e){
e.printStackTrace();
}
}
/*
* (non-Javadoc)
*/
public void invalidateCache(Object key){
try{
cacheMap.remove(key);
}catch(Exception e){
e.printStackTrace();
}
}
/*
* (non-Javadoc)
*/
public boolean containsKey(Object key) {
return cacheMap.containsKey(key);
}
/*
* (non-Javadoc)
*/
public int getSize() {
return cacheMap.size();
}
/*
* (non-Javadoc)
*/
public Iterator getKeyIterator() {
return cacheMap.keySet().iterator();
}
/*
* (non-Javadoc)
*/
public boolean isEnabled() {
return this.enabled;
}
/*
* (non-Javadoc)
*/
public void setEnabled(boolean enabled) {
this.enabled = enabled;
if(!this.enabled){
//清理缓存
this.invalidateCaches();
}
}
/**
* 清理过期缓存
*/
protected synchronized boolean gc(){
if(!isEnableGC()){
return false;
}
try{
iterateRemoveExpiredCache();
}catch(Exception e){
logger.error("gc() has error: " + e.getMessage(), e);
}
return true;
}
/**
* 迭代模式 - 移除过期的 key
* @param exceptKey
*/
private void iterateRemoveExpiredCache(){
long startTime = System.currentTimeMillis();
int size = cacheMap.size();
if(size ==0){
return;
}
int keyCount = 0;
int removedCount = 0 ;
int startIndex = 0;
int endIndex = 0;
try{
Object [] keys = cacheMap.keySet().toArray();
keyCount = keys.length;
int maxIndex = keyCount -1 ;
//初始化扫描下标
if(iterateScanAll){
startIndex = 0;
endIndex = maxIndex;
}else {
int gcThreshold = this.getGcThreshold();
int iterateLen = gcThreshold > this.maxIterateSize ? this.maxIterateSize : gcThreshold;
startIndex = this.iterateLastIndex;
startIndex = ( (startIndex < 0 || startIndex > maxIndex) ? 0 : startIndex );
endIndex = (startIndex + iterateLen);
endIndex = (endIndex > maxIndex ? maxIndex : endIndex);
}
//迭代清理
boolean flag = false;
for(int i=startIndex; i<= endIndex; i++){
flag = this.removeExpiredKey(keys[i]);
if(flag){
removedCount++;
}
}
this.iterateLastIndex = endIndex;
keys = null;
}catch(Exception e){
logger.error("iterateRemoveExpiredCache -- 移除过期的 key时出现异常: " + e.getMessage(), e);
}
long endTime = System.currentTimeMillis();
StringBuffer sb = new StringBuffer();
sb.append("iterateRemoveExpiredCache [ size: ").append(size).append(", keyCount: ").append(keyCount)
.append(", startIndex: ").append(startIndex).append(", endIndex: ").append(iterateLastIndex)
.append(", removedCount: ").append(removedCount).append(", currentSize: ").append(this.cacheMap.size())
.append(", timeConsuming: ").append(endTime - startTime).append(", nowTime: ").append(new Date())
.append(" ]");
this.lastGCInfo = sb.toString();
if(logger.isInfoEnabled()){
logger.info("iterateRemoveExpiredCache -- 清理结果 -- "+ lastGCInfo);
}
}
/**
* 随机模式 - 移除过期的 key
*/
private void randomRemoveExpiredCache(){
long startTime = System.currentTimeMillis();
int size = cacheMap.size();
if(size ==0){
return;
}
int removedCount = 0 ;
try{
Object [] keys = cacheMap.keySet().toArray();
int keyCount = keys.length;
boolean removeFlag = false;
int removeRandomTimes = this.getGcThreshold();
removeRandomTimes = ( removeRandomTimes > this.getMaxRandomTimes() ? this.getMaxRandomTimes() : removeRandomTimes );
while(removeRandomTimes-- > 0){
int index = random.nextInt(keyCount);
boolean flag = this.removeExpiredKey(keys[index]);
if(flag){
removeFlag = true;
removedCount ++;
}
}
//尝试 移除 首尾元素
if(!removeFlag){
this.removeExpiredKey(keys[0]);
this.removeExpiredKey(keys[keyCount-1]);
}
keys=null;
}catch(Exception e){
logger.error("randomRemoveExpiredCache -- 移除过期的 key时出现异常: " + e.getMessage(), e);
}
long endTime = System.currentTimeMillis();
StringBuffer sb = new StringBuffer();
sb.append("randomRemoveExpiredCache [ size: ").append(size).append(", removedCount: ").append(removedCount)
.append(", currentSize: ").append(this.cacheMap.size()).append(", timeConsuming: ").append(endTime - startTime)
.append(", nowTime: ").append(new Date())
.append(" ]");
this.lastGCInfo = sb.toString();
if(logger.isInfoEnabled()){
logger.info("randomRemoveExpiredCache -- 清理结果 -- "+ lastGCInfo);
}
}
private boolean removeExpiredKey(Object key){
boolean flag = false;
CacheElement cacheObj = null;
if(null != key){
try{
cacheObj = cacheMap.get(key);
boolean isExpiredCache = this.isExpiredCache(cacheObj);
if(isExpiredCache){
cacheMap.remove(key);
flag = true;
}
}catch(Exception e){
logger.error("removeExpired(" + key + ") -- error: " + e.getMessage(), e);
}
}
if(!flag && logGCDetail){
this.logger.warn("removeExpiredKey(" + key + ") return [" + flag + "]--" + cacheObj);
}
return flag;
}
public int getInitialCapacity() {
return initialCapacity;
}
public int getMaxCapacity() {
return maxCapacity;
}
public int getGcMode() {
return gcMode;
}
public void setGcMode(int gcMode) {
this.gcMode = gcMode;
}
public int getGcIntervalTime() {
return gcIntervalTime;
}
public void setGcIntervalTime(int gcIntervalTime) {
this.gcIntervalTime = gcIntervalTime;
}
public boolean isEnableGC() {
return enableGC;
}
public void setEnableGC(boolean enableGC) {
this.enableGC = enableGC;
}
public boolean isIterateScanAll() {
return iterateScanAll;
}
public void setIterateScanAll(boolean iterateScanAll) {
this.iterateScanAll = iterateScanAll;
}
public float getGcFactor() {
return gcFactor;
}
public void setGcFactor(float gcFactor) {
this.gcFactor = gcFactor;
}
/**
* gc 阀值
* @return
*/
public int getGcThreshold() {
int threshold = (int)( this.cacheMap.getMaxCapacity() * gcFactor );
return threshold;
}
public String getLastGCInfo() {
return lastGCInfo;
}
public void setLastGCInfo(String lastGCInfo) {
this.lastGCInfo = lastGCInfo;
}
public boolean isLogGCDetail() {
return logGCDetail;
}
public void setLogGCDetail(boolean logGCDetail) {
this.logGCDetail = logGCDetail;
}
public int getTimeout() {
return timeout;
}
public void setTimeout(int timeout) {
this.timeout = timeout;
}
public int getMaxIterateSize() {
return maxIterateSize;
}
public void setMaxIterateSize(int maxIterateSize) {
this.maxIterateSize = maxIterateSize;
}
public int getMaxRandomTimes() {
return maxRandomTimes;
}
public void setMaxRandomTimes(int maxRandomTimes) {
this.maxRandomTimes = maxRandomTimes;
}
public boolean isInitFlag() {
return initFlag;
}
public long getDefaultLifeTime() {
return defaultLifeTime;
}
public boolean isWarnLongerLifeTime() {
return warnLongerLifeTime;
}
public void setWarnLongerLifeTime(boolean warnLongerLifeTime) {
this.warnLongerLifeTime = warnLongerLifeTime;
}
//======================== dynMaxCapacity ========================
private int dynMaxCapacity = maxCapacity;
public int getDynMaxCapacity() {
return dynMaxCapacity;
}
public void setDynMaxCapacity(int dynMaxCapacity) {
this.dynMaxCapacity = dynMaxCapacity;
}
public void resetMaxCapacity(){
if(dynMaxCapacity > initialCapacity && dynMaxCapacity != maxCapacity){
if(logger.isInfoEnabled()){
logger.info("resetMaxCapacity( " + dynMaxCapacity + " ) start...");
}
synchronized(cacheMap){
LRULinkedHashMap cacheMap0 = new LRULinkedHashMap(initialCapacity ,dynMaxCapacity);
cacheMap.clear();
cacheMap = cacheMap0;
this.maxCapacity = dynMaxCapacity;
}
if(logger.isInfoEnabled()){
logger.info("resetMaxCapacity( " + dynMaxCapacity + " ) OK.");
}
}else {
if(logger.isWarnEnabled()){
logger.warn("resetMaxCapacity( " + dynMaxCapacity + " ) NO.");
}
}
}
//======================== showCacheElement ========================
private String showCacheKey;
public String getShowCacheKey() {
return showCacheKey;
}
public void setShowCacheKey(String showCacheKey) {
this.showCacheKey = showCacheKey;
}
public Object showCacheElement(){
Object v = null;
if(null != this.showCacheKey){
v = cacheMap.get(showCacheKey);
}
return v;
}
}
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持脚本之家。