微服务缓存漫谈之Guava Cache

原理

Cache 仿佛一直以来都是提高性能, 特别是I/O密集型服务提高性能的法宝

参见下面一组数字, 比较不同媒介的存取速度: 缓存 > 内存 > 网络 > 磁盘
(来自 Google Fellow Jeff Dean 的 PPT )

• 每个人都应知道的一组数字 :

存储媒介 storage	性能 performance
L1 cache reference	0.5 ns
Branch mispredict	5 nsv
L2 cache reference	7 ns
Mutex lock/unlock	100 ns (25)
Main memory reference	100 ns
Compress 1K bytes with Zippy	10,000 ns (3,000)
Send 2K bytes over 1 Gbps network	20,000 ns
Read 1 MB sequentially from memory	250,000 ns
Round trip within same datacenter	500,000 ns
Disk seek	10,000,000 ns
Read 1 MB sequentially from network	10,000,000 ns
Read 1 MB sequentially from disk	30,000,000 ns (20,000,000)
Send packet CA->Netherlands->CA	150,000,000 ns

从上面的数据, 我们知道数据缓存在 CPU cache 和内存中存取是最快的, 磁盘最慢, 顺序为

1. 本地内存中的缓存数据
2. 本地磁盘中的数据
3. 远程快速节点内存中的缓存数据
4. 远程快速节点磁盘中的数据
5. 远程慢速节点内存中的缓存数据
6. 远程慢速节点磁盘中的数据

第2, 第3取决于网络和磁盘的存取速度, 顺序可能会对调

远程节点分两种：
一种是在低延迟网络中的远程节点，或者说同一个局域网或城域网中的比较近的节点，又或虽然相隔距离远但是通过专用高速网络互连的节点，RTT一般在50ms 以内

一种是在高延迟网络中的远程节点，比如跨地区，跨网络，RTT大于100ms的网络，有时候，同一个城市不同的网络运营商之前的网络速度可能比不同城市同一个运营商之前速度更慢

Cache 的种类

• 本地 Cache: 放在本地内存中的缓存, 典型代表就是内存中的 map
• 远程 Cache: 放在远程服务器内存中的缓存, 比如 memcached, redis cache
• 分布式 Cache: 它其实有两种, 一种全部是远程分布式 cache , 还有一种是本地和远程cache 有同步机制, 存取都是在本地内存中

其实缓存无处不在, CPU 有L1/L2 缓存, 硬盘有读写缓存, 这里仅提及微服务常用的缓存

对于Cache 我们最关心的就是 Cache 的使用效率, 也就是命中率, 提高命中率的关键因素是

1. Cache key size 缓存的键值数量
2. Cache capacity 缓存的容量
3. Cache lifetime 缓存的生命周期

Cache 不可能无限增长, 不可能永远有效, 所以对于 Cache 的清除策略和失效策略要细细考量.
对于放在 Cache 中的数据也最好是读写比较高的, 即读得多, 写得少, 不会频繁地更新.

Cache的常用操作通常有

get，put，remove，clear

对于Cluster Cache来说，读操作（get）肯定是Local方法，只需要从本台计算机内存中获取数据。
Remove/clear两个写操作，肯定是Remote方法，需要和Cluster其他计算机进行同步。
Put这个写方法，可以是Local，也可以是Remote的。 Remote Put方法的场景是这样，一台计算机把数据放到Cache里面，这个数据就会被传播到Cluster其他计算机上。这个做法的好处是Cluster各台计算机的Cache数据可以及时得到补充，坏处是传播的数据量比较大，这个代价比较大 Local Put方法的场景是这样，一台计算机把数据放到Cache里面，这个数据不会被传播到Cluster其他计算机上。这个做法的好处是不需要传播数据，坏处是Cluster各台计算机的Cache数据不能及时得到补充，这个不是很明显的问题，从Cache中得不到数据，从数据库获取数据是很正常的现象。 Local Put比起Remote Put的优势很明显，所以，通常的Cluster Cache都采用Local Put的策略。各Cache一般都提供了Local Put的配置选项，如果你没有看到这个支持，那么请换一个Cache。

让我们看看下面几种不同的 cache 实现

• Guava Cache
• EhCache
• Memcached
• Redis

1. Guava

最简单的cache 实现莫过于在内存中维护一张 map 了, 按 key 检索或存储 , 不过内存毕竟有限, 要自己实现 Cache 和各种清除和失效策略, Guava Cache 是不错的选择, 可以很方便地设置最大容量, 清除和失效策略等等.

它的主要特性有:

• 自动加载数据项到缓存中
• 当最大限度到达时应用最近最少策略驱逐数据项
• 从上次的访问或修改时间开始计算的数据项过期
• 数据项的键自动封装于弱引用中
• 数据项的值自动封装于弱引用或软引用中
• 当数据项被驱逐或清除时通知
• 聚合缓存访问统计

我们来举一个天气预报数据的缓存例子, 天气预报是那种比较适合缓存的数据, 它在一定的时间范围内变化不大, 读写比很高.

• HelloCacheConfig 构建 Weather Cache 和相关依赖

package com.github.walterfan.hello;

import com.codahale.metrics.MetricRegistry;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.github.walterfan.dto.CityWeather;
import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import com.google.common.collect.ImmutableList;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.EnvironmentAware;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
import org.springframework.context.annotation.Lazy;
import org.springframework.core.env.Environment;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;
import org.springframework.web.client.RestTemplate;

import java.util.List;
import java.util.concurrent.TimeUnit;

/**
 * Created by yafan on 14/10/2017.
 */
@EnableAspectJAutoProxy
@ComponentScan
@Configuration
public class HelloCacheConfig  {//implements EnvironmentAware

    @Autowired
    private Environment environment;

    @Bean
    public HelloCacheLoader helloCacheLoader() {
        return new HelloCacheLoader();
    }

    @Bean
    public RestTemplate restTemplate() {
        RestTemplate restTemplate = new RestTemplate();

        List> converters = restTemplate.getMessageConverters();
        for (HttpMessageConverter converter : converters) {
            if (converter instanceof MappingJackson2HttpMessageConverter) {
                MappingJackson2HttpMessageConverter jsonConverter = (MappingJackson2HttpMessageConverter) converter;
                jsonConverter.setObjectMapper(new ObjectMapper());
                jsonConverter.setSupportedMediaTypes(ImmutableList.of(
                        new MediaType("application", "json", MappingJackson2HttpMessageConverter.DEFAULT_CHARSET),
                        new MediaType("text", "javascript", MappingJackson2HttpMessageConverter.DEFAULT_CHARSET)));
            }
        }
        return restTemplate;
    }

    @Bean
    public String appToken() {
        return this.environment.getProperty("ak");
    }


    @Bean
    public LoadingCache cityWeatherCache() {
        return CacheBuilder.newBuilder()
                .maximumSize(1000)
                .expireAfterWrite(60, TimeUnit.MINUTES)
                .build(helloCacheLoader());
    }

    @Bean
    public DurationTimerAspect durationTimerAspect() {
        return new DurationTimerAspect();
    }

    @Bean
    @Lazy
    public MetricRegistry metricRegistry() {
        return new MetricRegistry();
    }
}

• HelloCacheLoader 从远程的 Baidu API 获取天气预报数据放入缓存中
  同时把 main 函数放在这里, 也就是构建 Spring Application Context, 并从cache 中获取天气预报数据
  第一次花的时间比较长, 用了352 毫秒, 之后从cache 中获取数据, 都只了几十到几百微秒
  注意对于百度天气API, 你需要自己申请一个apptoken, 我申请了一个, 5000次/天的调用都免费的, 由于我用了一小时的缓存, 所以无需付出任何费用

package com.github.walterfan.hello;

import com.codahale.metrics.Histogram;
import com.codahale.metrics.MetricRegistry;
import com.codahale.metrics.Snapshot;
import com.github.walterfan.config.HelloConfig;
import com.github.walterfan.dto.CityWeather;
import com.google.common.base.Stopwatch;
import com.google.common.cache.Cache;
import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.util.UriComponentsBuilder;

import java.net.URL;
import java.util.HashMap;
import java.util.Map;
import java.util.SortedMap;
import java.util.concurrent.ExecutionException;

import static java.util.concurrent.TimeUnit.MILLISECONDS;

/**
 * Created by yafan on 11/10/2017.
 *
 * refer to api http://lbsyun.baidu.com/index.php?title=car/api/weather
 */


@Component
public class HelloCacheLoader extends CacheLoader {

    private static Logger logger = LoggerFactory.getLogger(HelloCacheLoader.class);

    @Autowired
    private RestTemplate restTemplate;

    @Autowired
    private String appToken;

    @Autowired
    private LoadingCache cityWeatherCache;

    @Autowired
    private MetricRegistry metricRegistry;

    @DurationTimer(name="getCityWeather")
    public CityWeather getCityWeather(String city) throws ExecutionException {
        return this.cityWeatherCache.get(city);
    }


    @Override
    public CityWeather load(String city) throws Exception {

        String url = "http://api.map.baidu.com/telematics/v3/weather";
        UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(url)
                .queryParam("location", city)
                .queryParam("output", "json")
                .queryParam("ak", appToken);

        ResponseEntity resp = restTemplate.getForEntity(builder.toUriString(), CityWeather.class);

        logger.debug("response status: " + resp.getStatusCode());
        return resp.getBody();
    }

    public static void main(String[] args) throws ExecutionException {
        try (AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(HelloCacheConfig.class)) {
            HelloCacheLoader helloCacheLoder = (HelloCacheLoader) context.getBean("helloCacheLoader");

            CityWeather cityWeather = helloCacheLoder.getCityWeather("hefei");
            for(int i=0;i<10;++i) {
                cityWeather = helloCacheLoder.getCityWeather("hefei");
            }

            logger.info("----- weather -----");
            logger.info(cityWeather.toString());

            MetricRegistry metricRegistry = (MetricRegistry) context.getBean("metricRegistry");


            SortedMap histograms =  metricRegistry.getHistograms();

            for(Map.Entry entry: histograms.entrySet()) {
                Snapshot snapshot = entry.getValue().getSnapshot();
                logger.info("{}: size={},values: {}",  entry.getKey(), snapshot.size(), snapshot.getValues());
                logger.info(" max={}, min={}, mean={}, median={}",
                        snapshot.getMax(), snapshot.getMin(), snapshot.getMean(), snapshot.getMedian());
            }
        }
    }


}

• output

13:02:03.273 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 352404 MICROSECONDS, threshold: 0 MICROSECONDS
13:02:03.274 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 251 MICROSECONDS, threshold: 0 MICROSECONDS
13:02:03.274 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 127 MICROSECONDS, threshold: 0 MICROSECONDS
13:02:03.274 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 87 MICROSECONDS, threshold: 0 MICROSECONDS
...
13:02:03.309 [main] INFO com.github.walterfan.hello.HelloCacheLoader - getCityWeather: size=11, count=[34, 36, 40, 45, 60, 63, 85, 87, 127, 251, 352404],values: {}
13:02:03.309 [main] INFO com.github.walterfan.hello.HelloCacheLoader -  max=352404, min=34, mean=32112.0, median=63.0

这里顺手写了两个 Metrics 相关的辅助类, 利用 AOP 和 Metrics 来记录调用时间

• class DurationTimer

ackage com.github.walterfan.hello;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.util.concurrent.TimeUnit;

/**
 * Created by yafan on 15/10/2017.
 */
@Target({ElementType.METHOD})
@Retention(value = RetentionPolicy.RUNTIME)
@Documented
public @interface DurationTimer {

    String name() default "";

    long logThreshold() default 0;

    //default timeunit μs
    TimeUnit thresholdTimeUnit() default TimeUnit.MICROSECONDS;
}

• class DurationTimerAspect

package com.github.walterfan.hello;

import com.codahale.metrics.MetricRegistry;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.aop.aspectj.annotation.AspectJProxyFactory;
import org.springframework.beans.factory.annotation.Autowired;

import java.util.concurrent.TimeUnit;

/**
 * Created by yafan on 15/10/2017.
 */
@Aspect
public class DurationTimerAspect {


    private final Logger logger = LoggerFactory.getLogger(getClass());

    @Autowired
    private MetricRegistry metricRegistry;

    public  T proxy(T o) {
        final AspectJProxyFactory factory = new AspectJProxyFactory(o);
        factory.setProxyTargetClass(true);
        factory.addAspect(this);
        return factory.getProxy();
    }

    @Around("@annotation( durationAnnotation ) ")
    public Object measureTimeRequest(final ProceedingJoinPoint pjp, DurationTimer durationAnnotation) throws Throwable {
        final long start = System.nanoTime();
        final Object retVal = pjp.proceed();

        String timerName = durationAnnotation.name();
        if("".equals(timerName)) {
            timerName = pjp.getSignature().toShortString();
        }
        TimeUnit timeUnit = durationAnnotation.thresholdTimeUnit();
        long threshold = durationAnnotation.logThreshold();
        //System.out.println("timerName=" + timerName);
        try {
            long difference = timeUnit.convert(System.nanoTime() - start, TimeUnit.NANOSECONDS);

            if(difference > threshold) {
                metricRegistry.histogram(timerName).update(difference);
                logger.info("Duration of {}: {} {}, threshold: {} {}", timerName, difference, timeUnit.name(), threshold, timeUnit.name());
            }

        } catch (Exception ex) {
            logger.error("Cannot measure api timing.... :" + ex.getMessage(), ex);
        }
        return retVal;
    }

}

相关的 DTO 如下

package com.github.walterfan.dto;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;

import java.util.Date;
import java.util.List;

/**
 * Created by yafan on 14/10/2017.
 */
@JsonIgnoreProperties(ignoreUnknown = true)
public class CityWeather  {
    private int error;

    private String status;

    private Date date;

    private List results;

    public int getError() {
        return error;
    }

    public void setError(int error) {
        this.error = error;
    }

    public String getStatus() {
        return status;
    }

    public void setStatus(String status) {
        this.status = status;
    }

    public Date getDate() {
        return date;
    }

    public void setDate(Date date) {
        this.date = date;
    }

    public List getResults() {
        return results;
    }

    public void setResults(List results) {
        this.results = results;
    }

    @Override
    public String toString() {
        try {
            return new ObjectMapper().writerWithDefaultPrettyPrinter().writeValueAsString(this);
        } catch (JsonProcessingException e) {
            e.printStackTrace();
        }
        return null;
    }
}


//---------------------
package com.github.walterfan.dto;

/**
 * Created by yafan on 14/10/2017.
 */
public class WeatherData {

    private String date;
    private String dayPictureUrl;
    private String nightPictureUrl;
    private String weather;
    private String wind;
    private String temperature;

    public String getDate() {
        return date;
    }

    public void setDate(String date) {
        this.date = date;
    }

    public String getDayPictureUrl() {
        return dayPictureUrl;
    }

    public void setDayPictureUrl(String dayPictureUrl) {
        this.dayPictureUrl = dayPictureUrl;
    }

    public String getNightPictureUrl() {
        return nightPictureUrl;
    }

    public void setNightPictureUrl(String nightPictureUrl) {
        this.nightPictureUrl = nightPictureUrl;
    }

    public String getWeather() {
        return weather;
    }

    public void setWeather(String weather) {
        this.weather = weather;
    }

    public String getWind() {
        return wind;
    }

    public void setWind(String wind) {
        this.wind = wind;
    }

    public String getTemperature() {
        return temperature;
    }

    public void setTemperature(String temperature) {
        this.temperature = temperature;
    }
}
//------------------
package com.github.walterfan.dto;

/**
 * Created by yafan on 14/10/2017.
 */
public class WeatherIndex {
    private String title;
    private String zs;
    private String tipt;
    private String des;

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }

    public String getZs() {
        return zs;
    }

    public void setZs(String zs) {
        this.zs = zs;
    }

    public String getTipt() {
        return tipt;
    }

    public void setTipt(String tipt) {
        this.tipt = tipt;
    }

    public String getDes() {
        return des;
    }

    public void setDes(String des) {
        this.des = des;
    }
}
//---------------------
package com.github.walterfan.dto;

import java.util.List;

/**
 * Created by yafan on 14/10/2017.
 */
public class WeatherResult {
    private String currentCity;
    private String pm25;

    private List index;

    private List weather_data;

    public String getCurrentCity() {
        return currentCity;
    }

    public void setCurrentCity(String currentCity) {
        this.currentCity = currentCity;
    }

    public String getPm25() {
        return pm25;
    }

    public void setPm25(String pm25) {
        this.pm25 = pm25;
    }

    public List getIndex() {
        return index;
    }

    public void setIndex(List index) {
        this.index = index;
    }

    public List getWeather_data() {
        return weather_data;
    }

    public void setWeather_data(List weather_data) {
        this.weather_data = weather_data;
    }
}

参考资料

• https://github.com/google/guava/wiki/CachesExplained