Dubbo + Zipkin + Brave 实现全链路追踪

阅读更多

       这两天看了好几篇帖子，写zipkin与dubbo整合的内容都不全面，忍不住亲自上手码一遍。

       利用zipkin可以对dubbo进行调用链监控，可以查到调用链中的dubbo服务的性能，并且dubbo提供了SPI的接口，能很容易完成并自定义相应的filter去监控dubbo服务。

ZipKin介绍 

Zipkin是一个致力于收集分布式服务的时间数据的分布式跟踪系统。

Zipkin 主要涉及四个组件:collector(数据采集),storage(数据存储),search(数据查询),UI(数据展示)。

github源码地址:https://github.com/openzipkin/zipkin。

Zipkin提供了可插拔数据存储方式:In-Memory,MySql, Cassandra, Elasticsearch

ZipKin部署与运行 （注意需要在linux下运行，JDK1.8）

1、下载zipkin

http://central.maven.org/maven2/io/zipkin/java/zipkin-server/2.11.7/zipkin-server-2.11.7-exec.jar

2、运行zipkin

(1)In-Memory方式 

nohup java -jar zipkin-server-2.11.7-exec.jar &

 

注意:内存存储,zipkin重启后数据会丢失,建议测试环境使用

(2)MySql方式 

目前只与MySQL的5.6-7。它的设计是易于理解,使用简单。但是,当数据量大时,查询很慢。性能不是很好。

创建数据库zipkin 
建表 

CREATE TABLE IF NOT EXISTS zipkin_spans ( 
`trace_id_high` BIGINT NOT NULL DEFAULT 0 COMMENT 'If non zero, this means the trace uses 128 bit traceIds instead of 64 bit', 
`trace_id` BIGINT NOT NULL, 
`id` BIGINT NOT NULL, 
`name` VARCHAR(255) NOT NULL, 
`parent_id` BIGINT, 
`debug` BIT(1), 
`start_ts` BIGINT COMMENT 'Span.timestamp(): epoch micros used for endTs query and to implement TTL', 
`duration` BIGINT COMMENT 'Span.duration(): micros used for minDuration and maxDuration query' 
) ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci; 
ALTER TABLE zipkin_spans ADD UNIQUE KEY(`trace_id_high`, `trace_id`, `id`) COMMENT 'ignore insert on duplicate'; 
ALTER TABLE zipkin_spans ADD INDEX(`trace_id_high`, `trace_id`, `id`) COMMENT 'for joining with zipkin_annotations'; 
ALTER TABLE zipkin_spans ADD INDEX(`trace_id_high`, `trace_id`) COMMENT 'for getTracesByIds'; 
ALTER TABLE zipkin_spans ADD INDEX(`name`) COMMENT 'for getTraces and getSpanNames'; 
ALTER TABLE zipkin_spans ADD INDEX(`start_ts`) COMMENT 'for getTraces ordering and range'; 
CREATE TABLE IF NOT EXISTS zipkin_annotations ( 
`trace_id_high` BIGINT NOT NULL DEFAULT 0 COMMENT 'If non zero, this means the trace uses 128 bit traceIds instead of 64 bit', 
`trace_id` BIGINT NOT NULL COMMENT 'coincides with zipkin_spans.trace_id', 
`span_id` BIGINT NOT NULL COMMENT 'coincides with zipkin_spans.id', 
`a_key` VARCHAR(255) NOT NULL COMMENT 'BinaryAnnotation.key or Annotation.value if type == -1', 
`a_value` BLOB COMMENT 'BinaryAnnotation.value(), which must be smaller than 64KB', 
`a_type` INT NOT NULL COMMENT 'BinaryAnnotation.type() or -1 if Annotation', 
`a_timestamp` BIGINT COMMENT 'Used to implement TTL; Annotation.timestamp or zipkin_spans.timestamp', 
`endpoint_ipv4` INT COMMENT 'Null when Binary/Annotation.endpoint is null', 
`endpoint_ipv6` BINARY(16) COMMENT 'Null when Binary/Annotation.endpoint is null, or no IPv6 address', 
`endpoint_port` SMALLINT COMMENT 'Null when Binary/Annotation.endpoint is null', 
`endpoint_service_name` VARCHAR(255) COMMENT 'Null when Binary/Annotation.endpoint is null' 
) ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci; 
ALTER TABLE zipkin_annotations ADD UNIQUE KEY(`trace_id_high`, `trace_id`, `span_id`, `a_key`, `a_timestamp`) COMMENT 'Ignore insert on duplicate'; 
ALTER TABLE zipkin_annotations ADD INDEX(`trace_id_high`, `trace_id`, `span_id`) COMMENT 'for joining with zipkin_spans'; 
ALTER TABLE zipkin_annotations ADD INDEX(`trace_id_high`, `trace_id`) COMMENT 'for getTraces/ByIds'; 
ALTER TABLE zipkin_annotations ADD INDEX(`endpoint_service_name`) COMMENT 'for getTraces and getServiceNames'; 
ALTER TABLE zipkin_annotations ADD INDEX(`a_type`) COMMENT 'for getTraces'; 
ALTER TABLE zipkin_annotations ADD INDEX(`a_key`) COMMENT 'for getTraces'; 
ALTER TABLE zipkin_annotations ADD INDEX(`trace_id`, `span_id`, `a_key`) COMMENT 'for dependencies job'; 
CREATE TABLE IF NOT EXISTS zipkin_dependencies ( 
`day` DATE NOT NULL, 
`parent` VARCHAR(255) NOT NULL, 
`child` VARCHAR(255) NOT NULL, 
`call_count` BIGINT 
) ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci; 
ALTER TABLE zipkin_dependencies ADD UNIQUE KEY(`day`, `parent`, `child`); 

 

启动zipkin命令 

 

STORAGE_TYPE=mysql MYSQL_HOST=IP MYSQL_TCP_PORT=3306 MYSQL_DB=zipkin MYSQL_USER=username MYSQL_PASS=password nohup java -jar zipkin-server-2.11.7-exec.jar &

 

(3)Elasticsearch方式 

创建elasticsearch用户,安装启动Elasticsearch服务 
官方文档:https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html 
zipkin启动命令

STORAGE_TYPE=elasticsearch ES_HOSTS=http://IP:9200 nohup java -jar zipkin-server-2.11.7-exec.jar &

 启动成功访问地址：http://192.168.20.15:9411/zipkin/（192.168.20.15替换为对应zipkin部署服务器地址）

 效果图如下：

整合

　　简单的描述一下，同ServletFilter一样，在dubbo中利用Filter过滤请求，传递TraceId等参数，生成相应的span传递给zipkin服务器。

　　引入brave-instrumentation-dubbo-rpc包，这是一个SPI的Filter包。

      
	    io.zipkin.reporter2
	    zipkin-sender-okhttp3
	    2.7.10
      

      
             io.zipkin.brave
             brave-instrumentation-dubbo-rpc
             5.4.3
       

 brave-instrumentation-dubbo-rpc包里面有一个TracingFilter的类，在其invoke方法中实现了对span的一些操作。

 

@Override
public Result invoke(Invoker invoker, Invocation invocation) throws RpcException {
    if (tracer == null) return invoker.invoke(invocation);

    RpcContext rpcContext = RpcContext.getContext();
    Kind kind = rpcContext.isProviderSide() ? Kind.SERVER : Kind.CLIENT;
    final Span span;
    if (kind.equals(Kind.CLIENT)) {
      span = tracer.nextSpan();
      injector.inject(span.context(), invocation.getAttachments());
    } else {
      TraceContextOrSamplingFlags extracted = extractor.extract(invocation.getAttachments());
      span = extracted.context() != null
          ? tracer.joinSpan(extracted.context())
          : tracer.nextSpan(extracted);
    }

    if (!span.isNoop()) {
      span.kind(kind);
      String service = invoker.getInterface().getSimpleName();
      String method = RpcUtils.getMethodName(invocation);
      span.name(service + "/" + method);
      parseRemoteAddress(rpcContext, span);
      span.start();
    }

    boolean isOneway = false, deferFinish = false;
    try (Tracer.SpanInScope scope = tracer.withSpanInScope(span)) {
      Result result = invoker.invoke(invocation);
      if (result.hasException()) {
        onError(result.getException(), span);
      }
      isOneway = RpcUtils.isOneway(invoker.getUrl(), invocation);
      Future