对于分布式应用来说,不可避免地要与系统周边的服务打交道,这个时候,需要对外部的服务调用进行监控,这里我们利用hystrix进行监控,并将其整合到dropwizard的metrics,方便统一输出到statsd。
引入hystrix-codahale-metrics-publisher
com.netflix.hystrix
hystrix-codahale-metrics-publisher
1.5.9
配置
@Bean
HystrixMetricsPublisher hystrixMetricsPublisher() {
HystrixCodaHaleMetricsPublisher publisher = new HystrixCodaHaleMetricsPublisher(metricRegistry);
HystrixPlugins.getInstance().registerMetricsPublisher(publisher);
return publisher;
}
指标实例
{
"HystrixDemoController.hello.countBadRequests": 0,
"HystrixDemoController.hello.countFallbackFailure": 0,
"HystrixDemoController.hello.rollingCountTimeout": 2,
"HystrixDemoController.hello.rollingCountFailure": 0,
"HystrixThreadPool.HystrixDemoController.rollingMaxActiveThreads": 1,
"HystrixDemoController.hello.latencyExecute_percentile_25": 0,
"HystrixDemoController.hello.currentTime": 1491203675091,
"HystrixDemoController.hello.rollingCountResponsesFromCache": 0,
"HystrixThreadPool.HystrixDemoController.completedTaskCount": 26,
"HystrixDemoController.hello.rollingCountFallbackMissing": 0,
"HystrixThreadPool.HystrixDemoController.currentTime": 1491203675091,
"HystrixDemoController.hello.latencyTotal_percentile_25": 0,
"HystrixDemoController.hello.rollingCountFallbackSuccess": 8191,
"HystrixThreadPool.HystrixDemoController.totalTaskCount": 26,
"HystrixDemoController.hello.rollingCountSemaphoreRejected": 0,
"HystrixDemoController.hello.countCollapsedRequests": 0,
"HystrixDemoController.hello.latencyExecute_percentile_995": 0,
"HystrixDemoController.hello.latencyTotal_percentile_50": 0,
"HystrixDemoController.hello.propertyValue_executionIsolationSemaphoreMaxConcurrentRequests": 10,
"HystrixDemoController.hello.propertyValue_circuitBreakerErrorThresholdPercentage": 50,
"HystrixDemoController.hello.latencyTotal_percentile_90": 0,
"HystrixDemoController.hello.countFallbackEmit": 0,
"HystrixDemoController.hello.latencyTotal_percentile_995": 0,
"HystrixDemoController.hello.countTimeout": 2,
"HystrixThreadPool.HystrixDemoController.propertyValue_queueSizeRejectionThreshold": 5,
"HystrixDemoController.hello.countSuccess": 6,
"HystrixDemoController.hello.latencyTotal_percentile_75": 0,
"HystrixThreadPool.HystrixDemoController.propertyValue_maximumSize": 10,
"HystrixDemoController.hello.propertyValue_fallbackIsolationSemaphoreMaxConcurrentRequests": 10,
"HystrixDemoController.hello.countEmit": 0,
"HystrixDemoController.hello.executionSemaphorePermitsInUse": 0,
"HystrixDemoController.hello.countFallbackSuccess": 8191,
"HystrixThreadPool.HystrixDemoController.largestPoolSize": 10,
"HystrixDemoController.hello.propertyValue_circuitBreakerSleepWindowInMilliseconds": 5000,
"HystrixDemoController.hello.propertyValue_circuitBreakerRequestVolumeThreshold": 20,
"HystrixDemoController.hello.latencyExecute_percentile_75": 0,
"HystrixDemoController.hello.latencyExecute_percentile_90": 0,
"HystrixDemoController.hello.latencyExecute_percentile_50": 0,
"HystrixDemoController.hello.propertyValue_executionTimeoutInMilliseconds": 100,
"HystrixDemoController.hello.latencyTotal_percentile_5": 0,
"HystrixDemoController.hello.rollingCountFallbackFailure": 0,
"HystrixDemoController.hello.latencyTotal_mean": 0,
"HystrixDemoController.hello.propertyValue_rollingStatisticalWindowInMilliseconds": 10000,
"HystrixThreadPool.HystrixDemoController.propertyValue_keepAliveTimeInMinutes": 1,
"HystrixDemoController.hello.rollingCountFallbackRejection": 198,
"HystrixDemoController.hello.countThreadPoolRejected": 0,
"HystrixDemoController.hello.countFallbackMissing": 0,
"HystrixDemoController.hello.rollingCountBadRequests": 0,
"HystrixDemoController.hello.countResponsesFromCache": 0,
"HystrixDemoController.hello.rollingCountSuccess": 6,
"HystrixThreadPool.HystrixDemoController.queueSize": 0,
"HystrixThreadPool.HystrixDemoController.threadActiveCount": 0,
"HystrixDemoController.hello.countShortCircuited": 8387,
"HystrixDemoController.hello.rollingCountExceptionsThrown": 198,
"HystrixDemoController.hello.rollingCountFallbackEmit": 0,
"HystrixDemoController.hello.rollingCountThreadPoolRejected": 0,
"HystrixDemoController.hello.rollingCountShortCircuited": 8387,
"HystrixThreadPool.HystrixDemoController.propertyValue_corePoolSize": 10,
"HystrixDemoController.hello.latencyExecute_mean": 0,
"HystrixDemoController.hello.countExceptionsThrown": 198,
"HystrixDemoController.hello.propertyValue_executionIsolationThreadTimeoutInMilliseconds": 100,
"HystrixThreadPool.HystrixDemoController.propertyValue_actualMaximumSize": 10,
"HystrixDemoController.hello.latencyExecute_percentile_5": 0,
"HystrixDemoController.hello.latencyTotal_percentile_99": 0,
"HystrixDemoController.hello.errorPercentage": 25,
"HystrixDemoController.hello.countFallbackRejection": 198,
"HystrixThreadPool.HystrixDemoController.rollingCountThreadsExecuted": 6,
"HystrixThreadPool.HystrixDemoController.propertyValue_maxQueueSize": -1,
"HystrixDemoController.hello.latencyExecute_percentile_99": 0,
"HystrixDemoController.hello.rollingCountEmit": 0,
"HystrixDemoController.hello.rollingCountCollapsedRequests": 0,
"HystrixThreadPool.HystrixDemoController.countThreadsExecuted": 7,
"HystrixDemoController.hello.countFailure": 0,
"HystrixThreadPool.HystrixDemoController.rollingCountCommandsRejected": 0,
"HystrixDemoController.hello.countSemaphoreRejected": 0
}
hystrix输出的指标比较多,这个需要注意一下后端存储容量。
hystrix除了输出latencyTotal相关分布的响应时间,还输出了fallback触发的相关数据,以及熔断的次数,带rolling前缀的为瞬时值,不带的一般为累积量。这里讲一下几个指标:
- countShortCircuited为累积量
- rollingCountShortCircuited为瞬时值,报警的时候,以后者为主,该值主要是上报间隔期间方法调用被熔断的次数。
- countFallbackSuccess为fallback成功执行的次数,累计量。