性能测试工具 nGrinder 项目剖析及二次开发

0.背景

组内需要一款轻量级的性能测试工具，之前考虑过LR（太笨重，单实例，当然它的地位是不容置疑的），阿里云的PTS(https://pts.aliyun.com/lite/index.htm， 仅支持阿里云内网和公网机器)，Gatling（http://gatling.io/#/，无TPS数据）等等，不适合我们；

nGrinderr是NAVER（韩国最大互联网公司NHN旗下搜索引擎网站）开源的性能测试工具，直接部署成web服务，支持多用户使用，可扩展性好，可自定义plugin（http://www.cubrid.org/wiki_ngrinder/entry/how-to-develop-plugin），wiki文档较丰富(http://www.cubrid.org/wiki_ngrinder/entry/ngrinder-devzone)，数据及图形化展示满足需求；但是展示的统计数据较简单，二次开发整合数据：TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数字段，并将这些数据展示在详细测试报告页中。

1.项目剖析

1-1. nGrinder架构

nGrinder是一款在一系列机器上执行Groovy或JPython测试脚本的应用，内部引擎是基于Grinder。
架构图：

层级图：

默认的NGRINDER_HOME为/root/.ngrinder， 大多是配置文件和数据文件。

目录/root/.ngrinder/perftest/0_999下，以每个test_id为名的文件夹对应的存储了执行性能测试时的采样数据：

*.data文件就是执行性能测试时对应的各种性能采样数据，性能测试详细报告页就是根据这些data文件，进行图形化展示（ajax）。

nGrinder包含2大组件：
1）Controller
为性能测试提供web interface
协同测试进程
收集和显示测试数据
新建和修改脚本

2）Agent
agent mode: 运行进程和线程，压测目标服务
monitor mode: 监控目标系统性能(cpu/memory)， 可以自定义收集的数据(比如 jvm数据)

http://www.cubrid.org/wiki_ngrinder/entry/general-architecture

1-2. 技术栈

1)Controller 层
FreeMarker: 基于Java的模板引擎
Spring Security
Spring Mvc：Spring MVC provides rich functionality for building robust web applications.
GSon
SVNKit Dav

2)Service 层
Grinder
Spring
EhCache: Ehcache has excellent Spring integration.

3)Data层
Spring Data
Hibernate:Hibernate is a powerful technology for persisting data,and it is Spring Data back-end within nGrinder.
H2: (nGrinder默认使用该DB)
Cubrid:(nGrinder同一家公司的DB)
Liquidase: Liquibase is an open source that automates database schema updates.
SVNKit

http://www.cubrid.org/wiki_ngrinder/entry/technology-stack

2.源码实现

需求：在详细测试报告页中展示TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数这些数据。

用到了工具包commons-math3的方法，在工程中添加相应的依赖：

<dependency>
    <groupId>org.apache.commonsgroupId>
    <artifactId>commons-math3artifactId>
    <version>3.6.1version>
dependency>

修改Controller层，增加数据处理业务逻辑(计算TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数)

在获取采样数据
ngrinder-core/src/main/java/net/grinder/SingleConsole.java中新增处理业务逻辑，核心修改代码片段：

    // tps list
    List tps = new CopyOnWriteArrayList();
    // rt list
    List meanTestTime = new CopyOnWriteArrayList();

    /**
     * 
     * 每次请求调用一次 Build up statistics for current sampling.
     *
     * @param accumulatedStatistics
     *            intervalStatistics
     * @param intervalStatistics
     *            accumulatedStatistics
     */
    protected void updateStatistics(StatisticsSet intervalStatistics,
            StatisticsSet accumulatedStatistics) {
        Map result = newHashMap();
        result.put("testTime", getCurrentRunningTime() / 1000);
        List> cumulativeStatistics = new ArrayList>();
        List> lastSampleStatistics = new ArrayList>();

        for (Test test : accumulatedStatisticMapPerTest.keySet()) {
            Map accumulatedStatisticMap = newHashMap();
            Map intervalStatisticsMap = newHashMap();
            StatisticsSet accumulatedSet = this.accumulatedStatisticMapPerTest
                    .get(test);
            StatisticsSet intervalSet = this.intervalStatisticMapPerTest
                    .get(test);

            accumulatedStatisticMap.put("testNumber", test.getNumber());
            accumulatedStatisticMap.put("testDescription",
                    test.getDescription());
            intervalStatisticsMap.put("testNumber", test.getNumber());
            intervalStatisticsMap.put("testDescription", test.getDescription());
            // When only 1 test is running, it's better to use the parametrized
            // snapshot.
            for (Entry each : getExpressionEntrySet()) {
                if (INTERESTING_STATISTICS.contains(each.getKey())) {
                    accumulatedStatisticMap.put(
                            each.getKey(),
                            getRealDoubleValue(each.getValue().getDoubleValue(
                                    accumulatedSet)));
                    intervalStatisticsMap.put(
                            each.getKey(),
                            getRealDoubleValue(each.getValue().getDoubleValue(
                                    intervalSet)));
                }
            }
            cumulativeStatistics.add(accumulatedStatisticMap);
            lastSampleStatistics.add(intervalStatisticsMap);
        }

        Map totalStatistics = newHashMap();

        for (Entry each : getExpressionEntrySet()) {
            if (INTERESTING_STATISTICS.contains(each.getKey())) {
                totalStatistics.put(each.getKey(), getRealDoubleValue(each
                        .getValue().getDoubleValue(accumulatedStatistics)));
            }
        }

        LOGGER.debug("hugang start get plug data");

        // 获取tps, rt集合
        for (Entry each : getExpressionEntrySet()) {
            if ("TPS".equals(each.getKey())) {
                tps.add((Double) getRealDoubleValue(each.getValue()
                        .getDoubleValue(intervalStatistics)));
            } else if ("Mean_Test_Time_(ms)".equals(each.getKey())) {
                meanTestTime.add((Double) getRealDoubleValue(each.getValue()
                        .getDoubleValue(intervalStatistics)));
            }
        }


        result.put("totalStatistics", totalStatistics);
        result.put("cumulativeStatistics", cumulativeStatistics);
        result.put("lastSampleStatistics", lastSampleStatistics);
        result.put("tpsChartData", getTpsValues());
        result.put("peakTpsForGraph", this.peakTpsForGraph);
        synchronized (this) {
            result.put(GrinderConstants.P_PROCESS, this.runningProcess);
            result.put(GrinderConstants.P_THREAD, this.runningThread);
            result.put("success", !isAllTestFinished());
        }
        // Finally overwrite.. current one.
        this.statisticData = result;
    }

    /**
     * 从updateStatistics()累加数据, list :rt 和 tps, 为成员变量
     * 
     * 再处理集合，放到statisticData中
     * 
     * @author hugang
     */
    public void getPlusResult(){

        LOGGER.debug("hugang getPlusResult() tpslist {}  rtlist is {}",
                tps.toString(), meanTestTime.toString());

        int i = 0;
        int j = 0;
        // list转成数组， 标准库使用数组作为参数
        double[] tpsArray = new double[tps.size()];
        for (double tpsNum : tps) {
            tpsArray[i++] = tpsNum;
        }

        // list转成数组
        double[] meanTestTimeArray = new double[meanTestTime.size()];
        for (double meanTime : meanTestTime) {
            meanTestTimeArray[j++] = meanTime;
        }

        // tps 标准差
        double tpsStd = new StandardDeviation().evaluate(tpsArray);
        // tps 平均值
        double tpsMean = new Mean().evaluate(tpsArray, 0, tpsArray.length);
        // tps 波动率= tps 标准差 / tps 平均值
        double tpsVix = 0;
        if(0 != tpsMean){
            tpsVix = tpsStd / tpsMean;
        }

        // meanTestTime 百分位数
        Percentile percentile = new Percentile();
        // 先排序
        Arrays.sort(meanTestTimeArray);
        // meanTestTime最小值
        double minMeanTime = meanTestTimeArray[0];
        double twentyFiveMeanTime = percentile.evaluate(meanTestTimeArray, 25);
        double fiftyMeanTime = percentile.evaluate(meanTestTimeArray, 50);
        double serventyFiveMeanTime = percentile
                .evaluate(meanTestTimeArray, 75);
        double eightyMeanTime = percentile.evaluate(meanTestTimeArray, 80);
        double eightyFiveMeanTime = percentile.evaluate(meanTestTimeArray, 85);
        double ninetyMeanTime = percentile.evaluate(meanTestTimeArray, 90);
        double ninetyFiveMeanTime = percentile.evaluate(meanTestTimeArray, 95);
        double ninetyNineMeanTime = percentile.evaluate(meanTestTimeArray, 99);

        int length = meanTestTimeArray.length;
        // meanTestTime最高值
        double maxMeanTime = meanTestTimeArray[length - 1];
        // meanTestTime平均值
        // double TimeMean = new Mean().evaluate(meanTestTimeArray, 0,
        // meanTestTimeArray.length);

        LOGGER.debug(
                "hugang plug Statistics MinMeanTime {}  MaxMeanTime is {}",
                minMeanTime, maxMeanTime);
        // 附加信息 hugang
        // tps 标准差, tps 波动率， 最小/最大RT, RT百分位数
        Map plusStatistics = newHashMap();
        plusStatistics.put("tpsStd", tpsStd);
//      plusStatistics.put("tpsMean", tpsMean);
        plusStatistics.put("tpsVix", tpsVix);
        plusStatistics.put("minMeanTime", minMeanTime);
        plusStatistics.put("twentyFiveMeanTime", twentyFiveMeanTime);
        plusStatistics.put("fiftyMeanTime", fiftyMeanTime);
        plusStatistics.put("serventyFiveMeanTime", serventyFiveMeanTime);
        plusStatistics.put("eightyMeanTime", eightyMeanTime);
        plusStatistics.put("eightyFiveMeanTime", eightyFiveMeanTime);
        plusStatistics.put("ninetyMeanTime", ninetyMeanTime);
        plusStatistics.put("ninetyFiveMeanTime", ninetyFiveMeanTime);
        plusStatistics.put("ninetyNineMeanTime", ninetyNineMeanTime);
        plusStatistics.put("maxMeanTime", maxMeanTime);


        LOGGER.debug("SingleConsole plug Statistics map plusStatistics {}", plusStatistics);


        this.statisticData.put("plusStatistics", plusStatistics);
    }



    /**
     * 
     * 停止采样数据
     * Stop sampling.
     */
    public void unregisterSampling() {
        this.currentNotFinishedProcessCount = 0;
        if (sampleModel != null) {
            this.sampleModel.reset();
            this.sampleModel.stop();
        }
        LOGGER.info("Sampling is stopped");
        informTestSamplingEnd();

        // 结束采样后，处理数据
        // hugang
        getPlusResult();
    }

Map statisticData为不同数据集集合。

Service层从SingleConsole类中获取数据集statisticData：
ngrinder-controller/src/main/java/org/ngrinder/perftest/server/PerfTestService.java 中Map result = consoleManager.getConsoleUsingPort(perfTest.getPort()).getStatisticsData();

/**
     * Update the given {@link PerfTest} properties after test finished.
     *
     * @param perfTest perfTest
     * 
     * getConsoleUsingPort()获取数据
     *
     * 
     * hugang
     */
    public void updatePerfTestAfterTestFinish(PerfTest perfTest) {
        checkNotNull(perfTest);
        Map result = consoleManager.getConsoleUsingPort(perfTest.getPort()).getStatisticsData();
        @SuppressWarnings("unchecked")
        Map totalStatistics = MapUtils.getMap(result, "totalStatistics", MapUtils.EMPTY_MAP);
        // 获取附加数据
        Map plusStatistics = MapUtils.getMap(result, "plusStatistics", MapUtils.EMPTY_MAP);

        LOGGER.info("Total Statistics for test {}  is {}", perfTest.getId(), totalStatistics);
        LOGGER.info("plug Statistics for test {}  is {}", perfTest.getId(), plusStatistics);

        perfTest.setTps(parseDoubleWithSafety(totalStatistics, "TPS", 0D));
        perfTest.setMeanTestTime(parseDoubleWithSafety(totalStatistics, "Mean_Test_Time_(ms)", 0D));
        perfTest.setPeakTps(parseDoubleWithSafety(totalStatistics, "Peak_TPS", 0D));
        perfTest.setTests(MapUtils.getDouble(totalStatistics, "Tests", 0D).longValue());
        perfTest.setErrors(MapUtils.getDouble(totalStatistics, "Errors", 0D).longValue());


        // 附加信息写到model， 持久化
        perfTest.setTpsStd(parseDoubleWithSafety(plusStatistics, "tpsStd", 0D));
        perfTest.setTpsVix(parseDoubleWithSafety(plusStatistics, "tpsVix", 0D));
        perfTest.setMinRT(parseDoubleWithSafety(plusStatistics, "minMeanTime", 0D));
        perfTest.setTwentyFiveMeanTime(parseDoubleWithSafety(plusStatistics, "twentyFiveMeanTime", 0D));
        perfTest.setFiftyMeanTime(parseDoubleWithSafety(plusStatistics, "fiftyMeanTime", 0D));
        perfTest.setServentyFiveMeanTime(parseDoubleWithSafety(plusStatistics, "serventyFiveMeanTime", 0D));
        perfTest.setEightyMeanTime(parseDoubleWithSafety(plusStatistics, "eightyMeanTime", 0D));
        perfTest.setEightyFiveMeanTime(parseDoubleWithSafety(plusStatistics, "eightyFiveMeanTime", 0D));
        perfTest.setNinetyMeanTime(parseDoubleWithSafety(plusStatistics, "ninetyMeanTime", 0D));
        perfTest.setNinetyFiveMeanTime(parseDoubleWithSafety(plusStatistics, "ninetyFiveMeanTime", 0D));
        perfTest.setNinetyNineMeanTime(parseDoubleWithSafety(plusStatistics, "ninetyNineMeanTime", 0D));
        perfTest.setMaxRT(parseDoubleWithSafety(plusStatistics, "maxMeanTime", 0D));


    }

修改Model层，在javabean中增加TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数, JPA持久化（H2 DB新增TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数字段)

model文件为：ngrinder-core/src/main/java/org/ngrinder/model/PerfTest.java

    /**
     * 新增字段，TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数
     * hugang
     */
    @Expose
    @Column(name = "tpsStd")
    private Double tpsStd;

    @Expose
    @Column(name = "tpsVix")
    private Double tpsVix;


    @Expose
    @Column(name = "minRT")
    private Double minRT;

    @Expose
    @Column(name = "twentyFiveMeanTime")
    private Double twentyFiveMeanTime;

    @Expose
    @Column(name = "fiftyMeanTime")
    private Double fiftyMeanTime;

    @Expose
    @Column(name = "serventyFiveMeanTime")
    private Double serventyFiveMeanTime;

    @Expose
    @Column(name = "eightyMeanTime")
    private Double eightyMeanTime;

    @Expose
    @Column(name = "eightyFiveMeanTime")
    private Double eightyFiveMeanTime;

    @Expose
    @Column(name = "ninetyMeanTime")
    private Double ninetyMeanTime;

    @Expose
    @Column(name = "ninetyFiveMeanTime")
    private Double ninetyFiveMeanTime;

    @Expose
    @Column(name = "ninetyNineMeanTime")
    private Double ninetyNineMeanTime;

    @Expose
    @Column(name = "maxRT")
    private Double maxRT;


    对应的set(), get()

还需修改db change文件(因为系统DB默认使用H2， 只需修改H2对应的xml)，ngrinder-controller/src/main/resources/ngrinder_datachange_logfile/db.changelog_schema_H2.xml

create table PERF_TEST (
            id bigint generated by default as identity unique,
            created_date timestamp,
            last_modified_date timestamp,
            agent_count integer,
            description varchar(2048),
            distribution_path varchar(255),
            duration bigint,
            errors integer,
            finish_time timestamp,
            ignore_sample_count integer,
            init_processes integer,
            init_sleep_time integer,
            last_progress_message varchar(2048),
            mean_test_time double,
            peak_tps double,
            errorRate double,
            tpsStd double,
            tpsVix double,
            minRT double,
            twentyFiveMeanTime double,
            fiftyMeanTime double,
            serventyFiveMeanTime double,
            eightyMeanTime double,
            eightyFiveMeanTime double,
            ninetyMeanTime double,
            ninetyFiveMeanTime double,
            ninetyNineMeanTime double,
            maxRT double,

系统重启加载时，Liquidase会自动更新DB。

修改View层，在详细报告对应的freemarker模板新增TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数字段，前端新增展示这些数据

ngrinder-controller/src/main/webapp/WEB-INF/ftl/perftest/detail_report.ftl

                <#-- hugang -->
                <#-- 新增 错误率，TPS标准差，TPS波动率，最小RT, 最大RT, RT 25/50/75/80/85/90/95/99百分位数 -->
                <tr>
                    <th><@spring.message "perfTest.report.errorRate"/>th>
                    <td>${(test.errors /(test.tests + test.errors))!""}td>
                tr>
                <tr>
                    <th><@spring.message "perfTest.report.tpsStd"/>th>
                    <td>${test.tpsStd!""}td>
                tr>
                <tr>
                    <th><@spring.message "perfTest.report.tpsVix"/>th>
                    <td>${test.tpsVix!""}td>
                tr>
                    <tr>
                    <th><@spring.message "perfTest.report.minRT"/>th>
                    <td>${test.minRT!""}   <code>mscode>td>
                tr>
                    <tr>
                    <th><@spring.message "perfTest.report.TwentyFiveMeanTime"/>th>
                    <td>${test.twentyFiveMeanTime!""}   <code>mscode>td>
                tr>
                    <tr>
                    <th><@spring.message "perfTest.report.FiftyMeanTime"/>th>
                    <td>${test.fiftyMeanTime!""}   <code>mscode>td>
                tr>
                    <tr>
                    <th><@spring.message "perfTest.report.ServentyFiveMeanTime"/>th>
                    <td>${test.serventyFiveMeanTime!""}   <code>mscode>td>
                tr>
                    <tr>
                    <th><@spring.message "perfTest.report.EightyMeanTime"/>th>
                    <td>${test.eightyMeanTime!""}   <code>mscode>td>
                tr>
                tr>
                    <tr>
                    <th><@spring.message "perfTest.report.EightyFiveMeanTime"/>th>
                    <td>${test.eightyFiveMeanTime!""}   <code>mscode>td>
                tr>
                tr>
                    <tr>
                    <th><@spring.message "perfTest.report.NinetyMeanTime"/>th>
                    <td>${test.ninetyMeanTime!""}   <code>mscode>td>
                tr>
                tr>
                    <tr>
                    <th><@spring.message "perfTest.report.NinetyFiveMeanTime"/>th>
                    <td>${test.ninetyFiveMeanTime!""}   <code>mscode>td>
                tr>
                tr>
                    <tr>
                    <th><@spring.message "perfTest.report.NinetyNineMeanTime"/>th>
                    <td>${test.ninetyNineMeanTime!""}   <code>mscode>td>
                tr>
                    tr>
                    <tr>
                    <th><@spring.message "perfTest.report.maxRT"/>th>
                    <td>${test.maxRT!""}   <code>mscode>td>
                tr>

还有个坑，就是从github拉下的代码，源码中pom.xml依赖的jar包不完整，直接打不了包，项目有的依赖的jar 公有maven仓库已经没有了，需要自己从网上找jar包，安装到本地仓库，我归整了下：

http://download.csdn.net/detail/neven7/9443895

直接在ngrinder根路径下执行打包命令：

mvn -Dmaven.test.skip=true clean package

部署生成的war即可。

3.结果展示

在详细报告页新增如下数据结果：