jacoco增量覆盖率实践
Jacoco增量覆盖率说明
能找到这里，说明对jacoco的原理和使用有了一定的了解，而我写这边文章主要是网络上基本没有完整文档加代码的jaocco增量覆盖说明，所以我想分享些东西让需要这方面的人快速去实现自己想要的功能，那么如果想实现增量代码覆盖率需要做到哪些工作呢？
大家在网络上找到的实现方式无外乎三种
- 获取到增量代码，在jacoco进行插桩时判断是否是增量代码后再进行插桩，这样需要两个步骤，一是获取增量代码，二是找到jacoco的插桩逻辑进行修改
- 获取增量代码，在report阶段去判断方法是否是增量，再去生成报告
- 获取差异代码，解析生成的report报告，再过滤出差异代码的报告
首先第一种需要对java字节码操作比较熟悉，难度较高，我们不谈，第三种去解析生成的报告，可能存在误差
所以我们一般选择第二种，而网络上所有的增量实现基本是基于第二种，我们先看看下面的图
上图说明了jacoco测试覆盖率的生成流程，而我们要做的是在report的时候加入我们的逻辑
根据我们的方案，我们需要三个动作
- 计算出两个版本的差异代码（基于git）
- 将差异代码在jacoco的report阶段传给jacoco
- 修改jacoco源码，生成报告时判断代码是否是增量代码，只有增量代码才去生成报告
下面我们逐步讲解上述步骤
计算差异代码
计算差异代码我实现了一个简单的工程：差异代码获取
主要用到了两个工具类
org.eclipse.jgit
org.eclipse.jgit
com.github.javaparser
javaparser-core
org.eclipse.jgit主要用于从git获取代码，并获取到存在变更的文件
javaparser-core是一个java解析类，能将class类文件解析成树状，方便我们去获取差异类
/**
* 获取差异类
*
* @param diffMethodParams
* @return
*/
public List diffMethods(DiffMethodParams diffMethodParams) {
try {
//原有代码git对象
Git baseGit = cloneRepository(diffMethodParams.getGitUrl(), localBaseRepoDir + diffMethodParams.getBaseVersion(), diffMethodParams.getBaseVersion());
//现有代码git对象
Git nowGit = cloneRepository(diffMethodParams.getGitUrl(), localBaseRepoDir + diffMethodParams.getNowVersion(), diffMethodParams.getNowVersion());
AbstractTreeIterator baseTree = prepareTreeParser(baseGit.getRepository(), diffMethodParams.getBaseVersion());
AbstractTreeIterator nowTree = prepareTreeParser(nowGit.getRepository(), diffMethodParams.getNowVersion());
//获取两个版本之间的差异代码
List diff = nowGit.diff().setOldTree(baseTree).setNewTree(nowTree).setShowNameAndStatusOnly(true).call();
//过滤出有效的差异代码
Collection validDiffList = diff.stream()
//只计算java文件
.filter(e -> e.getNewPath().endsWith(".java"))
//排除测试文件
.filter(e -> e.getNewPath().contains("src/main/java"))
//只计算新增和变更文件
.filter(e -> DiffEntry.ChangeType.ADD.equals(e.getChangeType()) || DiffEntry.ChangeType.MODIFY.equals(e.getChangeType()))
.collect(Collectors.toList());
if (CollectionUtils.isEmpty(validDiffList)) {
return null;
}
/**
* 多线程获取旧代码和新代码的差异类及差异方法
*/
List> priceFuture = validDiffList.stream().map(item -> getClassMethods(getClassFile(baseGit, item.getNewPath()), getClassFile(nowGit, item.getNewPath()), item)).collect(Collectors.toList());
return priceFuture.stream().map(CompletableFuture::join).filter(Objects::nonNull).collect(Collectors.toList());
} catch (GitAPIException e) {
e.printStackTrace();
}
return null;
} 以上代码为获取差异类的核心代码
/**
* 获取类的增量方法
*
* @param oldClassFile 旧类的本地地址
* @param mewClassFile 新类的本地地址
* @param diffEntry 差异类
* @return
*/
private CompletableFuture getClassMethods(String oldClassFile, String mewClassFile, DiffEntry diffEntry) {
//多线程获取差异方法，此处只要考虑增量代码太多的情况下，每个类都需要遍历所有方法，采用多线程方式加快速度
return CompletableFuture.supplyAsync(() -> {
String className = diffEntry.getNewPath().split("\\.")[0].split("src/main/java/")[1];
//新增类直接标记，不用计算方法
if (DiffEntry.ChangeType.ADD.equals(diffEntry.getChangeType())) {
return ClassInfoResult.builder()
.classFile(className)
.type(DiffEntry.ChangeType.ADD.name())
.build();
}
List diffMethods;
//获取新类的所有方法
List newMethodInfoResults = MethodParserUtils.parseMethods(mewClassFile);
//如果新类为空，没必要比较
if (CollectionUtils.isEmpty(newMethodInfoResults)) {
return null;
}
//获取旧类的所有方法
List oldMethodInfoResults = MethodParserUtils.parseMethods(oldClassFile);
//如果旧类为空，新类的方法所有为增量
if (CollectionUtils.isEmpty(oldMethodInfoResults)) {
diffMethods = newMethodInfoResults;
} else { //否则，计算增量方法
List md5s = oldMethodInfoResults.stream().map(MethodInfoResult::getMd5).collect(Collectors.toList());
diffMethods = newMethodInfoResults.stream().filter(m -> !md5s.contains(m.getMd5())).collect(Collectors.toList());
}
//没有增量方法，过滤掉
if (CollectionUtils.isEmpty(diffMethods)) {
return null;
}
ClassInfoResult result = ClassInfoResult.builder()
.classFile(className)
.methodInfos(diffMethods)
.type(DiffEntry.ChangeType.MODIFY.name())
.build();
return result;
}, executor);
} 以上代码为获取差异方法的核心代码
大家可以下载代码后运行，下面我们展示下，运行代码后获取到的差异代码内容（参数可以是两次commitId，也可以是两个分支，按自己的业务场景来）
{
"code": 10000,
"msg": "业务处理成功",
"data": [
{
"classFile": "com/dr/application/InstallCert",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/app/controller/Calculable",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/app/controller/JenkinsPluginController",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/app/controller/LoginController",
"methodInfos": [
{
"methodName": "captcha",
"parameters": "HttpServletRequest&HttpServletResponse"
},
{
"methodName": "login",
"parameters": "LoginUserParam&HttpServletRequest"
},
{
"methodName": "testInt",
"parameters": "int&char"
},
{
"methodName": "testInt",
"parameters": "String&int"
},
{
"methodName": "testInt",
"parameters": "short&int"
},
{
"methodName": "testInt",
"parameters": "int[]"
},
{
"methodName": "testInt",
"parameters": "T[]"
},
{
"methodName": "testInt",
"parameters": "Calculable&int&int"
},
{
"methodName": "testInt",
"parameters": "Map&List&Set"
},
{
"methodName": "display",
"parameters": ""
},
{
"methodName": "a",
"parameters": "InnerClass"
}
],
"type": "MODIFY"
},
{
"classFile": "com/dr/application/app/controller/RoleController",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/app/controller/TestController",
"methodInfos": [
{
"methodName": "test",
"parameters": ""
},
{
"methodName": "getPom",
"parameters": "HttpServletResponse"
},
{
"methodName": "getDeList",
"parameters": ""
}
],
"type": "MODIFY"
},
{
"classFile": "com/dr/application/app/controller/view/RoleViewController",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/app/param/AddRoleParam",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/app/vo/DependencyVO",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/app/vo/JenkinsPluginsVO",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/app/vo/RoleVO",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/config/ExceptionAdvice",
"methodInfos": [
{
"methodName": "handleException",
"parameters": "Exception"
},
{
"methodName": "handleMissingServletRequestParameterException",
"parameters": "MissingServletRequestParameterException"
}
],
"type": "MODIFY"
},
{
"classFile": "com/dr/application/config/GitConfig",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/config/JenkinsConfig",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/ddd/StaticTest",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/ddd/Test",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/application/util/GitAdapter",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/common/errorcode/BizCode",
"methodInfos": [
{
"methodName": "getCode",
"parameters": ""
}
],
"type": "MODIFY"
},
{
"classFile": "com/dr/common/response/ApiResponse",
"methodInfos": [
{
"methodName": "success",
"parameters": ""
}
],
"type": "MODIFY"
},
{
"classFile": "com/dr/jenkins/JenkinsApplication",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/jenkins/config/JenkinsConfigure",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/jenkins/controller/JenkinsController",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/jenkins/controller/TestApi",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/jenkins/dto/JobAddDto",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/jenkins/service/JenkinsService",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/jenkins/service/impl/JenkinsServiceImpl",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/jenkins/util/GenerateUniqueIdUtil",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/jenkins/vo/DeviceVo",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/jenkins/vo/GoodsVO",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/jenkins/vo/JobAddVo",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/repository/user/dto/query/RoleQueryDto",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/repository/user/dto/result/RoleResultDto",
"methodInfos": null,
"type": "ADD"
},
{
"classFile": "com/dr/user/service/impl/PermissionServiceImpl",
"methodInfos": [
{
"methodName": "getPermissionByRoles",
"parameters": "List"
},
{
"methodName": "buildMenuTree",
"parameters": "List"
},
{
"methodName": "getSubMenus",
"parameters": "Long&Map>"
}
],
"type": "MODIFY"
},
{
"classFile": "com/dr/user/service/impl/RoleServiceImpl",
"methodInfos": [
{
"methodName": "getByUserId",
"parameters": "Long"
},
{
"methodName": "getListByPage",
"parameters": "RoleQueryDto"
}
],
"type": "MODIFY"
}
],
"uniqueData": "[{\"classFile\":\"com/dr/application/InstallCert\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/app/controller/Calculable\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/app/controller/JenkinsPluginController\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/app/controller/LoginController\",\"methodInfos\":[{\"methodName\":\"captcha\",\"parameters\":\"HttpServletRequest&HttpServletResponse\"},{\"methodName\":\"login\",\"parameters\":\"LoginUserParam&HttpServletRequest\"},{\"methodName\":\"testInt\",\"parameters\":\"int&char\"},{\"methodName\":\"testInt\",\"parameters\":\"String&int\"},{\"methodName\":\"testInt\",\"parameters\":\"short&int\"},{\"methodName\":\"testInt\",\"parameters\":\"int[]\"},{\"methodName\":\"testInt\",\"parameters\":\"T[]\"},{\"methodName\":\"testInt\",\"parameters\":\"Calculable&int&int\"},{\"methodName\":\"testInt\",\"parameters\":\"Map&List&Set\"},{\"methodName\":\"display\",\"parameters\":\"\"},{\"methodName\":\"a\",\"parameters\":\"InnerClass\"}],\"type\":\"MODIFY\"},{\"classFile\":\"com/dr/application/app/controller/RoleController\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/app/controller/TestController\",\"methodInfos\":[{\"methodName\":\"test\",\"parameters\":\"\"},{\"methodName\":\"getPom\",\"parameters\":\"HttpServletResponse\"},{\"methodName\":\"getDeList\",\"parameters\":\"\"}],\"type\":\"MODIFY\"},{\"classFile\":\"com/dr/application/app/controller/view/RoleViewController\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/app/param/AddRoleParam\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/app/vo/DependencyVO\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/app/vo/JenkinsPluginsVO\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/app/vo/RoleVO\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/config/ExceptionAdvice\",\"methodInfos\":[{\"methodName\":\"handleException\",\"parameters\":\"Exception\"},{\"methodName\":\"handleMissingServletRequestParameterException\",\"parameters\":\"MissingServletRequestParameterException\"}],\"type\":\"MODIFY\"},{\"classFile\":\"com/dr/application/config/GitConfig\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/config/JenkinsConfig\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/ddd/StaticTest\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/ddd/Test\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/application/util/GitAdapter\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/common/errorcode/BizCode\",\"methodInfos\":[{\"methodName\":\"getCode\",\"parameters\":\"\"}],\"type\":\"MODIFY\"},{\"classFile\":\"com/dr/common/response/ApiResponse\",\"methodInfos\":[{\"methodName\":\"success\",\"parameters\":\"\"}],\"type\":\"MODIFY\"},{\"classFile\":\"com/dr/jenkins/JenkinsApplication\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/jenkins/config/JenkinsConfigure\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/jenkins/controller/JenkinsController\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/jenkins/controller/TestApi\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/jenkins/dto/JobAddDto\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/jenkins/service/JenkinsService\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/jenkins/service/impl/JenkinsServiceImpl\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/jenkins/util/GenerateUniqueIdUtil\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/jenkins/vo/DeviceVo\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/jenkins/vo/GoodsVO\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/jenkins/vo/JobAddVo\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/repository/user/dto/query/RoleQueryDto\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/repository/user/dto/result/RoleResultDto\",\"methodInfos\":[],\"type\":\"ADD\"},{\"classFile\":\"com/dr/user/service/impl/PermissionServiceImpl\",\"methodInfos\":[{\"methodName\":\"getPermissionByRoles\",\"parameters\":\"List\"},{\"methodName\":\"buildMenuTree\",\"parameters\":\"List\"},{\"methodName\":\"getSubMenus\",\"parameters\":\"Long&Map>\"}],\"type\":\"MODIFY\"},{\"classFile\":\"com/dr/user/service/impl/RoleServiceImpl\",\"methodInfos\":[{\"methodName\":\"getByUserId\",\"parameters\":\"Long\"},{\"methodName\":\"getListByPage\",\"parameters\":\"RoleQueryDto\"}],\"type\":\"MODIFY\"}]"
} data部分为差异代码的具体内容
将差异代码传递到jaocco
大家可以参考：jacoco增量代码改造
我们只需要找到Report类，加入可选参数
@Option(name = "--diffCode", usage = "input file for diff", metaVar = "") String diffCode; 
这样，我们就可以在jacoco内部接受到传递的参数了，如果report命令加上--diffCode就计算增量，不加则计算全量，不影响正常功能，灵活性高
我们这里改造了analyze方法，将增量代码塞给CoverageBuilder对象，我们需要用时直接去获取
private IBundleCoverage analyze(final ExecutionDataStore data,
final PrintWriter out) throws IOException {
CoverageBuilder builder;
// 如果有增量参数将其设置进去
if (null != this.diffCode) {
builder = new CoverageBuilder(this.diffCode);
} else {
builder = new CoverageBuilder();
}
final Analyzer analyzer = new Analyzer(data, builder);
for (final File f : classfiles) {
analyzer.analyzeAll(f);
}
printNoMatchWarning(builder.getNoMatchClasses(), out);
return builder.getBundle(name);
}差异代码匹配
jacoco采用AMS类去解析class类，我们需要去修改org.jacoco.core包下面的Analyzer类
private void analyzeClass(final byte[] source) {
final long classId = CRC64.classId(source);
final ClassReader reader = InstrSupport.classReaderFor(source);
if ((reader.getAccess() & Opcodes.ACC_MODULE) != 0) {
return;
}
if ((reader.getAccess() & Opcodes.ACC_SYNTHETIC) != 0) {
return;
}
// 字段不为空说明是增量覆盖
if (null != CoverageBuilder.classInfos
&& !CoverageBuilder.classInfos.isEmpty()) {
// 如果没有匹配到增量代码就无需解析类
if (!CodeDiffUtil.checkClassIn(reader.getClassName())) {
return;
}
}
final ClassVisitor visitor = createAnalyzingVisitor(classId,
reader.getClassName());
reader.accept(visitor, 0);
}主要是判断如果需要的是增量代码覆盖率，则匹配类是否是增量的（这里是jacoco遍历解析每个类的地方）
然后修改ClassProbesAdapter类的visitMethod方法（这个是遍历类里面每个方法的地方）
整个比较的代码逻辑在这里，注释写的比较详细了
修改完成后，大家只要构建出org.jacoco.cli-0.8.7-SNAPSHOT-nodeps.jar包，然后report时传入增量代码即可
全量报告
增量报告
所遇到问题
- 差异方法的参数匹配
由于我们使用javaparser解析出的参数格式为String a,int b
而ASM解析出的 为Ljava/lang/String，I;在匹配参数的时候遇到了问题，最终我找到了Type类的方法
Type.getArgumentTypes(desc)
然后
argumentTypes[i].getClassName()
将ASM的参数解析成String,int(做了截取)，然后再去匹配，就能正确匹配到参数的格式了
- 为什么不将整个生成报告做成一个平台
jacoco生成报告的时候，需要传入源码，编译后的class文件，而编译这些东西我们一般都有自己的ci平台去做，我们可以将我们的覆盖率功能集成到我们的devops平台，从那边去获取源码或编译出的class文件,而且可以做业务上的整合，所以没有像supper-jacoco那样做成一个平台
鉴于最近github不稳定，代码上传到了码云：
增量代码获取：https://gitee.com/Dray/code-diff.git
jacoco二开：https://gitee.com/Dray/jacoco.git
考资料： super-jacoco
jacoco-plus
欢迎大家一起探讨相关问题
微信群：
失效后加群主进群
