自定义ClassLoader的实现
背景
公司使用的是自己实现的RPC框架，有自己的传输协议和序列化方式，在RPC服务启动的时候，会扫描当前服务的lib目录，然后后加载以com.xxx开头的class文件，为接口的返回结果序列化为java Object做准备。
2019年开始，通过不断的调研和思考，RPC服务接口测试平台初具雏形，在开发测试平台的时候，一直存在一个问题，测试平台肯定要能够为所有的RPC服务都提供测试能力，但是每个RPC服务都有自己的lib目录，都有自己依赖的jar包和版本，所以每一次测试都需要使用被测服务当前版本依赖的jar包，类似与tomcat的每个应用都是互相隔离的，都使用自己的jar包。
思路
背景中也提到了，tomcat可以实现应用间jar包的隔离，那么就按照这个思路继续下去，先介绍一下普通java的双亲委托机制和普通java进程的classloader关系，BootStrap ClassLoader、ExtClassLoader、AppClassLoader为java的源码，他们的关系如下图
可以参考sun.misc.Launcher和java.lang.ClassLoader源码和博客深入理解 Tomcat（四）Tomcat 类加载器之为何违背双亲委派模型 【2. 什么是双亲委任模型】
sun.misc.Launcher.AppClassloader
public Class<?> loadClass(String var1, boolean var2) throws ClassNotFoundException {
...
//如果已知是不存在的类
if (this.ucp.knownToNotExist(var1)) {
// TODO 追到native方法，逻辑下次再补充上来
Class var5 = this.findLoadedClass(var1);
if (var5 != null) {
...
return var5;
} else {
throw new ClassNotFoundException(var1);
}
} else {
//委托给父类的loadClass方法加载
return super.loadClass(var1, var2);
}
}
java.lang.ClassLoader
protected Class<?> loadClass(String name, boolean resolve)
throws ClassNotFoundException
{
synchronized (getClassLoadingLock(name)) {
// First, check if the class has already been loaded
Class<?> c = findLoadedClass(name);
if (c == null) {
long t0 = System.nanoTime();
try {
// 如果有父classloader，委托给parent加载
if (parent != null) {
c = parent.loadClass(name, false);
} else {
// 如果parent为null，及当前的classloader是ExtClassloader，委托bootstrap classloader加载
c = findBootstrapClassOrNull(name);
}
} catch (ClassNotFoundException e) {
// ClassNotFoundException thrown if class not found
// from the non-null parent class loader
}
if (c == null) {
// If still not found, then invoke findClass in order
// to find the class.
long t1 = System.nanoTime();
c = findClass(name);
// this is the defining class loader; record the stats
sun.misc.PerfCounter.getParentDelegationTime().addTime(t1 - t0);
sun.misc.PerfCounter.getFindClassTime().addElapsedTimeFrom(t1);
sun.misc.PerfCounter.getFindClasses().increment();
}
}
if (resolve) {
resolveClass(c);
}
return c;
}
}
Tomcat打破了这种加载顺序，当前classloader加载类的时候，先尝试自己加载，加载失败的时候才委托parent加载，那么模仿tomcat的org.apache.catalina.loader.WebappClassLoader实现自己的classloader不久可以了？！
Apache Tomcat 8
实现
重载loadClass、findLoadedClass0、findClass、findResources
/**
* @author [email protected]
* @version 1.0
* @date 2020/2/9 19:48
*/
public class TestCaseClassLoader extends URLClassLoader {
@Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
return loadClass(name, false);
}
@Override
public Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
// 加锁
synchronized (this.getClassLoadingLockInternal(name)) {
Class<?> clazz = null;
// Check our previously loaded local class cache
// 检查本地缓存中是否已经加载过
clazz = this.findLoadedClass0(name);
if (clazz != null) {
if (resolve) {
this.resolveClass(clazz);
}
return clazz;
}
// Check our previously loaded class cache
// 还没找到和findLoadedClass0有啥差别
clazz = this.findLoadedClass(name);
if (clazz != null) {
if (resolve) {
this.resolveClass(clazz);
}
return clazz;
}
// Try loading the class with the system class loader, to prevent the webapp from overriding J2SE classes
// Bootstrap Classloader是否加载
try {
clazz = this.j2seClassLoader.loadClass(name);
if (clazz != null) {
if (resolve) {
this.resolveClass(clazz);
}
return clazz;
}
} catch (ClassNotFoundException var12) {
}
// Search local repositories
// 从当前的classloader 目录下加载
try {
clazz = this.findClass(name);
if (clazz != null) {
if (resolve) {
this.resolveClass(clazz);
}
return clazz;
}
} catch (ClassNotFoundException var14) {
}
// Delegate to parent unconditionally
// 委托给parent加载
try {
clazz = Class.forName(name, false, this.parent);
if (clazz != null) {
if (resolve) {
this.resolveClass(clazz);
}
return clazz;
}
} catch (ClassNotFoundException var10) {
}
}
throw new ClassNotFoundException(name);
}
// 查找缓存
private Class<?> findLoadedClass0(String name) {
String path = this.binaryNameToPath(name);
ResourceEntry entry = (ResourceEntry) this.resourceEntries.get(path);
return entry != null ? entry.loadedClass : null;
}
protected Class<?> findClass(String name) throws ClassNotFoundException {
Class<?> clazz = null;
if (!this.validate(name)) {
throw new ClassNotFoundException(name);
} else {
String path = this.binaryNameToPath(name);
// 从指定目录下查找
ResourceEntry entry = this.findResourceInternal(name, path);
if (entry == null) {
throw new ClassNotFoundException(name);
} else {
clazz = entry.loadedClass;
if (clazz == null) {
synchronized (this.getClassLoadingLockInternal(name)) {
clazz = entry.loadedClass;
if (clazz != null) {
return clazz;
} else if (entry.binaryContent == null) {
throw new ClassNotFoundException(name);
} else {
try {
// 从字节码转为java class对象
clazz = this.defineClass(name, entry.binaryContent, 0, entry.binaryContent.length, new CodeSource(entry.codeBase, entry.certificates));
} catch (UnsupportedClassVersionError var11) {
throw new UnsupportedClassVersionError(name);
}
entry.loadedClass = clazz;
entry.binaryContent = null;
entry.certificates = null;
}
}
}
}
}
if (clazz == null) {
throw new ClassNotFoundException(name);
}
return clazz;
}
...
/**
* SCFSerializableScanner.scan方法会调用classloader的getResources获取指定包名的class文件
*
* @param name
* @return
* @throws IOException
*/
public Enumeration<URL> findResources(String name) throws IOException {
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug(" findResources(" + name + ")");
}
LinkedHashSet<URL> result = new LinkedHashSet();
int jarFilesLength = this.jarFiles.size();
synchronized (this.jarFiles) {
for (int i = 0; i < jarFilesLength; ++i) {
JarFile jarFile = this.jarFiles.get(i);
JarEntry jarEntry = jarFile.getJarEntry(name);
if (jarEntry != null) {
try {
String jarFakeUrl = this.getURI(new File(jarFile.getName())).toString();
result.add(UriUtil.buildJarUrl(jarFakeUrl, name));
} catch (MalformedURLException var10) {
}
}
}
}
return Collections.enumeration(result);
}
}
效果
再测试平台上每一次测试都新建一个classloader，加载指定目录下的jar包，做到每次测试都是隔离的
完整源码
TestCaseClassLoader
import com.common.helper.LoggerHelper;
import org.apache.commons.io.FileUtils;
import javax.naming.NamingException;
import java.io.*;
import java.lang.reflect.Method;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.charset.Charset;
import java.security.*;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
/**
* @author [email protected]
* @version 1.0
* @date 2020/2/9 19:48
*/
public class TestCaseClassLoader extends URLClassLoader {
private static LoggerHelper loggerHelper = LoggerHelper.getLoggerHelper(TestCaseClassLoader.class);
private static final Method GET_CLASSLOADING_LOCK_METHOD;
// 缓存
private Map<String, ResourceEntry> resourceEntries = new ConcurrentHashMap();
// jvm根classloader
private ClassLoader j2seClassLoader;
// 创建TestCaseClassLoader时指定父classloader
private ClassLoader parent;
// 仓库地址，从哪些目录下查找
private String[] repositories;
protected HashMap<String, String> notFoundResources;
private static final Charset CHARSET_UTF8 = Charset.forName("UTF-8");
// 仓库地址下的所有jar
private List<JarFile> jarFiles;
static {
Method getClassLoadingLockMethod = null;
try {
final Method registerParallel = ClassLoader.class.getDeclaredMethod("registerAsParallelCapable");
AccessController.doPrivileged(new PrivilegedAction<Void>() {
public Void run() {
registerParallel.setAccessible(true);
return null;
}
});
registerParallel.invoke((Object) null);
getClassLoadingLockMethod = ClassLoader.class.getDeclaredMethod("getClassLoadingLock", String.class);
} catch (Exception var2) {
}
GET_CLASSLOADING_LOCK_METHOD = getClassLoadingLockMethod;
}
public TestCaseClassLoader(String[] lookupPaths, ClassLoader parent) {
super(new URL[0], parent);
this.parent = null;
ClassLoader p = this.getParent();
if (p == null) {
p = getSystemClassLoader();
}
this.parent = p;
ClassLoader j = String.class.getClassLoader();
// 获取bootstrap classloader
if (j == null) {
for (j = getSystemClassLoader(); j.getParent() != null; j = j.getParent()) {
}
}
this.j2seClassLoader = j;
this.jarFiles = new ArrayList<>();
this.repositories = lookupPaths;
//初始化资源
initRepositories();
class NamelessClass_1 extends LinkedHashMap<String, String> {
private static final long serialVersionUID = 1L;
NamelessClass_1() {
}
protected boolean removeEldestEntry(Map.Entry<String, String> eldest) {
return this.size() > 1000;
}
}
this.notFoundResources = new NamelessClass_1();
}
/**
* 扫描repositories中指定的目录下的所有jar包
*/
private void initRepositories() {
if (this.repositories.length > 0) {
for (String s : repositories) {
File file;
JarFile jarFile = null;
try {
file = new File(s);
Collection<File> jars = FileUtils.listFiles(file, new String[]{"jar"}, true);
for (File f : jars) {
// 过滤掉源码文件
if (!f.getName().contains("sources")) {
String dependentJar = f.getCanonicalPath();
jarFile = new JarFile(dependentJar);
jarFiles.add(jarFile);
}
}
} catch (Exception e) {
loggerHelper.warn("TestCaseClassLoader.jarOpenFail", jarFile, ",异常：", e);
}
}
}
}
private Object getClassLoadingLockInternal(String className) {
if (GET_CLASSLOADING_LOCK_METHOD != null) {
try {
return GET_CLASSLOADING_LOCK_METHOD.invoke(this, className);
} catch (Exception var3) {
}
}
return this;
}
/**
* 未没在jar包中的class文件提供加载方法，加载testcase class文件
*
* @param path case文件存在的目录
* @param name case文件名称，不包含.class
* @return
*/
public Class<?> loadTestCaseClass(String path, String name) {
if (path == null || name == null) {
return null;
}
//不能以.class结尾
if (name.endsWith(".class")) {
name = name.substring(0, name.length() - 6);
}
// 读取文件内容
File caseFile = new File(path, name + ".class");
// 文件不存在，直接返回null
if (caseFile.exists()) {
int contentLength = (int) caseFile.length();
byte[] buf = new byte[contentLength];
int pos = 0;
try {
InputStream binaryStream = new FileInputStream(caseFile);
while (true) {
int n = binaryStream.read(buf, pos, buf.length - pos);
if (n <= 0) {
break;
}
pos += n;
}
} catch (IOException var96) {
loggerHelper.error("TestCaseClassLoader.readError", name, var96);
return null;
}
// name中不能包含.class
// 字节码转为name名称的class对象
Class clazz = this.defineClass(name, buf, 0, contentLength);
return clazz;
}
return null;
}
@Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
return loadClass(name, false);
}
@Override
public Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
synchronized (this.getClassLoadingLockInternal(name)) {
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug("loadClass(" + name + ", " + resolve + ")");
}
Class<?> clazz = null;
// Check our previously loaded local class cache
clazz = this.findLoadedClass0(name);
if (clazz != null) {
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug(" Returning class from cache");
}
if (resolve) {
this.resolveClass(clazz);
}
return clazz;
}
// Check our previously loaded class cache
clazz = this.findLoadedClass(name);
if (clazz != null) {
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug(" Returning class from cache");
}
if (resolve) {
this.resolveClass(clazz);
}
return clazz;
}
// Try loading the class with the system class loader, to prevent
// the webapp from overriding J2SE classes
try {
clazz = this.j2seClassLoader.loadClass(name);
if (clazz != null) {
if (resolve) {
this.resolveClass(clazz);
}
return clazz;
}
} catch (ClassNotFoundException var12) {
}
// Search local repositories
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug(" Searching local repositories");
}
try {
clazz = this.findClass(name);
if (clazz != null) {
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug(" Loading class from local repository");
}
if (resolve) {
this.resolveClass(clazz);
}
return clazz;
}
} catch (ClassNotFoundException var14) {
}
// Delegate to parent unconditionally
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug(" Delegating to parent classloader at end: " + this.parent);
}
try {
clazz = Class.forName(name, false, this.parent);
if (clazz != null) {
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug(" Loading class from parent");
}
if (resolve) {
this.resolveClass(clazz);
}
return clazz;
}
} catch (ClassNotFoundException var10) {
}
}
throw new ClassNotFoundException(name);
}
private Class<?> findLoadedClass0(String name) {
String path = this.binaryNameToPath(name);
ResourceEntry entry = (ResourceEntry) this.resourceEntries.get(path);
return entry != null ? entry.loadedClass : null;
}
protected Class<?> findClass(String name) throws ClassNotFoundException {
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug(" findClass(" + name + ")");
}
Class<?> clazz = null;
if (!this.validate(name)) {
throw new ClassNotFoundException(name);
} else {
String path = this.binaryNameToPath(name);
// 查找对应的资源
ResourceEntry entry = this.findResourceInternal(name, path);
if (entry == null) {
throw new ClassNotFoundException(name);
} else {
clazz = entry.loadedClass;
if (clazz == null) {
synchronized (this.getClassLoadingLockInternal(name)) {
clazz = entry.loadedClass;
if (clazz != null) {
return clazz;
} else if (entry.binaryContent == null) {
throw new ClassNotFoundException(name);
} else {
try {
// 字节码转为name名称的class对象
clazz = this.defineClass(name, entry.binaryContent, 0, entry.binaryContent.length, new CodeSource(entry.codeBase, entry.certificates));
} catch (UnsupportedClassVersionError var11) {
throw new UnsupportedClassVersionError(name);
}
entry.loadedClass = clazz;
entry.binaryContent = null;
entry.certificates = null;
}
}
}
}
}
if (clazz == null) {
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug(" --> Returning ClassNotFoundException");
}
throw new ClassNotFoundException(name);
}
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug(" Returning class " + clazz);
ClassLoader cl = clazz.getClassLoader();
loggerHelper.debug(" Loaded by " + cl.toString());
}
return clazz;
}
private URL getURI(File file) throws MalformedURLException {
File realFile = file;
try {
realFile = realFile.getCanonicalFile();
} catch (IOException var4) {
}
return realFile.toURI().toURL();
}
private ResourceEntry findResourceInternal(File file, String path) {
ResourceEntry entry = new ResourceEntry();
try {
entry.source = this.getURI(new File(file, path));
return entry;
} catch (MalformedURLException var5) {
return null;
}
}
private ResourceEntry findResourceInternal(String name, String path) {
if (name != null && path != null) {
JarEntry jarEntry = null;
String jarEntryPath = path.substring(1);
// path对应的资源是否已经加载
ResourceEntry entry = (ResourceEntry) this.resourceEntries.get(path);
// 不存在，就去查找
if (entry == null) {
int contentLength = -1;
InputStream binaryStream = null;
int jarFilesLength = this.jarFiles.size();
// name之前就没有查找到，直接返回null
if (this.notFoundResources.containsKey(name)) {
return null;
} else {
if (jarFiles.size() > 0) {
synchronized (this.jarFiles) {
try {
// 所有jar包都检查一遍，过程中如果找到就退出遍历
for (int i = 0; entry == null && i < jarFilesLength; ++i) {
JarFile jarFile = this.jarFiles.get(i);
jarEntry = jarFile.getJarEntry(jarEntryPath);
if (jarEntry != null) {
entry = new ResourceEntry();
entry.codeBase = this.getURI(new File(jarFile.getName()));
entry.source = UriUtil.buildJarUrl(entry.codeBase.toString(), jarEntryPath);
contentLength = (int) jarEntry.getSize();
binaryStream = jarFile.getInputStream(jarEntry);
}
}
// 所有jar包都没有找到，放到notFoundResources中
if (entry == null) {
synchronized (this.notFoundResources) {
this.notFoundResources.put(name, name);
}
return (ResourceEntry) null;
}
if (binaryStream != null) {
byte[] buf = new byte[contentLength];
int pos = 0;
try {
// 读取文件
while (true) {
int n = binaryStream.read(buf, pos, buf.length - pos);
if (n <= 0) {
break;
}
pos += n;
}
} catch (IOException var96) {
loggerHelper.error("TestCaseClassLoader.readError", name, var96);
return null;
}
entry.binaryContent = buf;
if (jarEntry != null) {
entry.certificates = jarEntry.getCertificates();
}
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (binaryStream != null) {
try {
binaryStream.close();
} catch (IOException var85) {
}
}
}
}
// 放到缓存中
synchronized (this.resourceEntries) {
ResourceEntry entry2 = (ResourceEntry) this.resourceEntries.get(path);
if (entry2 == null) {
this.resourceEntries.put(path, entry);
} else {
entry = entry2;
}
return entry;
}
}
}
}
return entry;
}
return null;
}
private boolean validate(String name) {
if (name == null) {
return false;
} else if (name.startsWith("java.")) {
return false;
} else if (name.startsWith("javax.servlet.jsp.jstl")) {
return true;
} else if (name.startsWith("javax.servlet.")) {
return false;
} else {
return !name.startsWith("javax.el");
}
}
private String binaryNameToPath(String binaryName) {
StringBuilder path = new StringBuilder(7 + binaryName.length());
path.append('/');
path.append(binaryName.replace('.', '/'));
path.append(".class");
return path.toString();
}
/**
* SCFSerializableScanner.scan方法会调用classloader的getResources获取指定包名的class文件
*
* @param name
* @return
* @throws IOException
*/
public Enumeration<URL> findResources(String name) throws IOException {
if (loggerHelper.isDebugEnabled()) {
loggerHelper.debug(" findResources(" + name + ")");
}
LinkedHashSet<URL> result = new LinkedHashSet();
int jarFilesLength = this.jarFiles.size();
synchronized (this.jarFiles) {
for (int i = 0; i < jarFilesLength; ++i) {
JarFile jarFile = this.jarFiles.get(i);
JarEntry jarEntry = jarFile.getJarEntry(name);
if (jarEntry != null) {
try {
String jarFakeUrl = this.getURI(new File(jarFile.getName())).toString();
result.add(UriUtil.buildJarUrl(jarFakeUrl, name));
} catch (MalformedURLException var10) {
}
}
}
}
return Collections.enumeration(result);
}
}
ResourceEntry
import java.net.URL;
import java.security.cert.Certificate;
import java.util.jar.Manifest;
/**
* @author [email protected]
* @version 1.0
* @date 2020/2/9 20:02
*/
public class ResourceEntry {
public byte[] binaryContent = null;
public volatile Class<?> loadedClass = null;
public URL source = null;
public URL codeBase = null;
public Certificate[] certificates = null;
public ResourceEntry() {
}
}
UriUtil
import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.regex.Pattern;
/**
* @author [email protected]
* @version 1.0
* @date 2020/2/10 17:27
*/
public final class UriUtil {
private static Pattern PATTERN_EXCLAMATION_MARK = Pattern.compile("!/");
private static Pattern PATTERN_CARET = Pattern.compile("\\^/");
private static Pattern PATTERN_ASTERISK = Pattern.compile("\\*/");
private UriUtil() {
}
private static boolean isSchemeChar(char c) {
return Character.isLetterOrDigit(c) || c == '+' || c == '-' || c == '.';
}
public static boolean hasScheme(CharSequence uri) {
int len = uri.length();
for (int i = 0; i < len; ++i) {
char c = uri.charAt(i);
if (c == ':') {
return i > 0;
}
if (!isSchemeChar(c)) {
return false;
}
}
return false;
}
public static URL buildJarUrl(File jarFile) throws MalformedURLException {
return buildJarUrl((File) jarFile, (String) null);
}
public static URL buildJarUrl(File jarFile, String entryPath) throws MalformedURLException {
return buildJarUrl(jarFile.toURI().toString(), entryPath);
}
public static URL buildJarUrl(String fileUrlString) throws MalformedURLException {
return buildJarUrl((String) fileUrlString, (String) null);
}
public static URL buildJarUrl(String fileUrlString, String entryPath) throws MalformedURLException {
String safeString = makeSafeForJarUrl(fileUrlString);
StringBuilder sb = new StringBuilder();
sb.append(safeString);
sb.append("!/");
if (entryPath != null) {
sb.append(makeSafeForJarUrl(entryPath));
}
return new URL("jar", (String) null, -1, sb.toString());
}
public static URL buildJarSafeUrl(File file) throws MalformedURLException {
String safe = makeSafeForJarUrl(file.toURI().toString());
return new URL(safe);
}
private static String makeSafeForJarUrl(String input) {
String tmp = PATTERN_EXCLAMATION_MARK.matcher(input).replaceAll("%21/");
tmp = PATTERN_CARET.matcher(tmp).replaceAll("%5e/");
return PATTERN_ASTERISK.matcher(tmp).replaceAll("%2a/");
}
}
参考
深入理解 Tomcat（四）Tomcat 类加载器之为何违背双亲委派模型
Tomcat源码研究之ClassLoader
违反ClassLoader双亲委派机制三部曲第二部——Tomcat类加载机制
Java中自定义ClassLoader和ClassLoader的使用
Tomcat类加载器以及应用间class隔离与共享