kettle运行时jar包动态加载的实现

大家好，我是帅气小伙。今天为大家分享一下有关kettle需要导入第三方jar包时，动态加载的经验。我们在使用kettle的时候，经常需要用一些第三方的jar包，例如 mssql的驱动，我们平常可以把它放到kettle_home的lib目录下，然后重启。这样其实挺不方便的，尤其是对于集成了kettle调度的应用来说。又是往往就是一个jar包的问题，导致平台要重启N次，显示会对一些生成环境的业务造成巨大的影响。于是实现kettle运行时jar包的动态加载还是有价值的。

思路

• 为什么kettle客户端重启就能加载到lib的jar包呢？
  • 1.kettle的入口 spoon.bat/spoon.sh

if %STARTTITLE%!==! SET STARTTITLE="Spoon"
REM Eventually call java instead of javaw and do not run in a separate window
if not "%SPOON_CONSOLE%"=="1" set SPOON_START_OPTION=start %STARTTITLE%

@echo on
%SPOON_START_OPTION% "%_PENTAHO_JAVA%" %OPT% -jar launcher\pentaho-application-launcher-5.4.0.1-130.jar -lib ..\%LIBSPATH% %_cmdline%
@echo off
if "%SPOON_PAUSE%"=="1" pause

显然，kettle的入口程序是 pentaho-application-launcher-5.4.0.1-130.jar，然后传入了lib参数

• 2.反编译 pentaho-application-launcher-5.4.0.1-130.jar

  public static void main(String[] args) throws Exception {
        Parameters parameters = Parameters.fromArgs(args, System.err);
        URL location = Launcher.class.getProtectionDomain().getCodeSource().getLocation();
        File appDir = FileUtil.computeApplicationDir(location, new File("."), System.err);
        File configurationFile = new File(appDir, "launcher.properties");
        Properties configProperties = new Properties();

        try {
            configProperties.load(new FileReader(configurationFile));
        } catch (Exception var13) {
            ;
        }

        Configuration configuration = Configuration.create(configProperties, appDir, parameters);
        if (configuration.isUninstallSecurityManager()) {
            System.setSecurityManager((SecurityManager)null);
        }

        Iterator i$ = configuration.getSystemProperties().entrySet().iterator();

        while(i$.hasNext()) {
            Entry systemProperty = (Entry)i$.next();
            System.setProperty((String)systemProperty.getKey(), (String)systemProperty.getValue());
        }

        List jars = FileUtil.populateClasspath(configuration.getClasspath(), appDir, System.err);
        jars.addAll(FileUtil.populateLibraries(configuration.getLibraries(), appDir, System.err));
        URL[] classpathEntries = (URL[])((URL[])jars.toArray(new URL[jars.size()]));
        ClassLoader cl = new URLClassLoader(classpathEntries);
        Thread.currentThread().setContextClassLoader(cl);
        if (StringUtil.isEmpty(configuration.getMainClass())) {
            System.err.println("Invalid main-class entry, cannot proceed.");
            System.err.println("Application Directory: " + appDir);
            System.exit(1);
        }

        if (configuration.isDebug()) {
            System.out.println("Application Directory: " + appDir);

            for(int i = 0; i < classpathEntries.length; ++i) {
                URL url = classpathEntries[i];
                System.out.println("ClassPath[" + i + "] = " + url);
            }
        }

        Class mainClass = cl.loadClass(configuration.getMainClass());
        String[] newArgs = new String[args.length - parameters.getParsedArgs()];
        System.arraycopy(args, parameters.getParsedArgs(), newArgs, 0, newArgs.length);
        Method method = mainClass.getMethod("main", String[].class);
        method.invoke((Object)null, newArgs);
    }

• 3.关键性代码

  List jars = FileUtil.populateClasspath(configuration.getClasspath(), appDir, System.err);
        jars.addAll(FileUtil.populateLibraries(configuration.getLibraries(), appDir, System.err));
        URL[] classpathEntries = (URL[])((URL[])jars.toArray(new URL[jars.size()]));
        ClassLoader cl = new URLClassLoader(classpathEntries);
        Thread.currentThread().setContextClassLoader(cl);

也就是说，只需把jar包动态加载到Thread.currentThread()的classpath就可以了。

实现方式

    private static ConcurrentHashMap jarMaps = new ConcurrentHashMap();//防止重复加载

    /**
     * 动态加载jar到kettle运行时classpath
     * @param libs
     * @return
     */
    public static List loadExtLib(List libs){
        List success = new ArrayList();
        for(String path : libs){
            ClassLoader cl = Thread.currentThread().getContextClassLoader();
            try {
                //反射调用 addURL
                Method addURL = URLClassLoader.class.getDeclaredMethod("addURL", new Class[] {URL.class});
                addURL.setAccessible(true);
                File file = new File(path);
                List paths = new ArrayList();
                findJars(file,paths);//寻找路径下的jar包
                for(String jarPath : paths){
                    File jar = new File(jarPath);
                    FileInputStream inputStream = new FileInputStream(jar);
                    String jarHash = md5HashCode(inputStream);
                    if(!jarMaps.containsKey(jarHash)){
                        jarMaps.put(jarHash,jarPath);
                        URL url = jar.toURI().toURL();
                        addURL.invoke(cl, new Object[] { url });
                        success.add(jar.getName());
                        log.info("成功加载："+jarPath);
                    }
                    inputStream.close();
                }
            } catch (Exception e) {
                e.printStackTrace();
                log.error("加载kettle类库失败,原因",path,e);
            }
        }
        return success;
    }

    /**
     * 遍历文件jar包
     * @param file 文件夹
     * @param jarPaths jar包路径集合
     */
    public static void findJars(File file,List jarPaths){
        File[] files = file.listFiles();
        for(File a :files){
            String path = a.getPath();
            if(a.isFile() && (path.endsWith(".jar") || path.endsWith(".zip"))){
                jarPaths.add(path);
            }
            if(a.isDirectory()){
                findJars(a,jarPaths);
            }
        }
    }

    /**
     * java获取文件的md5值
     * @param fis 输入流
     * @return
     */
    public static String md5HashCode(InputStream fis) {
        try {
            //拿到一个MD5转换器,如果想使用SHA-1或SHA-256，则传入SHA-1,SHA-256
            MessageDigest md = MessageDigest.getInstance("MD5");

            //分多次将一个文件读入，对于大型文件而言，比较推荐这种方式，占用内存比较少。
            byte[] buffer = new byte[1024];
            int length = -1;
            while ((length = fis.read(buffer, 0, 1024)) != -1) {
                md.update(buffer, 0, length);
            }
            fis.close();
            //转换并返回包含16个元素字节数组,返回数值范围为-128到127
            byte[] md5Bytes  = md.digest();
            BigInteger bigInt = new BigInteger(1, md5Bytes);//1代表绝对值
            return bigInt.toString(16);//转换为16进制
        } catch (Exception e) {
            e.printStackTrace();
            return "";
        }
    }

写个controller调用下就ok了。

    @RequestMapping("/load/lib")
    @ResponseBody
    public List loadLib(){
        List success = KettleUtils.loadExtLib(config.getLibs());
        return success;
    }

总结

就说这么多，有关于kettle调度这个项目，我尽量把它完善下吧，等一个spring-boot-starter-kettle吧。