kettle 9.1.0.0-324源码编译及IDEA调试运行

一、项目依赖导入:

用的是本地的maven3.3.9,setting.xml文件如下:







  
  C:\Users\ZH\Work\apache-maven-3.3.9\repository

  

  

  
  
    
  

  
  
    
  

  
  
    

    
  

  
  
    
    
        central
        Maven Repository Switchboard
        https://repo1.maven.org/maven2/
        central
    

    
        repo2
        central
        Human Readable Name for this Mirror.
        http://repo2.maven.org/maven2/
    

    
        alimaven
        aliyun maven
        http://maven.aliyun.com/nexus/content/groups/public/
        central
    
  

  
  
    

    
  

  


kettle工程的pom.xml文件如下



  4.0.0

  org.pentaho.di
  pdi
  9.1.0.0-324
  pom

  Pentaho Data Integration
  Container pom for Pentaho Data Integration modules
  http://www.pentaho.com

  
    org.pentaho
    pentaho-ce-jar-parent-pom
    9.1.0.0-324
  

  
    scm:git:[email protected]:pentaho/pentaho-kettle.git
    HEAD
  

  
    
      Apache License, Version 2.0
      https://www.apache.org/licenses/LICENSE-2.0.txt
      repo
      A business-friendly OSS license
    
    
      GNU Lesser General Public License, Version 2.1
      https://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt
      repo
      
    
    
      GNU Lesser General Public License, Version 3.0
      https://www.gnu.org/licenses/lgpl-3.0.txt
      repo
      
    
  

  
    1.8
    1.8
    yyyy-MM-dd hh.mm.ss
    -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Duser.language=en -Duser.country=US

    
    apache_v2

    
    9.1.0.0-324
    9.1.0.0-324

    
    4.6
    1.7R3
    1.9.3
    17.0
    1.2
    1.2.131
    3.2.4

    
    1.10.19

    1.7.3
    1.7.3
    2.0.0.0
  

  
    
      
        org.pentaho
        commons-xul-core
        ${commons-xul.version}
      
      
        org.pentaho
        commons-xul-swing
        ${commons-xul.version}
      
      
        org.pentaho
        commons-xul-swt
        ${commons-xul.version}
        
          
            org.eclipse.swt
            *
          
        
      
      
        pentaho
        metastore
        ${pentaho-metastore.version}
      

      
      
        org.eclipse.swt
        org.eclipse.swt.gtk.linux.x86_64
        ${org.eclipse.swt.version}
        
          
            *
            *
          
        
      
      
        rhino
        js
        ${js.version}
        
          
            *
            *
          
        
      
      
        commons-beanutils
        commons-beanutils
        ${commons-beanutils.version}
        
          
            *
            *
          
        
      
      
        com.google.guava
        guava
        ${guava.version}
        
          
            *
            *
          
        
      
      
        org.owasp.encoder
        encoder
        ${encoder.version}
        
          
            *
            *
          
        
      
      
        com.h2database
        h2
        ${h2.version}
        
          
            *
            *
          
        
      
      
        org.codehaus.woodstox
        wstx-asl
        ${wstx-asl.version}
        provided
      

      
      
        org.mockito
        mockito-all
        ${mockito-all.version}
        test
        
          
            *
            *
          
        
      
      
        org.hamcrest
        java-hamcrest
        ${java-hamcrest.version}
        test
        
          
            *
            *
          
        
      
      
        org.powermock
        powermock-module-junit4
        ${powermock-module-junit4.version}
        test
      
      
        org.powermock
        powermock-api-mockito
        ${powermock-api-mockito.version}
        test
      
    
  

  
    
      
        maven-compiler-plugin
      
      
        maven-jar-plugin
        
          
            
              ${maven.build.timestamp}
              ${user.name}
              ${project.artifactId}
            
          
        
      
    
  

  
    
      base
      
        
          !skipDefault
        
      
      
        engine-ext
        core
        engine
        dbdialog
        ui
      
    
    
      plugins
      
        
          !skipDefault
        
      
      
        plugins
      
    
    
      assemblies
      
        
          !skipDefault
        
      
      
        assemblies
      
    
  


  
    
      pentaho-nexus
      https://nexus.pentaho.org/content/groups/omni/
    

    
      pentaho-releases
      http://repository.pentaho.org/artifactory/repo/
    
  



小技巧

如果卡在某个文件的下载,实在自动下载不下来的,可以手动下载放到相应的目录下,并把该目录下的_remote.repositories文件删除,这样idea会优先读取本地仓库的文件;删除maven库中的.lastUpdated文件,然后update maven project,循环往复直到直到仓库中没有.lastUpdated文件,项目所有依赖导入完毕。

二、异常处理

依赖全部成功下载后,启动运行遇到以下几个异常:

1.java.lang.UnsatisfiedLinkError: Could not load SWT library.

java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons: 
    no swt-gtk-4623 in java.library.path
    no swt-gtk in java.library.path
    Can't load library: C:\Users\ZH\.swt\lib\win32\x86_64\swt-gtk-4623.dll
    Can't load library: C:\Users\ZH\.swt\lib\win32\x86_64\swt-gtk.dll

    at org.eclipse.swt.internal.Library.loadLibrary(Unknown Source)
    at org.eclipse.swt.internal.Library.loadLibrary(Unknown Source)
    at org.eclipse.swt.internal.C.(Unknown Source)
    at org.eclipse.swt.internal.Converter.wcsToMbcs(Unknown Source)
    at org.eclipse.swt.internal.Converter.wcsToMbcs(Unknown Source)
    at org.eclipse.swt.widgets.Display.(Unknown Source)
    at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:668)
原因:

swt相关依赖默认为linux环境,需要修改为win环境,笔者的机器为win11 64位 专业版

解决方案:

打开kettle-ui-swt的pom.xml文件,注释掉org.eclipse.swt.gtk.linux.x86_64的依赖,并在dependencys的开头,添加org.eclipse.swt.win32.win32.x86_64的依赖

 

    
      org.eclipse.swt
      org.eclipse.swt.win32.win32.x86_64
      4.6
    
    
    
      pentaho-kettle
      kettle-core
      ${project.version}
    
...
...
    




    
      org.eclipse.equinox
      common
      ${common.version}
...
...

参考链接:https://blog.csdn.net/gongchengshiv/article/details/105211351

2.ERROR Could not parse file [plugins\kettle5-log4j-plugin\log4j.xml].

log4j:ERROR Could not parse file [plugins\kettle5-log4j-plugin\log4j.xml].
java.io.FileNotFoundException: C:\Users\ZH\Work\Gitlab\pentaho-kettle-9.1.0.0\plugins\kettle5-log4j-plugin\log4j.xml (系统找不到指定的路径。)
    at java.io.FileInputStream.open0(Native Method)
    at java.io.FileInputStream.open(FileInputStream.java:195)
    at java.io.FileInputStream.(FileInputStream.java:138)
    at java.io.FileInputStream.(FileInputStream.java:93)
    at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
    at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
    at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
原因:

源码中Spoon的方法registerUIPluginObjectTypes()中会注册环境根目录下的plugins目录,后续扫描插件jar包时会用到。
如:BasePluginType类的方法registerPluginJars()与findAnnotatedClassFiles( String x),会对plugins下的jar文件扫描。
而编译后的项目会在target目录下存在插件jar包,但是并没有在plugins目录下存在正确的插件目录如日志插件目录kettle5-log4j-plugin、json插件目录kettle-json-plugin
所以在扫描到这些jar包后初始化时找不到对应目录下的某些文件导致出错。

解决方案:

启动失败后, IDEA 会自动添加一个启动配置,点击工具栏上的启动配置【Spoon】然后点击【Edit Configurations】
配置Spoon的启动configuration,设置Working directory项为kettle-ui-swt项目的目录,其下没有plugins目录自然扫描不到jar包,不会执行日志插件初始化操作。
这样启动的kettle是没有插件的,如果想加载插件则需要将编译后的zip包的plugins和system目录拷贝至kettle-ui-swt根目录下


参考链接:https://blog.csdn.net/weixin_40479337/article/details/105402671

3.org.pentaho.ui.xul.XulException: Can not locate Xul document [ui/spoon.xul]

2022/09/09 11:56:09 - ERROR (version Unknown, build 0 from 2022/09/09 11:56:06.969 by ZH) : Error initializing transformation
2022/09/09 11:56:09 - ERROR (version Unknown, build 0 from 2022/09/09 11:56:06.969 by ZH) : org.pentaho.ui.xul.XulException: Can not locate Xul document [ui/spoon.xul]
2022/09/09 11:56:09 -   at org.pentaho.ui.xul.impl.AbstractXulLoader.findDocument(AbstractXulLoader.java:512)
2022/09/09 11:56:09 -   at org.pentaho.ui.xul.impl.AbstractXulLoader.loadXul(AbstractXulLoader.java:229)
2022/09/09 11:56:09 -   at org.pentaho.di.ui.spoon.Spoon.init(Spoon.java:869)
2022/09/09 11:56:09 -   at org.pentaho.di.ui.spoon.Spoon.createContents(Spoon.java:9335)
2022/09/09 11:56:09 -   at org.eclipse.jface.window.Window.create(Window.java:426)
2022/09/09 11:56:09 -   at org.eclipse.jface.window.Window.open(Window.java:785)
2022/09/09 11:56:09 -   at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9366)
原因:

静态xul文件未找到

解决方案:

找到根项目的assemblies目录下的子项目pdi-static的resources下的ui下的所有xul文件,复制至kettle-ui-swt项目相同位置
参考链接:https://blog.csdn.net/weixin_40479337/article/details/105402671

4.If this is live make sure a kettle-password-encoder-plugins.xml exits in the classpath

org.pentaho.di.core.exception.KettleException:
Unable to find plugin with ID 'Kettle'. If this is a test, make sure kettle-core tests jar is a dependency. If this is live make sure a kettle-password-encoder-plugins.xml exits in the classpath
at org.pentaho.di.core.encryption.Encr.init([Encr.java:61])
at org.pentaho.di.core.KettleClientEnvironment.init([KettleClientEnvironment.java:124])
at org.pentaho.di.core.KettleClientEnvironment.init([KettleClientEnvironment.java:80])
...
解决方案:

在kettle-ui-swt项目的resources文件夹下面新建文件:kettle-password-encoder-plugins.xml
文件内容:


    
        Kettle Password Encoder
        org.pentaho.di.core.encryption.KettleTwoWayPasswordEncoder
    

参考链接:https://www.zhihu.com/question/412703202

5.java.lang.NoClassDefFoundError: org/jaxen/JaxenException

2022/09/09 11:16:10 - ERROR (version Unknown, build 0 from 2022/09/09 11:16:08.147 by ZH) : Error starting Spoon shell
2022/09/09 11:16:10 - ERROR (version Unknown, build 0 from 2022/09/09 11:16:08.147 by ZH) : java.lang.NoClassDefFoundError: org/jaxen/JaxenException
2022/09/09 11:16:10 -   at org.pentaho.ui.xul.impl.AbstractXulLoader.preProcess(AbstractXulLoader.java:357)
2022/09/09 11:16:10 -   at org.pentaho.ui.xul.impl.AbstractXulLoader.loadXul(AbstractXulLoader.java:113)
2022/09/09 11:16:10 -   at org.pentaho.ui.xul.swt.SwtXulLoader.loadXul(SwtXulLoader.java:128)
2022/09/09 11:16:10 -   at org.pentaho.ui.xul.swt.SwtXulLoader.loadXul(SwtXulLoader.java:122)
2022/09/09 11:16:10 -   at org.pentaho.ui.xul.impl.AbstractXulLoader.loadXul(AbstractXulLoader.java:239)
2022/09/09 11:16:10 -   at org.pentaho.di.ui.spoon.Spoon.init(Spoon.java:869)
2022/09/09 11:16:10 -   at org.pentaho.di.ui.spoon.Spoon.createContents(Spoon.java:9335)
2022/09/09 11:16:10 -   at org.eclipse.jface.window.Window.create(Window.java:426)
2022/09/09 11:16:10 -   at org.eclipse.jface.window.Window.open(Window.java:785)
2022/09/09 11:16:10 -   at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9366)
2022/09/09 11:16:10 -   at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:711)
2022/09/09 11:16:10 - Caused by: java.lang.ClassNotFoundException: org.jaxen.JaxenException
2022/09/09 11:16:10 -   at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
2022/09/09 11:16:10 -   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
2022/09/09 11:16:10 -   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
2022/09/09 11:16:10 -   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
2022/09/09 11:16:10 -   ... 11 more
原因:

Jaxen依赖包依赖未传递成功

解决方案:

找到kettle-engine项目下的pom.xml,找到如下依赖,注释掉test域可见的配置,使其将依赖传递至kettle-ui-swt,reload一下

    
    
      jaxen
      jaxen

    

参考链接:https://blog.csdn.net/weixin_40479337/article/details/105402671

最终运行成功:

你可能感兴趣的:(kettle 9.1.0.0-324源码编译及IDEA调试运行)