PDI/Kettle-9.2.0.0-R（对应jdk1.8）源码编译问题记录及源码结构简介

第一章 前言

背景

昨天下载的Kettle源码，需要JDK11,今天按照官方发布的9.2版本，重新下载源码，地址为9.2.0.0-R/290，对应JDK1.8,至于源码下载及编译可参照PDI/Kettle-9.4.0.0-343源码下载及编译，步骤都是一样的。如果确实使用到JDK11,可以选择9.2之后的版本

目的

研究代码，尝试二开（PS:下载完代码，编译了一下，结果全是错，详见问题记录）

总体方向

把kettle任务编排提取出来做成web界面（PS:编译项目干了一天）

第二章 代码结构初识

基本结构

代码模块详情

• integration：测试代码模块，据说很多已过时，用处不大。
  
• kettle-core:PDI的核心库，包含了所有基本的数据处理功能。
  • 该模块提供了数据源连接、数据流设计、转换步骤定义、元数据管理等功能。
  • 该模块不依赖于任何特定的UI框架或数据库驱动程序，因此可以被其他应用程序以编程方式使用。
• kettle-dbdialog:对话框
  • 该模块包含了一个用于与用户交互的对话框，用于配置数据库连接参数。
  • 该模块允许用户输入数据库URL、用户名、密码和其他必要的信息。
• kettle-engine:核心引擎，执行作业
  • 该模块负责执行作业和转换。
  • 该模块包含了所有的核心逻辑，包括错误处理、日志记录、插件管理和线程调度等。
• kettle-ui-swt:图形用户界面
  • 该模块为Kettle提供了一套基于SWT（Standard Widget Toolkit）的图形用户界面。
  • 用户可以通过这个GUI来创建、编辑和运行作业和转换。
• pdi-assemblies:
  • 该模块包含了PDI的所有组件的组装和打包脚本。
  • 该模块通过这些脚本，可以将各个模块组合成一个完整的可部署的应用程序。
• pdi-engine-ext:扩展
  • 该模块包含了对引擎的扩展。
  • 这些扩展可能包括新的数据源连接器、新的步骤类型或者自定义的行为。
• pdi-plugins:插件
  • 该模块包含了各种插件，它们扩展了PDI的功能。
  • 插件可以是预定义的步骤、输入/输出插件、自定义函数库等。

⁉️问题记录

❓问题一：代码分支哪些是发布版本

❗答：后缀-R的版本

在软件开发中，-R和-RC通常代表不同的版本状态：

• -R：这个后缀可能表示“Release”，即正式发布的版本。这通常是软件经过多次测试和优化后的稳定版本，可供广大用户下载和使用。
• -RC：这个后缀通常表示“Release Candidate”，即候选发布版本。这是软件发布周期中的一个阶段，在这个阶段，软件已经基本完成开发，并且通过了内部的测试，但还没有正式发布。RC版本通常用来收集最后阶段的反馈和错误报告，以便在最终发布之前进行修复。

本文的版本9.2.0.0-290对应的就是9.2.0.0-R

❓问题二：503 Service Temporarily Unavailable and 'parent.relativePath' points at wrong local POM

[INFO] Scanning for projects...
[INFO] Downloading from pentaho-public: http://nexus.pentaho.org/content/groups/omni/org/pentaho/pentaho-ce-jar-parent-pom/9.2.0.0-290/pentaho-ce-jar-parent-pom-9.2.0.0-290.pom
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.pentaho.di:pdi:9.2.0.0-290: Could not transfer artifact org.pentaho:pentaho-ce-jar-parent-pom:pom:9.2.0.0-290 from/to pentaho-public (http://nexus.pentaho.org/content/groups/omni/): transfer failed for http://nexus.pentaho.org/content/groups/omni/org/pentaho/pentaho-ce-jar-parent-pom/9.2.0.0-290/pentaho-ce-jar-parent-pom-9.2.0.0-290.pom, status: 503 Service Temporarily Unavailable and 'parent.relativePath' points at wrong local POM @ line 16, column 11
 @ 
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.pentaho.di:pdi:9.2.0.0-290 (D:\code\opensource\ETL\pentaho-kettle-9.2.0.0-R\pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM for org.pentaho.di:pdi:9.2.0.0-290: Could not transfer artifact org.pentaho:pentaho-ce-jar-parent-pom:pom:9.2.0.0-290 from/to pentaho-public (http://nexus.pentaho.org/content/groups/omni/): transfer failed for http://nexus.pentaho.org/content/groups/omni/org/pentaho/pentaho-ce-jar-parent-pom/9.2.0.0-290/pentaho-ce-jar-parent-pom-9.2.0.0-290.pom, status: 503 Service Temporarily Unavailable and 'parent.relativePath' points at wrong local POM @ line 16, column 11 -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

❗解决方式：更换根目录下pom.xml中的仓库地址

  
  
   <repositories>
    <repository>
      <id>pentaho-publicid>
      <name>Pentaho Publicname>
      <url>https://repo.orl.eng.hitachivantara.com/artifactory/pnt-mvn/url>
      <releases>
        <enabled>trueenabled>
        <updatePolicy>dailyupdatePolicy>
      releases>
      <snapshots>
        <enabled>trueenabled>
        <updatePolicy>interval:15updatePolicy>
      snapshots>
    repository>
  repositories>

更换完地址，进行update maven

❓问题三：karaf-maven-plugin:4.2.6-R2 or one of its dependencies could not be resolved

[INFO] Scanning for projects...
[WARNING] The POM for org.hitachivantara.karaf.tooling:karaf-maven-plugin:jar:4.2.6-R2 is missing, no dependency information available
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[ERROR] Unresolveable build extension: Plugin org.hitachivantara.karaf.tooling:karaf-maven-plugin:4.2.6-R2 or one of its dependencies could not be resolved: org.hitachivantara.karaf.tooling:karaf-maven-plugin:jar:4.2.6-R2 was not found in http://maven.aliyun.com/nexus/content/groups/public/ during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of alimaven has elapsed or updates are forced @ 
[ERROR] Unknown packaging: feature @ line 9, column 14
 @ 
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.pentaho.di.plugins:pentaho-streaming-feature:9.2.0.0-290 (D:\code\opensource\ETL\pentaho-kettle-9.2.0.0-R\plugins\streaming\assemblies\feature\pom.xml) has 2 errors
[ERROR]     Unresolveable build extension: Plugin org.hitachivantara.karaf.tooling:karaf-maven-plugin:4.2.6-R2 or one of its dependencies could not be resolved: org.hitachivantara.karaf.tooling:karaf-maven-plugin:jar:4.2.6-R2 was not found in http://maven.aliyun.com/nexus/content/groups/public/ during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of alimaven has elapsed or updates are forced -> [Help 2]
[ERROR]     Unknown packaging: feature @ line 9, column 14
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/PluginManagerException

❗解决方式：阿里云仓库地址已变更，重新配置settings.xml中仓库地址

详情：https://developer.aliyun.com/mvn/guide

    
    <mirror>
      <id>aliyunmavenid>
      <mirrorOf>*mirrorOf>
      <name>阿里云公共仓库name>
      <url>https://maven.aliyun.com/repository/publicurl>
    mirror>

❓问题四：org.pentaho:pentaho-ce-jar-parent-pom:pom:9.2.0.0-290 was not found

[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.pentaho.di:pdi:9.2.0.0-290: org.pentaho:pentaho-ce-jar-parent-pom:pom:9.2.0.0-290 was not found in https://maven.aliyun.com/repository/public during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of aliyunmaven has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM @ line 16, column 11
 @ 
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.pentaho.di:pdi:9.2.0.0-290 (D:\code\opensource\ETL\pentaho-kettle-9.2.0.0-R\pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM for org.pentaho.di:pdi:9.2.0.0-290: org.pentaho:pentaho-ce-jar-parent-pom:pom:9.2.0.0-290 was not found in https://maven.aliyun.com/repository/public during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of aliyunmaven has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM @ line 16, column 11 -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

❗版本最高就只到8.1

❗setting.xml（直接按照这种方式配置，上面问题二三四应该都解决了）

<settings xmlns="http://maven.apache.org/SETTINGS/1.2.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.2.0 https://maven.apache.org/xsd/settings-1.2.0.xsd">

    <localRepository>D:\soft\apache-maven-3.8.5\apache-maven-3.8.5\repositorylocalRepository>
    <pluginGroups>pluginGroups>
    <proxies>proxies>
    <servers>servers>
    <mirrors>
        <mirror>
            <id>pentaho-publicid>
            <name>Pentaho Public Mirrorname>
            <url>https://repo.orl.eng.hitachivantara.com/artifactory/pnt-mvn/url>
            <mirrorOf>*mirrorOf>
        mirror>
        <mirror>
            <id>aliyunmavenid>
            <mirrorOf>*mirrorOf>
            <name>阿里云公共仓库name>
            <url>https://maven.aliyun.com/repository/publicurl>
        mirror>
    mirrors>
    <profiles>profiles>

settings>

❓问题五：1.7G,为什么那么大？版本之间差距这么大嘛…

下面是官网下载下来的，和9.4相差也太大了

❗ 对比pdi-ce-9.2.0.0-290和pdi-ce-9.4.0.0-343

9.2多出来的两个目录文件，占了1G

9.2的插件目录又多出160M,但是个数并不多


9.2的中的额外文件作用：文件是为KARaf容器准备的，用于扩展和配置PDI服务器的功能（嗯，不懂！）