java代码调用kettle的一次实践
-
说明
第一次写博客，没经验，比较乱，抱歉！下面步入正文。
使用kettle版本为8.2，开始时使用7.1但是7.1中写java代码片段时出现不少问题，换成8.2问题都消失了。
-
所遇问题
- jar包问题（缺少jar包，jar包版本低）
- kettle插件问题
先上代码
package com.code;
import org.pentaho.di.core.KettleEnvironment;
import org.pentaho.di.core.logging.LogLevel;
import org.pentaho.di.core.plugins.PluginFolder;
import org.pentaho.di.core.plugins.StepPluginType;
import org.pentaho.di.job.Job;
import org.pentaho.di.job.JobMeta;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import java.util.Set;
public class ExecuteJob {
public static void main(String[] args) {
//为作业设置变量
Map map = new HashMap<>();
//手动参数(手动:1 定时:2)定时执行无需设置 默认为2
map.put("sdcs", "");
//实时上传设置流水号
map.put("ywczlsh", "");
String path = "D:\\KETTLE数据上传\\糖尿病\\糖尿病报卡\\糖尿病报告卡作业.kjb";
ExecuteJob.runJob(map, path);
}
/**
* 执行作业
* @param maps 设置作业参数
* @param jobPath Job脚本的路径
*/
private static void runJob(Map maps, String jobPath) {
try {
//在kettle环境init前，将所需要的插件加载到环境中(本作业执行只用到了pdi-xml-plugin插件,用于处理xml)
StepPluginType.getInstance().getPluginFolders().
add(new PluginFolder("D:\\TIM\\kettle8.2\\data-integration\\plugins\\pdi-xml-plugin", false, true));
//加载环境
KettleEnvironment.init();
// jobPath 是Job脚本的路径及名称
JobMeta jobMeta = new JobMeta(jobPath, null);
//初始化job
Job job = new Job(null, jobMeta);
//日志级别
job.setLogLevel(LogLevel.DEBUG);
// 向Job 脚本传递参数job.setVariable(paraname, paravalue)
Set> set=maps.entrySet();
for(Iterator> it = set.iterator(); it.hasNext();){
Map.Entry ent=it.next();
job.setVariable(ent.getKey(), ent.getValue());
}
//开始执行
job.start();
//等待结束
job.waitUntilFinished();
//判断执行是否成功
if (job.getErrors() > 0) {
throw new Exception("执行job发生异常!");
}
} catch (Exception e) {
e.printStackTrace();
}
}
} 当时由于缺少kettle中的插件一直报错
2019/02/15 10:20:35 - 添加根节点.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Can't run transformation due to plugin missing
2019/02/15 10:20:35 - 添加根节点.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : 错误初始化步骤[添加根节点]
2019/02/15 10:20:35 - 组合xml.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Can't run transformation due to plugin missing
2019/02/15 10:20:35 - 组合xml.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : 错误初始化步骤[组合xml]在网上找了相关资料，究其原因是缺少插件，于是将kettle中的插件在java代码中kettle环境初始化之前加进去，报错解决
说明（由于我的出错地方全是在xml相关的步骤，于是想起是不是缺少xml相关插件，将xml插件放进去后错误解决了。。。）
插件位置：kettle安装包中的plugins文件夹中
代码：
//在kettle环境init前，将所需要的插件加载到环境中(本作业执行只用到了pdi-xml-plugin插件,用于处理xml)
StepPluginType.getInstance().getPluginFolders().
add(new PluginFolder("D:\\TIM\\kettle8.2\\data-integration\\plugins\\pdi-xml-plugin", false, true));相关的maven依赖如下：
4.0.0
kettle-java
kettle-java
1.0-SNAPSHOT
UTF-8
UTF-8
8.2.0.0-342
commons-codec
commons-codec
1.9
commons-lang
commons-lang
2.6
commons-logging
commons-logging
1.1.3
org.apache.commons
commons-vfs2
2.3
commons-logging
commons-logging
com.google.guava
guava
17.0
pentaho-kettle
kettle-core
${kettle.version}
jug-lgpl
jug-lgpl
secondstring
secondstring
org.slf4j
slf4j-log4j12
xercesImpl
xerces
org.apache.xmlgraphics
batik-js
pentaho-kettle
kettle-engine
${kettle.version}
pentaho-kettle
metastore
${kettle.version}
pentaho-kettle
javassist
3.20.0-GA
pentaho-kettle
js
1.7R3
pentaho-kettle
jsch
0.1.54
pentaho-kettle
pentaho-vfs-browser
${kettle.version}
net.sourceforge.jtds
jtds
1.3.0
mysql
mysql-connector-java
5.1.40
junit
junit
4.12
test
org.mybatis
mybatis
3.4.6
org.apache.commons
commons-io
1.3.2
org.apache.commons
commons-lang3
3.8.1
org.slf4j
slf4j-log4j12
1.8.0-beta2
test
org.slf4j
slf4j-api
1.8.0-beta2
org.mozilla
rhino
1.7.10
org.codehaus.janino
janino
2.5.16
org.jaxen
com.springsource.org.jaxen
1.1.1
org.scannotation
scannotation
1.0.3
org.apache.httpcomponents
httpclient
4.5.3
org.apache.httpcomponents
httpcore
4.4.6
dom4j
dom4j
1.6.1
org.codehaus.janino
commons-compiler
3.0.12
org.apache.maven.plugins
maven-compiler-plugin
3.1
1.7
1.7
UTF-8
org.apache.maven.plugins
maven-surefire-plugin
2.18.1
true