java代码调用kettle的一次实践

  • 说明

第一次写博客,没经验,比较乱,抱歉!下面步入正文。

使用kettle版本为8.2,开始时使用7.1但是7.1中写java代码片段时出现不少问题,换成8.2问题都消失了。

  • 所遇问题

  1. jar包问题(缺少jar包,jar包版本低)
  2. kettle插件问题

先上代码

package com.code;
import org.pentaho.di.core.KettleEnvironment;
import org.pentaho.di.core.logging.LogLevel;
import org.pentaho.di.core.plugins.PluginFolder;
import org.pentaho.di.core.plugins.StepPluginType;
import org.pentaho.di.job.Job;
import org.pentaho.di.job.JobMeta;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import java.util.Set;

public class ExecuteJob {
    public static void main(String[] args) {
        //为作业设置变量
        Map map = new HashMap<>();
        //手动参数(手动:1 定时:2)定时执行无需设置  默认为2
        map.put("sdcs", "");
        //实时上传设置流水号
        map.put("ywczlsh", "");
        String path = "D:\\KETTLE数据上传\\糖尿病\\糖尿病报卡\\糖尿病报告卡作业.kjb";
        ExecuteJob.runJob(map, path);
    }

    /**
     * 执行作业
     * @param maps 设置作业参数
     * @param jobPath Job脚本的路径
     */
    private static void runJob(Map maps, String jobPath) {
        try {
            //在kettle环境init前,将所需要的插件加载到环境中(本作业执行只用到了pdi-xml-plugin插件,用于处理xml)
            StepPluginType.getInstance().getPluginFolders().
                    add(new PluginFolder("D:\\TIM\\kettle8.2\\data-integration\\plugins\\pdi-xml-plugin", false, true));
            //加载环境
            KettleEnvironment.init();
            // jobPath 是Job脚本的路径及名称
            JobMeta jobMeta = new JobMeta(jobPath, null);
            //初始化job
            Job job = new Job(null, jobMeta);
            //日志级别
            job.setLogLevel(LogLevel.DEBUG);
            // 向Job 脚本传递参数job.setVariable(paraname, paravalue)
            Set> set=maps.entrySet();
            for(Iterator> it = set.iterator(); it.hasNext();){
                Map.Entry ent=it.next();
                job.setVariable(ent.getKey(), ent.getValue());
            }
            //开始执行
            job.start();
            //等待结束
            job.waitUntilFinished();
            //判断执行是否成功
            if (job.getErrors() > 0) {
                throw new Exception("执行job发生异常!");
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

当时由于缺少kettle中的插件一直报错

2019/02/15 10:20:35 - 添加根节点.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Can't run transformation due to plugin missing
2019/02/15 10:20:35 - 添加根节点.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : 错误初始化步骤[添加根节点]
2019/02/15 10:20:35 - 组合xml.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Can't run transformation due to plugin missing
2019/02/15 10:20:35 - 组合xml.0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : 错误初始化步骤[组合xml]

在网上找了相关资料,究其原因是缺少插件,于是将kettle中的插件在java代码中kettle环境初始化之前加进去,报错解决

说明(由于我的出错地方全是在xml相关的步骤,于是想起是不是缺少xml相关插件,将xml插件放进去后错误解决了。。。)

插件位置:kettle安装包中的plugins文件夹中

代码:

//在kettle环境init前,将所需要的插件加载到环境中(本作业执行只用到了pdi-xml-plugin插件,用于处理xml)
            StepPluginType.getInstance().getPluginFolders().
                    add(new PluginFolder("D:\\TIM\\kettle8.2\\data-integration\\plugins\\pdi-xml-plugin", false, true));

相关的maven依赖如下:



    4.0.0

    kettle-java
    kettle-java
    1.0-SNAPSHOT
    
        
        UTF-8
        UTF-8

        
        8.2.0.0-342
    

    
        
            commons-codec
            commons-codec
            1.9
        
        
            commons-lang
            commons-lang
            2.6
        
        
            commons-logging
            commons-logging
            1.1.3
        
        
            org.apache.commons
            commons-vfs2
            2.3
            
                
                    commons-logging
                    commons-logging
                
            
        
        
            com.google.guava
            guava
            17.0
        
        
        
            pentaho-kettle
            kettle-core
            ${kettle.version}
            
                
                    jug-lgpl
                    jug-lgpl
                
                
                    secondstring
                    secondstring
                
                
                    org.slf4j
                    slf4j-log4j12
                
                
                    xercesImpl
                    xerces
                
                
                    org.apache.xmlgraphics
                    batik-js
                
            
        
        
            pentaho-kettle
            kettle-engine
            ${kettle.version}
        
        
            pentaho-kettle
            metastore
            ${kettle.version}
        
        
            pentaho-kettle
            javassist
            3.20.0-GA
        
        
            pentaho-kettle
            js
            1.7R3
        
        
            pentaho-kettle
            jsch
            0.1.54
        
        
            pentaho-kettle
            pentaho-vfs-browser
            ${kettle.version}
        
        
        
            net.sourceforge.jtds
            jtds
            1.3.0
        
        
            mysql
            mysql-connector-java
            5.1.40
        
        
        
            junit
            junit
            4.12
            test
        
        
            org.mybatis
            mybatis
            3.4.6
        
        
        
            org.apache.commons
            commons-io
            1.3.2
        
        
        
            org.apache.commons
            commons-lang3
            3.8.1
        
        
        
            org.slf4j
            slf4j-log4j12
            1.8.0-beta2
            test
        
        
        
            org.slf4j
            slf4j-api
            1.8.0-beta2
        
        
        
            org.mozilla
            rhino
            1.7.10
        
        
        
            org.codehaus.janino
            janino
            2.5.16
        
        
        
            org.jaxen
            com.springsource.org.jaxen
            1.1.1
        
        
        
            org.scannotation
            scannotation
            1.0.3
        
        
        
            org.apache.httpcomponents
            httpclient
            4.5.3
        
        
        
            org.apache.httpcomponents
            httpcore
            4.4.6
        
        
        
            dom4j
            dom4j
            1.6.1
        
        
            org.codehaus.janino
            commons-compiler
            3.0.12
        
    

    
    
        
        
            
            
                org.apache.maven.plugins
                maven-compiler-plugin
                3.1
                
                    1.7
                    1.7
                    UTF-8
                
            
            
            
                org.apache.maven.plugins
                maven-surefire-plugin
                2.18.1
                
                    true
                
            
        
    


 

你可能感兴趣的:(java,SqlServer,kettle)