在非图形界面的linux下部署kettle并使用示例

1. 下载kettle的zip包
2. 上传至linux对应的目录下
3. 将windows上.kettle目录下的所有文件copy至linux对应的目录下
4. 在linux上创建目录作为资源文件目录以便保存.ktr和.kjb文件
5. 修改repositories.xml文件中的资源库路径

6. 
   
     
       KettleFileRepository
       product-id
       product
       /usr/local/wonhigh/o2o/kettle-respository/
       N
       N
     
    

   
   
   
7. 将需要的db驱动包放入$KETTLE_HOME/lib目录下
8. 将windows上配置的db的连接配置信息放入资源库根目录下
9. 执行命令：

10. 执行转换：./data-integration/pan.sh -rep:product-id -user:admin -pass:admin -file:/usr/local/wonhigh/o2o/kettle-respository/coloth_t-skap-i2.ktr -level:Debug -logfile:/data/wonhigh/kettle-test.log
    执行job：./data-integration/kitchen.sh -rep:product-id -user:admin -pass:admin -file:/usr/local/wonhigh/o2o/kettle-respository/coloth_t-skap-i2.kjb -level:Debug -logfile:/data/wonhigh/kettle-test.log

11. 
    如何指定系统配置文件路径(kettle.properties、respositories.xml...)

12. vim /etc/profile
    
        export KETTLE_HOME=/usr/local/wonhigh/o2o/data-integration
    
    . /etc/profile
    
    cd /usr/local/wonhigh/o2o/data-integration
    
    //这里随便执行以下kettle的指令即可，不要求跑正确
    pan.sh -file= ##你会发现.kettle目录已经在$KETTLE_HOME目录下生成/usr/local/wonhigh/o2o/data-integration/.kettle/kettle.properties当然还有其他配置文件需要你手动简历，或者从开发环境拷贝了

• 集群方案

增加master1.ksl   slave1-6101.ksl  slave1-6102.ksl master-slaver-cluster.kcs等文件

        

master1.ksl：

master1${WONHIGH.MASTER.IP}${WONHIGH.MASTER.PORT}${WONHIGH.KETTLE.USER}${WONHIGH.KETTLE.PASS}Y

slave1-6101.ksl：

master1${WONHIGH.MASTER.IP}${WONHIGH.MASTER.PORT}${WONHIGH.KETTLE.USER}${WONHIGH.KETTLE.PASS}Y

slave1-6102.ksl：

slave1-6102${WONHIGH.SLAVE2.IP}${WONHIGH.SLAVE2.PORT}${WONHIGH.KETTLE.USER}${WONHIGH.KETTLE.PASS}N

master-slaver-cluster.kcs：

         
          master-slaver-cluster
          40000
          2000
          5000
          Y
          N
          
            master1
            slave1-6101
            slave1-6102
          
        

kettle.properties：

#resposity
WONHIGH.KETTLE.USER = wonhigh
WONHIGH.KETTLE.PASS = test123

#WONHIGH.KETTLE.USER = cluster
#WONHIGH.KETTLE.PASS = cluster


#master
WONHIGH.MASTER.IP = 172.17.210.95
WONHIGH.MASTER.PORT = 6100


#salve1
WONHIGH.SLAVE1.IP = 172.17.210.95
WONHIGH.SLAVE1.PORT = 6101


#salve2
WONHIGH.SLAVE2.IP = 172.17.210.95
WONHIGH.SLAVE2.PORT = 6102

kettle.pwd：
备注：密码加密使用 encr.sh -carte test123

# Please note that the default password (cluster) is obfuscated using the Encr script provided in this release
# Passwords can also be entered in plain text as before
# 
#cluster: OBF:1v8w1uh21z7k1ym71z7i1ugo1v9q
wonhigh: OBF:1m801j1u1lmp1z0f1lj11iz01m4e

kitchen.sh
Options:
  -rep            = Repository name
  -user           = Repository username
  -pass           = Repository password
  -job            = The name of the job to launch
  -dir            = The directory (dont forget the leading /)
  -file           = The filename (Job XML) to launch
  -level          = The logging level (Basic, Detailed, Debug, Rowlevel, Error, Nothing)
  -logfile        = The logging file to write to
  -listdir        = List the directories in the repository
  -listjobs       = List the jobs in the specified directory
  -listrep        = List the available repositories
  -norep          = Do not log into the repository
  -version        = show the version, revision and build date
  -param          = Set a named parameter =. For example -param:FILE=customers.csv
  -listparam      = List information concerning the defined parameters in the specified job.
  -export         = Exports all linked resources of the specified job. The argument is the name of a ZIP file.
  -custom         = Set a custom plugin specific option as a String value in the job using =, for example: -custom:COLOR=Red
  -maxloglines    = The maximum number of log lines that are kept internally by Kettle. Set to 0 to keep all rows (default)
  -maxlogtimeout  = The maximum age (in minutes) of a log line while being kept internally by Kettle. Set to 0 to keep all rows indefinitely (default)

[root@WL-APP-001 data-integration]# ./kitchen.sh -rep:product-id -job:./skap/test-skap -level:Basic -logfile:/data/wonhigh/kettle.log 
2014/07/17 10:03:52 - Kitchen - Logging is at level : 基本日志
2014/07/17 10:03:52 - Kitchen - Start of run.
2014/07/17 10:03:52 - RepositoriesMeta - Reading repositories XML file: /usr/local/wonhigh/o2o/data-integration/.kettle/repositories.xml
2014/07/17 10:03:54 - ./skap/test-skap - 开始执行任务
2014/07/17 10:03:54 - ./skap/test-skap - 开始项[coloth_t-skap-i2]
2014/07/17 10:03:54 - coloth_t-skap-i2 - Loading transformation from repository [coloth_t-skap-i2] in directory [/skap]
2014/07/17 10:03:54 - coloth_t-skap-i2 - 为了转换解除补丁开始  [coloth_t-skap-i2]
2014/07/17 10:03:54 - coloth_t-output.0 - Connected to database [I2STG] (commit=1000)
2014/07/17 10:04:17 - coloth_t.0 - Finished reading query, closing connection.
2014/07/17 10:04:17 - coloth_t.0 - 完成处理 (I=34786, O=0, R=0, W=34786, U=0, E=0
2014/07/17 10:04:25 - coloth_t-output.0 - 完成处理 (I=0, O=34786, R=34786, W=34786, U=0, E=0
2014/07/17 10:04:25 - ./skap/test-skap - 完成作业项[coloth_t-skap-i2] (结果=[true])
2014/07/17 10:04:25 - ./skap/test-skap - 任务执行完毕
2014/07/17 10:04:25 - Kitchen - Finished!
2014/07/17 10:04:25 - Kitchen - Start=2014/07/17 10:03:52.683, Stop=2014/07/17 10:04:25.704
2014/07/17 10:04:25 - Kitchen - Processing ended after 33 seconds

注：在无UI界面的情况下要使用集群必须以job为入口，在job的配置中设置