Oozie WorkFlow中sqoop的命令file案例

复制样例

[beifeng@hadoop-senior oozie-4.0.0-cdh5.3.6]$ cp -r examples/apps/sqoop oozie-apps/

样例重新命令

mv sqoop sqoop-import

复制mysql依赖包

[beifeng@hadoop-senior oozie-apps]$ mkdir sqoop-import/lib
[beifeng@hadoop-senior oozie-apps]$ cp hive-select/lib/mysql-connector-java-5.1.27-bin.jar sqoop-import/lib/

修改job.properties

nameNode=hdfs://hadoop-senior.beifeng.com:8020
jobTracker=hadoop-senior.beifeng.com:8032
queueName=default
examplesRoot=examples
oozieAppsRoot=user/beifeng/oozie-apps
oozieDataRoot=user/beifeng/oozie/datas

oozie.use.system.libpath=true

oozie.wf.application.path=${nameNode}/${oozieAppsRoot}/sqoop-import/workflow.xml
outputDir=sqoop-import-user/output

修改流程文件

    

    
        
            ${jobTracker}
            ${nameNode}
            
                
            
            
                
                    mapred.job.queue.name
                    ${queueName}
                
            
            import --options-file /opt/cdh5.3.6/oozie-4.0.0-cdh5.3.6/oozie-apps/sqoop-import/import-user.sql
        
        
        
    

    
        Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]
    
    

注意：import --options-file path的路径必须是本地路径，而不是hdfs路径 浪费我2个小时。 中的命令不是以 sqoop开头，而是以sqoop的命令开始

创建hdfs的输出目录

bin/hdfs dfs -mkdir -p /user/beifeng/oozie/datas/sqoop-import-user

上传sqoop流程到hdfs系统

bin/hdfs dfs -put ../oozie-4.0.0-cdh5.3.6/oozie-apps/sqoop-import  /user/beifeng/oozie-apps

import-user.sql

--connect 
jdbc:mysql://hadoop-senior.beifeng.com:3306/test 
--username 
root 
--password 
123456 
--table 
my_user_0321 
--num-mappers 
1  
--delete-target-dir  
--fields-terminated-by 
"\t" 
--target-dir 
/user/beifeng/oozie/datas/sqoop-import/output

运行sqoop流程job

export OOZIE_URL=http://hadoop-senior.beifeng.com:11000/oozie
bin/oozie job -config oozie-apps/sqoop-import2/job.properties  -run

查看运行结果

bin/oozie job -info JobId

JobId：0000016-180315133250705-oozie-beif-W 来自上一步运行结果