【转】sqoop2 1.99.3安装

【转自】www.68idc.cn/help/jiabenmake/qita/2014042391332.html
下载的sqoop包为sqoop-1.99.3-bin-hadoop200,下载地址：

1.解压文件到工作目录:

tar -xzvf sqoop-1.99.3-bin-hadoop200.tar.gz

mv sqoop-1.99.3-bin-hadoop200 /usr/app/sqoop


2.修改环境变量：

vim /etc/profile

增加SQOOP主目录

export SQOOP_HOME=/use/app/sqoop

export PATH=$PATH:$SQOOP_HOME/bin

export CATALINA_BASE=$SQOOP_HOME/server

export LOGDIR=$SQOOP_HOME/logs/


即时生效：

source /etc/profile


3.修改sqoop配置：

vi server/conf/sqoop.properties

将org.apache.sqoop.submission.engine.mapreduce.configuration.directory后面hadoop的位置修改为自己安装的hadoop配置文件位置，我的为:/usr/app/hadoop-2.2.0/etc/hadoop/


4.修改sqoop读取hadoop的jar包的路径：

vi /sqoop/server/conf/catalina.properties

将common.loader行后的/usr/lib/hadoop/lib/*.jar改成自己的hadoop jar 包目录，我的为：

/usr/app/hadoop-2.2.0/share/hadoop/common/*.jar,

/usr/app/hadoop-2.2.0/share/hadoop/common/lib/*.jar,

/usr/app/hadoop-2.2.0/share/hadoop/hdfs/*.jar,

/usr/app/hadoop-2.2.0/share/hadoop/hdfs/lib/*.jar,

/usr/app/hadoop-2.2.0/share/hadoop/mapreduce/*.jar,

/usr/app/hadoop-2.2.0/share/hadoop/mapreduce/lib/*.jar,

/usr/app/hadoop-2.2.0/share/hadoop/tools/*.jar,

/usr/app/hadoop-2.2.0/share/hadoop/tools/lib/*.jar,

/usr/app/hadoop-2.2.0/share/hadoop/yarn/*.jar,

/usr/app/hadoop-2.2.0/share/hadoop/yarn/lib/*.jar

到此sqoop就基本配置完成可以直接运行./bin/sqoop.sh server start启动sqoop了


5.使用SQOOP：


启动SQOOP:

$./bin/sqoop.sh server start

停止SQOOP：

$./bin/sqoop.sh server stop


进入客户端交互模式:

$./bin/sqoop.sh client

Sqoop home directory: /usr/lib/sqoop

Sqoop Shell: Type 'help' or '\h' for help.


sqoop:000>


为客户端配置服务器：

sqoop:000> set server --host master --port 12000 --webapp sqoop

Server is set successfully


查看版本信息：

sqoop:000> show version --all

client version:

Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b

Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013

server version:

Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b

Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013

Protocol version:

[1]


显示连接器：

sqoop:000> show connector --all

1 connector(s) to show:

Connector with id 1:

Name: generic-jdbc-connector

Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector

Supported job types: [EXPORT, IMPORT]
...

创建数据库连接

sqoop:000> create connection --cid 1 

Creating connection for connector with id 1 

Please fill following values to create new connection object 

Name: Mysql-H216 


Connection configuration 


JDBC Driver Class: com.mysql.jdbc.Driver 

JDBC Connection String: jdbc:mysql://192.168.1.120:3306/test

Username: admin 

Password: ***** 

JDBC Connection Properties: 

There are currently 0 values in the map: 

entry# 


Security related configuration options 


Max connections: 100 


New connection was successfully created with validation status FINE and persistent id 1


创建导入任务

sqoop:000> create job --xid 1 --type import 

Creating job for connection with id 1 

Please fill following values to create new job object 

Name: HeartBeat 


Database configuration 


Schema name: mic_db_out 

Table name: t_heart_beat 

Table SQL statement: 

Table column names: 

Partition column name: 

Nulls in partition column: 

Boundary query: 


Output configuration 


Storage type: 

0 : HDFS 

Choose: 0 

Output format: 

0 : TEXT_FILE 

1 : SEQUENCE_FILE 

Choose: 0 

Compression format: 

0 : NONE 

1 : DEFAULT 

2 : DEFLATE 

3 : GZIP 

4 : BZIP2 

5 : LZO 

6 : LZ4 

7 : SNAPPY 

Choose: 0 

Output directory: /user/jarcec/users


Throttling resources 


Extractors: 

Loaders: 

New job was successfully created with validation status FINE  and persistent id 1 

sqoop:000>


查看导入状态：

sqoop:000> status job --jid 1 

Submission details 

Job ID: 1 

Server URL: :12000/sqoop/ 

Created by: dev 

Creation date: 2014-04-19 18:54:25 CST 

Lastly updated by: dev 

External ID: job_local1638775039_0002 

2014-04-19 18:54:50 CST: UNKNOWN 


相关命令：

启动任务同步执行:start job --jid 1 -s

显示任务：status job --jid 1

显示所有任务：show job -a

停止任务：stop job --jid 1

克隆连接：clone connection --xid 1

克隆任务：clone job --jid 1

删除连接：delete connection --xid 1