1.安装准备工作:
已经装好的 hadoop 环境是 hadoop-2.5.1 64位
下载的sqoop安装包(注意是hadoop200)
http://www.us.apache.org/dist/sqoop/1.99.4/sqoop-1.99.4-bin-hadoop200.tar.gz
2.解压文件到工作目录:
tar -xvf sqoop-1.99.3-bin-hadoop200.tar.gz
mv sqoop-1.99.4-bin-hadoop200 /usr/local/software/
3.修改环境变量:
vim /etc/profile
添加如下内容:
#sqoop
export SQOOP_HOME/usr/local/software/sqoop-1.99.4-bin-hadoop200
export PATH=$SQOOP_HOME/bin:$PATH
export CATALINA_HOME=$SQOOP_HOME/server
export LOGDIR=$SQOOP_HOME/logs
保存退出即时生效:
source /etc/profile
4.修改sqoop配置:
vim /usr/local/sqoop-1.99.4-bin-hadoop200/server/conf/sqoop.properties
#修改指向我的hadoop安装目录
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/software/hadoop-2.5.1-64/
#把hadoop目录下的jar包都引进来
vim /usr/local/software/sqoop-1.99.4-bin-hadoop200/server/conf/catalina.properties
common.loader=/usr/local/software/hadoop-2.5.1-64/share/hadoop/common/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/common/lib/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/hdfs/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/hdfs/lib/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/mapreduce/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/mapreduce/lib/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/tools/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/tools/lib/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/yarn/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/yarn/lib/*.jar,/usr/local/software/hadoop-2.5.1-64/share/hadoop/httpfs/tomcat/lib/*.jar
5.下载mysql驱动包
mysql-connector-java-5.1.16-bin.jar
并放到 /usr/local/software/sqoop-1.99.4-bin-hadoop200/server/lib/ 目录下
[注意:下载的是 mysql-5.6.13.tar.gz 只需要把里面的 mysql-connector-java.5.1.34-bin.jar 考出来即可,这是个坑啊]
6.启动/停止sqoop200
/usr/local/software/sqoop-1.99.4-bin-hadoop200/bin/sqoop.sh server start/stop
查看启动日志:
vim usr/local/software/sqoop-1.99.4-bin-hadoop200/server/logs/catalina.out
7.进入客户端交互目录
/usr/local/software/sqoop-1.99.4-bin-hadoop200/bin/sqoop.sh client
Sqoop home directory: /usr/local/software/sqoop-1.99.4-bin-hadoop200
Sqoop Shell: Type 'help' or '\h' for help.
sqoop:000>
为客户端配置服务器:set server --host hadoopMaster --port 12000 --webapp sqoop 【红色部分为 本机hostname 主机名】
Server is set successfully
sqoop:000> show version --all 查看版本信息
client version:
Sqoop 1.99.4 source revision 2475a76ef70a0660f381c75c3d47d0d24f00b57f
Compiled by gshapira on Sun Nov 16 02:50:00 PST 2014
server version:
Sqoop 1.99.4 source revision 2475a76ef70a0660f381c75c3d47d0d24f00b57f
Compiled by gshapira on Sun Nov 16 02:50:00 PST 2014
API versions:
[v1]
sqoop:000> show connector --all 显示连接器 【会显示很长一串。。。】
【注意:以下是 1.99.4版本用的命令,要是 1.99.3以下版本请看最底部链接】
sqoop:000> create link --cid 1 创建数据库连接
Creating link for connector with id 1
Please fill following values to create new link object Name: mysql --输入名称
Link configuration
JDBC Driver Class: com.mysql.jdbc.Driver --输入
JDBC Connection String: jdbc:mysql://localhost:3306/widgets
Username: root
Password: ***
JDBC Connection Properties:
There are currently 0 values in the map:
entry#
sqoop:001> create job --from 1 --to 1
To database configuration
Schema name: traceweb
Table name: widgets
Table SQL statement:
Table column names:
Stage table name:
Should clear stage table:
Throttling resources
Extractors:
Loaders:
New job was successfully created with validation status OK and persistent id 1
sqoop:001> show link 显示所有链接
sqoop:001> show job 显示所有job
sqoop:000> start job --jid 1 启动job
sqoop:000> status job --jid 1 查看导入状态
ll /usr/local/software/sqoop-1.99.4-bin-hadoop200/sqoop_output/
下面命令一看就知道是什么意思了:
sqoop:000> show job
sqoop:000> delete job --jid 1
sqoop:000> show job
sqoop:000> delete job --jid 2
sqoop:000> show job
sqoop:000> show connection
报错:The specified function "connection" is not recognized.
原因是版本不一样,SQOOP 1.99.4以后命令就变了: 具体看这里:https://sqoop.apache.org/docs/1.99.4/CommandLineClient.html#create-link-function
SQOOP 1.99.3 安装 参考页面:http://houshangxiao.iteye.com/blog/2070057