Sqoop2学习（一）—Sqoop1.99.3介绍与安装
问题导读：
1、Sqoop2 server安装需要依赖什么条件？
2、common.loader值怎么配置？
3、Sqoop默认两个端口是12000、12001
一、Sqoop2简介
Sqoop2是用来在Hadoop与结构化数据存储如关系型数据库之间进行批量数据传输的一个有效工具。Sqoop2 1.99.3目前支持将数据导入到HDFS中，期待更多的功能。
Sqoop2安装包分为两个部分：服务器和客户端，你需要在Hadoop集群上的一个节点上（一般是Master节点）上安装Sqoop服务器，客户端安装不依赖Hadoop集群。Sqoop服务端扮演着从客户端收集的入口点，同时也充当着MapReduce的客户机。
二、下载Sqoop2安装包
1、点击下载sqoop-1.99.3-bin-hadoop200.tar.gz，解压到对应目录，我的目录如下：
[hadoopUser@secondmgt sqoop-1.99.3-bin-hadoop200]$ pwd
/home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200
2、修改.bashrc，添加以下配置
#Sqoop Configure
export SQOOP_HOME=/home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200
export PATH=$PATH:$SQOOP_HOME/bin
export CATALINA_HOME=$SQOOP_HOME/server
export LOGDIR=$SQOOP_HOME/logs
1、切换到安装包的server目录下的conf中
[hadoopUser@secondmgt ~]$ cd cloud/sqoop/sqoop-1.99.3-bin-hadoop200/server/conf/
[hadoopUser@secondmgt conf]$ pwd
/home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200/server/conf
[hadoopUser@secondmgt conf]$ ls
catalina.policy catalina.properties context.xml logging.properties server.xml sqoop_bootstrap.properties sqoop.properties tomcat-users.xml web.xml
2、首先修改catalina.properties文件
修改common.loader值，将Hadoop相关包引进来，具体如下：
# common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../lib/*.jar,/usr/lib/hadoop/*.jar,/usr/lib/hadoop/lib/*.jar,/usr/lib/hadoop-hdfs/*.jar,/usr/lib/hadoop-hdfs/lib/*.jar,/usr/lib/hadoop-mapreduce/*.jar,/usr/lib/hadoop-mapreduce/lib/*.jar,/usr/lib/hadoop-yarn/*.jar,/usr/lib/hadoop-yarn/lib/*.jar
common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../lib/*.jar,/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/share/hadoop/common/*.jar,/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/share/hadoop/common/lib/*.jar,/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/share/hadoop/hdfs/*.jar,/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/share/hadoop/hdfs/lib/*.jar,/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/share/hadoop/mapreduce/*.jar,/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/share/hadoop/mapreduce/lib/*.jar,/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/share/hadoop/tools/*.jar,/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/share/hadoop/tools/lib/*.jar,/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/share/hadoop/yarn/*.jar,/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/share/hadoop/yarn/lib/*.jar,/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/share/hadoop/httpfs/tomcat/lib/*.jar
sqoop.config.provider=org.apache.sqoop.core.PropertiesConfigurationProviderDerby数据库的JDBC相关配置
# JDBC repository provider configuration
org.apache.sqoop.repository.jdbc.handler=org.apache.sqoop.repository.derby.DerbyRepositoryHandler
org.apache.sqoop.repository.jdbc.transaction.isolation=READ_COMMITTED
org.apache.sqoop.repository.jdbc.maximum.connections=10
org.apache.sqoop.repository.jdbc.url=jdbc:derby:@BASEDIR@/repository/sqoopdb;create=true
org.apache.sqoop.repository.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver
org.apache.sqoop.repository.jdbc.user=sa
org.apache.sqoop.repository.jdbc.password=
# Hadoop configuration directory
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/home/hadoopUser/cloud/hadoop/programs/hadoop-2.2.0/etc/hadoop/5、将MySQL JDBC驱动包拷贝到server/lib目录下
[hadoopUser@secondmgt lib]$ pwd
/home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200/server/lib
[hadoopUser@secondmgt lib]$ ls
annotations-api.jar catalina.jar el-api.jar jsp-api.jar tomcat-coyote.jar tomcat-i18n-fr.jar
catalina-ant.jar catalina-tribes.jar jasper-el.jar mysql-connector-java-5.1.18-bin.jar tomcat-dbcp.jar tomcat-i18n-ja.jar
catalina-ha.jar ecj-3.7.2.jar jasper.jar servlet-api.jar tomcat-i18n-es.jar
四、启动与停止Sqoop2 Server服务
1、启动
[hadoopUser@secondmgt sqoop-1.99.3-bin-hadoop200]$ sqoop.sh server start
Sqoop home directory: /home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200
Setting SQOOP_HTTP_PORT: 12000
Setting SQOOP_ADMIN_PORT: 12001
Using CATALINA_OPTS:
Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
Using CATALINA_BASE: /home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200/server
Using CATALINA_HOME: /home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200/server
Using CATALINA_TMPDIR: /home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200/server/temp
Using JRE_HOME: /home/hadoopUser/cloud/jdk/jdk1.7.0_60
Using CLASSPATH: /home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200/server/bin/bootstrap.jar使用jps查看进程，发现多了一个Bootstrap进程，启动成功
[hadoopUser@secondmgt ~]$ jps
19877 ResourceManager
19637 SecondaryNameNode
16591 org.eclipse.equinox.launcher_1.3.0.v20130327-1440.jar
27234 Jps
19424 NameNode
27149 Bootstrap2、停止
[hadoopUser@secondmgt sqoop-1.99.3-bin-hadoop200]$ sqoop.sh server stop
Sqoop home directory: /home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200
Setting SQOOP_HTTP_PORT: 12000
Setting SQOOP_ADMIN_PORT: 12001
Using CATALINA_OPTS:
Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
Using CATALINA_BASE: /home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200/server
Using CATALINA_HOME: /home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200/server
Using CATALINA_TMPDIR: /home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200/server/temp
Using JRE_HOME: /home/hadoopUser/cloud/jdk/jdk1.7.0_60
Using CLASSPATH: /home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200/server/bin/bootstrap.jar
Jan 16, 2015 12:30:24 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200/lib], exists: [false], isDirectory: [false], canRead: [false]五、Sqoop客户端安装
Sqoop客户端不需要额外的配置和安装步骤，只需要将安装包解压拷贝到指定目录，配置.bashrc文件，直接执行如下命令即可：
[hadoopUser@secondmgt ~]$ sqoop.sh client
Sqoop home directory: /home/hadoopUser/cloud/sqoop/sqoop-1.99.3-bin-hadoop200
Sqoop Shell: Type 'help' or '\h' for help.
sqoop:000>