sqoop-1.4.同步数据到hdfs
sqoop list-databases -connect jdbc:mysql://192.168.1.133:3306/ --username root -password root
2. 显示所有表
sqoop list-tables --connect jdbc:mysql://192.168.1.133:3306/ue_incas --username root -P
3. 数据从mysql导入hdfs
sqoop import --connect jdbc:mysql://192.168.1.133:3306/ue_incas --username root --P --table sys_param --target-dir /mysqltohdfs2/sys_param -m 4
mysql导入hdfs时注意事项：
(1) --target-dir /mysqltohdfs2/sys_param 中的 “sys_param”目录不能存在；
(2) 异常信息：
Application application_1531365865313_0001 failed 2 times due to AM Container for appattempt_1531365865313_0001_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://localhost:8088/cluster/app/application_1531365865313_0001Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1531365865313_0001_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:585)
at org.apache.hadoop.util.Shell.run(Shell.java:482)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:776)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
解决方案：
（1）检查 core-site.xml、mapred-site.xml中配置的文件路径是否正确
mapred-site.xml中的日志路径：
中的 “/hadoop_log” 是通过 hdfs dfs -mkdir /hadoop_log 命令创建的，非linux中文件系统的路径
（2）yarn-site.xml中的参数 yarn.application.classpath 需要注释
4. hadoop-2.7.6的配置文件
（1） core-site.xml
(2) hdfs-site.xml
(3) yarn-site.xml
(4) mapred-site.xml
5. sqoop1.4.X 配置文件sqoop-env.sh：
export HADOOP_COMMON_HOME=/opt/modules/hadoop-2.7.6
export HADOOP_MAPRED_HOME=/opt/modules/hadoop-2.7.6
#export HBASE_HOME=
#export HIVE_HOME=
export ZOOCFGDIR=/opt/modules/zookeeper-3.4.12
6. 下载地址：
（1）hadoop2.7.6：http://mirrors.hust.edu.cn/apache/
（hadoop-2.6.0-cdh5.9.3.tar.gz下载地址：http://archive.cloudera.com/cdh5/cdh/5/）
（2）sqoop下载地址：http://mirrors.hust.edu.cn/apache/sqoop/
（3）zookeeper下载地址：https://www.apache.org/dyn/closer.cgi/zookeeper/
（4）JDK下载地址：http://www.oracle.com/technetwork/java/javase/downloads/index.html
（5）storm下载地址：http://storm.apache.org/downloads.html
7. sqoop导入Mysql到hdfs其他方式
（1）指定分隔符和导入路径
sqoop import \
--connect jdbc:mysql://hadoop1:3306/mysql \
--username root \
--password root \
--table help_keyword \
--target-dir /user/hadoop11/my_help_keyword1 \
--fields-terminated-by '\t' \
-m 2
（2）带where条件
sqoop import \
--connect jdbc:mysql://hadoop1:3306/mysql \
--username root \
--password root \
--where "name='STRING' " \
--table help_keyword \
--target-dir /sqoop/hadoop11/myoutport1 \
-m 1
（3）查询指定列
sqoop import \
--connect jdbc:mysql://hadoop1:3306/mysql \
--username root \
--password root \
--columns "name" \
--where "name='STRING' " \
--table help_keyword \
--target-dir /sqoop/hadoop11/myoutport22 \
-m 1
selct name from help_keyword where name = "string"
（4）指定自定义查询SQL
sqoop import \
--connect jdbc:mysql://hadoop1:3306/ \
--username root \
--password root \
--target-dir /user/hadoop/myimport33_1 \
--query 'select help_keyword_id,name from mysql.help_keyword where $CONDITIONS and name = "STRING"' \
--split-by help_keyword_id \
--fields-terminated-by '\t' \
-m 4