将HDFS数据通过Sqoop导入MySQL

1.去官网下载Sqoop

2.复制一份conf/下的sqoop-env-template.sh，修改名为 sqoop-env.sh

guo@drguo1:/opt/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/conf$ cp sqoop-env-template.sh sqoop-env.sh

3.在sqoop-env.sh添加各种home

#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME=/opt/Hadoop/hadoop-2.7.2

#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/opt/Hadoop/hadoop-2.7.2

#set the path to where bin/hbase is available
export HBASE_HOME=/opt/hbase

#Set the path to where bin/hive is available
export HIVE_HOME=/opt/apache-hive-2.0.0-bin

#Set the path for where zookeper config dir is
export ZOOCFGDIR=/opt/zookeeper-3.4.8/conf

4.把MySQL驱动jar包导入lib/下

5.建表

mysql> use hive;
Database changed
mysql> create table urlrule(
    -> url varchar(1024),
    -> info varchar(50)
    -> );
Query OK, 0 rows affected (0.12 sec)

mysql> show tables;
+---------------+
| Tables_in_hive |
+---------------+
| urlrule       |
+---------------+
1 row in set (0.00 sec)

6.将数据从HDFS导入MySQL

sqoop export --connect jdbc:mysql://drguo1:3306/url --username hive --password  guo \
--table urlrule \
--export-dir /flow/topkurl/ \
--columns url \
--input-fields-terminated-by '\t'

它会转化成mapreduce任务执行。