【Sqoop1】Sqoop1实战之导出Sqoop export
承接上篇，本篇主要演示Sqoop1的导出，涉及到的命令是Sqoop export。
1 查看export帮助信息
[hadoop@strong ~]$ sqoop help export
2 默认导出
1）在Mysql创建表
mysql> create table lang as select *from language where 1=2;
Query OK, 0 rows affected (1.56 sec)
Records: 0 Duplicates: 0 Warnings: 0
2）查看HDFS文件内容
[hadoop@strong ~]$ hdfs dfs -cat /user/sqoop1/language/part-r-00000
1,2006-02-15 05:02:19.0,English
2,2006-02-15 05:02:19.0,Italian
3,2006-02-15 05:02:19.0,Japanese
4,2006-02-15 05:02:19.0,Mandarin
5,2006-02-15 05:02:19.0,French
6,2006-02-15 05:02:19.0,German
7,2018-06-26 16:51:34.0,Chinese
8,2018-06-26 17:54:54.0,GD
3）执行导出命令
[hadoop@strong ~]$ sqoop export --connect jdbc:mysql://strong.hadoop.com:3306/sakila --username root --password root --table lang --export-dir /user/sqoop1/language
注：由于表没有主键，重复执行后会出现重复记录，避免此情况，可设主键。
4）查看导出结果
mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name | last_update |
+-------------+----------+---------------------+
| 1 | English | 2006-02-15 05:02:19 |
| 2 | Italian | 2006-02-15 05:02:19 |
| 3 | Japanese | 2006-02-15 05:02:19 |
| 4 | Mandarin | 2006-02-15 05:02:19 |
| 5 | French | 2006-02-15 05:02:19 |
| 6 | German | 2006-02-15 05:02:19 |
| 7 | Chinese | 2018-06-26 16:51:34 |
| 8 | GD | 2018-06-26 17:54:54 |
+-------------+----------+---------------------+
8 rows in set (0.00 sec)
3 以update_key方式导出
1）修改数据
mysql> update lang set name='GuangDongHua',last_update=current_timestamp where language_id=8;
Query OK, 1 row affected (0.45 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select *from lang order by 1;
+-------------+--------------+---------------------+
| language_id | name | last_update |
+-------------+--------------+---------------------+
| 1 | English | 2006-02-15 05:02:19 |
| 2 | Italian | 2006-02-15 05:02:19 |
| 3 | Japanese | 2006-02-15 05:02:19 |
| 4 | Mandarin | 2006-02-15 05:02:19 |
| 5 | French | 2006-02-15 05:02:19 |
| 6 | German | 2006-02-15 05:02:19 |
| 7 | Chinese | 2018-06-26 16:51:34 |
| 8 | GuangDongHua | 2018-06-27 12:05:22 |
+-------------+--------------+---------------------+
8 rows in set (0.01 sec)
2）执行导出
[hadoop@strong ~]$ sqoop export --connect jdbc:mysql://strong.hadoop.com:3306/sakila --username root --password root --table lang --export-dir /user/sqoop1/language --update-key language_id
3）查看数据
mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name | last_update |
+-------------+----------+---------------------+
| 1 | English | 2006-02-15 05:02:19 |
| 2 | Italian | 2006-02-15 05:02:19 |
| 3 | Japanese | 2006-02-15 05:02:19 |
| 4 | Mandarin | 2006-02-15 05:02:19 |
| 5 | French | 2006-02-15 05:02:19 |
| 6 | German | 2006-02-15 05:02:19 |
| 7 | Chinese | 2018-06-26 16:51:34 |
| 8 | GD | 2018-06-26 17:54:54 |
+-------------+----------+---------------------+
8 rows in set (0.03 sec)
注：以更新模式对数据进行了更新。
4 指定update-mode为updateonly方式导出
1）修改数据
mysql> delete from lang where language_id=8;
Query OK, 1 row affected (0.26 sec)
mysql> update lang set name='CN',last_update=current_timestamp where language_id=7;
Query OK, 1 row affected (0.12 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name | last_update |
+-------------+----------+---------------------+
| 1 | English | 2006-02-15 05:02:19 |
| 2 | Italian | 2006-02-15 05:02:19 |
| 3 | Japanese | 2006-02-15 05:02:19 |
| 4 | Mandarin | 2006-02-15 05:02:19 |
| 5 | French | 2006-02-15 05:02:19 |
| 6 | German | 2006-02-15 05:02:19 |
| 7 | CN | 2018-06-27 12:16:57 |
+-------------+----------+---------------------+
7 rows in set (0.00 sec)
2）进行导出
[hadoop@strong ~]$ sqoop export --connect jdbc:mysql://strong.hadoop.com:3306/sakila --username root --password root --table lang --export-dir /user/sqoop1/language --update-key language_id --update-mode updateonly
3）查看数据
mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name | last_update |
+-------------+----------+---------------------+
| 1 | English | 2006-02-15 05:02:19 |
| 2 | Italian | 2006-02-15 05:02:19 |
| 3 | Japanese | 2006-02-15 05:02:19 |
| 4 | Mandarin | 2006-02-15 05:02:19 |
| 5 | French | 2006-02-15 05:02:19 |
| 6 | German | 2006-02-15 05:02:19 |
| 7 | Chinese | 2018-06-26 16:51:34 |
+-------------+----------+---------------------+
7 rows in set (0.06 sec)
注：发现以updateonly方式导出时，只更新，不会插入新行。
5 指定update-mode为allowinsert方式导出
1）修改数据
mysql> update lang set name='CN',last_update=current_timestamp where language_id=7;
Query OK, 1 row affected (0.14 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name | last_update |
+-------------+----------+---------------------+
| 1 | English | 2006-02-15 05:02:19 |
| 2 | Italian | 2006-02-15 05:02:19 |
| 3 | Japanese | 2006-02-15 05:02:19 |
| 4 | Mandarin | 2006-02-15 05:02:19 |
| 5 | French | 2006-02-15 05:02:19 |
| 6 | German | 2006-02-15 05:02:19 |
| 7 | CN | 2018-06-27 12:22:51 |
+-------------+----------+---------------------+
7 rows in set (0.01 sec)
2）进行导出
[hadoop@strong ~]$ sqoop export --connect jdbc:mysql://strong.hadoop.com:3306/sakila --username root --password root --table lang --export-dir /user/sqoop1/language --update-key language_id --update-mode allowinsert
3）查看数据
mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name | last_update |
+-------------+----------+---------------------+
| 1 | English | 2006-02-15 05:02:19 |
| 2 | Italian | 2006-02-15 05:02:19 |
| 3 | Japanese | 2006-02-15 05:02:19 |
| 4 | Mandarin | 2006-02-15 05:02:19 |
| 5 | French | 2006-02-15 05:02:19 |
| 6 | German | 2006-02-15 05:02:19 |
| 7 | Chinese | 2018-06-26 16:51:34 |
| 8 | GD | 2018-06-26 17:54:54 |
+-------------+----------+---------------------+
8 rows in set (0.00 sec)
注：以allowinsert方式作为更新模式时，可以进行更新和插入操作。