sqoop全增量导出到mysql

全量导出
HQL示例：insert overwrite directory ‘/user/root/export/test’ row format delimited fields terminated by ‘,’ STORED AS textfile select F1,F2,F3 from ;

SQOOP脚本：sqoop export --connect jdbc:mysql://localhost:3306/wht --username root --password cloudera --table --fields-terminated-by ‘,’ --columns F1,F2,F3 --export-dir /user/root/export/test

增量导出（insert模式）
HQL示例：insert overwrite directory ‘/user/root/export/test’ row format delimited fields terminated by ‘,’ STORED AS textfile select F1,F2,F3 from where ;

SQOOP脚本：sqoop export --connect jdbc:mysql://47.99.89.210:3306/phpmanager --username datamanager --password 4duL%*iALC --table t_stock_equipment1 --fields-terminated-by ‘\001’ --update-key hp_stat_date --update-mode allowinsert --export-dir /user/hive/warehouse/cdm.db/t_stock_equipment1

更新导出（update模式）
HQL示例：insert overwrite directory ‘/user/root/export/test’ row format delimited fields terminated by ‘,’ STORED AS textfile select F1,F2,F3 from where ;

SQOOP脚本：sqoop export --connect jdbc:mysql://localhost:3306/wht --username root --password cloudera --table --fields-terminated-by ‘,’ --columns F1,F2,F3 --update-key F4 --update-mode updateonly --export-dir /user/root/export/test

注意事项：
增量导出到mysql时候，hive要根据不同的存储格式来进行设置，mysql表必须设置主键，否则增量会无效
hive
text格式：
sqoop export --connect jdbc:mysql://197.0.0.1:3306/tg --username tgg --password 4duL%*iALC --table t_stock --fields-terminated-by ‘\001’ --export-dir /user/hive/warehouse/cdm.db/t_stock --update-key hp_stat_date --update-mode allowinsert

parquet格式存储：
sqoop export --connect jdbc:mysql://197.1.0.1:3306/tg–username tgg --password 4duL%*iALC --table t_stock --input-fields-terminated-by ‘\001’ --update-key hp_stat_date --update-mode allowinsert --hcatalog-database cdm --hcatalog-table t_stock --num-mappers 1