Mysql表center_pub_merchant数据导出为center_pub_merchant.tsv
mysql -h 172.xx.x.175 -P 3307 -uUSER -pPASSWORD -e "SELECT concat('merchant:',id), parentId, id, cascade_label AS cascadeLabel, cascade_level AS cascadeLevel, account_id AS accountId, merchant_name AS merchantName,merchant_type AS merchantType,merchant_project AS merchantProject, full_name AS fullName, telephone, email, thumb, industry, user_id AS userId, province,city, county FROM center_pub_merchant" awifi_dc > /opt/gofunTest/center_pub_merchant.tsv
hdfs dfs -put center_pub_merchant.tsv /tmp/center_pub_merchant.tsv
hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns=HBASE_ROW_KEY,base_data:parentId,base_data:id,base_data:cascadeLabel,base_data:cascadeLevel,base_data:accountId,base_data:merchantName,base_data:merchantType,base_data:merchantProject,base_data:fullName,base_data:telephone,base_data:email,base_data:thumb,base_data:industry,base_data:userId,base_data:province,base_data:city,base_data:county tj_base_merchant /tmp/center_pub_merchant.tsv
Mysql表center_pub_device数据导出为center_pub_device.tsv
mysql -h 172.XX.X.175 -P 3307 -uUSER -pPASSWORD -e "SELECT concat('device:',d.device_id),d.province,d.city,d.county,e.entity_type as entityType,d.device_id as deviceId,d.project_id as projectId,d.belongto FROM center_pub_device d INNER JOIN center_pub_entity e ON d.OUT_ID = e.ID WHERE d.OUT_TYPE_ID = '00'" awifi_dc > /opt/gofunTest/center_pub_device.tsv
hdfs dfs -put center_pub_device.tsv /tmp/center_pub_device.tsv
hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns=HBASE_ROW_KEY,base_data:province,base_data:city,base_data:county,base_data:entityType,base_data:deviceId,base_data:projectId,base_data:belongto tj_base_device /tmp/center_pub_device.tsv
注意：
- 若Permission denied: user=hbase, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x
修改 hadoop fs -chmod 777 /user - 若报min.user.id。。。的错
找到这个文件/etc/hadoop/conf.cloudera.yarn/container-executor.cfg
chmod 755 container-executor.cfg
将 min.user.id=900改小
chmod 400 container-executor.cfg - 若报banned.users。。。的错
chmod 755 container-executor.cfg
则将banned.users=hdfs,yarn,mapred,bin中hdfs删掉
chmod 400 container-executor.cfg