StreamSet mysql数据导入到hive异常问题记录
1. 异常问题Permission denied: user [anonymous] does not have [USE] privilege on [default]
com.streamsets.pipeline.api.base.OnRecordErrorException: HIVE_23 - TBL Properties 'com.streamsets.pipeline.stage.lib.hive.exceptions.HiveStageCheckedException: HIVE_20 - Error executing SQL: DESCRIBE DATABASE `default`, Reason:Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [anonymous] does not have [USE] privilege on [default]' Mismatch: Actual: {} , Expected: {}
at com.streamsets.pipeline.stage.processor.hive.HiveMetadataProcessor.process(HiveMetadataProcessor.java:585)
at com.streamsets.pipeline.api.base.RecordProcessor.process(RecordProcessor.java:52)
at com.streamsets.pipeline.api.base.configurablestage.DProcessor.process(DProcessor.java:35)
at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:286)
at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:235)
at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:298)
at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:244)
解决办法：
在集群的ranger中为hive服务添加{user} 群组的权限。
2.异常问题二
using Storage Format Type org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, but Avro requested instead
原因：因为Steamsets中配置的hive metadata存在格式是AVRO格式，而实际上创建的表格式和AVRO格式不一致。
解决办法：删掉之前的表，重新建一个存储格式是AVRO的表
CREATE TABLE `test3`(
`id` int,
`name` string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n'
STORED AS AVRO
LOCATION
'hdfs://ns1/apps/hive/warehouse/test3'；