hive0.14-insert、update、delete操作测试

问题导读

1.测试insert报错,该如何解决?
2.hive delete和update报错,该如何解决?
3.什么情况下才允许delete和update?






首先用最普通的建表语句建一个表:
  1. hive>create table test(id int,name string)row format delimited fields terminated by ',';  
复制代码
测试insert:

  1. insert into table test values (1,'row1'),(2,'row2');  
复制代码


结果报错:
  1. java.io.FileNotFoundException: File does not exist: hdfs://127.0.0.1:9000/home/hadoop/git/hive/packaging/target/apache-hive-0.14.0-SNAPSHOT-bin/
  2. apache-hive-0.14.0-SNAPSHOT-bin/lib/curator-client-2.6.0.jar
  3.         at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1128)
  4.         at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
  5.         at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
  6.         at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1120)
  7.         at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)
  8.         at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)
  9.         at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:99)
  10.         at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)
  11.         at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:265)
  12.         at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
  13.         at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
  14.         at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
  15.         at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
  16.         at java.security.AccessController.doPrivileged(Native Method)
  17.         ......
复制代码


貌似往hdfs上找jar包了,小问题,直接把lib下的jar包上传到hdfs
  1. hadoop fs -mkdir -p /home/hadoop/git/hive/packaging/target/apache-hive-0.14.0-SNAPSHOT-bin/apache-hive-0.14.0-SNAPSHOT-bin/lib/
  2. hadoop fs -put $HIVE_HOME/lib/* /home/hadoop/git/hive/packaging/target/apache-hive-0.14.0-SNAPSHOT-bin/apache-hive-0.14.0-SNAPSHOT-bin/lib/
复制代码


接着运行insert,没有问题,接下来测试delete
  1. hive>delete from test where id = 1;  
复制代码


报错!:
FAILED: SemanticException [Error 10294]: Attempt to do update or delete using transaction manager that does not support these operations.
说是在使用的转换管理器不支持update跟delete操作。
原来要支持update操作跟delete操作,必须额外再配置一些东西,见:
https://cwiki.apache.org/conflue ... tersforTransactions
根据提示配置hive-site.xml:


  1.     hive.support.concurrency – true
  2.     hive.enforce.bucketing – true
  3.     hive.exec.dynamic.partition.mode – nonstrict
  4.     hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager
  5.     hive.compactor.initiator.on – true
  6.     hive.compactor.worker.threads – 1
复制代码


配置完以为能够顺利运行了,谁知开始报下面这个错误:
  1. FAILED: LockException [Error 10280]: Error communicating with the metastore  
复制代码


与元数据库出现了问题,修改log为DEBUG查看具体错误:
  1. 4-11-04 14:20:14,367 DEBUG [Thread-8]: txn.CompactionTxnHandler (CompactionTxnHandler.java:findReadyToClean(265)) - Going to execute query