今天在自己的虚拟机上安装apache-hive-3.1.1时启动hive时出现了很多错误,经过不断的资料查询及测试最终可以正常运行了,特记录下,加深自己的印象分享给大家,也以便以后出现同样的错误时可以查看笔记解决。
第一条错误:
MetaException(message:Error creating transactional connection factory)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8661)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8656)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8926)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8843)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: MetaException(message:Error creating transactional connection factory)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:208)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:80)
... 11 more
原因:${HIVE_HOME}/lib目录下没有找到mysql-connector的jar包,因为我配置的源数据是MySQL数据库,然后把mysql-connector-java-5.1.27-bin.jar下载放到${HIVE_HOME}/lib目录下再次启动。发现报错如下:
MetaException(message:Hive Schema version 3.1.0 does not match metastore's schema version 1.2.0 Metastore is not upgraded or corrupt)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8661)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8656)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8926)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8843)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
原因:MySQL中的MetaStore数据库中存储的源数据版本和现在用的hive版本不匹配(我以前安装了apache-hive-1.2.1版本,然后配置文件拷贝以前的,忘记改hive-site.xml配置文件中javax.jdo.option.ConnectionURL属性的值,此属性是指定MetaStore连接地址及数据库的,将其设置成MySQL的连接地址及指定数据库就OK)
如果你只有一个hive版本,可以有以下两种解决方案:
1.登陆mysql,修改hive metastore版本:
进行mysql:mysql -uroot -p 123456
use hive; -- 配置文件中javax.jdo.option.ConnectionURL属性值指定的数据库名称
select * from version;
update VERSION set SCHEMA_VERSION='3.1.1' where VER_ID=1;
2.简单粗暴:在hvie-site.xml中关闭版本验证
<property>
<name>hive.metastore.schema.verificationname>
<value>falsevalue>
property>
我按照上面将javax.jdo.option.ConnectionURL的属性值改成另外一个数据库后启动MetaStore发现报错如下(MySQL数据库没有没有):
MetaException(message:Version information not found in metastore.)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8661)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8656)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8926)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8843)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
解决方案:将下列两个属性设置为false
<property>
<name>datanucleus.metadata.validatename>
<value>falsevalue>
property>
<property>
<name>hive.metastore.schema.verificationname>
<value>falsevalue>
property>
接下来报错:
MetaException(message:Required table missing : "`DBS`" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.schema.autoCreateTables")
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8661)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8656)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8926)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8843)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
因为在此实验环境中,hive的元数据保存在远程的mysql中,所以该错误意思是:在远程的数据库中没有找到相应的数据对象。
解决办法:将下列属性设置为true
<property>
<name>datanucleus.schema.autoCreateAllname>
<value>truevalue>
property>
这个属性的含义是:当元数据库中必要的数据对象不存在是,会自动创建。
最后提醒:在xml配置文件中,在标签之间不要有空格!!!因为读取xml文件时是按标签读取的。