hiveserver2启动：javax.jdo.JDODataStoreException: Error executing SQL query "select "DB_ID" from "DBS""
原文链接：http://www.cnblogs.com/zwgblog/p/6063993.html
启动hive-metastore和hive-server2 用beeline连接hive报错
|
1
2
3
4
5
6
7
8
|
[root
@node04
hive]# beeline
Beeline version
0.13
.
1
-cdh5.
3.0
by Apache Hive
beeline> !connect jdbc:hive2:
//172.16.145.114:10000 hive
scan complete in 3ms
Connecting to jdbc:hive2:
//172.16.145.114:10000
Enter password
for
jdbc:hive2:
//172.16.145.114:10000: ****
Error: Could not open connection to jdbc:hive2:
//172.16.145.114:10000: java.net.ConnectException: Connection refused (state=08S01,code=0)
0
: jdbc:hive2:
//172.16.145.114:10000 (closed)> !connect jdbc:hive2://172.16.145.114:10000 hive
|
确认已经赋予了hive用户在当前主机链接数据库的权限：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
mysql> use mysql;
Reading table information
for
completion of table and column names
You can turn off
this
feature to get a quicker startup with -A
Database changed
mysql> select host,user from user;
+-----------+------+
| host | user |
+-----------+------+
| % | hive |
|
127.0
.
0.1
| root |
| localhost | hive |
| localhost | root |
| node04 | root |
+-----------+------+
5
rows in set (
0.00
sec)
|
查看metastore的日志，报错信息如下：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
|
...
47
more
2016
-
11
-
14
16
:
10
:
00
,
881
ERROR [main]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:
135
)) - Self-test query [select
"DB_ID"
from
"DBS"
] failed; direc
t SQL is disabled
javax.jdo.JDODataStoreException: Error executing SQL query
"select "
DB_ID
" from "
DBS
""
.
at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:
451
)
at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:
230
)
at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.
131
)
at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:
253
)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:
224
)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:
73
)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
133
)
at org.apache.hadoop.hive.metastore.RawStoreProxy.
58
)
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:
67
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:
506
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:
484
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:
532
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:
406
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.
365
)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
55
)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:
60
)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:
4953
)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:
5174
)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:
5093
)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
57
)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
43
)
at java.lang.reflect.Method.invoke(Method.java:
606
)
at org.apache.hadoop.util.RunJar.main(RunJar.java:
212
)
NestedThrowablesStackTrace:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table
'metastore.DBS'
doesn't exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:
57
)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:
45
)
at java.lang.reflect.Constructor.newInstance(Constructor.java:
526
)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:
411
)
at com.mysql.jdbc.Util.getInstance(Util.java:
386
)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:
1052
)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:
3597
)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:
3529
)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:
1990
)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:
2151
)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:
2625
)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:
2119
)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:
2283
)
at com.jolbox.bonecp.PreparedStatementHandle.executeQuery(PreparedStatementHandle.java:
174
)
at org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeQuery(ParamLoggingPreparedStatement.java:
381
)
at org.datanucleus.store.rdbms.SQLController.executeStatementQuery(SQLController.java:
504
)
at org.datanucleus.store.rdbms.query.SQLQuery.performExecute(SQLQuery.java:
280
)
at org.datanucleus.store.query.Query.executeQuery(Query.java:
1786
)
at org.datanucleus.store.query.AbstractSQLQuery.executeWithArray(AbstractSQLQuery.java:
339
)
at org.datanucleus.store.query.Query.execute(Query.java:
1654
)
at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:
221
)
at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.
131
)
at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:
253
)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:
224
)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:
73
)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
133
)
at org.apache.hadoop.hive.metastore.RawStoreProxy.
58
)
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:
67
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:
506
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:
484
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:
532
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:
406
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.
365
)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
55
)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:
60
)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:
4953
)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:
5174
)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:
5093
)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
57
)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
43
)
at java.lang.reflect.Method.invoke(Method.java:
606
)
at org.apache.hadoop.util.RunJar.main(RunJar.java:
212
)
2016
-
11
-
14
16
:
10
:
01
,
019
WARN [main]: metastore.ObjectStore (ObjectStore.java:checkSchema(
6304
)) - Version information not found in metastore. hive.metastore.schema
.verification is not enabled so recording the schema version
0.13
.
0
2016
-
11
-
14
16
:
10
:
01
,
063
ERROR [main]: metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(
5215
)) - javax.jdo.JDODataStoreException: Required table missing :
"`VERSION`"
in Catalog
""
Schema
""
. DataNucleus requires
this
table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable
"datanucleus.autoCreateTables"
at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:
461
)
at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:
732
)
at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:
752
)
at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:
6398
)
at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:
6308
)
at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:
6286
)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
57
)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
43
)
at java.lang.reflect.Method.invoke(Method.java:
606
)
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:
108
)
at com.sun.proxy.$Proxy0.verifySchema(Unknown Source)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:
485
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:
532
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:
406
)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.
365
)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
55
)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:
60
)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:
4953
)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:
5174
)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:
5093
)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
57
)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
43
)
at java.lang.reflect.Method.invoke(Method.java:
606
)
at org.apache.hadoop.util.RunJar.main(RunJar.java:
212
)
NestedThrowablesStackTrace:|
|
由报错信息猜想hive元数据表创建失败，通过查看mysql中hive的元数据库，果真里面一张表都没有。
通过”Either your MetaData is incorrect, or you need to enable "datanucleus.autoCreateTables" 初步判断可能是因为这个属性没有设置。一顿百度后，看到一个帖子http://blog.csdn.net/eason_oracle/article/details/52273954，说是修改hive-site.xml:
照着做修改完后重启hive-metastore和hiveserver2，还是没有生成元数据表。
纠结了好久，确定了hive元数据库没有初始化是罪魁祸首，最后，得知hive可以通过命令初始化元数据库
第一次需要执行初始化命令：
schematool -dbType mysql -initSchema
查看那初始化后的信息 schematool -dbType mysql -info
进入hive的bin目录下执行schematool脚本 :./schematool -dbType mysql -initSchema
|
1
2
3
4
5
6
7
8
|
[root
@node04
bin]# ./schematool -dbType mysql -initSchema
Metastore connection URL: jdbc:mysql:
//node04:3306/metastore?createDatabaseIfNotExist=true&characterEncoding=UTF-8
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: hive
Starting metastore schema initialization to
0.13
.
0
Initialization script hive-schema-
0.13
.
0
.mysql.sql
Initialization script completed
schemaTool completed
|
查看初始化信息：
|
1
2
3
4
5
6
7
|
[root
@node04
bin]# ./schematool -dbType mysql -info
Metastore connection URL: jdbc:mysql:
//node04:3306/metastore?createDatabaseIfNotExist=true&characterEncoding=UTF-8
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: hive
Hive distribution version:
0.13
.
0
Metastore schema version:
0.13
.
0
schemaTool completed
|
查看hive的元数据库，发现表已经生成了
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
|
mysql> show tables;
+---------------------------+
| Tables_in_metastore |
+---------------------------+
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| COMPACTION_QUEUE |
| COMPLETED_TXN_COMPONENTS |
| DATABASE_PARAMS |
| DBS |
| DB_PRIVS |
| DELEGATION_TOKENS |
| FUNCS |
| FUNC_RU |
| GLOBAL_PRIVS |
| HIVE_LOCKS |
| IDXS |
| INDEX_PARAMS |
| MASTER_KEYS |
| NEXT_COMPACTION_QUEUE_ID |
| NEXT_LOCK_ID |
| NEXT_TXN_ID |
| NUCLEUS_TABLES |
| PARTITIONS |
| PARTITION_EVENTS |
| PARTITION_KEYS |
| PARTITION_KEY_VALS |
| PARTITION_PARAMS |
| PART_COL_PRIVS |
| PART_COL_STATS |
| PART_PRIVS |
| ROLES |
| ROLE_MAP |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SKEWED_COL_NAMES |
| SKEWED_COL_VALUE_LOC_MAP |
| SKEWED_STRING_LIST |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES |
| SORT_COLS
|
重启metastore和hiveserver2
beeline连接hiveserve2成功
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
[root
@node05
~]# beeline
Beeline version
0.13
.
1
-cdh5.
3.0
by Apache Hive
beeline> !connect jdbc:hive2:
//node04:10000 hive
scan complete in 4ms
Connecting to jdbc:hive2:
//node04:10000
Enter password
for
jdbc:hive2:
//node04:10000: ****
Connected to: Apache Hive (version
0.13
.
1
-cdh5.
3.0
)
Driver: Hive JDBC (version
0.13
.
1
-cdh5.
3.0
)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0
: jdbc:hive2:
//node04:10000> show tables;
+-----------+--+
| tab_name |
+-----------+--+
+-----------+--+
No rows selected (
0.261
seconds)
|
参考：http://blog.csdn.net/diudiu2025/article/details/51890673
http://blog.csdn.net/eason_oracle/article/details/52273954