sparkSession访问hive,PARTITIONS表不可用

为分布式搭建hive on spark;

记录耗时最久的一个问题:太气人了

 

2019-08-15 18:53:03 ERROR Datastore:115 - Error thrown executing ALTER TABLE `PARTITIONS` ADD COLUMN `TBL_ID` BIGINT NULL : Table 'hive.PARTITIONS' doesn't exist
java.sql.SQLSyntaxErrorException: Table 'hive.PARTITIONS' doesn't exist

-----------------------------

2019-08-15 18:53:04 ERROR ConnectionHandle:388 - Database access problem. Killing off this connection and all remaining connections in the connection pool. SQL State = HY000
2019-08-15 18:53:04 ERROR Datastore:115 - Error thrown executing ALTER TABLE `PARTITIONS` ADD COLUMN `TBL_ID` BIGINT NULL : Table 'hive.PARTITIONS' doesn't exist
java.sql.SQLSyntaxErrorException: Table 'hive.PARTITIONS' doesn't exist

-------------------------------

2019-08-15 18:53:04 ERROR ConnectionHandle:388 - Database access problem. Killing off this connection and all remaining connections in the connection pool. SQL State = HY000
2019-08-15 18:53:04 ERROR Datastore:115 - Error thrown executing ALTER TABLE `PARTITIONS` ADD COLUMN `TBL_ID` BIGINT NULL : Table 'hive.PARTITIONS' doesn't exist
java.sql.SQLSyntaxErrorException: Table 'hive.PARTITIONS' doesn't exist


--------------------------------------

2019-08-15 18:53:05 ERROR ConnectionHandle:388 - Database access problem. Killing off this connection and all remaining connections in the connection pool. SQL State = HY000
2019-08-15 18:53:05 WARN  Query:106 - Query for candidates of org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics and subclasses resulted in no possible candidates
Schema Transaction threw exception "Add classes to Catalog "hive", Schema """
org.datanucleus.exceptions.NucleusDataStoreException: Schema Transaction threw exception "Add classes to Catalog "hive", Schema """

Caused by: java.sql.SQLException: Connection is closed!
    at com.jolbox.bonecp.ConnectionHandle.checkClosed(ConnectionHandle.java:459)
    at com.jolbox.bonecp.ConnectionHandle.getAutoCommit(ConnectionHandle.java:776)
    at org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:131)
    ... 122 more
Nested Throwables StackTrace:
java.sql.SQLException: Connection is closed!

-----------------------

以上是报错信息-------太糊人

很明显是mysql的问题,过程太愚蠢就不写了,直接上结论:

高版本的mysql.driver的jar包不兼容一些就的sql脚本,导致创建表执行失败,

解决办法:

将driver的jar,换回低版本,我换回了5.1的jar。

 

注解:本来应用的是与mysql完全对应的driver,就一直没有想到真的是它的问题。郑重警告有类似洁癖的猩猩们。

 

你可能感兴趣的:(大数据,spark,hive)