cdh5.13 spark连接hive数据源

1通过hive直接连接

cdh拷贝的客户端文件缺少相关配置需要修改hive-site.xml 添加

  
    javax.jdo.option.ConnectionURL
    jdbc:mysql://master01:3306/hive?createDatabaseIfNotExist=true
    JDBC connect string for a JDBC metastore
  
  
    javax.jdo.option.ConnectionDriverName
    com.mysql.jdbc.Driver
    Driver class name for a JDBC metastore
  
  
    javax.jdo.option.ConnectionUserName
    root
    username to use against metastore database
  
  
    javax.jdo.option.ConnectionPassword
    root
    password to use against metastore database
  

cdh5.13 spark通过hiveserver2连接hive数据源
11引入依赖包

            org.apache.hive
            hive-jdbc
             1.1.0
 
``

def main(args: Array[String]): Unit = {

Class.forName("org.apache.hive.jdbc.HiveDriver")
val conn = DriverManager.getConnection("jdbc:hive2://master01:10000", "root", "")
// val stmt = conn.prepareStatement("select * from tmp.gr_p_customer")
val sql = "show databases"
val rd1= conn.prepareStatement(sql)
val rusult = rd1.execute()
conn.close()

}

注意可能同时通过hiveserver和hive直接相互连接会报一个错误

java.lang.NoSuchFieldError: HIVE_SCRATCH_DIR_LOCK

解决方法：

pom文件中spark-hive以及hive-jdbc包冲突，注释上hive-jdbc即可