Spark之thriftserver/beeline的使用

  1. 启动thriftserver: 默认端口是10000 ,可以修改
  2. 启动beeline
    beeline -u jdbc:hive2://localhost:10000 -n hadoop

修改thriftserver启动占用的默认端口号:
./start-thriftserver.sh
--master local[2]
--jars ~/software/mysql-connector-java-5.1.27-bin.jar
--hiveconf hive.server2.thrift.port=14000

beeline -u jdbc:hive2://localhost:14000 -n hadoop

thriftserver和普通的spark-shell/spark-sql有什么区别?
1)spark-shell、spark-sql都是一个spark application;
2)thriftserver, 不管你启动多少个客户端(beeline/开发环境code),永远都是一个spark application
解决了一个数据共享的问题,多个客户端可以共享数据;

开发时代码可以这么写:


        org.spark-project.hive
        hive-jdbc
        1.2.1.spark2

    Class.forName("org.apache.hive.jdbc.HiveDriver")
    val conn = DriverManager.getConnection("jdbc:hive2://hadoop001:14000","hadoop","")
    val pstmt = conn.prepareStatement("select empno,ename,sal from emp")
    val rs = pstmt.executeQuery()
    while(rs.next()){
      println("empno:"+rs.getInt("empno")+" ,ename:"+rs.getString("ename")+" ,sal:"+rs.getDouble("sal"))
    }

注意事项:在使用jdbc开发时,一定要先启动thriftserver
Exception in thread "main" java.sql.SQLException:
Could not open client transport with JDBC Uri: jdbc:hive2://hadoop001:14000:
java.net.ConnectException: Connection refused

你可能感兴趣的:(Spark之thriftserver/beeline的使用)