spark的thrift-jdbcodbc-server+kerberos编程

1.官网地址

http://spark.apache.org/docs/2.3.3/sql-programming-guide.html#running-the-thrift-jdbcodbc-server

2.启动和连接测试

2.1在找到spark的sbin目录，启动start-thriftserver.sh

./start-thriftserver.sh \
--name SparkJDBC \
--master yarn \
--num-executors 2 \
--executor-memory 2g \
--executor-cores 4 \
--driver-memory 2g \
--driver-cores 2  \
--conf spark.storage.memoryFraction=0.2 \
--conf spark.shuffle.memoryFraction=0.6 \
--hiveconf hive.server2.thrift.port=10001 \
--hiveconf hive.server2.authentication.kerberos.principal=hive/[email protected] \
--hiveconf hive.server2.authentication.kerberos.keytab /home/keydir/hive.keytab

2.2beeline用bdp用户连接测试

[bdp@host66 ~]$/opt/cloudera/parcels/CDH/bin/beeline -u 'jdbc:hive2://host66:10001/smart_test;principal=hive/[email protected]' -n bdp
0: jdbc:hive2://host66:10001/smart_test> show databases;
|       databaseName       |
+--------------------------+--+
| wid_bigdata_980          |
| wid_bigdata_981          |