hive 集成sentry

环境

apache-hive-2.3.3-bin
apache-sentry-2.1.0-bin

sentry是目前最新的版本，支持hive的最高版本为2.3.3，hive版本如果高于2.3.3，会出一些版本兼容问题[亲测]

hive快速安装

wget http://mirrors.shu.edu.cn/apache/hive/hive-2.3.3/apache-hive-2.3.3-bin.tar.gz
tar -zxvf apache-hive-2.3.3-bin.tar.gz

配置hive-site.xml

 mv hive-default.xml.template hive-site.xml
 mkdir -p /home/xiaobin/soft/apache-hive-2.3.3-bin/tmpdir
 vi hive-site.xml
 
   
    system:java.io.tmpdir
    /home/xiaobin/soft/apache-hive-2.3.3-bin/tmpdir
  
  
    system:user.name
    master
  
  
    javax.jdo.option.ConnectionURL
    jdbc:mysql://192.168.1.115/hive2?createDatabaseIfNotExist=true&useUnicode=true
  
  
    javax.jdo.option.ConnectionUserName
    root
  
  
    javax.jdo.option.ConnectionPassword
    123456
  
  
    javax.jdo.option.ConnectionDriverName
    com.mysql.jdbc.Driver
  

下载驱动

yum install mysql-connector-java -y
查看安装路径
[root@cluster-master conf]# rpm -qa|grep  mysql-connector-java  
mysql-connector-java-5.1.25-3.el7.noarch
[root@cluster-master conf]# rpm -ql  mysql-connector-java-5.1.25-3.el7.noarch
/usr/share/doc/mysql-connector-java-5.1.25
/usr/share/doc/mysql-connector-java-5.1.25/CHANGES
/usr/share/doc/mysql-connector-java-5.1.25/COPYING
/usr/share/doc/mysql-connector-java-5.1.25/docs
/usr/share/doc/mysql-connector-java-5.1.25/docs/README.txt
/usr/share/doc/mysql-connector-java-5.1.25/docs/connector-j.html
/usr/share/doc/mysql-connector-java-5.1.25/docs/connector-j.pdf
/usr/share/java/mysql-connector-java.jar
/usr/share/maven-fragments/mysql-connector-java
/usr/share/maven-poms/JPP-mysql-connector-java.pom

copy mysql-connector驱动

cp /usr/share/java/mysql-connector-java.jar apache-hive-2.3.3-bin/lib/

创建元数据数据库

mysql> create database hive2;
Query OK, 1 row affected (0.01 sec)

初始化元数据

schematool -dbType mysql -initSchema

sentry安装

下载

http://sentry.apache.org/general/downloads.html
wget http://apache.01link.hk/sentry/2.1.0/apache-sentry-2.1.0-bin.tar.gz
tar -zxvf apache-sentry-2.1.0-bin.tar.gz

config

cp sentry-site.xml.service.example sentry-site.xml
vi sentry-site.xml



    sentry.hive.server
    server1
  

  
    sentry.verify.schema.version
    true
  


  
    sentry.service.allow.connect
    hive,impala,hue,hdfs
    comma separated list of users - List of users that are allowed to connect to the service (eg Hive, Impala) 
  

  
    sentry.store.jdbc.url
    jdbc:mysql://localhost:3306/sentry
    JDBC connection URL for the backed DB
  

  
    sentry.store.jdbc.user
    sentry
    The username of the user that connects to the Sentry database
  

  
    sentry.store.jdbc.password
    sentry
    Sentry password for backend JDBC user 
  

  
    sentry.service.server.keytab
    
    Keytab for service principal
  

  
    sentry.service.server.rpcport
    8038
     TCP port number for service
  

  
    sentry.service.server.rpcaddress
    0.0.0.0
     TCP interface for service to bind to
  

  
    sentry.store.jdbc.driver
    com.mysql.jdbc.Driver
    Backend JDBC driver - org.apache.derby.jdbc.EmbeddedDriver (only when dbtype = derby) JDBC Driver class for the backed DB
  
 
  
    sentry.service.admin.group
    hive,impala,hue,hdfs
    Comma separates list of groups.  List of groups allowed to make policy updates
  

  
    sentry.store.group.mapping
    org.apache.sentry.provider.common.HadoopGroupMappingService
    
	Group mapping class for Sentry service. org.apache.sentry.provider.file.LocalGroupMapping service can be used for local group mapping. 
  

  
    sentry.store.group.mapping.resource
     
     Policy file for group mapping. Policy file path for local group mapping, when sentry.store.group.mapping is set to LocalGroupMapping Service class.
  

  
    sentry.service.security.mode
    none
    Options: kerberos, none.  Authentication mode for Sentry service. Currently supports Kerberos and trusted mode 
  
 
  
    sentry.service.server.principal
     
    Service Kerberos principal
  

  
    sentry.service.web.enable
    true
    Enable web service
  

  
    sentry.service.web.authentication.type
    NONE
    Options: kerberos, NONE.  Authentication mode for Sentry web service.
  

  
    sentry.service.web.authentication.kerberos.keytab
    
    Keytab for web service principal
  

  
    sentry.service.web.authentication.kerberos.principal
    
    Web service Kerberos principal
  

  
    sentry.service.web.authentication.allow.connect.users
    
    comma separated list of users - List of users that are allowed to connect to the web service (eg Hive, Impala) 
  

创建sentry元数据数据库

Create Database sentry;
Create User sentry Identified By 'sentry';
Grant All On sentry.* To sentry@'localhost' Identified By 'sentry';
Grant All On sentry.* To sentry@'%' Identified By 'sentry';
flush privileges;

复制mysql-connector驱动

cp mysql-connector-java.jar apache-sentry-2.1.0-bin/lib/

初始化元数据

sentry --command schema-tool --conffile apache-sentry-2.1.0-bin/conf/sentry-site.xml --dbType mysql --initSchema

启动service

./sentry --command service --conffile apache-sentry-2.1.0-bin/conf/sentry-site.xml

查看是否启动成功

netstat -anpl|grep 8038
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 0.0.0.0:8038            0.0.0.0:*               LISTEN      11950/java

hive集成sentry

copy sentry 客户端配置文件

cp apache-sentry-2.1.0-bin/conf/sentry-site.xml.hive-client.example apache-hive-2.3.3-bin/conf/
cd apache-hive-2.3.3-bin/conf/
mv sentry-site.xml.hive-client.example sentry-site.xml

配置$HIVE_HOME/conf/sentry-site.xml

    sentry.service.security.mode
    none
    Options: kerberos, none.  Authentication mode for Sentry service. Currently supports Kerberos and trusted mode 
  
 

  
    sentry.service.client.server.rpc-addresses
    localhost
     TCP address of the sentry store server
  

  
    sentry.service.client.server.rpc-port
    8038
    Port # of the sentry store server
  

  
    sentry.service.client.server.rpc-connection-timeout
    200000
    Client timeout default(200000) RPC connection timeout in milisecs
  

  
    sentry.metastore.service.users
    hive
    
      Comma separated list of users
      List of service users (eg hive, impala) to bypass
      the Sentry metastore authorization. These
      services handle the metadata authorization
      on their side.
    
  



  
    sentry.hive.provider
    org.apache.sentry.provider.file.HadoopGroupResourceAuthorizationProvider
     Deprecated name: hive.sentry.provider.  Group mapping which should be used at client side
  

  
    sentry.hive.server
    server1
     Deprecated name: hive.sentry.server.  Defaut: HS2.  Hive Server2 Server identifier like "server1"
  

  
    sentry.hive.failure.hooks
     
    Deprecated Name:  hive.sentry.failure.hooks
  
  
  
        sentry.hive.testing.mode
        true
  

  
    sentry.hive.provider.backend
    org.apache.sentry.provider.db.SimpleDBProviderBackend
     Options: {org.apache.sentry.provider.db.SimpleDBProviderBackend, org.apache.sentry.provider.file.SimpleFileProviderBackend}
      Privilege provider to be used, we support file based or db based
    
  

vi hive-site.xml

    hive.metastore.pre.event.listeners
    org.apache.sentry.binding.metastore.MetastoreAuthzBinding



    hive.metastore.event.listeners
    org.apache.sentry.binding.metastore.SentrySyncHMSNotificationsPostEventListener




    hive.server2.enable.impersonation
    true



    hive.security.authorization.task.factory
     org.apache.sentry.binding.hive.SentryHiveAuthorizationTaskFactoryImpl



     hive.server2.session.hook
     org.apache.sentry.binding.hive.HiveAuthzBindingSessionHook



     hive.sentry.conf.url
     file:///home/xiaobin/soft/apache-hive-2.3.3-bin/conf/sentry-site.xml

copy sentry jars

cp apache-sentry-2.1.0-bin/lib/sentry-*.jar  apache-hive-2.3.3-bin/lib/
cp apache-sentry-2.1.0-bin/lib/shiro-*  apache-hive-2.3.3-bin/lib/

启动hiveserver2

hiveserver2 --hiveconf hive.root.logger=INFO,console

查看hiveserver2监听端口

 netstat -anpl|grep 10000
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 0.0.0.0:10000           0.0.0.0:*               LISTEN      12231/java