1、ranger-hive-plugin架构说明
image.png
ranger将hive的鉴权分为了两部分
- ranger-hive-plugin-shim 通过shim的classloader加载真正的ranger实现，这样能够通过classloader做到java类环境的隔离，避免与hive的类环境冲突
- ranger-hive-plugin 真正的鉴权实现
2、ranger-plugin执行安装脚本时的操作
这一步是保证hive正确的加载hive的ranger鉴权插件
- 将hive的ranger-hive-plugin-shim-1.2.0.jar、ranger-plugin-classloader-1.2.0.jar复制到hive的lib目录
- 复制配置到$HIVE_HOME/conf目录下
image.png
hive启动会加载hiveserver2-site.xml，ranger也主要是通过该配置设置的
主要属性
hive.security.authorization.enabled:true
hive.security.authorization.manager:org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory
3、hive鉴权类加载过程示意图
3e7928b5cdf74de9bdcf534c0bc4f9ef.png
4、hive加载鉴权插件代码片段说明
4.1、Driver类的compile，
有一部分hook也在编译前触发
public int compile(String command, boolean resetTaskIds, boolean deferClose) {
PerfLogger perfLogger = SessionState.getPerfLogger(true);
perfLogger.PerfLogBegin(CLASS_NAME, PerfLogger.DRIVER_RUN);
perfLogger.PerfLogBegin(CLASS_NAME, PerfLogger.COMPILE);
stateLock.lock();
try {
driverState = DriverState.COMPILING;
} finally {
stateLock.unlock();
}
//省略。。。。
//do the authorization check
if (!sem.skipAuthorization() &&
HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_AUTHORIZATION_ENABLED)) {
try {
perfLogger.PerfLogBegin(CLASS_NAME, PerfLogger.DO_AUTHORIZATION);
//调用鉴权
doAuthorization(queryState.getHiveOperation(), sem, command);
} catch (AuthorizationException authExp) {
console.printError("Authorization failed:" + authExp.getMessage()
+ ". Use SHOW GRANT to get more details.");
errorMessage = authExp.getMessage();
SQLState = "42000";
return 403;
} finally {
perfLogger.PerfLogEnd(CLASS_NAME, PerfLogger.DO_AUTHORIZATION);
}
}
image.png
4.2、Driver类的doAuthorization
public static void doAuthorization(HiveOperation op, BaseSemanticAnalyzer sem, String command)
throws HiveException, AuthorizationException {
if("SWITCHMETASTORE".equals(op.getOperationName())) {
return;
}
//省略...
HiveAuthorizationProvider authorizer = ss.getAuthorizer();
if (op.equals(HiveOperation.CREATEDATABASE)) {
authorizer.authorize(
op.getInputRequiredPrivileges(), op.getOutputRequiredPrivileges());
} else if (op.equals(HiveOperation.CREATETABLE_AS_SELECT)
|| op.equals(HiveOperation.CREATETABLE)) {
authorizer.authorize(
db.getDatabase(SessionState.get().getCurrentDatabase()), null,
HiveOperation.CREATETABLE_AS_SELECT.getOutputRequiredPrivileges());
} else {
if (op.equals(HiveOperation.IMPORT)) {
ImportSemanticAnalyzer isa = (ImportSemanticAnalyzer) sem;
if (!isa.existsTable()) {
authorizer.authorize(
db.getDatabase(SessionState.get().getCurrentDatabase()), null,
HiveOperation.CREATETABLE_AS_SELECT.getOutputRequiredPrivileges());
}
}
}
4.3、SessionState类的getAuthorizer
public HiveAuthorizationProvider getAuthorizer() {
setupAuth();
return authorizer;
}
4.4、SessionState类的setupAuth
private void setupAuth() {
//直接加载授权类
authorizer = HiveUtils.getAuthorizeProviderManager(sessionConf,
clsStr, authenticator, true);
//通过工厂模式加载授权类
if (authorizer == null) {
// if it was null, the new (V2) authorization plugin must be specified in
// config
HiveAuthorizerFactory authorizerFactory = HiveUtils.getAuthorizerFactory(sessionConf,
HiveConf.ConfVars.HIVE_AUTHORIZATION_MANAGER);
HiveAuthzSessionContext.Builder authzContextBuilder = new HiveAuthzSessionContext.Builder();
authzContextBuilder.setClientType(isHiveServerQuery() ? CLIENT_TYPE.HIVESERVER2
: CLIENT_TYPE.HIVECLI);
authzContextBuilder.setSessionString(getSessionId());
authorizerV2 = authorizerFactory.createHiveAuthorizer(new HiveMetastoreClientFactoryImpl(),
sessionConf, authenticator, authzContextBuilder.build());
setAuthorizerV2Config();
}
5、hive鉴权的V1与V2
根据代码分析，v1是直接放在lib下的鉴权类（也要安装V1的写法）
V2是通过工厂模式创建的鉴权类ranger 是走的V2
public AuthorizationMode getAuthorizationMode(){
setupAuth();
if(authorizer != null){
return AuthorizationMode.V1;
}else if(authorizerV2 != null){
return AuthorizationMode.V2;
}
//should not happen - this should not get called before this.start() is called
throw new AssertionError("Authorization plugins not initialized!");
}
6、ranger代码的加载过程
image.png
这样处理的好处是将ranger-hive-plugin的插件环境与hive的环境做隔离，这个有很好的借鉴作用
7、ranger代码片段
7.1、ranger shim 下RangerHiveAuthorizerFactory的init方法
public void init(){
if(LOG.isDebugEnabled()) {
LOG.debug("==> RangerHiveAuthorizerFactory.init()");
}
try {
rangerPluginClassLoader = RangerPluginClassLoader.getInstance(RANGER_PLUGIN_TYPE, this.getClass());
@SuppressWarnings("unchecked")
Class cls = (Class) Class.forName(RANGER_HIVE_AUTHORIZER_IMPL_CLASSNAME, true, rangerPluginClassLoader);
activatePluginClassLoader();
rangerHiveAuthorizerFactoryImpl = cls.newInstance();
7.2、ranger shim 下RangerPluginClassLoader的getInstance
public static RangerPluginClassLoader getInstance(final String pluginType, final Class pluginClass ) throws Exception {
RangerPluginClassLoader ret = me;
if ( ret == null) {
synchronized(RangerPluginClassLoader.class) {
ret = me;
if (ret == null && pluginClass != null) {
me = ret = AccessController.doPrivileged(
new PrivilegedExceptionAction(){
public RangerPluginClassLoader run() throws Exception {
return new RangerPluginClassLoader(pluginType,pluginClass);
}
}
);
}
}
}
return ret;
}
7.3、ranger shim 下RangerPluginClassLoader的构造函数
public RangerPluginClassLoader(String pluginType, Class pluginClass ) throws Exception {
//这里getPluginFilesForServiceTypeAndPluginclass的方法将ranger-hive-impl下的包全都加载
super(RangerPluginClassLoaderUtil.getInstance().getPluginFilesForServiceTypeAndPluginclass(pluginType, pluginClass), null);
componentClassLoader = AccessController.doPrivileged(
new PrivilegedAction() {
public MyClassLoader run() {
return new MyClassLoader(Thread.currentThread().getContextClassLoader());
}
}
);
}
7.4、ranger shim 下RangerPluginClassLoaderUtil的getPluginImplLibPath
private String getPluginImplLibPath(String serviceType, Class pluginClass) throws Exception {
// 这里有个变量，这里是用ranger的classloader加载所有的 private static String rangerPluginLibDir = "ranger-hive-plugin-impl";
String ret = null;
if(LOG.isDebugEnabled()) {
LOG.debug("==> RangerPluginClassLoaderUtil.getPluginImplLibPath for Class (" + pluginClass.getName() + ")");
}
URI uri = pluginClass.getProtectionDomain().getCodeSource().getLocation().toURI();
Path path = Paths.get(URI.create(uri.toString()));
ret = path.getParent().toString() + File.separatorChar + rangerPluginLibDir.replaceAll("%", serviceType);
if(LOG.isDebugEnabled()) {
LOG.debug("<== RangerPluginClassLoaderUtil.getPluginImplLibPath for Class (" + pluginClass.getName() + " PATH :" + ret + ")");
}
return ret;
}
7.5、ranger shim 下RangerPluginClassLoaderUtil的createHiveAuthorizer
public HiveAuthorizer createHiveAuthorizer(HiveMetastoreClientFactory metastoreClientFactory,
HiveConf conf,
HiveAuthenticationProvider hiveAuthenticator,
HiveAuthzSessionContext sessionContext)
throws HiveAuthzPluginException {
HiveAuthorizer ret = null;
if(LOG.isDebugEnabled()) {
LOG.debug("==> RangerHiveAuthorizerFactory.createHiveAuthorizer()");
}
try {
activatePluginClassLoader();
//最终获取了ranger-hive-plugin-impl下的RangerHiveAuthorizerFactory，并调用了createHiveAuthorizer做到了环境隔离，真的是很赞
ret = rangerHiveAuthorizerFactoryImpl.createHiveAuthorizer(metastoreClientFactory, conf, hiveAuthenticator, sessionContext);
} finally {
deactivatePluginClassLoader();
}
if(LOG.isDebugEnabled()) {
LOG.debug("<== RangerHiveAuthorizerFactory.createHiveAuthorizer()");
}
return ret;
}