Kerberos
Kerberos搭建（基于Ambari环境）
搭建KDC
- 安装
yum install krb5-server krb5-libs krb5-workstation -
修改配置文件
vi /etc/krb5.conf[libdefaults] renew_lifetime = 7d forwardable = true default_realm = EXAMPLE.COM ticket_lifetime = 24h dns_lookup_realm = false dns_lookup_kdc = false default_ccache_name = /tmp/krb5cc_%{uid} #default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5 #default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5 [logging] default = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log kdc = FILE:/var/log/krb5kdc.log [realms] EXAMPLE.COM = { admin_server = master.hadoop//此处是你的主机名 kdc = master.hadoop//此处是你的主机名 } - 创建Kerberos数据库
kdb5_util create -s
终端会提示您输入密码，该密码管理Kerberos数据库必须
- 启动KDC
systemctl start krb5kdcsystemctl start kadmin - 启动自动开机服务
systemctl enable krb5kdcsystemctl enable kadmin - 创建Kerberos管理员
kadmin.local -q "addprinc admin/admin" - 重启kadmin进程
systemctl restart kadmin
安装JCE
-
获取适用于集群中JDK版本的JCE策略文件
- 对于Oracle JDK 1.8：
http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html
- 对于Oracle JDK 1.7：
http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html- 解压到安装的jdk中（此步骤每台集群都要安装！谨记！切记！）
unzip -o -j -q jce_policy-8.zip -d /opt/jdk1.8.0_111/jre/lib/security/ - 重启ambari服务器
ambari-server restart
进入Ambari向导启用Kerbores
我是做完了才截图的，Kadmin少了两个配置选项，从网上补截一张补进来吧！
一路next即可。
Kerbores使用
- kadmin.local与kadmin
kadmin.local和kadmin至于用哪个，取决于账户和访问权限:
kadmin.local（on the KDC machine）or kadmin （on others machine）
如果有访问kdc服务器的root权限，但是没有kerberos admin账户，使用kadmin.local
如果没有访问kdc服务器的root权限，但是有kerberos admin账户，使用kadmin -
添加票据
$ kadmin.local addprinc -randkey test/[email protected] xst -norandkey -k /etc/security/keytabs/test.service.keytab test/[email protected] -
获取票据信息
kadmin.local: getprinc test/[email protected] Principal: test/[email protected] Expiration date: [never] Last password change: Wed Apr 03 16:05:50 CST 2019 Password expiration date: [none] Maximum ticket life: 1 day 00:00:00 Maximum renewable life: 0 days 00:00:00 Last modified: Wed Apr 03 16:05:50 CST 2019 (ljk/[email protected]) Last successful authentication: [never] Last failed authentication: [never] Failed password attempts: 0 Number of keys: 8 Key: vno 1, aes256-cts-hmac-sha1-96 Key: vno 1, aes128-cts-hmac-sha1-96 Key: vno 1, des3-cbc-sha1 Key: vno 1, arcfour-hmac Key: vno 1, camellia256-cts-cmac Key: vno 1, camellia128-cts-cmac Key: vno 1, des-hmac-sha1 Key: vno 1, des-cbc-md5 MKey: vno 1 Attributes: Policy: [none] - 列出KDC所有票据
listprincs - 删除票据
delprinc test/[email protected] - 修改属性
modprinc -maxlife 30days test/[email protected] -
缓存票据
klist -k -t /etc/security/keytabs/test.service.keytab kinit -k -t /etc/security/keytabs/test.service.keytab test/[email protected] kinit -k -t /etc/security/keytabs/test.service.keytab -c /tmp/testkeytab test/[email protected] - 更新票据
kinit -R - 查看或者删除用户缓存的票据
klistkdestroy -
合并票据
ktutil ktutil: rkt test.service.keytab ktutil: rkt test1.service.keytab ktutil: wkt test-test1.service.keytab
client如何访问Kerbores的HDP集群
网上大量方法都是自己去生成一张票据，使用这张票据作为client验证基础的。
但是事实上HDP已经为集群生成了大量票据和keytab,可以直接使用的。
接下来就来验证一下。
-
先查看下HDP生成的Principal
可以看到HDP生成的Principal非常规范，以 $服务/$主机名的方式。
kadmin.local: listprincs HTTP/[email protected] HTTP/[email protected] HTTP/[email protected] K/[email protected] admin/[email protected] [email protected] [email protected] amshbase/[email protected] amshbase/[email protected] amszk/[email protected] dn/[email protected] dn/[email protected] [email protected] hbase/[email protected] hbase/[email protected] [email protected] hive/[email protected] hive/[email protected] hive/[email protected] jhs/[email protected] [email protected] jn/[email protected] jn/[email protected] jn/[email protected] kadmin/[email protected] kadmin/[email protected] kadmin/[email protected] kiprop/[email protected] krbtgt/[email protected] nm/[email protected] nm/[email protected] nn/[email protected] nn/[email protected] rm/[email protected] yarn/[email protected] zookeeper/[email protected] zookeeper/[email protected] zookeeper/[email protected] - 下载Active NN的keytab
在配置信息可以找到它的位置，位于/etc/security/keytabs下，找到nn.service.keytab,并下载到本地。 -
java写第三方Client访问
现在hadoop集群有了Kerbores的保护，你按照往常访问，是会报错的。Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]在往常基础上加上验证就可以了。
代码如下：
public static void main(String[] args) throws IOException { final String USER_KEY = "nn/nn2.ambari"; final String KEY_TAB_PATH = "/Users/LJK/Downloads/nn.service.keytab"; Configuration conf = new Configuration(); System.setProperty("java.security.krb5.conf", "/Users/LJK/Downloads/krb5.conf"); // System.setProperty("sun.security.krb5.debug", "true"); conf.set("fs.defaultFS", "hdfs://mycluster:8020"); conf.set("hadoop.security.authentication", "KERBEROS"); conf.set("dfs.client.block.write.replace-datanode-on-failure.policy", "NEVER"); UserGroupInformation.setConfiguration(conf); UserGroupInformation.loginUserFromKeytab(USER_KEY, KEY_TAB_PATH); FileSystem fileSystem = FileSystem.get(conf); FileStatus[] fileStatuses = fileSystem.listStatus(new Path("/LJKTEST")); for (FileStatus fileStatus : fileStatuses) { Path path = fileStatus.getPath(); System.out.println(path.toString()); } }这里有个坑，我一直以为keytab文件放到resource目录下，就可以直接访问。会一直报错，这个和你把这个路径地址赋值空是一样的道理，报错也比较恶心，不说文件找不到，一直让我以为是权限的问题。有点坑~
Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user
访问Kerberos HBase集群
@Before
public void init() throws IOException {
BasicConfigurator.configure();
final String USER_KEY = "hbase/[email protected]";
String keyTabPath = Objects.requireNonNull(
HBaseSoulTest.class.getClassLoader().getResource("hbase.service.keytab")).getPath();
String krb5Path = Objects.requireNonNull(
HBaseSoulTest.class.getClassLoader().getResource("krb5.conf")).getPath();
System.setProperty("java.security.krb5.conf", krb5Path);
Configuration conf = HBaseConfiguration.create();
conf.set("hbase.master.kerberos.principal", "hbase/[email protected]");
conf.set("hbase.regionserver.kerberos.principal", "hbase/[email protected]");
conf.set("hbase.zookeeper.quorum", "nn1.ambari");
conf.set("hbase.zookeeper.property.clientPort", "2181");
conf.set("zookeeper.znode.parent", "/hbase-secure");
conf.set("hadoop.security.authentication", "Kerberos");
conf.set("hbase.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
UserGroupInformation.loginUserFromKeytab(USER_KEY, keyTabPath);
connection = ConnectionFactory.createConnection(conf);
}如果是自己建立的keytab，还要去hbase shell做一步授权的动作。否则你的账户权限是不够的。
hbase shell授权动作语句
permissions is either zero or more letters from the set "RWXCA".
READ('R'), WRITE('W'), EXEC('X'), CREATE('C'), ADMIN('A')
grant 'test', 'RWXCA'