在 CDH 集群启用 Kerberos认证
Kerberos原理介绍
kerberos主要是用来做网络通信时候的身份认证，最主要的特点就是“复杂”。所以在入坑kerberos之前，最好先熟悉一下其原理。这里推荐一些别人写的文章内容来进行简单汇总：
1.链接：
kerberos认证原理
用对话场景来解释kerbeors的设计过程
Kerberos的组件和术语(翻译和注解)
2.Kerberos认证流程：
KDC服务安装及配置
本文档中将KDC服务安装在worker3节点服务器上（KDC服务可根据自己需要安装在其他服务器，最好安装在Cloudera Manager Server所在的服务器上）
1.在worker3节点上安装KDC服务
[root@worker3 ~]# yum -y install krb5-serverkrb5-libs krb5-auth-dialog krb5-workstation
2.修改/etc/krb5.conf配置(标红部分为需要修改的信息)
[root@worker3 ~]# vim /etc/krb5.conf
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
dns_lookup_realm = false
dns_lookup_kdc = false
clockskew = 600
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
rdns = false
default_realm = WANWEI.COM
# default_ccache_name = KEYRING:persistent:%{uid}
[realms]
WANWEI.COM = {
kdc = worker3
admin_server = worker3
}
[domain_realm]
# .example.com = EXAMPLE.COM
# example.com = EXAMPLE.COM
.worker3 = WANWEI.COM
worker3 = WANWEI.COM
这个配置kdc，kerberos客户端，以及调用kerberos api时都会使用到。
几个重要的配置项目包括：
- ticket_lifetime和renew_lifetime：指定了kdc授权ticket的过期时长，和允许更新现有ticket的时长。
- realms的section：指定了kdc和admin_server的路径
3.修改/var/kerberos/krb5kdc/kadm5.acl配置
[root@worker3 ~]# vi /var/kerberos/krb5kdc/kadm5.acl
*/[email protected] *
4.修改/var/kerberos/krb5kdc/kdc.conf配置
[root@worker3 ~]# vim /var/kerberos/krb5kdc/kdc.conf
[kdcdefaults]
kdc_ports = 88
kdc_tcp_ports = 88
[realms]
WANWEI.COM = {
# # master_key_type = aes256-cts
max_renewable_life= 7d
acl_file = /var/kerberos/krb5kdc/kadm5.acl
dict_file = /usr/share/dict/words
admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-c
ts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
}
这是kdc的专属配置，可以根据自己的需求修改下kdc数据库的存放目录。
5.创建Kerberos数据库
[root@worker3 krb5kdc]# kdb5_util create -r WANWEI.COM –s
Loading random data
Initializing database '/var/kerberos/krb5kdc/principal' for realm 'WANWEI.COM',
master key name 'K/[email protected]'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key:
Re-enter KDC database master key to verify: 
红框处需要输入Kerberos数据库的密码。
6.创建Kerberos的管理账号
[root@worker3 krb5kdc]# kadmin.local
Authenticating as principal root/[email protected] with password.
kadmin.local: addprinc admin/[email protected]
WARNING: no policy specified for admin/[email protected]; defaulting to no policy
Enter password for principal "admin/[email protected]":
Re-enter password for principal "admin/[email protected]":
Principal "admin/[email protected]" created.
kadmin.local: exit
标红部分为Kerberos管理员账号，需要输入管理员密码
7.将Kerberos服务添加到自启动服务，并启动krb5kdc和kadmin服务
[root@worker3 ~]# systemctl enable krb5kdc.service
[root@worker3 ~]# systemctl enable kadmin.service
[root@worker3 ~]# systemctl start krb5kdc.service
Starting Kerberos 5 KDC: [ OK ]
[root@worker3 ~]# systemctl start kadmin.service
Starting Kerberos 5 Admin Server: [ OK ]8.测试Kerberos的管理员账号
[root@worker3 ~]# kinit admin/[email protected]
Password for admin/[email protected]:
[root@worker3 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: admin/[email protected]
Valid starting Expires Service principal
2019-03-01T11:53:20 2019-03-02T11:53:20 krbtgt/[email protected]
renew until 2019-03-08T11:53:20
9.为集群安装所有Kerberos客户端，包括Cloudera Manager
yum -y install krb5-libs krb5-workstation10.在Cloudera Manager Server服务器上安装额外的包
yum -y install openldap-clients11.将KDC Server上的krb5.conf文件拷贝到所有Kerberos客户端
scp -r /etc/krb5.conf root@master:/etc/CDH集群启用Kerberos
1.在KDC中给Cloudera Manager添加管理员账号
[root@worker3 ~]# kadmin.local
Authenticating as principal admin/[email protected] with password.
kadmin.local: addprinc cloudera-scm/[email protected]
WARNING: no policy specified for cloudera-scm/[email protected]; defaulting to no policy
Enter password for principal "cloudera-scm/[email protected]":
Re-enter password for principal "cloudera-scm/[email protected]":
Principal "cloudera-scm/[email protected]" created.
kadmin.local: exit
2.进入Cloudera Manager的“管理”-> “安全”界面 -> “启用Kerberos”
3.选择“启用Kerberos”，进入如下界面确保如下列出的所有检查项都已完成
4.点击“继续”，配置相关的KDC信息，包括类型、KDC服务器、KDC Realm、加密类型以及待创建的Service Principal（hdfs，yarn,，hbase，hive等）的更新生命期等。
5.取消“通过Cloudera manager管理”，不建议让Cloudera Manager来管理krb5.conf.点击“继续”：
6.输入Cloudera Manager的Kerbers管理员账号，必须和之前创建的账号一致。点击“继续”:
7.等待启用Kerberos完成，点击“继续”
8.重启集群，点击“继续”
9.等待集群重启成功，点击“继续”
至此已成功启用Kerberos。
Kerberos使用
1.使用root用户运行MapReduce任务及操作Hive，需要在集群所有节点创建root用户。
[root@worker3 ~]# kadmin.local
Authenticating as principal admin/[email protected] with password.
kadmin.local: addprinc [email protected]
WARNING: no policy specified for [email protected]; defaulting to no policy
Enter password for principal "[email protected]":
Re-enter password for principal "[email protected]":
Principal "[email protected]" created.
kadmin.local: exit2.使用root用户登录Kerberos
[root@worker3 ~]# kdestroy
[root@worker3 ~]# kinit root
Password for [email protected]:
[root@worker3 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: [email protected]
Valid starting Expires Service principal
2019-03-01T15:51:09 2019-03-02T15:51:09 krbtgt/[email protected]
renew until 2019-03-08T15:51:093.验证namenode是否正确启动
两种方法：
（1）访问http://machine:50070
（2）
[root@worker3 ~]# hadoop fs -ls /注：如果在你的凭据缓存中没有有效的kerberos ticket，执行hadoop fs -ls将会失败。
可以使用klist来查看是否有有有效的ticket。
可以通过kinit来获取ticket.
kinit -k -t /usr/local/hadoop/conf/hdfs.ketab hdfs/nn.hadoop.local@for_hadoop