centos7下调试集群三台机器实现免密登陆--hadoop安装系列之一

一、实验目标

安装三台centos7虚拟机，组建hadoop实验集群，centos是从centos7官网下载的最新版本，默认系统安装，创建hadoop用户组，新建用户hadoop并加入hadoop组。

二、实验环境介绍

三台机器的网络主机配置如下：

192.168.10.166 master

192.168.10.167 slave01

192.168.10.168 slave02

hadoop官网下载链接：选择2.7.3稳定版

http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz

javasdk需要在oralce的官方网站下载，目前是版本是jdk1.8

http://download.oracle.com/otn-pub/java/jdk/8u121-b13/e9e7ea248e2c4826b92b3f075a80e441/jdk-8u121-linux-x64.rpm?AuthParam=1491112154_2109c06de9b8b6eb3cd4e31e09df1780

操作系统版本如下：

[hadoop@master .ssh]$ uname -a
Linux master 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

[hadoop@master .ssh]$ java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
[hadoop@master .ssh]$ javac -version
javac 1.8.0_121

二、实验步骤

默认系统安装完后，分别进入master执行以下操作

1、生成密钥文件.
$ ssh-keygen 
然后一路回车.

[hadoop@master .ssh]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
ca:b9:86:98:4a:d8:8a:c5:c7:d8:f0:68:dd:c4:9f:f5 hadoop@master
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|                 |
|     .           |
|  .   o S .      |
|.o O + + o .     |
|..Bo*.= o   E    |
|o+o.. ..         |
|+.   ..          |
+-----------------+
然后~/.ssh下会生成id_rsa.pub和id_rsa, 其中id_rsa文件起到唯一标识你的客户机的作用.

2、进入~/.ssh目录生成rsa默认配置文件authorized_keys 

[hadoop@master .ssh]$ cat id_rsa.pub >> authorized_keys 

--这里是一个坑，authorized_keys 文件的权限必须是600
[hadoop@master .ssh]$ chmod 600 authorized_keys 
[hadoop@master .ssh]$ ll
total 16
-rw-------. 1 hadoop hadoop  790 Apr  2 13:00 authorized_keys
-rw-------. 1 hadoop hadoop 1675 Apr  2 12:58 id_rsa
-rw-r--r--. 1 hadoop hadoop  395 Apr  2 12:58 id_rsa.pub

3、修改sshd配置文件(/etc/ssh/sshd_config).

--/etc/ssh/sshd_config文件是sshd服务的系统配置文件，修改这个文件需要root权限
找到以下内容，并去掉注释符”#“
=========================
　　RSAAuthentication yes
　　PubkeyAuthentication yes
　　AuthorizedKeysFile  .ssh/authorized_keys
=========================

默认配置前两项被注释掉，第一项RSAAuthentication表示允许进行RSA方式进行认证，第二项PubkeyAuthentication允许进行公钥认证，

第三项默认打开，但是如果不打开前两项，第三项不会生效，第三项配置的就是公钥认证文件的文件名及其所属目录，默认名字就是authorized_keys，你也可以修改，但是不建议这么做，保持约定俗成的约定对后期维护成本有不可低估的好处。

4、重启sshd服务，测试ssh本机是否可以免密登陆

[root@master .ssh]# service sshd restart

--重启sshd服务需要root权限
Redirecting to /bin/systemctl restart  sshd.service

[hadoop@master .ssh]$ ssh master
Last login: Sun Apr  2 12:10:41 2017 from master

--这一步只要不需要输入密码就成功了

5、~/.ssh目录如何自动生成

使用hadoop账户登陆服务器，然后使用ssh命令登陆一下任意一个其他主机，例如：

[root@slave02 .ssh]# ssh slave01

--在slave02使用hadoop账户只要执行一次ssh命令，系统就会自动在对应账户下生成一个隐形文件夹.ssh，用于保存ssh登陆的相关信息，一般不需要自己手工创建
The authenticity of host 'slave01 (192.168.10.167)' can't be established.
ECDSA key fingerprint is 1b:50:d1:5f:66:98:11:9f:38:ef:2c:2f:18:ea:d1:43.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'slave01,192.168.10.167' (ECDSA) to the list of known hosts.
root@slave01's password: 

6、拷贝master的公钥文件到集群的其他两台主机上；

完成这一步前一定要先在slave01及02上确保hadoop目录下存在.ssh目录，否则拷贝会提示失败。

[hadoop@master .ssh]$ scp authorized_keys hadoop@slave01:/home/hadoop/.ssh/
hadoop@slave01's password: 
authorized_keys                                                                                           100%  790     0.8KB/s   00:00    
[hadoop@master .ssh]$ scp authorized_keys hadoop@slave02:/home/hadoop/.ssh/
hadoop@slave02's password: 
authorized_keys                                                                                           100%  790     0.8KB/s   00:00    

7、登陆到slave01与slave02上修改各自/etc/ssh/sshd_config

具体修改步骤与在master上修改的步骤完全一致，参见第3步；

8、测试免密登陆是否成功

[hadoop@master .ssh]$ ssh slave01
Last login: Sun Apr  2 12:15:36 2017 from master
[hadoop@slave01 ~]$ exit
logout
Connection to slave01 closed.
[hadoop@master .ssh]$ ssh slave02
Last login: Sun Apr  2 12:17:16 2017 from master
[hadoop@slave02 ~]$ exit
logout
Connection to slave02 closed.
[hadoop@master .ssh]$ 

三、故障处理

1、无法实现免密登陆故障检查思路

  首先从master侧检查是否可以免密登陆自己，

   a）检查sshd的配置文件是否修改正确；

  b）sshd服务是否在修改文件重启成功；

  c）authorized_keys文件权限是否是600；

2、调试方法,使用ssh加上 -v参数进入调试模式观察调试信息寻找故障点

[hadoop@master .ssh]$ ssh -v slave01

OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 56: Applying options for *
debug1: Connecting to slave01 [192.168.10.167] port 22.
debug1: Connection established.
debug1: identity file /home/hadoop/.ssh/id_rsa type 1
debug1: identity file /home/hadoop/.ssh/id_rsa-cert type -1
debug1: identity file /home/hadoop/.ssh/id_dsa type -1
debug1: identity file /home/hadoop/.ssh/id_dsa-cert type -1
debug1: identity file /home/hadoop/.ssh/id_ecdsa type -1
debug1: identity file /home/hadoop/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/hadoop/.ssh/id_ed25519 type -1
debug1: identity file /home/hadoop/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr [email protected] none
debug1: kex: client->server aes128-ctr [email protected] none
debug1: kex: [email protected] need=16 dh_need=16
debug1: kex: [email protected] need=16 dh_need=16
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA 1b:50:d1:5f:66:98:11:9f:38:ef:2c:2f:18:ea:d1:43
debug1: Host 'slave01' is known and matches the ECDSA host key.
debug1: Found key in /home/hadoop/.ssh/known_hosts:1
--从本地known_hosts文件中发现曾经登陆过slave01，保存的有slave01的指纹信息

debug1: ssh_ecdsa_verify: signature correct

debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available (default cache: KEYRING:persistent:1000)


debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available (default cache: KEYRING:persistent:1000)


debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/hadoop/.ssh/id_rsa

--如果没有这两步检查配置文件是否修改正确并且是否重启过sshd服务；

debug1: Server accepts key: pkalg ssh-rsa blen 279
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type RSA
debug1: Authentication succeeded (publickey).

--如果没有这两步检查文件authorized_keys是否存在并且权限是否是600
Authenticated to slave01 ([192.168.10.167]:22).
debug1: channel 0: new [client-session]
debug1: Requesting [email protected]
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env XMODIFIERS = @im=ibus
debug1: Sending env LANG = en_US.UTF-8
Last login: Sun Apr  2 13:18:27 2017 from master

四、总结

免密登陆是hadoop集群机器间，master主机对两台slave主机上的服务进行无交互自动调度所需要的操作系统约束，所以在部署hadoop集群前必须首先解决这个问题，

免密登陆的原理是把自己的rsa公钥文件送给需要免密登陆的主机.ssh目录下，让对方认得自己，比如本例中只是需要master可以免密登陆其它两台主机，所以只是把master的公钥文件送到了slave机器上，所以从slave主机上向master方向是不能免密登陆的，除非把slave主机的公钥文件送到master主机的.ssh目录下。