2.修改/etc/hosts文件 /安装SSH、配置SSH无密码登陆(全程切换到hadoop用户下操作)

修改/etc/hosts文件

sudo gedit /etc/hosts

127.0.0.1   localhost
192.168.222.129 master     下面分别对应三个机器的IP地址,用ifconfig查看
192.168.222.130 slave1
192.168.222.131 slave2

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

测试是否能pingping master/slave1/slave2

==========

分别给3台机器都创建hadoop用户,方便进行管理

2.修改/etc/hosts文件 /安装SSH、配置SSH无密码登陆(全程切换到hadoop用户下操作)_第1张图片

二、           建立hadoop运行帐号
即为hadoop集群专门设置一个用户组及用户
(1)创建hadoop用户组
(2)创建hadoop用户
     sudo adduser -ingroup hadoop hadoop
    回车后会提示输入新的UNIX密码,这是新建用户hadoop的密码,输入回车即可。
最后确认信息是否正确,如果没问题,输入 Y,回车即可。
(3)为hadoop用户添加权限(给hadoop用户赋予和root用户同样的权限)
输入:
    sudo gedit /etc/sudoers


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

================================================

用新增加的hadoop用户登录Ubuntu系统,进行SSH的配置

================================================

配置ssh免密码登录

    hadoop工作时,各节点要相互通讯,正常情况下linux之间通讯要提供用户名、密码(目的是保证通讯安全),如果
需要人工干预输入密码,显然不方便,做这一步的目的,是让各节点能自动通过安全认证,不影响正常通讯。



安装ssh: sudo apt-get install openssh-server
安装完成后,启动服务:sudo /etc/init.d/ssh start
查看服务是否正确启动:ps -e | grep ssh

一.使用SSH使每一台机器免密码登录自己本机

//下面以mster机器为例:

1.使登录身份始终是hadoop,操作目录始终是/home/master
master@master:~$ su hadoop     #切换成hadoop(以hadoop身份登录到系统)
hadoop@master:~$cd /home/hadoop(进入个人主目录,默认为/home/hadoop)

2.生成公钥私钥id_rsa  id_rsa.pub
hadoop@master:~$ ssh-keygen -t rsa -P ''   (注:最后是二个单引号,生成公钥私钥)
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): 
Created directory '/home/hadoop/.ssh'.
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
cd:67:f5:0f:15:a1:42:ad:7a:29:78:3b:ca:8c:3a:b7 hadoop@slave1
The key's randomart image is:
+--[ RSA 2048]----+
|           ..  o.|
|          .  .. .|
|           .... .|
|         o ... o |
|        S + + . .|
|       . + =   ..|
|        . +     .|
|   . .+  o       |
|   .+E.+. .      |
+-----------------+

解释:以rsa算法,生成公钥、私钥对,-P ''表示空密码。该命令
运行完后,会在个人主目录/home/hadoop下生成.ssh目录,里面会
有二个文件id_rsa(私钥) ,id_rsa.pub(公钥),即:
hadoop@master:~$ ls .ssh  #查看是否生成公钥私钥
id_rsa  id_rsa.pub


3.导入公钥

hadoop@master:cat .ssh/id_rsa.pub >> .ssh/authorized_keys
解释:会在个人主目录/home/hadoop/.ssh目录下生成authorized_keys,即
hadoop@master:~$ ls .ssh #查看是否生成authorized_keys
authorized_keys  id_rsa  id_rsa.pub


4.修改文件权限
hadoop@master:chmod 600 .ssh/authorized_keys

5.进行SSH测试是否能免密码连接本机
    执行完以后,可以在本机上测试下,用ssh连接自己,即:ssh localhost (或ssh master)

【注解】废话不多说,直接上下面中3个中括号中的命令
【hadoop@master:~$ ssh localhost】 #连接自己
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is d5:f7:47:44:7b:9f:55:ac:e7:ed:6c:8d:f6:98:68:72.
Are you sure you want to continue connecting (yes/no)? yes #输入yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

New release '16.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.


The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

【hadoop@master:~$ exit】  #连接成功后,退出exit
注销
Connection to localhost closed.

【hadoop@master:~$ ssh localhost】  #重新链接,不需要密码,此时成功配置了master机器的SSH
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

281 packages can be updated.
193 updates are security updates.

New release '16.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Fri Aug 11 10:22:27 2017 from localhost

hadoop@master:~$ 

【终于】
    到此位置,成功搞定一台机器:master,下面我们再进行slave1,slave2机器的SSH免密码登录的配置。(配置方法同上)

二. 将slave公钥文件复制到master(只操作slave机器,不操作master)

a)用scp命令,把公钥文件发放给master(即:刚才已经搞定的那台机器)
#在slave1上:
scp .ssh/id_rsa.pub hadoop@master:/home/hadoop/id_rsa_01.pub

#在slave2上:
scp .ssh/id_rsa.pub hadoop@master:/home/hadoop/id_rsa_02.pub

b)这二行执行完后,回到master中,查看下/home/hadoop目录,应该有二个新文
件id_rsa_01.pub、id_rsa_02.pub,然后在master上,导入这二个公钥

cat id_rsa_01.pub >> .ssh/authorized_keys
cat id_rsa_02.pub >> .ssh/authorized_keys

这样,master这台机器就有所有3台机器的公钥了。

三. 将master上的“最全”公钥,复制到其它机器slave1.slave2

a) 继续保持在master上,

scp .ssh/authorized_keys hadoop@slave1:/home/hadoop/.ssh/authorized_keys

scp .ssh/authorized_keys hadoop@slave2:/home/hadoop/.ssh/authorized_keys

b) 修改其它机器上authorized_keys文件的权限

slave1以及slave2机器上,均执行命令

chmod 600 .ssh/authorized_keys

四. 验证能否通过SSH无密码连接其他机器

    在每个虚拟机上,均用 ssh 其它机器的hostname 验证下,如果能正常无密码连接成功,表示ok

以master机器,连接slave1,slave2机器为例:
(1)切换到hadoop用户
master@master:~$ su hadoop
(2)测试无密码连接slave1
hadoop@master:/home/master$ ssh slave1
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

281 packages can be updated.
193 updates are security updates.

New release '16.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Fri Aug 11 11:02:00 2017 from slave2
(3)测试无密码连接slave2
hadoop@slave1:~$ ssh slave2
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

281 packages can be updated.
193 updates are security updates.

New release '16.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Fri Aug 11 11:02:11 2017 from slave2

【发现】master可以无密码连接slave1,slave2,同理,下面
分别进行测试slave1,slave2

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

【重点】非常容易犯的错误
    配置SSH成功后,每次都要"先切换到hadoop用户"下,才可以SSH成功

(1)切换到用户hadoop
master@master:~$ su hadoop  
密码: 

(2SSH本机器
hadoop@master:/home/master$ ssh master
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

New release '16.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Fri Aug 11 18:12:54 2017 from master
hadoop@master:~$ 3SSH slave1机器
hadoop@master:~$ ssh slave1
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

New release '16.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Fri Aug 11 18:16:14 2017 from master
hadoop@slave1:~$ 3SSH slave2机器
hadoop@slave1:~$ ssh slave2
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

New release '16.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Fri Aug 11 17:35:47 2017 from slave1
hadoop@slave2:~$ 

[小结]

    小结:该步骤非常重要,主要思路是在各节点上生成公钥、私钥,然后将公钥发放其它所有节点。RSA算法是非
对称加密算法,仅公布“公钥”,只要私钥不外泄,还是不能解密的,所以安全性依然有保障。

    如果本步骤失败,根据我个人经验,多半是权限问题,请检查hadoop是否具有足够权限,建议将hadoop加
入sudoers列表及root用户组。另外,这里也整理了一些SSH免密码设置失败的原因,请移步 ssh 免密码设置失败原因总结

Hadoop集群安装配置教程_Hadoop2.6.0_Ubuntu/CentOShttp://dblab.xmu.edu.cn/blog/285/

你可能感兴趣的:(hadoop/spark)