Hadoop 1.2.1 安装笔记01 : Linux 与免密码
目标： 配置一个hadoop 1.2.1 测试环境 
用的JDK是: jdk-7u65-linux-x64.gz
选的hadoop 是： hadoop-1.2.1.tar.gz
均来源Apache和oracle网站
主机规划：
Linux版本 ： Centos 6.5 x64位
/boot：用来存放与Linux系统启动有关的程序，比如启动引导装载程序等，建议大小为100MB。
/usr：用来存放Linux系统中的应用程序，其相关数据较多，建议大于3GB以上。
/var：用来存放Linux系统中经常变化的数据以及日志文件，建议大于1GB以上。
/home：存放普通用户的数据，是普通用户的宿主目录，建议大小为剩下的空间。
/：Linux系统的根目录，所有的目录都挂在这个目录下面，建议大小为5GB以上。
/tmp：将临时盘在独立的分区，可避免在文件系统被塞满时影响到系统的稳定性。建议大小为500MB以上。
swap：实现虚拟内存，建议大小是物理内存的1~2倍。
Boot Loader密码是 D*****2014
安装选择 Basic Server 不然所有的包得一个个打，确实很麻烦.
配置sudo ，赋予hadoop用户权限 ，并配置sudo免密
useradd hadoop
passwd hadoop
vi /etc/sudoers
root ALL=(ALL) ALL
hadoop ALL=(ALL) ALL 添加一行
hadoop ALL=(ALL) NOPASSWD: ALL 添加一行免密码
![clipboard[1]](http://img1.51cto.com/attachment/201408/12/8976580_1407803506sQuI.png "clipboard[1]")
配置本机的解析
[hadoop@master /]$ cat /etc/hosts 10.15.5.200 master.hadoop 10.15.5.201 slave01.hadoop 10.15.5.202 slave02.hadoop
配置主机名
vi etc/sysconfig/network HOSTNAME=master.hadoop
配置IP 地址
[hadoop@master /]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 TYPE=Ethernet UUID=721f9261-45d5-4335-9b47-64459173b2a9 ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=none HWADDR=00:50:56:82:00:0F IPADDR=10.15.5.200 PREFIX=24 GATEWAY=10.15.5.1 DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME="System eth0"
挂载 安装盘，以便所需要的安装包。打上ftp包 方便安装软件
[root@master home]# mount -t auto /dev/cdrom /home/cdrom mount: block device /dev/sr0 is write-protected, mounting read-only [root@master Packages]# rpm -ivh ftp-0.17-54.el6.x86_64.rpm warning: ftp-0.17-54.el6.x86_64.rpm: Header V3 RSA/SHA1 Signature, key ID c105b9de: NOKEY Preparing... ########################################### [100%] 1:ftp ########################################### [100%]
目标： master和slaves hadoop账户无密码互访
原理：
可以粗糙的理解为，我制造一把钥匙，把样子发给你，你在门禁系统里记下我钥匙的样子，我拿这把钥匙去开门时候你的门禁系统直接放行。
Master上，登录hadoop用户，并在/home/hadoop目录下操作
[hadoop@master ~]$ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): Created directory '/home/hadoop/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/hadoop/.ssh/id_rsa. Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub. The key fingerprint is: 8d:81:09:c8:45:3f:c0:fb:3b:a0:cf:95:b6:dd:e9:b1 [email protected] The key's randomart image is: +--[ RSA 2048]----+ | . == | | o .+ o | | .= . | | . . + | | . S . | | . .. | | . .+. . | | .. ooo . + | | .o ....E | +-----------------+
该隐藏文件夹下，生成了一个 id_rsa.pub文件
[hadoop@master .ssh]$ ls -al /home/hadoop/.ssh total 16 drwx------. 2 hadoop hadoop 4096 Jul 30 22:09 . drwx------. 3 hadoop hadoop 4096 Jul 30 22:09 .. -rw-------. 1 hadoop hadoop 1671 Jul 30 22:09 id_rsa -rw-r--r--. 1 hadoop hadoop 402 Jul 30 22:09 id_rsa.pub [hadoop@master .ssh]$ cat id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAugxDdYElWX8URmERVTPsNtSW2mZoEMZyzmIKEe3UQjMbgYOCx0jV15/vtnOXJf4K+S6hccajih2oemcC4BmMI99nFWcyd9zcrvfJvn/DZHnw0Yog6MyMd9Qw2BQWUL265dhW2FNCaECuyG2U1CxR0W9wZlVz54JlToCRX6yUvWZzzGqUw/oR3ZWE7pUpQiOHV0ZNypUtbWEw/zj7N01LbVskNLqyY164APIVfZBONPMXjs/H6B8/VcxsA0lDwAqndnMxJ1iQHFkMntNgqcLkhs8oAmvl+/A6NH2I0ZBW+VOCUIjbNKNRO9BKwGvUQUZGcKthEPU0jK5ErsS6RPBrmQ== [email protected]
把这个文件的内容，追加到 authorized_keys 的文件里
[ hadoop@master .ssh]$ cat id_rsa.pub >> authorized_keys [ hadoop@master .ssh]$ ls authorized_keys id_rsa id_rsa.pub [ hadoop@master .ssh]$ cat authorized_keys ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAugxDdYElWX8URmERVTPsNtSW2mZoEMZyzmIKEe3UQjMbgYOCx0jV15/vtnOXJf4K+S6hccajih2oemcC4BmMI99nFWcyd9zcrvfJvn/DZHnw0Yog6MyMd9Qw2BQWUL265dhW2FNCaECuyG2U1CxR0W9wZlVz54JlToCRX6yUvWZzzGqUw/oR3ZWE7pUpQiOHV0ZNypUtbWEw/zj7N01LbVskNLqyY164APIVfZBONPMXjs/H6B8/VcxsA0lDwAqndnMxJ1iQHFkMntNgqcLkhs8oAmvl+/A6NH2I0ZBW+VOCUIjbNKNRO9BKwGvUQUZGcKthEPU0jK5ErsS6RPBrmQ== [email protected]
此时需要修改authorized_keys的文件权限 ，不安全的设置安全设置，会让你不能使用RSA功能
[ hadoop@master .ssh]$ ls -al total 20 drwx------. 2 hadoop hadoop 4096 Jul 30 22:20 . drwx------. 3 hadoop hadoop 4096 Jul 30 22:09 .. -rw-rw-r--. 1 hadoop hadoop 402 Jul 30 22:20 authorized_keys -rw-------. 1 hadoop hadoop 1671 Jul 30 22:09 id_rsa -rw-r--r--. 1 hadoop hadoop 402 Jul 30 22:09 id_rsa.pub [ hadoop@master .ssh]$ sudo chmod 600 authorized_keys [ hadoop@master .ssh]$ ls -al total 20 drwx------. 2 hadoop hadoop 4096 Jul 30 22:20 . drwx------. 3 hadoop hadoop 4096 Jul 30 22:09 .. -rw-------. 1 hadoop hadoop 402 Jul 30 22:20 authorized_keys -rw-------. 1 hadoop hadoop 1671 Jul 30 22:09 id_rsa -rw-r--r--. 1 hadoop hadoop 402 Jul 30 22:09 id_rsa.pub
修改 /etc/ssh/sshd_config 并且确认其中三个选项 然后sudo service sshd restart 服务
[ hadoop@master .ssh]$ sudo cat /etc/ssh/sshd_config RSAAuthentication yes PubkeyAuthentication yes AuthorizedKeysFile .ssh/authorized_keys
在所有Slave上建立 .ssh文件夹,改变权限
[hadoop@slave02 ~]$ mkdir .ssh [hadoop@slave02 ~]$ ls -al total 32 drwx------. 3 hadoop hadoop 4096 Jul 30 22:47 . drwxr-xr-x. 4 root root 4096 Jul 30 20:10 .. -rw-------. 1 hadoop hadoop 401 Jul 30 20:58 .bash_history -rw-r--r--. 1 hadoop hadoop 18 Jul 18 2013 .bash_logout -rw-r--r--. 1 hadoop hadoop 176 Jul 18 2013 .bash_profile -rw-r--r--. 1 hadoop hadoop 124 Jul 18 2013 .bashrc drwxrwxr-x. 2 hadoop hadoop 4096 Jul 30 22:47 .ssh -rw-------. 1 hadoop hadoop 557 Jul 30 20:28 .viminfo [hadoop@slave02 ~]$ sudo chmod 700 .ssh [hadoop@slave02 ~]$ ls -al total 32 drwx------. 3 hadoop hadoop 4096 Jul 30 22:47 . drwxr-xr-x. 4 root root 4096 Jul 30 20:10 .. -rw-------. 1 hadoop hadoop 401 Jul 30 20:58 .bash_history -rw-r--r--. 1 hadoop hadoop 18 Jul 18 2013 .bash_logout -rw-r--r--. 1 hadoop hadoop 176 Jul 18 2013 .bash_profile -rw-r--r--. 1 hadoop hadoop 124 Jul 18 2013 .bashrc drwx------. 2 hadoop hadoop 4096 Jul 30 22:47 .ssh -rw-------. 1 hadoop hadoop 557 Jul 30 20:28 .viminfo [hadoop@slave02 ~]$
把id_rsa.pub发送给slaves
[ hadoop@master .ssh]$ scp id_rsa.pub [email protected] :/home/hadoop/.ssh/id_rsa_frommaster.pub The authenticity of host 'slave01.hadoop (10.15.5.201)' can't be established. RSA key fingerprint is 76:14:2f:f9:d9:03:07:17:7c:d1:ad:1e:af:55:45:00. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'slave01.hadoop,10.15.5.201' (RSA) to the list of known hosts. [email protected] 's password: id_rsa.pub 100% 402 0.4KB/s 00:00 [ hadoop@master .ssh]$ scp id_rsa.pub [email protected] :/home/hadoop/.ssh/id_rsa_frommaster.pub The authenticity of host 'slave02.hadoop (10.15.5.202)' can't be established. RSA key fingerprint is 76:14:2f:f9:d9:03:07:17:7c:d1:ad:1e:af:55:45:00. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'slave02.hadoop,10.15.5.202' (RSA) to the list of known hosts. [email protected] 's password: id_rsa.pub 100% 402 0.4KB/s 00:00
把这个值追加到slaves上的authorized_keys里面,并修改安全权限
[hadoop@slave02 .ssh]$ cat id_rsa_frommaster.pub >> authorized_keys [hadoop@slave02 .ssh]$ sudo chmod 600 authorized_keys
修改slaves主机上的 /etc/ssh/sshd_config内容，并重启sshd服务
[hadoop@slave01 .ssh]$ sudo cat /etc/ssh/sshd_config RSAAuthentication yes PubkeyAuthentication yes AuthorizedKeysFile .ssh/authorized_keys [hadoop@slave01 .ssh]$ sudo service sshd restart Stopping sshd: [ OK ] Starting sshd: [ OK ]
最终验证方法
[hadoop@master .ssh]$ ssh slave01.hadoop Last login: Wed Jul 30 23:00:25 2014 from master.hadoop [hadoop@slave01 ~]$ logout Connection to slave01.hadoop closed. [hadoop@master .ssh]$ ssh slave02.hadoop [hadoop@slave02 ~]$ logout Connection to slave02.hadoop closed.
同理： 从slaves 到master
slaves 生成公钥
把slaves把Slave的公钥追加到Master的".ssh"文件夹下的"authorized_keys"中
[hadoop@slave01 .ssh]$ cat id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAqkRUrgiyD3W36WPRpL9ifJjgnVe+1R4x4mRhimKXOfmW+D3dcbi9FwE2j0/H+NaMdLWjQYw685ITdfHni0X5La7YXy6EIe0FQB/NxDKVslc44RuzJQJKQkosqSY/HGVfefF7OzIFCEcqVQDqN+OpWlrbXNtmO1UUhG2TFvj3MsbgwWHf7fc+uSN7Y7bMzLJpKhAVEjljjyTaHekJ8WMNzgT160vgr0mIZawDlRxKRlS2HtWQnndf74ZJDCeQKgJA6RUKuqBLop5x/0lvHBbUkn3MADnJUlIzd5pJW/aFEVySypfkGZTspj3+m8gNfQkqYJiB6sBfFgUYt1iPL/GNRw== [email protected]
[hadoop@slave02 ~]$ cat .ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA3mHGmqOGjIkC/Elc/4CopgvKCQ7AdtqV87dhQPWnDdlljePVY/GbCWYqp4h8CmtmsMNreTva8RLBPSIP3jGMrbvNDCCuMo+mZGa034yR6JwekI9zVikzYscCTqWCk6w5Hs3u/pnb1Ym6a46ho+DNEm42qIaYzrhRn9Fc1F9Hd3/dXRq0KZlO/5xMKuhFt1GKy+gs+l7MiK6y7pTISx+ox/mDjDQfzxfpguRO68XX54+dD0gONSb/MaVVMdqiWK3Fh88ouN23sKi/cnzD1vLSm55kpT6ZRo792qcFmMe7CIYc6dwdkKZBgkdheayWy5PPfsZrGfDd/erVCWrmofDhiw== [email protected]
可以用cat >>的方式追加，或者vi直接写入。钥匙么，让master知道就行了.
![clipboard[6]](http://img1.51cto.com/attachment/201408/12/8976580_1407803506N2fv.png "clipboard[6]")