Hadoop搭建_01
安装vmware
安装linux
配置linux
配置IP
192.168.1.100 master
192.168.1.102 slave2
192.168.1.103 slave1
配置主机名
[root@localhost ~]# vim /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=master
[root@localhost ~]# hostname master
三台主机都配置主机名
[root@localhost ~]# vim /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=slave1
[root@localhost ~]# hostnameslave1
[root@localhost ~]# vim /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=slave2
[root@localhost ~]# hostnameslave2
重启才能生效
Reboot
然后使用远程连接工具，这里使用xshell
关闭防火墙和selinux
[root@master ~]# vim /etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - SELinux is fully disabled.
SELINUX=disabled
# SELINUXTYPE= type of policy in use. Possible values are:
# targeted - Only targeted network daemons are protected.
# strict - Full SELinux protection.
SELINUXTYPE=targeted
[root@master ~]# serviceiptables status
表格：filter
Chain INPUT (policy ACCEPT)
num targetprot opt source destination
Chain FORWARD (policy ACCEPT)
num targetprot opt source destination
Chain OUTPUT (policy ACCEPT)
num targetprot opt source destination
域名解析
使得master和slave1 slave2 连接对方
三台都要做
[root@master ~]# vim /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomainlocalhost
::1 localhost6.localdomain6 localhost6
192.168.7.100 master
192.168.7.101 slave1
192.168.7.102 slave2
配置java
启动ftp
[root@master ~]# /etc/init.d/vsftpd restart
关闭vsftpd： [失败]
为vsftpd启动vsftpd： [确定]
默认情况下 root不允许使用ftp
[root@mastervsftpd]# pwd
/etc/vsftpd
[root@mastervsftpd]# ls
ftpusersuser_list
将这两个文件中的root注释掉。
然后重启ftp
在三个机器上创建文件夹
[root@master ~]# mkdir installer
[root@master ~]#
上传jdk
E:\开发工具
jdk-6u27-linux-i586-rpm.bin
到installer目录
[root@master installer]# ll
总计 78876
-rw-r--r-- 1 root root 80680219 12-01 12:50 jdk-6u27-linux-i586-rpm.bin
[root@master installer]# chmoda+x jdk-6u27-linux-i586-rpm.bin
[root@master installer]#
[root@master installer]# ./jdk-6u27-linux-i586-rpm.bin
Unpacking...
Checksumming...
Extracting...
UnZipSFX 5.50 of 17 February 2002, by Info-ZIP ([email protected]).
inflating: jdk-6u27-linux-i586.rpm
这时候可以再开一个连接进行远程拷贝
[root@master installer]# scp jdk-6u27-linux-i586-rpm.bin slave1:/root/installer
root@slave1's password:
jdk-6u27-linux-i586-rpm.bin
拷贝完在三个机器上都要安装jdk。
[root@master installer]# java -version
java version "1.6.0_27"
添加用户add
三台机器都做
[root@master ~]# useraddhadoop
[root@master ~]# passwdhadoop
Changing password for user hadoop.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
[root@master ~]#
并且在slave1和slave2使用hadoop用户创建installers目录
[root@slave1 ~]# su - hadoop
[hadoop@slave1 ~]$ mkdir installer
配置ssh等效性
[hadoop@master ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
5b:d6:30:95:6f:37:b1:d8:e0:ec:b4:cb:94:cc:3f:ccroot@master
[hadoop @master ~]#
一路回车
这一个动作在三台机器上都执行
[hadoop@master ~]$ cd .ssh/
[hadoop@master .ssh]$ ls
id_rsa id_rsa.pub
[hadoop@master .ssh]$
[hadoop@master .ssh]$ cat id_rsa.pub >authorized_keys
[hadoop@master .ssh]$ ls
authorized_keysid_rsa id_rsa.pub
[hadoop@master .ssh]$
将生成的authorized_keys文件拷贝到slave1和slave2
[hadoop@master .ssh]$ scpauthorized_keys slave1:~/.ssh/
The authenticity of host 'slave1 (192.168.1.103)' can't be established.
RSA key fingerprint is 61:e5:be:d1:92:41:b4:22:4e:88:ff:b1:b1:a1:64:bb.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'slave1,192.168.1.103' (RSA) to the list of known hosts.
hadoop@slave1's password:
authorized_keys 100% 395 0.4KB/s 00:00
[hadoop@master .ssh]$
然后进入slave1
[hadoop@slave1 .ssh]$ cat id_rsa.pub >>authorized_keys
[hadoop@slave1 .ssh]$ catauthorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAw8taarZ+/ndWV04MqGsnT5cKcYs5LqMmtocWSsIxfUttYpMjwcgktjEPSByb/SFPE3alx0/Te7bjG8nFu2HHV4v++2jNfraqoBjIrO3/ITzHOSGduYmM4xbvBcXCAX5BSawwbpKn8RifPM5M1ZbExFhdZ0njsYSBlq6ZAMV+2F77enfwCI6jB/WhtfClj4QpWuMTQ8O/gqaMbM0OMrIuY84ssoYfDSpl2uUtGBBGY3cyyTDEbQukRH5doapSNPwZQs6lJSVIO7JWLGMfOQbvsqlS0r1nly57I1b7hAMZcGdVWZy2CGclQX3s8a7vjpJ8+iTFtwiAdydFsP+aQ9ldUw== hadoop@master
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAqhiMNhNlBZ1+aC+tU9O8HKTd7lSMmqhi7FcBKue/q/H37hyMp+PqS/BVYStvEhtHzcy+1/SJWKqSV0ut1Qh8zUo42w81KW/g1xCt5fAJLe61/XtC2WyTrwfVQbFVXCPTpAarYJTlgy+ZgarD8Qg4hS642dmXKbSUQf/Mjbxd7PpcAZx1GCVOX3wck+7LIQJuLInlAFIXhyP0rq+I80CX9u40utkgJQd6ZVvsqJdnB+eeXr08w16GEOSY8ER2Vksbw69PGJjjKz1eMFpCUNatlf3bgmLp+JBOnlbgEizc21ogwcnyTXKCP9j3ZHTO2pDxAaHJ2hYJnOjr2+GSALzeOw== hadoop@slave1
[hadoop@slave1 .ssh]$
然后在slave1 传输到slave2
[hadoop@slave1 .ssh]$ scpauthorized_keys slave2:~/.ssh
The authenticity of host 'slave2 (192.168.1.102)' can't be established.
RSA key fingerprint is 61:e5:be:d1:92:41:b4:22:4e:88:ff:b1:b1:a1:64:bb.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'slave2,192.168.1.102' (RSA) to the list of known hosts.
hadoop@slave2's password:
authorized_keys 100% 790 0.8KB/s 00:00
[hadoop@slave1 .ssh]$
到slave2上
[hadoop@slave2 .ssh]$ cat id_rsa.pub >>authorized_keys
[hadoop@slave2 .ssh]$ catauthorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAw8taarZ+/ndWV04MqGsnT5cKcYs5LqMmtocWSsIxfUttYpMjwcgktjEPSByb/SFPE3alx0/Te7bjG8nFu2HHV4v++2jNfraqoBjIrO3/ITzHOSGduYmM4xbvBcXCAX5BSawwbpKn8RifPM5M1ZbExFhdZ0njsYSBlq6ZAMV+2F77enfwCI6jB/WhtfClj4QpWuMTQ8O/gqaMbM0OMrIuY84ssoYfDSpl2uUtGBBGY3cyyTDEbQukRH5doapSNPwZQs6lJSVIO7JWLGMfOQbvsqlS0r1nly57I1b7hAMZcGdVWZy2CGclQX3s8a7vjpJ8+iTFtwiAdydFsP+aQ9ldUw== hadoop@master
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAqhiMNhNlBZ1+aC+tU9O8HKTd7lSMmqhi7FcBKue/q/H37hyMp+PqS/BVYStvEhtHzcy+1/SJWKqSV0ut1Qh8zUo42w81KW/g1xCt5fAJLe61/XtC2WyTrwfVQbFVXCPTpAarYJTlgy+ZgarD8Qg4hS642dmXKbSUQf/Mjbxd7PpcAZx1GCVOX3wck+7LIQJuLInlAFIXhyP0rq+I80CX9u40utkgJQd6ZVvsqJdnB+eeXr08w16GEOSY8ER2Vksbw69PGJjjKz1eMFpCUNatlf3bgmLp+JBOnlbgEizc21ogwcnyTXKCP9j3ZHTO2pDxAaHJ2hYJnOjr2+GSALzeOw== hadoop@slave1
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzyFZKYRXh1HIm+p//kh/P268u6CHQJ88M+vEcb0fEjpXhNoDaVDceuYhQZxc0E/3dJRd86jaRNWnV+G+IPN00ykV2+UJhE2yjsdMa+Yqwy6XU14H25lMaImJGtxpoXO+3kWKJZ1uGB0E2TU2nS+Epb8EI+6ezZ0ilQhgwpc0kQR/jN6d6hUKKK5yTxKZg4agn4QsOZhyBNQZX7tLofHELR970T5n7to19UejB1j09AVdME+TYf7q3reLYHtVA1NsD7+wQcPB3WOKCRhHU5Uas+Rd3ukIP2/H8h13mJ5NHhq5FzxdVa62OPw9BKZVVO2vXp7SvxJG0MW0Aw8fO+AuRQ== hadoop@slave2
[hadoop@slave2 .ssh]$
然后将这个文件传回slave1和master
[hadoop@slave2 .ssh]$ scpauthorized_keys master:~/.ssh/
The authenticity of host 'master (192.168.1.100)' can't be established.
RSA key fingerprint is 61:e5:be:d1:92:41:b4:22:4e:88:ff:b1:b1:a1:64:bb.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'master,192.168.1.100' (RSA) to the list of known hosts.
hadoop@master's password:
authorized_keys 100% 1185 1.2KB/s 00:00
[hadoop@slave2 .ssh]$
[hadoop@slave2 .ssh]$ scpauthorized_keys slave1:~/.ssh/
The authenticity of host 'slave1 (192.168.1.103)' can't be established.
RSA key fingerprint is 61:e5:be:d1:92:41:b4:22:4e:88:ff:b1:b1:a1:64:bb.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'slave1,192.168.1.103' (RSA) to the list of known hosts.
hadoop@slave1's password:
authorized_keys 100% 1185 1.2KB/s 00:00
[hadoop@slave2 .ssh]$
在三台机器上修改权限
[hadoop@master .ssh]$ chmod 600 authorized_keys
到这里配置完毕，可以直接使用
Ssh slave1 链接不需要提示密码。