伪分布模式的搭建

配置ubuntu环境：

1. 防火墙

》查看防火墙状态：sudo ufw status

》关闭本机防火墙：控制面板——window防火墙——打开或关闭windows防火墙——关闭防火墙

2. jdk和scp

》下载32位jdk:

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

安装winscp,配置虚拟机的ip和访问用户/密码：

-rw-rw-r-- 1 hadoop hadoop 188157665 Sep 10 05:21 jdk-8u144-linux-i586.tar.gz

hadoop@ubuntu:~/profile/download$ pwd

/home/hadoop/profile/download

》安装jdk：

root@ubuntu:/home/hadoop/profile/download# tar zxvf  jdk-8u144-linux-i586.tar.gz -C /root/directory/java

》查看安装情况：

root@ubuntu:~/directory/java# pwd

/root/directory/java

root@ubuntu:~/directory/java# ls

jdk1.8.0_144

》配置环境变量：

export JAVA_HOME=/root/directory/java/jdk1.8.0_144

export PATH=${JAVA_HOME}/bin:$PATH

》加载及测试：

root@ubuntu:/etc# source profile

root@ubuntu:/etc# java -version

java version "1.8.0_144"

Java(TM) SE Runtime Environment (build 1.8.0_144-b01)

Java HotSpot(TM) Client VM (build 25.144-b01, mixed mode)

3. 搭建伪分布模式

》设置主机名称

sed -i 's/ubuntu/dj/g' /etc/hostname

sudo gedit /etc/hostname

sed -i 's/ubuntu/dj/g' /etc/hosts

sudo gedit /etc/hosts

》下载包：http://www.apache.org/dyn/closer.cgi/hadoop/common/

选择下载:http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.7.4/hadoop-2.7.4.tar.gz

通过scp上传，解压：

tar -zxvf  /home/hadoop/profile/download/hadoop-2.7.4.tar.gz-C /root/directory/

》设置环境变量：

vi ~/.bash_profile

export HADOOP_HOME=/root/directory/hadoop-2.7.4

export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

》加载环境变量：

source ~/.bash_profile

》配置 hadoop-env.sh

export JAVA_HOME=/root/directory/java/jdk1.8.0_144

配置hdfs-site.xml

配置core-site.xml:

配置mapred-site.xml:

cp  mapred-site.xml.template mapred-site.xml

vi mapred-site.xml

配置yarn-site.xml

对hdfs进行格式化：hdfs namenode -format

启动hdfs和yarn：start-all.sh

查看进程：

root@dj:~/directory/hadoop-2.7.4/sbin# jps

3890 ResourceManager

2837 NameNode

3142 SecondaryNameNode

2957 DataNode

4013 NodeManager

4045 Jps

查看页面：

后台日志报错：

doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager.

修改配置文件；

yarn-site.xml

配置host：vi /etc/hosts

添加：192.168.1.119  dj

修改配置：

core-site.xml

arn

yarn-site.xml

配置免秘钥登陆：

ssh-keygen -t rsa
ssh-copy-id -i .ssh/id_rsa.pub root@dj
检测免秘钥是否成功：start-all.sh

root@dj:~# jps

2980 DataNode

3527 Jps

2857 NameNode

3353 ResourceManager

3162 SecondaryNameNode

3483 NodeManager

root@dj:~#

无需输入密码，登陆成功