伪分布模式的搭建

配置ubuntu环境:

1. 防火墙

》查看防火墙状态:sudo ufw status

》关闭本机防火墙:控制面板——window防火墙——打开或关闭windows防火墙——关闭防火墙

2. jdk和scp

》下载32位jdk:

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

安装winscp,配置虚拟机的ip和访问用户/密码:

-rw-rw-r-- 1 hadoop hadoop 188157665 Sep 10 05:21 jdk-8u144-linux-i586.tar.gz

hadoop@ubuntu:~/profile/download$ pwd

/home/hadoop/profile/download

》安装jdk:

root@ubuntu:/home/hadoop/profile/download# tar zxvf  jdk-8u144-linux-i586.tar.gz -C /root/directory/java

》查看安装情况:

root@ubuntu:~/directory/java# pwd

/root/directory/java

root@ubuntu:~/directory/java# ls

jdk1.8.0_144

》配置环境变量:

export JAVA_HOME=/root/directory/java/jdk1.8.0_144

export PATH=${JAVA_HOME}/bin:$PATH

》加载及测试:

root@ubuntu:/etc# source profile

root@ubuntu:/etc# java -version

java version "1.8.0_144"

Java(TM) SE Runtime Environment (build 1.8.0_144-b01)

Java HotSpot(TM) Client VM (build 25.144-b01, mixed mode)

3. 搭建伪分布模式

》设置主机名称

sed -i 's/ubuntu/dj/g' /etc/hostname

sudo gedit /etc/hostname

sed -i 's/ubuntu/dj/g' /etc/hosts

sudo gedit /etc/hosts

》下载包:http://www.apache.org/dyn/closer.cgi/hadoop/common/

选择下载:http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.7.4/hadoop-2.7.4.tar.gz

通过scp上传,解压:

tar -zxvf  /home/hadoop/profile/download/hadoop-2.7.4.tar.gz-C /root/directory/

》设置环境变量:

vi ~/.bash_profile

export HADOOP_HOME=/root/directory/hadoop-2.7.4

export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

》加载环境变量:

source ~/.bash_profile

》配置 hadoop-env.sh

export JAVA_HOME=/root/directory/java/jdk1.8.0_144

配置hdfs-site.xml

伪分布模式的搭建_第1张图片

配置core-site.xml:

伪分布模式的搭建_第2张图片

配置mapred-site.xml:

cp  mapred-site.xml.template mapred-site.xml

vi mapred-site.xml

伪分布模式的搭建_第3张图片

配置yarn-site.xml

伪分布模式的搭建_第4张图片

对hdfs进行格式化:hdfs namenode -format

启动hdfs和yarn:start-all.sh

查看进程:

root@dj:~/directory/hadoop-2.7.4/sbin# jps

3890 ResourceManager

2837 NameNode

3142 SecondaryNameNode

2957 DataNode

4013 NodeManager

4045 Jps

查看页面:

伪分布模式的搭建_第5张图片


伪分布模式的搭建_第6张图片

后台日志报错:

doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager.

修改配置文件;

yarn-site.xml

伪分布模式的搭建_第7张图片

配置host:vi /etc/hosts

添加:192.168.1.119  dj

修改配置:

core-site.xml


arn

yarn-site.xml


配置免秘钥登陆:

ssh-keygen -t rsa
ssh-copy-id -i .ssh/id_rsa.pub root@dj
检测免秘钥是否成功:start-all.sh

root@dj:~# jps

2980 DataNode

3527 Jps

2857 NameNode

3353 ResourceManager

3162 SecondaryNameNode

3483 NodeManager

root@dj:~#

无需输入密码,登陆成功

你可能感兴趣的:(伪分布模式的搭建)