黑猴子的家：Hadoop 伪分布式Yarn运行MapReduce

1、配置集群

（1）配置yarn-env.sh

获取jdk 的安装路径
[victor@node1 hadoop-2.8.2]$ echo $JAVA_HOME
/opt/module/jdk1.8.0_144

修改JAVA_HOME 路径
[victor@node1 hadoop-2.8.2]$ vim etc/hadoop/yarn-env.sh
export JAVA_HOME=/opt/module/jdk1.8.0_144

（2）配置yarn-site.xml

    yarn.nodemanager.aux-services
    mapreduce_shuffle




    yarn.resourcemanager.hostname
    node1


    yarn.resourcemanager.nodes.include-path
    /opt/module/hadoop-2.8.2/etc/hadoop/dnrm

（3）配置dnrm

[victor@node1 hadoop]$ vim dnrm
node1
node2
node3

（4）配置mapred-env.sh

修改JAVA_HOME 路径
[victor@node1 hadoop-2.8.2]$ vim etc/hadoop/mapred-env.sh
export JAVA_HOME=/opt/module/jdk1.8.0_144

（5）配置mapred-site.xml

对mapred-site.xml.template重命名为mapred-site.xml 
重命名mapred-site.xml文件
[victor@node1 hadoop-2.8.2]$ cd etc/hadoop/
[victor@node1 hadoop-2.8.2]$ pwd
/opt/module/hadoop-2.8.2/etc/hadoop
[victor@node1 hadoop]$ mv mapred-site.xml.template mapred-site.xml

添加配置


    mapreduce.framework.name
    yarn

2、启动集群

（1）启动resourcemanager

[victor@node1 hadoop-2.8.2]$ sbin/yarn-daemon.sh start resourcemanager

（2）启动nodemanager

[victor@node1 hadoop-2.8.2]$ sbin/yarn-daemon.sh start nodemanager

(3)集群操作

（1）yarn的浏览器页面查看

$ http://node1:8088/cluster

image.png

（2）删除文件系统上的output文件

[victor@node1 hadoop-2.8.2]$ bin/hdfs dfs -rm -R /user/victor/wordcount/output

（3）执行mapreduce程序

[victor@node1 hadoop-2.8.2]$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.2.jar wordcount /user/victor/wordcount/input  /user/victor/wordcount/output

（4）查看运行结果

[victor@node1 hadoop-2.8.2]$ bin/hdfs dfs -cat /user/victor/wordcount/output/*
![image.png](https://upload-images.jianshu.io/upload_images/9193428-2364994192887eee.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)