Hadoop---yarn单机安装文档

本文紧接上文Hadoop---hdfs单机安装文档

1.修改mapred-site.xml文件

[hadoop@Hadoop001 hadoop]$ cp mapred-site.xml.template mapred-site.xml
[hadoop@Hadoop001 hadoop]$ vi mapred-site.xml

   
       mapreduce.framework.name
       yarn
   

2.修改yarn-site.xml文件

[hadoop@Hadoop001 hadoop]$ vi yarn-site.xml 

   
       yarn.nodemanager.aux-services
       mapreduce_shuffle
   

3.开启ResourceManager和NodeManager进程

[hadoop@Hadoop001 hadoop]$ sbin/start-yarn.sh 

4.浏览ResourceManager的web界面，默认为8088端口

2019-07-04_170301.png

5.运行一个mr任务

[hadoop@Hadoop001 hadoop]$ find ./ -name '*example*.jar'
./share/hadoop/mapreduce1/hadoop-examples-2.6.0-mr1-cdh5.7.0.jar
./share/hadoop/mapreduce2/sources/hadoop-mapreduce-examples-2.6.0-cdh5.7.0-test-sources.jar
./share/hadoop/mapreduce2/sources/hadoop-mapreduce-examples-2.6.0-cdh5.7.0-sources.jar
./share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -mkdir -p /examples/input
19/07/04 17:06:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -mkdir -p /examples/output
19/07/04 17:07:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -put wordcount.txt /examples/input/
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -cat /examples/input/wordcount.txt
19/07/04 17:18:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
hello alvin
alvin fure
[hadoop@Hadoop001 hadoop]$ bin/hadoop jar ./share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar wordcount /examples/input/* /examples/output/1
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -cat /examples/output/1/*
19/07/04 17:19:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
alvin   2
fure    1
hello   1

6.关闭yarn

[hadoop@Hadoop001 hadoop]$ sbin/stop-yarn.sh 
stopping yarn daemons
stopping resourcemanager
Hadoop001: stopping nodemanager
Hadoop001: nodemanager did not stop gracefully after 5 seconds: killing with kill -9
no proxyserver to stop