Hadoop笔记：YARN环境搭建

Hadoop笔记：YARN环境搭建

标签： 大数据

---

使用版本：hadoop-2.6.0-cdh5.7.0

配置文件：
yarn-site.xml
mapred-site.xml

伪分布式YARN环境搭建

You can run a MapReduce job on YARN in a pseudo-distributed mode by setting a few parameters and running ResourceManager daemon and NodeManager daemon in addition.

配置参数

Configure parameters as follows:etc/hadoop/mapred-site.xml:

配置mapred-site.xml文件，说明MapReduce是执行在YARN框架上的

<configuration>
    <property>
        <name>mapreduce.framework.namename>
        <value>yarnvalue>
    property>
configuration>

我们可以发现，它只有一个模板：

[japson@localhost hadoop]$ vim mapred-site.xml.template 

我们可以根据这个模板先拷贝一下：

[japson@localhost hadoop]$ cp mapred-site.xml.template mapred-site.xml
[japson@localhost hadoop]$ vim mapred-site.xml

再配置etc/hadoop/yarn-site.xml，告诉框架，yarn的nodemanager用的是哪一个service，用的是mapreduce_shuffle。如果到spark中，就是spark_shuffle:

<configuration>
    <property>
        <name>yarn.nodemanager.aux-servicesname>
        <value>mapreduce_shufflevalue>
    property>
configuration>

启动进程

启动ResourceManager进程和NodeManager进程

$ sbin/start-yarn.sh

[japson@localhost sbin]$ ./start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /home/japson/app/hadoop-2.6.0-cdh5.7.0/logs/yarn-japson-resourcemanager-localhost.localdomain.out
localhost: starting nodemanager, logging to /home/japson/app/hadoop-2.6.0-cdh5.7.0/logs/yarn-japson-nodemanager-localhost.localdomain.out

验证

jps：

[japson@localhost sbin]$ jps
5161 SecondaryNameNode
4998 DataNode
5399 NodeManager
5310 ResourceManager
4874 NameNode
5626 Jps

http://localhost:8088/

定制YARN相关的进程

$ sbin/stop-yarn.sh