hadoop yarn HA 高可用架构简易集群配置

架构信息:

 

帮助文档：

http://hadoop.apache.org/docs/r2.7.6/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html

yarn-site.xml配置。

 

  yarn.resourcemanager.ha.enabled

  true

  yarn.resourcemanager.cluster-id

  cluster1

  yarn.resourcemanager.ha.rm-ids

  rm1,rm2

  yarn.resourcemanager.hostname.rm1

  master1

  yarn.resourcemanager.hostname.rm2

  master2

  yarn.resourcemanager.zk-address

  zk1:2181,zk2:2181,zk3:2181

  yarn.resourcemanager.recovery.enabled

  true

  yarn.resourcemanager.store.class

  org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore

  yarn.resourcemanager.zk-state-store.parent-path

  /rmstore

  yarn.nodemanager.recovery.enabled

  true

  yarn.nodemanager.address

  ${yarn.nodemanager.hostname}:45454

 

 

yarn.nodemanager.aux-services

mapreduce_shuffle

yarn.resourcemanager.hostname

slave1

yarn.web-proxy.address

slave1:8888

yarn.log-aggregation-enable

true

yarn.log-aggregation.retain-seconds

604800

 

yarn.nodemanager.resource.memory-mb

4192

yarn.nodemanager.resource.cpu-vcores

4

 

 

1.启动整个集群 sbin/start-dfs.sh sbin/start-yarn.sh

2.在令一台resouceManager机器：启动resoruceManager sbin/yarn-daemon.sh start resourcemanager

3.问题：sbin/start-yarn.sh nodemanager 启动失败：java.lang.IllegalArgumentException: Cannot support recovery with an ephemeral server port. Check the setting of yarn.nodemanager.address

解决方式：参考文档:http://hadoop.apache.org/docs/r2.7.6/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

 

  yarn.nodemanager.address

  ${yarn.nodemanager.hostname}:45454

转载于:https://my.oschina.net/undermoonlightperson/blog/1812927