（十三）大数据实战——hadoop集群之YARN高可用实现自动故障转移

前言

本节内容是关于hadoop集群下yarn服务的高可用搭建，以及其发生故障转移的处理，同样需要依赖zookeeper集群的实现，实现该集群搭建时，我们要预先保证zookeeper集群是启动状态。yarn的高可用同样依赖zookeeper的临时节点及监控，实现服务的故障转移。其ResourceManager的节点任务同样存储于zookeeper集群中，实现数据的共享。

正文

• 集群规划

YARN高可用

hadoop101	hadoop102	hadoop103
ResourceManager	ResourceManager	ResourceManager
NodeManager	NodeManager	NodeManager
Zookeeper	Zookeeper	Zookeeper

•  修改yarn服务配置文件yarn-site.xml

- yarn-site.xml

    
    
        yarn.nodemanager.aux-services
        mapreduce_shuffle
    
    
    
        yarn.resourcemanager.ha.enabled
        true
    
    
    
        yarn.resourcemanager.cluster-id
        cluster-yarn
    
    
    
        yarn.resourcemanager.ha.rm-ids
        rm1,rm2,rm3
    
    
    
    
        yarn.resourcemanager.hostname.rm1
        hadoop101
    
    
    
        yarn.resourcemanager.webapp.address.rm1
        hadoop101:8088
    
    
    
        yarn.resourcemanager.address.rm1
        hadoop101:8032
    
    
    
        yarn.resourcemanager.scheduler.address.rm1
        hadoop101:8030
    
    
    
        yarn.resourcemanager.resource-tracker.address.rm1
        hadoop101:8031
    
    
    
    
        yarn.resourcemanager.hostname.rm2
        hadoop102
    
    
        yarn.resourcemanager.webapp.address.rm2
        hadoop102:8088
    
    
        yarn.resourcemanager.address.rm2
        hadoop102:8032
    
    
        yarn.resourcemanager.scheduler.address.rm2
        hadoop102:8030
    
    
        yarn.resourcemanager.resource-tracker.address.rm2
        hadoop102:8031
    
    
    
    
        yarn.resourcemanager.hostname.rm3
        hadoop103
    
    
    
        yarn.resourcemanager.webapp.address.rm3
        hadoop103:8088
    
    
    
        yarn.resourcemanager.address.rm3
        hadoop103:8032
    
    
    
        yarn.resourcemanager.scheduler.address.rm3
        hadoop103:8030
    
    
    
        yarn.resourcemanager.resource-tracker.address.rm3
        hadoop103:8031
    
    
    
        yarn.resourcemanager.zk-address
        hadoop101:2181,hadoop102:2181,hadoop103:2181
    
    
    
        yarn.resourcemanager.recovery.enabled
        true
    
    
    
        yarn.resourcemanager.store.class
        org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore
    
    
    
        yarn.nodemanager.env-whitelist
        JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME
    

• 同步配置文件yarn-site.xml 

- 命令：hsync yarn-site.xml

• 启动yarn集群服务

 -命令：start-yarn.sh

•  访问yarn集群服务

-访问yarn集群服务地址查看yarn服务状态

 - 通过命令查看

• yarn服务故障转移

- 将hadoop101节点下线，查看ResourceManager的节点切换

- hadoop102切换为了主节点

结语

关于hadoop集群之YARN高可用实现自动故障转移的内容到这里就结束了，我们下期见。。。。。。