<官网学大数据>YARN

YARN官网介绍地址：http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YARN.html

The fundamental(基本的) idea of YARN is to split up the functionalities of resource management and job scheduling/monitoring(调度/监视)into separate daemons(守护进程). The idea is to have a global ResourceManager (RM) and per-application ApplicationMaster (AM). An application is either a single job or a DAG of jobs.

YARN把资源管理和作业调度/监视两个功能切分，交由两个不同的守护进程进行管理。全局资源管理（RM）和每个应用的管理者（AM）。一个应用可以是一个单独的job，也可以是job的DAG图。

The ResourceManager and the NodeManager form the data-computation framework. The ResourceManager is the ultimate authority(最终权限) that arbitrates(仲裁) resources among all the applications in the system. The NodeManager is the per-machine framework agent who is responsible for containers, monitoring their resource usage (cpu, memory, disk, network) and reporting the same to the ResourceManager/Scheduler.

RM负责系统中所有应用资源的管理，NM是RM在每台机器上的代理，负责该机器容器（containers）的管理，监视并且向RM汇报容器资源（cpu、内存、磁盘、网络）使用情况。

The per-application ApplicationMaster is, in effect, a framework specific library and is tasked with negotiating(协调) resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks.

AM的任务是向RM申请资源、和NM协作执行和监视任务

The ResourceManager has two main components: Scheduler and ApplicationsManager.

RM主要由调度器（Scheduler）和应用管理器（ApplicationsManager）两部分组成

The Scheduler is responsible for allocating(分配) resources to the various running applications subject to familiar constraints(限制) of capacities(容量), queues etc. The Scheduler is pure scheduler in the sense that it performs no monitoring or tracking(追踪) of status for the application. Also, it offers no guarantees(保证) about restarting failed tasks either due to application failure or hardware failures. The Scheduler performs its scheduling function based on the resource requirements of the applications; it does so based on the abstract notion(概念) of a resource Container which incorporates(包含) elements such as memory, cpu, disk, network etc.

调度器负责给各种正在运行的和有相似的约束如容量、队列等的应用分配资源。调度器只是一个纯粹的调度器而不负责应用状态的监视和追踪。并且它也不负责重启由于程序故障和硬件故障导致失败的任务。调度器根据应用的资源需求来执行它的调度功能，主要是基于资源“container”的抽象概念，包含内存、cpu、硬盘和网络等元素，

The Scheduler has a pluggable(可插拔) policy(策略) which is responsible for partitioning the cluster resources among the various queues, applications etc. The current schedulers such as the CapacityScheduler and the FairScheduler would be some examples of plug-ins(插件).

调度器是可插拔的组件，负责将集群资源分配各种队列、应用等。目前有 CapacityScheduler和FairScheduler可作为示例。

The ApplicationsManager is responsible for accepting job-submissions, negotiating(协商) the first container for executing the application specific ApplicationMaster and provides the service for restarting the ApplicationMaster container on failure. The per-application ApplicationMaster has the responsibility of negotiating appropriate resource containers from the Scheduler, tracking their status and monitoring for progress.

应用管理器负责接收作业的提交，选择第一个容器来运行AM并提供AM容器失败的重启服务。AM负责向调度器申请合适的资源容器，追踪和监视他们的状态

MapReduce in hadoop-2.x maintains(维持) API compatibility(兼容性) with previous stable release(版本) (hadoop-1.x). This means that all MapReduce jobs should still run unchanged on top of YARN with just a recompile.

hadoop2.x完全兼容hadoop1.x的api，所有的MapReduce作业只需重新编译就可以在YARN上运行。

YARN supports the notion of resource reservation via (通过)the ReservationSystem, a component that allows users to specify a profile(指定配置文件) of resources over-time and temporal constraints (时间约束)(e.g., deadlines), and reserve resources to ensure the predictable execution of important jobs.The ReservationSystem tracks resources over-time, performs admission control for reservations, and dynamically (动态)instruct the underlying scheduler(基础调度) to ensure that the reservation is fullfilled.

YARN通过ReservationSystem支持资源预留的概念，允许用户通过配置文件指定时间资源和时间约束（例：截止日期），并预留资源以确保重要的job能可预测执行。ReservationSystem跟踪资源超时，执行预留的准入控制，并动态指示基础调度程序确保预留已满。

In order to scale YARN beyond few thousands nodes, YARN supports the notion of Federation via the YARN Federation feature. Federation allows to transparently(透明的) wire together(连在一起) multiple yarn (sub-)clusters, and make them appear as a single massive(大规模的) cluster. This can be used to achieve larger scale, and/or to allow multiple independent clusters to be used together for very large jobs, or for tenants(租户) who have capacity across all of them.

为了将YARN扩展到超过几千节点，YARN通过YARN Federation特性来支持联邦的概念。联邦允许透明的将yarn的子集群连接在一起,让他们表现的像大规模集群。这样可以达到很大的扩展，多个独立的集群可以一起用于很大的job。