基于ambari的yarn-site.xml的分析与介绍
| key | value | 描述 |
|---|---|---|
| hadoop.registry.rm.enabled | true | yarn资源管理器中是否启用了注册表 |
| hadoop.registry.zk.quorum | node1:2181,node2:2181,node3:2181 | 用于定义注册表的zookeeper仲裁绑定 |
| manage.include.files | false | |
| yarn.acl.enable | false | 是否启用了acls |
| yarn.admin.acl | activity_analyzer,yarn | 谁可以成为YARN集群管理员的ACL |
| yarn.application.classpath | /usr/hdp/2.6.5.0-292/hadoop/conf,/usr/hdp/2.6.5.0-292/hadoop/,/usr/hdp/2.6.5.0-292/hadoop/lib/,/usr/hdp/current/hadoop-hdfs-client/,/usr/hdp/current/hadoop-hdfs-client/lib/,/usr/hdp/current/hadoop-yarn-client/,/usr/hdp/current/hadoop-yarn-client/lib/,/usr/hdp/current/ext/hadoop/* | |
| yarn.client.failover-proxy-provider | org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider | 在HA模式下配置故障转移的代理程序 |
| yarn.client.nodemanager-connect.max-wait-ms | 60000 | 连接到NM的最大等待时间 |
| yarn.client.nodemanager-connect.retry-interval-ms | 10000 | 每次尝试连接到NM的时间间隔 |
| yarn.http.policy | HTTP_ONLY | 设置是支持HTTP还是HTTPS |
| yarn.log-aggregation-enable | true | 是否启用日志聚合功能 |
| yarn.log-aggregation.file-controller.IndexedFormat.class | org.apache.hadoop.yarn.logaggregation.filecontroller.ifile.LogAggregationIndexedFileController | |
| yarn.log-aggregation.file-controller.TFile.class | org.apache.hadoop.yarn.logaggregation.filecontroller.tfile.LogAggregationTFileController | 支持TFile的读写操作类 |
| yarn.log-aggregation.file-formats | IndexedFormat,TFile | 收集日志的控制器必须要有TFile |
| yarn.log-aggregation.retain-seconds | 2592000 | 多久删除聚合日志 |
| yarn.log.server.url | http://node1:19888/jobhistory/logs | |
| yarn.log.server.web-service.url | http://node1:8188/ws/v1/applicationhistory | 日志聚合服务器Web服务的URL |
| yarn.node-labels.enabled | false | 启用节点标签功能 |
| yarn.node-labels.fs-store.retry-policy-spec | 2000, 500 | |
| yarn.node-labels.fs-store.root-dir | /system/yarn/node-labels | 节点标签的路径，/tmp/hadoop-yarn-${user}/node-labels/ |
| yarn.nodemanager.address | 0.0.0.0:45454 | |
| yarn.nodemanager.admin-env | MALLOC_ARENA_MAX=$MALLOC_ARENA_MAX | nodemanager的环境变量 |
| yarn.nodemanager.aux-services | mapreduce_shuffle,spark2_shuffle | shuffle服务的列表 |
| yarn.nodemanager.aux-services.mapreduce_shuffle.class | org.apache.hadoop.mapred.ShuffleHandler | MR的shuffle配置 |
| yarn.nodemanager.aux-services.spark2_shuffle.class | org.apache.spark.network.yarn.YarnShuffleService | |
| yarn.nodemanager.aux-services.spark2_shuffle.classpath | /usr/hdp/${hdp.version}/spark2/aux/* | |
| yarn.nodemanager.aux-services.spark_shuffle.class | org.apache.spark.network.yarn.YarnShuffleService | |
| yarn.nodemanager.aux-services.spark_shuffle.classpath | /usr/hdp/${hdp.version}/spark/aux/* | |
| yarn.nodemanager.bind-host | 0.0.0.0 | 绑定的主机 |
| yarn.nodemanager.container-executor.class | org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor | 谁将执行（启动）容器 |
| yarn.nodemanager.container-metrics.unregister-delay-ms | 60000 | 完成后取消注册容器指标的延迟时间ms。默认值是10000 |
| yarn.nodemanager.container-monitor.interval-ms | 3000 | 检查的间隔时间默认值3000ms |
| yarn.nodemanager.delete.debug-delay-sec | 0 | 本配置就是在程序运行完后是否立即删除所缓存的配置文件以及环境信息，默认值0 |
| yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage | 90 | 配置磁盘的最大使用比例，默认值90 |
| yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb | 1000 | 磁盘上必须使用的最小空间，默认值0 |
| yarn.nodemanager.disk-health-checker.min-healthy-disks | 0.25 | 启动新容器的最佳磁盘数量的最小比例，默认值0.25 |
| yarn.nodemanager.health-checker.interval-ms | 135000 | 运行节点运行状况脚本的频率，默认值600000 |
| yarn.nodemanager.health-checker.script.timeout-ms | 60000 | 脚本超时期限，默认值1200000 |
| yarn.nodemanager.kill-escape.launch-command-line | slider-agent,LLAP | |
| yarn.nodemanager.kill-escape.user | hive | |
| yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage | false | 此标志确定应用程序是应以严格的资源限制运行，还是允许在需要时使用备用资源，打开此值会降低吞吐量 |
| yarn.nodemanager.linux-container-executor.group | hadoop | |
| yarn.nodemanager.local-dirs | /data/hadoop/yarn/local | 在运行时对资源的一个本地化路径 |
| yarn.nodemanager.log-aggregation.compression-type | gz | 日志的压缩方式 |
| yarn.nodemanager.log-aggregation.debug-enabled | false | 是否开启dubug的日志 |
| yarn.nodemanager.log-aggregation.num-log-files-per-app | 336 | 定义远程文件系统中每个NM可以拥有的每个应用程序的聚合日志文件数，默认值30 |
| yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds | 3600 | 上传聚合日志的时间间隔，默认值3600 |
| yarn.nodemanager.log-dirs | /data/hadoop/yarn/log | 本地存储日志的目录 |
| yarn.nodemanager.log.retain-seconds | 604800 | 保留用户日志的时间（以秒为单位）仅适用于禁用日志聚合的情况 |
| yarn.nodemanager.recovery.dir | /data/var/log/hadoop-yarn/nodemanager/recovery-state | 启用恢复时节点管理器将在其中存储状态的本地文件系统目录 |
| yarn.nodemanager.recovery.enabled | true | 启动后启用节点管理器以进行恢复，默认值flase |
| yarn.nodemanager.remote-app-log-dir | /app-logs | 将日志聚合后的位置 |
| yarn.nodemanager.remote-app-log-dir-suffix | logs | 后缀名 |
| yarn.nodemanager.resource.cpu-vcores | 24 | 设置可用CPU的数量 |
| yarn.nodemanager.resource.detect-hardware-capabilities | false | 是否启用节点的自检功能，当内存与CPU配置为-1时将通过自检的方式获取 |
| yarn.nodemanager.resource.memory-mb | 57344 | |
| yarn.nodemanager.resource.percentage-physical-cpu-limit | 100 | 可以为容器分配的比例 |
| yarn.nodemanager.vmem-check-enabled | false | 是否将对容器强制实施虚拟内存限制，默认值是ture |
| yarn.nodemanager.vmem-pmem-ratio | 2.1 | 虚拟内存与分配的物理内存的比例 |
| yarn.resourcemanager.address | node:8050 | |
| yarn.resourcemanager.admin.address | node:8141 | RM管理界面的地址 |
| yarn.resourcemanager.am.max-attempts | 2 | AM全局重试次数，默认值2 |
| yarn.resourcemanager.bind-host | 0.0.0.0 | |
| yarn.resourcemanager.connect.max-wait.ms | -1 | 等待与ResourceManager建立连接的最长时间 |
| yarn.resourcemanager.connect.retry-interval.ms | 15000 | 间隔时间，默认值30000 |
| yarn.resourcemanager.fs.state-store.retry-policy-spec | 2000, 500 | |
| yarn.resourcemanager.fs.state-store.uri | 当使用文件存储时必须设置，对于RM的数据备份机制 | |
| yarn.resourcemanager.ha.enabled | false | 是否启用HA模式 |
| yarn.resourcemanager.hostname | localhost | RM主机地址 |
| yarn.resourcemanager.monitor.capacity.preemption.natural_termination_factor | 1 | |
| yarn.resourcemanager.monitor.capacity.preemption.total_preemption_per_round | 0.13 | |
| yarn.resourcemanager.nodes.exclude-path | /etc/hadoop/conf/yarn.exclude | 包含要排除的节点的文件路径 |
| yarn.resourcemanager.recovery.enabled | true | yarn.resourcemanager.store.class必须设置。启用RM恢复机制，默认值false |
| yarn.resourcemanager.resource-tracker.address | node:8025 | |
| yarn.resourcemanager.scheduler.address | node:8030 | |
| yarn.resourcemanager.scheduler.class | org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler | 用作资源调度程序的类 |
| yarn.resourcemanager.scheduler.monitor.enable | false | 启用一组影响调度程序的定期监视器，默认值false |
| yarn.resourcemanager.state-store.max-completed-applications | ${yarn.resourcemanager.max-completed-applications} | 保存完成应用的数量，默认值1000 |
| yarn.resourcemanager.store.class | org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore | |
| yarn.resourcemanager.system-metrics-publisher.dispatcher.pool-size | 10 | 发送yarn系统指标数据的工作线程数 |
| yarn.resourcemanager.system-metrics-publisher.enabled | true | 控制纱线系统指标是否在RM和NM上在时间轴服务上发布的设置，默认值false |
| yarn.resourcemanager.webapp.address | node1:8088 | |
| yarn.resourcemanager.webapp.delegation-token-auth-filter.enabled | false | 一种身份验证策略，默认true |
| yarn.resourcemanager.webapp.https.address | node:8090 | |
| yarn.resourcemanager.work-preserving-recovery.enabled | true | 启用RM工作以保留恢复。此配置对YARN是专用的，用于试验该功能 |
| yarn.resourcemanager.work-preserving-recovery.scheduling-wait-ms | 10000 | 设置RM在为工作保留 - 恢复分配新容器之前等待的时间。在将新容器分配给应用程序之前，这样的等待期使RM有机会在恢复时解决与集群中的NM重新同步的问题 |
| yarn.resourcemanager.zk-acl | world:anyone:rwcda | |
| yarn.resourcemanager.zk-address | node:2181 | zk的连接地址 |
| yarn.resourcemanager.zk-num-retries | 1000 | |
| yarn.resourcemanager.zk-retry-interval-ms | 1000 | |
| yarn.resourcemanager.zk-state-store.parent-path | /rmstore | |
| yarn.resourcemanager.zk-timeout-ms | 10000 | |
| yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled | false | |
| yarn.scheduler.maximum-allocation-mb | 57344 | RM中每个容器请求的最大分配，以MB为单位 |
| yarn.scheduler.maximum-allocation-vcores | 12 | |
| yarn.scheduler.minimum-allocation-mb | 1024 | RM中每个容器请求的最小分配（MB）。低于此值的内存请求将设置为此属性的值。 |
| yarn.scheduler.minimum-allocation-vcores | 1 | |
| yarn.timeline-service.address | node:10200 | |
| yarn.timeline-service.bind-host | 0.0.0.0 | |
| yarn.timeline-service.client.fd-flush-interval-secs | 5 | |
| yarn.timeline-service.client.max-retries | 30 | |
| yarn.timeline-service.client.retry-interval-ms | 1000 | |
| yarn.timeline-service.enabled | true | |
| yarn.timeline-service.entity-group-fs-store.active-dir | /ats/active/ | |
| yarn.timeline-service.entity-group-fs-store.app-cache-size | 10 | |
| yarn.timeline-service.entity-group-fs-store.cleaner-interval-seconds | 3600 | |
| yarn.timeline-service.entity-group-fs-store.done-dir | /ats/done/ | |
| yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes | org.apache.tez.dag.history.logging.ats.TimelineCachePluginImpl | |
| yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath | ||
| yarn.timeline-service.entity-group-fs-store.retain-seconds | 604800 | |
| yarn.timeline-service.entity-group-fs-store.scan-interval-seconds | 15 | |
| yarn.timeline-service.entity-group-fs-store.summary-store | org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore | |
| yarn.timeline-service.generic-application-history.store-class | org.apache.hadoop.yarn.server.applicationhistoryservice.NullApplicationHistoryStore | |
| yarn.timeline-service.http-authentication.proxyuser.root.groups | * | |
| yarn.timeline-service.http-authentication.proxyuser.root.hosts | node | |
| yarn.timeline-service.http-authentication.simple.anonymous.allowed | true | |
| yarn.timeline-service.http-authentication.type | simple | |
| yarn.timeline-service.leveldb-state-store.path | /hadoop/yarn/timeline | |
| yarn.timeline-service.leveldb-timeline-store.path | /hadoop/yarn/timeline | |
| yarn.timeline-service.leveldb-timeline-store.read-cache-size | 104857600 | |
| yarn.timeline-service.leveldb-timeline-store.start-time-read-cache-size | 10000 | |
| yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size | 10000 | |
| yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms | 300000 | |
| yarn.timeline-service.recovery.enabled | true | |
| yarn.timeline-service.state-store-class | org.apache.hadoop.yarn.server.timeline.recovery.LeveldbTimelineStateStore | |
| yarn.timeline-service.store-class | org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore | |
| yarn.timeline-service.ttl-enable | true | |
| yarn.timeline-service.ttl-ms | 2678400000 | |
| yarn.timeline-service.version | 1.5 | |
| yarn.timeline-service.webapp.address | node:8188 | |
| yarn.timeline-service.webapp.https.address | node:8190 |
注：关于时间线的不做过多介绍。