Hadoop基础教程-第5章 YARN：资源调度平台（5.6 YARN的命令）（草稿）
yarn命令的用法
[root@node1 ~]# yarn
Usage: yarn [--config confdir] [COMMAND | CLASSNAME]
CLASSNAME run the class named CLASSNAME
or
where COMMAND is one of:
resourcemanager -format-state-store deletes the RMStateStore
resourcemanager run the ResourceManager
nodemanager run a nodemanager on each slave
timelineserver run the timeline server
rmadmin admin tools
sharedcachemanager run the SharedCacheManager daemon
scmadmin SharedCacheManager admin tools
version print the version
jar run a jar file
application prints application(s)
report/kill application
applicationattempt prints applicationattempt(s)
report
container prints container(s) report
node prints node report(s)
queue prints queue information
logs dump container logs
classpath prints the class path needed to
get the Hadoop jar and the
required libraries
cluster prints cluster information
daemonlog get/set the log level for each
daemon classpath
使用: yarn classpath
打印需要得到Hadoop的jar和所需要的lib包路径
[root@node1 ~]# yarn classpath
/opt/hadoop-2.7.3/etc/hadoop:/opt/hadoop-2.7.3/etc/hadoop:/opt/hadoop-2.7.3/etc/hadoop:/opt/hadoop-2.7.3/share/hadoop/common/lib/*:/opt/hadoop-2.7.3/share/hadoop/common/*:/opt/hadoop-2.7.3/share/hadoop/hdfs:/opt/hadoop-2.7.3/share/hadoop/hdfs/lib/*:/opt/hadoop-2.7.3/share/hadoop/hdfs/*:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/*:/opt/hadoop-2.7.3/share/hadoop/yarn/*:/opt/hadoop-2.7.3/share/hadoop/mapreduce/lib/*:/opt/hadoop-2.7.3/share/hadoop/mapreduce/*:/opt/hadoop-2.7.3/contrib/capacity-scheduler/*.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/*:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/*查看节点列表
[root@node1 ~]# yarn node -list
17/05/26 08:00:08 INFO client.RMProxy: Connecting to ResourceManager at node1/192.168.80.131:8032
Total Nodes:3
Node-Id Node-State Node-Http-Address Number-of-Running-Containers
node3:37523 RUNNING node3:8042 0
node2:35644 RUNNING node2:8042 0
node1:33105 RUNNING node1:8042 0查看节点状况
[root@node1 ~]# yarn node -status node3:37523
17/05/26 08:01:33 INFO client.RMProxy: Connecting to ResourceManager at node1/192.168.80.131:8032
Node Report :
Node-Id : node3:37523
Rack : /default-rack
Node-State : RUNNING
Node-Http-Address : node3:8042
Last-Health-Update : Fri 26/May/17 08:00:17:963EDT
Health-Report :
Containers : 0
Memory-Used : 0MB
Memory-Capacity : 8192MB
CPU-Used : 0 vcores
CPU-Capacity : 8 vcores
Node-Labels : 先停止YARN，然后通过yarn命令来启动resourcemanager和nodemanager
[root@node1 ~]# stop-yarn.sh
stopping yarn daemons
stopping resourcemanager
node2: stopping nodemanager
node1: stopping nodemanager
node3: stopping nodemanager
node2: nodemanager did not stop gracefully after 5 seconds: killing with kill -9
node1: nodemanager did not stop gracefully after 5 seconds: killing with kill -9
node3: nodemanager did not stop gracefully after 5 seconds: killing with kill -9JPS命令，已经没有YARN的相关服务了
[root@node1 ~]# jps
2247 NameNode
2348 DataNode
4686 Jps[root@node2 ~]# jps
2137 DataNode
2201 SecondaryNameNode
3660 Jps[root@node3 ~]# jps
8167 Jps
5179 DataNode[root@node1 ~]# yarn resourcemanager
17/05/26 09:08:07 INFO resourcemanager.ResourceManager: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting ResourceManager
STARTUP_MSG: host = node1/192.168.80.131
STARTUP_MSG: args = []
STARTUP_MSG: version = 2.7.3
STARTUP_MSG: classpath = /opt/hadoop-2.7.3/etc/hadoop:/opt/hadoop-2.7.3/etc/hadoop:/opt/hadoop-2.7.3/etc/hadoop:/opt/hadoop-2.7.3/share/hadoop/common/lib/jsr305-3.0.0.jar:/opt/hadoop-:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-api-2.7.3.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-common-2.7.3.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-server-common-2.7.3.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.3.jar:/opt/hadoop-2.7.3.jar:/opt/hadoop-2.7.3/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.7.3/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/opt/hadoop-2.7.3/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/asm-3.2.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/jersey-json-1.9.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/jettison-1.1.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/hadoop-2.7.3/etc/hadoop/rm-config/log4j.properties
STARTUP_MSG: build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r baa91f7c6bc9cb92be5982de4719c1c8af91ccff; compiled by 'root' on 2016-08-18T01:41Z
STARTUP_MSG: java = 1.8.0_112
************************************************************/
17/05/26 09:08:07 INFO resourcemanager.ResourceManager: registered UNIX signal handlers for [TERM, HUP, INT]
17/05/26 09:08:08 INFO conf.Configuration: found resource core-site.xml at file:/opt/hadoop-2.7.3/etc/hadoop/core-site.xml
17/05/26 09:08:08 INFO security.Groups: clearing userToGroupsMap cache
17/05/26 09:08:08 INFO conf.Configuration: found resource yarn-site.xml at file:/opt/hadoop-2.7.3/etc/hadoop/yarn-site.xml
17/05/26 09:08:09 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher
17/05/26 09:08:09 INFO security.NMTokenSecretManagerInRM: NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms
17/05/26 09:08:09 INFO security.RMContainerTokenSecretManager: ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms
17/05/26 09:08:09 INFO security.AMRMTokenSecretManager: AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms
17/05/26 09:08:09 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler
17/05/26 09:08:09 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager
17/05/26 09:08:09 INFO resourcemanager.ResourceManager: Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
17/05/26 09:08:09 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher
17/05/26 09:08:09 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher
17/05/26 09:08:09 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher
17/05/26 09:08:09 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher
17/05/26 09:08:10 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
17/05/26 09:08:10 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
17/05/26 09:08:10 INFO impl.MetricsSystemImpl: ResourceManager metrics system started
17/05/26 09:08:10 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager
17/05/26 09:08:10 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher
17/05/26 09:08:10 INFO resourcemanager.RMNMInfo: Registered RMNMInfo MBean
17/05/26 09:08:10 INFO security.YarnAuthorizationProvider: org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer is instiantiated.
17/05/26 09:08:10 INFO util.HostsFileReader: Refreshing hosts (include/exclude) list
17/05/26 09:08:10 INFO conf.Configuration: found resource capacity-scheduler.xml at file:/opt/hadoop-2.7.3/etc/hadoop/capacity-scheduler.xml
17/05/26 09:08:10 INFO capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root is undefined
17/05/26 09:08:10 INFO capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root is undefined
17/05/26 09:08:10 INFO capacity.ParentQueue: root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*,
, reservationsContinueLooking=true
17/05/26 09:08:10 INFO capacity.ParentQueue: Initialized parent-queue root name=root, fullname=root
17/05/26 09:08:10 INFO capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root.default is undefined
17/05/26 09:08:10 INFO capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root.default is undefined
17/05/26 09:08:10 INFO capacity.LeafQueue: Initializing default
capacity = 1.0 [= (float) configuredCapacity / 100 ]
asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ]
maxCapacity = 1.0 [= configuredMaxCapacity ]
absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ]
userLimit = 100 [= configuredUserLimit ]
userLimitFactor = 1.0 [= configuredUserLimitFactor ]
maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)]
maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ]
usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)]
absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory]
maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ]
minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ]
maximumAllocation = [= configuredMaxAllocation ]
numContainers = 0 [= currentNumContainers ]
state = RUNNING [= configuredState ]
acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ]
nodeLocalityDelay = 40
labels=*,
nodeLocalityDelay = 40
reservationsContinueLooking = true
preemptionDisabled = true
17/05/26 09:08:10 INFO capacity.CapacityScheduler: Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0
17/05/26 09:08:10 INFO capacity.CapacityScheduler: Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0
17/05/26 09:08:10 INFO capacity.CapacityScheduler: Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0
17/05/26 09:08:10 INFO capacity.CapacityScheduler: Initialized queue mappings, override: false
17/05/26 09:08:10 INFO capacity.CapacityScheduler: Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<>, asynchronousScheduling=false, asyncScheduleInterval=5ms
17/05/26 09:08:10 INFO metrics.SystemMetricsPublisher: YARN system metrics publishing service is not enabled
17/05/26 09:08:10 INFO resourcemanager.ResourceManager: Transitioning to active state
17/05/26 09:08:10 INFO recovery.RMStateStore: Updating AMRMToken
17/05/26 09:08:10 INFO security.RMContainerTokenSecretManager: Rolling master-key for container-tokens
17/05/26 09:08:10 INFO security.NMTokenSecretManagerInRM: Rolling master-key for nm-tokens
17/05/26 09:08:10 INFO delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
17/05/26 09:08:10 INFO security.RMDelegationTokenSecretManager: storing master key with keyID 1
17/05/26 09:08:10 INFO recovery.RMStateStore: Storing RMDTMasterKey.
17/05/26 09:08:10 INFO delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s)
17/05/26 09:08:10 INFO delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
17/05/26 09:08:10 INFO security.RMDelegationTokenSecretManager: storing master key with keyID 2
17/05/26 09:08:10 INFO recovery.RMStateStore: Storing RMDTMasterKey.
17/05/26 09:08:10 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler
17/05/26 09:08:10 INFO ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
17/05/26 09:08:10 INFO ipc.Server: Starting Socket Reader #1 for port 8031
17/05/26 09:08:10 INFO pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server
17/05/26 09:08:10 INFO ipc.Server: IPC Server Responder: starting
17/05/26 09:08:10 INFO ipc.Server: IPC Server listener on 8031: starting
17/05/26 09:08:10 INFO ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
17/05/26 09:08:10 INFO ipc.Server: Starting Socket Reader #1 for port 8030
17/05/26 09:08:10 INFO pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server
17/05/26 09:08:10 INFO ipc.Server: IPC Server Responder: starting
17/05/26 09:08:10 INFO ipc.Server: IPC Server listener on 8030: starting
17/05/26 09:08:10 INFO ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
17/05/26 09:08:10 INFO ipc.Server: Starting Socket Reader #1 for port 8032
17/05/26 09:08:10 INFO pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server
17/05/26 09:08:10 INFO ipc.Server: IPC Server Responder: starting
17/05/26 09:08:10 INFO ipc.Server: IPC Server listener on 8032: starting
17/05/26 09:08:10 INFO resourcemanager.ResourceManager: Transitioned to active state
17/05/26 09:08:11 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
17/05/26 09:08:11 INFO server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
17/05/26 09:08:11 INFO http.HttpRequestLog: Http request log for http.requests.resourcemanager is not defined
17/05/26 09:08:11 INFO http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
17/05/26 09:08:11 INFO http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster
17/05/26 09:08:11 INFO http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs
17/05/26 09:08:11 INFO http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static
17/05/26 09:08:11 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster
17/05/26 09:08:11 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
17/05/26 09:08:11 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
17/05/26 09:08:11 INFO http.HttpServer2: adding path spec: /cluster/*
17/05/26 09:08:11 INFO http.HttpServer2: adding path spec: /ws/*
17/05/26 09:08:11 INFO webapp.WebApps: Registered webapp guice modules
17/05/26 09:08:11 INFO http.HttpServer2: Jetty bound to port 8088
17/05/26 09:08:11 INFO mortbay.log: jetty-6.1.26
17/05/26 09:08:12 INFO mortbay.log: Extract jar:file:/opt/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-common-2.7.3.jar!/webapps/cluster to /tmp/Jetty_node1_8088_cluster____uqk9cr/webapp
17/05/26 09:08:12 INFO delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
17/05/26 09:08:12 INFO delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s)
17/05/26 09:08:12 INFO delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
May 26, 2017 9:08:12 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class
May 26, 2017 9:08:12 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class
May 26, 2017 9:08:12 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
May 26, 2017 9:08:12 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
May 26, 2017 9:08:13 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
May 26, 2017 9:08:13 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
May 26, 2017 9:08:14 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton"
17/05/26 09:08:14 INFO mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@node1:8088
17/05/26 09:08:14 INFO webapp.WebApps: Web app cluster started at 8088
17/05/26 09:08:14 INFO ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
17/05/26 09:08:14 INFO ipc.Server: Starting Socket Reader #1 for port 8033
17/05/26 09:08:14 INFO pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server
17/05/26 09:08:14 INFO ipc.Server: IPC Server Responder: starting
17/05/26 09:08:14 INFO ipc.Server: IPC Server listener on 8033: starting [root@node2 ~]# yarn nodemanager
17/05/26 09:11:19 INFO nodemanager.NodeManager: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NodeManager
STARTUP_MSG: host = node2/192.168.80.132
STARTUP_MSG: args = []
STARTUP_MSG: version = 2.7.3
STARTUP_MSG: classpath = /opt/hadoop-2.7.3/etc/hadoop:/opt/hadoop-2.7.3/etc/hadoop:/opt/hadoop-2.7.3/etc/hadoop:/opt/hadoop-2.7.3/share/hadoop/common/lib/jsr305-3.0.0.jar:/opt/hadoop-2.7.3/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/hadoop-2.7.3/share/hadoop/common/lib/commons-math3-3.1.1.jar:/opt/hadoop-2.7.3/share/hadoop/common/lib/xmlenc-0.52.jar:/opt/hadoop-2.7.3/share/hadoop/common/lib/commons-httpclient-3.1.jar:/opt/hadoop-2.7.3/share/hadoop/common/lib/commons-logging-1.1.3.jar:/opt/hadoop-2.7.3/share/hadoop/common/lib/commons-codec-1.4.jar:/opt/hadoop-2.7.3/share/hadoop/common/lib/commons-io-2.4.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop-2.7.3/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/hadoop-2.7.3/etc/hadoop/nm-config/log4j.properties
STARTUP_MSG: build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r baa91f7c6bc9cb92be5982de4719c1c8af91ccff; compiled by 'root' on 2016-08-18T01:41Z
STARTUP_MSG: java = 1.8.0_112
************************************************************/
17/05/26 09:11:19 INFO nodemanager.NodeManager: registered UNIX signal handlers for [TERM, HUP, INT]
17/05/26 09:11:21 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher
17/05/26 09:11:21 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher
17/05/26 09:11:21 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService
17/05/26 09:11:21 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices
17/05/26 09:11:21 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
17/05/26 09:11:21 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher
17/05/26 09:11:21 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl
17/05/26 09:11:21 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.NodeManager
17/05/26 09:11:21 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
17/05/26 09:11:21 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
17/05/26 09:11:21 INFO impl.MetricsSystemImpl: NodeManager metrics system started
17/05/26 09:11:21 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler
17/05/26 09:11:21 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService
17/05/26 09:11:21 INFO localizer.ResourceLocalizationService: per directory file limit = 8192
17/05/26 09:11:22 INFO localizer.ResourceLocalizationService: usercache path : file:/var/data/hadoop/nm-local-dir/usercache_DEL_1495804281967
17/05/26 09:11:22 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker
17/05/26 09:11:22 WARN containermanager.AuxServices: The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
17/05/26 09:11:22 INFO containermanager.AuxServices: Adding auxiliary service httpshuffle, "mapreduce_shuffle"
17/05/26 09:11:22 INFO monitor.ContainersMonitorImpl: Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@a8ef162
17/05/26 09:11:22 INFO monitor.ContainersMonitorImpl: Using ResourceCalculatorProcessTree : null
17/05/26 09:11:22 INFO monitor.ContainersMonitorImpl: Physical memory check enabled: true
17/05/26 09:11:22 INFO monitor.ContainersMonitorImpl: Virtual memory check enabled: true
17/05/26 09:11:22 WARN monitor.ContainersMonitorImpl: NodeManager configured with 8 G physical memory allocated to containers, which is more than 80% of the total physical memory available (1.8 G). Thrashing might happen.
17/05/26 09:11:22 INFO nodemanager.NodeStatusUpdaterImpl: Initialized nodemanager for null: physical-memory=8192 virtual-memory=17204 virtual-cores=8
17/05/26 09:11:22 INFO ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
17/05/26 09:11:22 INFO ipc.Server: Starting Socket Reader #1 for port 38556
17/05/26 09:11:22 INFO pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server
17/05/26 09:11:22 INFO containermanager.ContainerManagerImpl: Blocking new container-requests as container manager rpc server is still starting.
17/05/26 09:11:22 INFO security.NMContainerTokenSecretManager: Updating node address : node2:38556
17/05/26 09:11:22 INFO ipc.Server: IPC Server Responder: starting
17/05/26 09:11:22 INFO ipc.Server: IPC Server listener on 38556: starting
17/05/26 09:11:22 INFO ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
17/05/26 09:11:22 INFO ipc.Server: Starting Socket Reader #1 for port 8040
17/05/26 09:11:22 INFO pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server
17/05/26 09:11:22 INFO ipc.Server: IPC Server Responder: starting
17/05/26 09:11:22 INFO ipc.Server: IPC Server listener on 8040: starting
17/05/26 09:11:22 INFO localizer.ResourceLocalizationService: Localizer started on port 8040
17/05/26 09:11:22 INFO mapred.IndexCache: IndexCache created with max memory = 10485760
17/05/26 09:11:22 INFO mapred.ShuffleHandler: httpshuffle listening on port 13562
17/05/26 09:11:22 INFO containermanager.ContainerManagerImpl: ContainerManager started at node2/192.168.80.132:38556
17/05/26 09:11:22 INFO containermanager.ContainerManagerImpl: ContainerManager bound to 0.0.0.0/0.0.0.0:0
17/05/26 09:11:22 INFO webapp.WebServer: Instantiating NMWebApp at 0.0.0.0:8042
17/05/26 09:11:22 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
17/05/26 09:11:22 INFO server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
17/05/26 09:11:22 INFO http.HttpRequestLog: Http request log for http.requests.nodemanager is not defined
17/05/26 09:11:22 INFO http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
17/05/26 09:11:22 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node
17/05/26 09:11:22 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
17/05/26 09:11:22 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
17/05/26 09:11:22 INFO http.HttpServer2: adding path spec: /node/*
17/05/26 09:11:22 INFO http.HttpServer2: adding path spec: /ws/*
17/05/26 09:11:23 INFO webapp.WebApps: Registered webapp guice modules
17/05/26 09:11:23 INFO http.HttpServer2: Jetty bound to port 8042
17/05/26 09:11:23 INFO mortbay.log: jetty-6.1.26
17/05/26 09:11:23 INFO mortbay.log: Extract jar:file:/opt/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-common-2.7.3.jar!/webapps/node to /tmp/Jetty_0_0_0_0_8042_node____19tj0x/webapp
May 26, 2017 9:11:24 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class
May 26, 2017 9:11:24 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
May 26, 2017 9:11:24 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class
May 26, 2017 9:11:24 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
May 26, 2017 9:11:24 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
May 26, 2017 9:11:24 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
May 26, 2017 9:11:25 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton"
17/05/26 09:11:25 INFO mortbay.log: Started [email protected]:8042
17/05/26 09:11:25 INFO webapp.WebApps: Web app node started at 8042
17/05/26 09:11:25 INFO client.RMProxy: Connecting to ResourceManager at node1/192.168.80.131:8031
17/05/26 09:11:26 INFO nodemanager.NodeStatusUpdaterImpl: Sending out 0 NM container statuses: []
17/05/26 09:11:26 INFO nodemanager.NodeStatusUpdaterImpl: Registering with RM using containers :[]
17/05/26 09:11:26 INFO security.NMContainerTokenSecretManager: Rolling master-key for container-tokens, got key with id -1439781476
17/05/26 09:11:26 INFO security.NMTokenSecretManagerInNM: Rolling master-key for container-tokens, got key with id 559454324
17/05/26 09:11:26 INFO nodemanager.NodeStatusUpdaterImpl: Registered with ResourceManager as node2:38556 with total resource of
17/05/26 09:11:26 INFO nodemanager.NodeStatusUpdaterImpl: Notifying ContainerManager to unblock new container-requests 查看YARN版本
[root@node1 ~]# yarn version
Hadoop 2.7.3
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r baa91f7c6bc9cb92be5982de4719c1c8af91ccff
Compiled by root on 2016-08-18T01:41Z
Compiled with protoc 2.5.0
From source with checksum 2e4ce5f957ea4db193bce3734ff29ff4
This command was run using /opt/hadoop-2.7.3/share/hadoop/common/hadoop-common-2.7.3.jar