Flink on yarn Couldn't deploy Yarn cluster或者The YARN application unexpectedly switched to state FAIL

Error while deploying YARN cluster: Couldn't deploy Yarn cluster
java.lang.RuntimeException: Couldn't deploy Yarn cluster
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:443)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:630)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:486)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:483)
	at org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:483)
Caused by: org.apache.flink.yarn.AbstractYarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment. 
Diagnostics from YARN: Application application_1553768779226_0005 failed 1 times due to AM Container for appattempt_1553768779226_0005_000001 exited with  exitCode: -103
For more detailed output, check application tracking page:http://uplooking01:8088/proxy/application_1553768779226_0005/Then, click on links to logs of each attempt.
Diagnostics: Container [pid=2322,containerID=container_1553768779226_0005_01_000001] is running beyond virtual memory limits. Current usage: 75.1 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memor
y used. Killing container.Dump of the process-tree for container_1553768779226_0005_01_000001 :
	|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
	|- 2322 2321 2322 2322 (bash) 0 0 108650496 300 /bin/bash -c /opt/jdk/bin/java -Xmx424m  -Dlog.file=/opt/hadoop/logs/userlogs/application_1553768779226_0005/container_1553768779226_0005_01_000001/jobma
nager.log -Dlogback.configurationFile=file:logback.xml -Dlog4j.configuration=file:log4j.properties org.apache.flink.yarn.YarnApplicationMasterRunner  1> /opt/hadoop/logs/userlogs/application_1553768779226_0005/container_1553768779226_0005_01_000001/jobmanager.out 2> /opt/hadoop/logs/userlogs/application_1553768779226_0005/container_1553768779226_0005_01_000001/jobmanager.err 	|- 2330 2322 2322 2322 (java) 267 71 2165399552 18925 /opt/jdk/bin/java -Xmx424m -Dlog.file=/opt/hadoop/logs/userlogs/application_1553768779226_0005/container_1553768779226_0005_01_000001/jobmanager.lo
g -Dlogback.configurationFile=file:logback.xml -Dlog4j.configuration=file:log4j.properties org.apache.flink.yarn.YarnApplicationMasterRunner 
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
Failing this attempt. Failing the application.
If log aggregation is enabled on your cluster, use this command to further investigate the issue:
yarn logs -applicationId application_1553768779226_0005
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.startAppMaster(AbstractYarnClusterDescriptor.java:902)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:582)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:441)
	... 9 more
2019-03-28 06:46:32,070 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Cancelling deployment from Deployment Failure Hook
2019-03-28 06:46:32,070 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Killing YARN application
2019-03-28 06:46:32,082 INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl         - Killed application application_1553768779226_0005

出现此类错误,主要的原因是Current usage: 75.1 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memor
y used. Killing container.

字面原因是容器内存不够,实际上是flink on yarn启动时检查虚拟内存造成的

所以修改配置文件,让它不检查就没事了

修改etc/hadoop/yarn-site.xml

 

    yarn.nodemanager.vmem-check-enabled 

    false 

 

你可能感兴趣的:(Flink on yarn Couldn't deploy Yarn cluster或者The YARN application unexpectedly switched to state FAIL)