spark错误日志查看方法
查看错误日志yarn logs -applicationId application_1512098667219_5898489
yarn logs -applicationId xxx执行方式
spark-submit \
--master yarn-cluster \
--class com.jd.astron.jrdm.ClusterTest \
--num-executors 1 \
--executor-memory 1g \
--driver-memory 1g \
--executor-cores 1 \
--queue bdp_jmart_astron_union.bdp_jmart_astron_formal \
/home/mart_astron/data_dir/astron/sparktest/jrdm_astron_test-1.0-SNAPSHOT.jar出错日志
[mart_astron@BJLFRZ-Client-9-199 sparktest]$ spark-submit --master yarn-cluster --class com.astron.astron.jrdm.ClusterTest --num-executors 1 --executor-memory 1g --driver-memory 1g --executor-cores 1 --queue bdp_jmart_astron_union.bdp_jmart_astron_formal /home/mart_astron/data_dir/shida6/sparktest/jrdm_astron_test-1.0-SNAPSHOT.jar
Warning: Master yarn-cluster is deprecated since 2.0. Please use master "yarn" with specified deploy mode instead.
18/01/03 21:07:05 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/01/03 21:07:06 INFO ConfiguredRMFailoverProxyProvider: Failing over to rm2
18/01/03 21:07:06 INFO Client: Requesting a new application from cluster with 4145 NodeManagers
18/01/03 21:07:06 INFO Client: Verifying our application has not requested more than the maximum memory capability of the cluster (53248 MB per container)
18/01/03 21:07:06 INFO Client: Will allocate AM container, with 1408 MB memory including 384 MB overhead
18/01/03 21:07:06 INFO Client: Setting up container launch context for our AM
18/01/03 21:07:06 INFO Client: Setting up the launch environment for our AM container
18/01/03 21:07:06 INFO Client: Preparing resources for our AM container
18/01/03 21:07:07 INFO Client: Uploading resource hdfs://ns2/user/spark/lib/spark-2.1.0-hadoop-2.7.1.zip -> hdfs://ns12/user/mart_astron/.sparkStaging/application_1512098667219_5899268/spark-2.1.0-hadoop-2.7.1.zip
18/01/03 21:07:08 INFO Client: Uploading resource file:/home/mart_astron/data_dir/shida6/sparktest/jrdm_astron_test-1.0-SNAPSHOT.jar -> hdfs://ns12/user/mart_astron/.sparkStaging/application_1512098667219_5899268/jrdm_astron_test-1.0-SNAPSHOT.jar
18/01/03 21:07:08 INFO Client: Uploading resource file:/data0/tmp/spark-46f0c288-8591-49eb-8273-7140b3f3e9c5/__spark_conf__7338282222566816143.zip -> hdfs://ns12/user/mart_astron/.sparkStaging/application_1512098667219_5899268/__spark_conf__.zip
18/01/03 21:07:08 INFO Client: ===============================================================================
18/01/03 21:07:08 INFO Client: YARN AM launch context:
18/01/03 21:07:08 INFO Client: user class: com.astron.astron.jrdm.ClusterTest
18/01/03 21:07:08 INFO Client: env:
18/01/03 21:07:08 INFO Client: CLASSPATH -> {{PWD}}{{PWD}}/__spark_conf__{{PWD}}/__spark_libs__/*<CPS>$HADOOP_CONF_DIR$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*
18/01/03 21:07:08 INFO Client: SPARK_YARN_STAGING_DIR -> hdfs://ns12/user/mart_astron/.sparkStaging/application_1512098667219_5899268
18/01/03 21:07:08 INFO Client: SPARK_USER -> mart_astron
18/01/03 21:07:08 INFO Client: SPARK_YARN_MODE -> true
18/01/03 21:07:08 INFO Client: resources:
18/01/03 21:07:08 INFO Client: __app__.jar -> resource { scheme: "hdfs" host: "ns12" port: -1 file: "/user/mart_astron/.sparkStaging/application_1512098667219_5899268/jrdm_astron_test-1.0-SNAPSHOT.jar" } size: 48852 timestamp: 1514984828432 type: FILE visibility: PRIVATE
18/01/03 21:07:08 INFO Client: __spark_libs__ -> resource { scheme: "hdfs" host: "ns12" port: -1 file: "/user/mart_astron/.sparkStaging/application_1512098667219_5899268/spark-2.1.0-hadoop-2.7.1.zip" } size: 178778732 timestamp: 1514984828309 type: ARCHIVE visibility: PRIVATE
18/01/03 21:07:08 INFO Client: __spark_conf__ -> resource { scheme: "hdfs" host: "ns12" port: -1 file: "/user/mart_astron/.sparkStaging/application_1512098667219_5899268/__spark_conf__.zip" } size: 273316 timestamp: 1514984828499 type: ARCHIVE visibility: PRIVATE
18/01/03 21:07:08 INFO Client: command:
18/01/03 21:07:08 INFO Client: LD_LIBRARY_PATH="/software/servers/hadoop-2.7.1/lib/native:$LD_LIBRARY_PATH" {{JAVA_HOME}}/bin/java -server -Xmx1024m -Djava.io.tmpdir={{PWD}}/tmp -Dspark.yarn.app.container.log.dir= org.apache.spark.deploy.yarn.ApplicationMaster --class 'com.astron.astron.jrdm.ClusterTest' --jar file:/home/mart_astron/data_dir/shida6/sparktest/jrdm_astron_test-1.0-SNAPSHOT.jar --properties-file {{PWD}}/__spark_conf__/__spark_conf__.properties 1> /spark_stdout 2> /spark _stderr
18/01/03 21:07:08 INFO Client: ===============================================================================
18/01/03 21:07:08 INFO SecurityManager: Changing view acls to: mart_astron
18/01/03 21:07:08 INFO SecurityManager: Changing modify acls to: mart_astron
18/01/03 21:07:08 INFO SecurityManager: Changing view acls groups to:
18/01/03 21:07:08 INFO SecurityManager: Changing modify acls groups to:
18/01/03 21:07:08 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(mart_astron); groups with view permissions: Set(); users with modify permissions: Set(mart_astron); groups with modify permissions: Set()
18/01/03 21:07:08 INFO Client: Submitting application application_1512098667219_5899268 to ResourceManager
18/01/03 21:07:10 INFO YarnClientImpl: Application submission is not finished, submitted application application_1512098667219_5899268 is still in SUBMITTED
18/01/03 21:07:12 INFO YarnClientImpl: Application submission is not finished, submitted application application_1512098667219_5899268 is still in SUBMITTED
18/01/03 21:07:13 INFO YarnClientImpl: Submitted application application_1512098667219_5899268
18/01/03 21:07:14 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:14 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.bdp_jmart_astron_union.bdp_jmart_astron_formal
start time: 1514984828664
final status: UNDEFINED
tracking URL: http://BJLFRZ-10k-234.hadoop.astron.local:50320/proxy/application_1512098667219_5899268/
user: mart_astron
18/01/03 21:07:15 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:16 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:17 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:18 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:19 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:20 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:21 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:22 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:23 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:24 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:25 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:26 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:27 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:28 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:29 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:29 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 172.21.83.43
ApplicationMaster RPC port: 0
queue: root.bdp_jmart_astron_union.bdp_jmart_astron_formal
start time: 1514984828664
final status: UNDEFINED
tracking URL: http://BJLFRZ-10k-234.hadoop.astron.local:50320/proxy/application_1512098667219_5899268/
user: mart_astron
18/01/03 21:07:30 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:31 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:32 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:33 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:34 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:35 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:36 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:37 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:38 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:39 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:40 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:41 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:07:42 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:42 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.bdp_jmart_astron_union.bdp_jmart_astron_formal
start time: 1514984828664
final status: UNDEFINED
tracking URL: http://BJLFRZ-10k-234.hadoop.astron.local:50320/proxy/application_1512098667219_5899268/
user: mart_astron
18/01/03 21:07:43 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:44 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:45 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:46 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:47 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:48 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:49 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:50 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:51 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:52 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:53 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:54 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:55 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:56 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:57 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:58 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:07:59 INFO Client: Application report for application_1512098667219_5899268 (state: ACCEPTED)
18/01/03 21:08:00 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:08:00 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 172.21.83.43
ApplicationMaster RPC port: 0
queue: root.bdp_jmart_astron_union.bdp_jmart_astron_formal
start time: 1514984828664
final status: UNDEFINED
tracking URL: http://BJLFRZ-10k-234.hadoop.astron.local:50320/proxy/application_1512098667219_5899268/
user: mart_astron
18/01/03 21:08:01 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:08:02 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:08:03 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:08:04 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:08:05 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:08:06 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:08:07 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:08:08 INFO Client: Application report for application_1512098667219_5899268 (state: RUNNING)
18/01/03 21:08:09 INFO Client: Application report for application_1512098667219_5899268 (state: FINISHED)
18/01/03 21:08:09 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 172.21.83.43
ApplicationMaster RPC port: 0
queue: root.bdp_jmart_astron_union.bdp_jmart_astron_formal
start time: 1514984828664
final status: FAILED
tracking URL: http://BJLFRZ-10k-234.hadoop.astron.local:50320/proxy/application_1512098667219_5899268/
user: mart_astron
18/01/03 21:08:09 INFO Client: Deleted staging directory hdfs://ns12/user/mart_astron/.sparkStaging/application_1512098667219_5899268
Exception in thread "main" org.apache.spark.SparkException: Application application_1512098667219_5899268 finished with failed status
at org.apache.spark.deploy.yarn.Client.run(Client.scala:1179)
at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1225)
at org.apache.spark.deploy.yarn.Client.main(Client.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:739)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:186)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:211)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:125)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)