20/07/30 15:35:02 WARN Utils: Service ‘sparkWorker‘ could not bind on a random free port.

$ startspark

org.apache.spark.deploy.master.Master running as process 1067300.  Stop it first.
Desktop: org.apache.spark.deploy.worker.Worker running as process 1067817.  Stop it first.
Laptop: starting org.apache.spark.deploy.worker.Worker, logging to /home/appleyuchi/bigdata/spark-3.0.0-bin-hadoop3.2/logs/spark-appleyuchi-org.apache.spark.deploy.worker.Worker-1-Laptop.out
Laptop: failed to launch: nice -n 0 /home/appleyuchi/bigdata/spark-3.0.0-bin-hadoop3.2/bin/spark-class org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://Desktop:7077
Laptop:       at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:248)
Laptop:       at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:356)
Laptop:       at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
Laptop:       at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
Laptop:       at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
Laptop:       at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
Laptop:       at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
Laptop:       at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
Laptop:       at java.lang.Thread.run(Thread.java:748)
Laptop:   20/07/30 15:35:03 INFO ShutdownHookManager: Shutdown hook called
Laptop: full log in /home/appleyuchi/bigdata/spark-3.0.0-bin-hadoop3.2/logs/spark-appleyuchi-org.apache.spark.deploy.worker.Worker-1-Laptop.out

启动spark集群失败,想起来自己更新过spark集群

cat spark-appleyuchi-org.apache.spark.deploy.worker.Worker-1-Laptop.out

20/07/30 15:35:02 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(appleyuchi); groups with view permissions: Set(); users  with modify permissions: Set(appleyuchi); groups with modify permissions: Set()
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 WARN Utils: Service 'sparkWorker' could not bind on a random free port. You may check whether configuring an appropriate binding address.
20/07/30 15:35:02 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[main,5,main]

解决方案:

Laptop节点的$SPARK_HOME/conf/spark-env.sh

export SPARK_LOCAL_IP=Laptop

每个节点的这个SPARK_LOCAL_IP变量都必须是节点所在的局域网的域名或者IP

然后重启spark集群报错消失。

 

你可能感兴趣的:(Scala与Spark)