zeppelin出现ConnectException: Connection refused解决方法

zeppelin出现ConnectException: Connection refused

问题

一直用zeppelin做分析工作台，最近想把现用的环境整体打包复制到另一个集群，在新的集群启动spark interpreter时出现了ConnectException，详细出错信息如下：

ERROR [2018-03-05 17:33:01,109] ({Thread-31} RemoteInterpreterEventPoller.java[run]:77) - Can't get RemoteInterpreterEvent
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)
        at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
        at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
        at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
        at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:189)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:75)
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)
        at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
        at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
        ... 7 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
        ... 8 more

解决方法

老的zeppelin环境下，在conf/zeppelin-env.sh下配置了export SPARK_SUBMIT_OPTIONS="--packages com.databricks:spark-csv_2.10:1.4.0"， 在spark job提交时需要从远程仓库下载指定的packages包，这里的远程仓库也可能是私有的配置，因此利用以下方法解决：

• 去掉zeppelin-env.sh中的submit option中的packages配置
  export SPARK_SUBMIT_OPTIONS="--packages com.databricks:spark-csv_2.10:1.4.0"
  如果尝试spark interpreter是否可用，不行再去掉repo

• 去掉interpreter中的Repositories私有配置