sparkling-water Unsupported argument: (spark.dynamicAllocation.enabled,true) on CDH

PYSPARK_DRIVER_PYTHON="ipython" bin/pysparkling
from pysparkling import *
import h2o
hc = H2OContext.getOrCreate(spark)
此时会出现错误:
Unsupported argument:(spark.dynamicAllocation.enabled,true)

查到如下解决方法:这里 和 这里
right now Sparkling Water/RSparkling does not support dynamic Spark cluster. SO you just need to disable it:
config = list(“spark.dynamicAllocation.enabled” = “false”)
所以这样使用:
PYSPARK_DRIVER_PYTHON="ipython" bin/pysparkling --conf spark.dynamicAllocation.enabled=false

你可能感兴趣的:(错误处理,sparkling,water,pysparkling,spark)