java.lang.IllegalArgumentException : System memory 259522560 must be at least 471859200.

Exception in thread "main" java.lang.IllegalArgumentException: System memory 259522560 must be at least 471859200. Please increase heap size using the --driver-memory option or spark.driver.memory in Spark configuration.

问题:

JVM内存不够, 无法启动sc

解决:

通过SparkContext增加内存 > least 471859200

eg:

val conf = new SparkConf().setAppName("AggregateOperator").setMaster("local").set("spark.testing.memory", "512000000")