使用idea编辑sparkSql的程序运行出现java.lang.IllegalArgumentException: java.net.URISyntaxException

这个是具体错误:

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: file:D:/idea/work/sparkTest/spark-warehouse
    at org.apache.hadoop.fs.Path.initialize(Path.java:206)
    at org.apache.hadoop.fs.Path.(Path.java:172)
    at org.apache.spark.sql.catalyst.catalog.SessionCatalog.makeQualifiedPath(SessionCatalog.scala:114)
    at org.apache.spark.sql.catalyst.catalog.SessionCatalog.createDatabase(SessionCatalog.scala:145)
    at org.apache.spark.sql.catalyst.catalog.SessionCatalog.(SessionCatalog.scala:89)
    at org.apache.spark.sql.internal.SessionState.catalog$lzycompute(SessionState.scala:95)
    at org.apache.spark.sql.internal.SessionState.catalog(SessionState.scala:95)
    at org.apache.spark.sql.internal.SessionState$$anon$1.(SessionState.scala:112)
    at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:112)
    at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:111)
    at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:49)
    at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64)
    at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:252)
    at org.apache.spark.sql.SQLContext.createDataFrame(SQLContext.scala:278)
    at com.Sql_on_spar$.main(Sql_on_spar.scala:20)
    at com.Sql_on_spar.main(Sql_on_spar.scala)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: file:D:/idea/work/sparkTest/spark-warehouse
    at java.net.URI.checkPath(URI.java:1823)
    at java.net.URI.(URI.java:745)
    at org.apache.hadoop.fs.Path.initialize(Path.java:203)
    ... 15 more

虽然我不知道问什么这么做,但能解决:

var spark=SparkSession.builder().master(“spark://spark02:7077”).appName(“spark_mllib”).config(“spark.sql.warehouse.dir”,”file:///D://test”).getOrCreate()
//需要设置spark.sql.warehouse.dir的路径

你可能感兴趣的:(spark)