Spark——Spark读写Oracle

Maven依赖

本地IDEA执行Spark应用程序读取Oracle需要在pom.xml文件中添加如下依赖：

<dependency>
    <groupId>com.oracle.database.jdbcgroupId>
    <artifactId>ojdbc8artifactId>
    <version>12.2.0.1version>
dependency>

在集群上使用spark-submit脚本提交应用程序需要指定依赖包——ojdbc8-12.2.0.1.jar。

代码实现

	def readOracle(spark: SparkSession): Unit = {
		val prop = new Properties()
		prop.put("user", "")
		prop.put("password", "")
		prop.put("driver", "oracle.jdbc.driver.OracleDriver")
		val url = "jdbc:oracle:thin:@host:1521:gtora" //url
		val table = "db.table" //表名

		//下推条件
		val predicates = Array[String](
			"yearv <= '2015'", 
			"yearv >= '2016' and yearv < '2020'",
			"yearv >= '2021'")
			
		spark.read.jdbc(url, table, predicates, prop).show()
	}