ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 1) java.sql.BatchUpdateException: Duplicate

 sparksql把JDBC 从关系型数据库中读取数据的方式创建 DataFrame报错：

20/08/26 15:29:37 ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 1)
java.sql.BatchUpdateException: Duplicate entry '269' for key 'PRIMARY'
 com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '269' for key 'PRIMARY'
Job aborted due to stage failure: Task 0 in stage 1.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID 1, localhost, executor driver): java.sq

 去掉数据库里的主键递增

idea中sparksql把JDBC 从关系型数据库中读取数据的方式创建 DataFrame,代码部分

import org.apache.log4j.{Level, Logger}
import org.apache.spark.SparkConf
import org.apache.spark.sql.SparkSession

object readFile {
  def main(args: Array[String]): Unit = {
    Logger.getLogger("org").setLevel(Level.ERROR)

    val conf: SparkConf = new SparkConf().setAppName(this.getClass.getSimpleName).setMaster("local[2]")

    val spark = SparkSession.builder().config(conf).getOrCreate()
    
//      加载/保存方法
//     val frame = spark.read.json("D:/JulyMounth/spark/sparkData/people.json")
//    frame.show()
//    frame.write.format("csv").mode("append").save("D:/JulyMounth/spark/sparkData/900")

//JDBC 从关系型数据库中读取数据的方式创建 DataFrame
    val df = spark.read.format("jdbc")
      .option("url", "jdbc:mysql://localhost:3306/mydata")
      .option("driver", "com.mysql.jdbc.Driver")
      .option("dbtable", "a_users")
      .option("user", "root")
      .option("password", "123456")
      .load()
    
    df.show()
    
    df.write.format("jdbc")
      .option("url","jdbc:mysql://localhost:3306/mydata")
      .option("driver", "com.mysql.jdbc.Driver")
      .option("dbtable","a_users")
      .option("user","root")
      .option("password","root")
      .mode("append")
      .save()

    spark.close()
  }
}