Spark学习之使用spark-shell读取mysql数据库

1. 启动spark
   cd /export/server/spark/sbin
   start-all.sh
   查看spark
   jps
   确保有worker进程
2. 执行命令：

spark-shell \
--master spark://node2:7077 \
--executor-memory 1g \
--total-executor-cores  2 \
--jars /export/server/hive/lib/mysql-connector-java-5.1.32.jar \
--driver-class-path /export/server/hive/lib/mysql-connector-java-5.1.32.jar

3. 读取mysql文件，返回一个dataFrame

val mysqlDF = spark.read.format("jdbc").options(Map("url" -> "jdbc:mysql://192.168.214.101:3306/test", "driver" -> "com.mysql.jdbc.Driver", "dbtable" -> "iplocation", "user" -> "root", "password" -> "root")).load()
	   

指定mysql服务器地址，用户名和密码，还有表名，数据库名

4. 查看mysqlDF数据
   mysqlDF.show