ClickHouse（二十三）：Java &Spark读写ClickHouse API
进入正文前，感谢宝子们订阅专题、点赞、评论、收藏！关注IT贫道，获取高质量博客内容！
个人主页：含各种IT体系技术，IT贫道_Apache Doris,大数据OLAP体系技术栈,Kerberos安全认证-CSDN博客
订阅：拥抱独家专题，你的订阅将点燃我的创作热情！
点赞：赞同优秀创作，你的点赞是对我创作最大的认可！
⭐️ 收藏：收藏原创博文，让我们一起打造IT界的荣耀与辉煌！
✏️评论：留下心声墨迹，你的评论将是我努力改进的方向！
目录
1. Java 读写ClickHouse API
2. Spark 写入 ClickHouse API
1. Java 读写ClickHouse API
Java读取ClickHouse中的数据API 。
1） 首先需要加入maven依赖
ru.yandex.clickhouse
clickhouse-jdbc
0.2.4
2) Java 读取ClickHouse单节点表数据
ClickHouseProperties props = new ClickHouseProperties();
props.setUser("default");
props.setPassword("");
BalancedClickhouseDataSource dataSource = new BalancedClickhouseDataSource("jdbc:clickhouse://node1:8123/default", props);
ClickHouseConnection conn = dataSource.getConnection();
ClickHouseStatement statement = conn.createStatement();
ResultSet rs = statement.executeQuery("select id,name,age from test");
while(rs.next()){
int id = rs.getInt("id");
String name = rs.getString("name");
int age = rs.getInt("age");
System.out.println("id = "+id+",name = "+name +",age = "+age);
}3. Java 读取ClickHouse集群表数据
ClickHouseProperties props = new ClickHouseProperties();
props.setUser("default");
props.setPassword("");
BalancedClickhouseDataSource dataSource = new BalancedClickhouseDataSource("jdbc:clickhouse://node1:8123,node2:8123,node3:8123/default", props);
ClickHouseConnection conn = dataSource.getConnection();
ClickHouseStatement statement = conn.createStatement();
ResultSet rs = statement.executeQuery("select id,name from t_cluster");
while(rs.next()){
int id = rs.getInt("id");
String name = rs.getString("name");
System.out.println("id = "+id+",name = "+name );
}4) Java向ClickHouse 表中写入数据。
# API 操作：
ClickHouseProperties props = new ClickHouseProperties();
props.setUser("default");
props.setPassword("");
BalancedClickhouseDataSource dataSource = new BalancedClickhouseDataSource("jdbc:clickhouse://node1:8123/default", props);
ClickHouseConnection conn = dataSource.getConnection();
ClickHouseStatement statement = conn.createStatement();
statement.execute("insert into test values (100,'王五',30)");//可以拼接批量插入多条
#查询default库下 test表 数据：
node1 :) select * from test;
┌──id─┬─name─┬─age─┐
│ 100 │ 王五 │ 30 │
└─────┴──────┴─────┘
┌─id─┬─name─┬─age─┐
│ 1 │ 张三 │ 18 │
│ 2 │ 李四 │ 19 │
└────┴──────┴─────┘2. Spark 写入 ClickHouse API
SparkCore写入ClickHouse，可以直接采用写入方式。下面案例是使用SparkSQL将结果存入ClickHouse对应的表中。在ClickHouse中需要预先创建好对应的结果表。
1) 导入依赖
ru.yandex.clickhouse
clickhouse-jdbc
0.2.4
com.fasterxml.jackson.core
jackson-databind
net.jpountz.lz4
lz4
org.apache.spark
spark-core_2.11
2.3.1
org.apache.spark
spark-sql_2.11
2.3.1
org.apache.spark
spark-hive_2.11
2.3.1
2) 代码编写：
val session: SparkSession = SparkSession.builder().master("local").appName("test").getOrCreate()
val jsonList = List[String](
"{\"id\":1,\"name\":\"张三\",\"age\":18}",
"{\"id\":2,\"name\":\"李四\",\"age\":19}",
"{\"id\":3,\"name\":\"王五\",\"age\":20}"
)
//将jsonList数据转换成DataSet
import session.implicits._
val ds: Dataset[String] = jsonList.toDS()
val df: DataFrame = session.read.json(ds)
df.show()
//将结果写往ClickHouse
val url = "jdbc:clickhouse://node1:8123/default"
val table = "test"
val properties = new Properties()
properties.put("driver", "ru.yandex.clickhouse.ClickHouseDriver")
properties.put("user", "default")
properties.put("password", "")
properties.put("socket_timeout", "300000")
df.write.mode(SaveMode.Append).option(JDBCOptions.JDBC_BATCH_INSERT_SIZE, 100000).jdbc(url, table, properties)如需博文中的资料请私信博主。