1-3 java操作Elasticsearch
java操作Elasticsearch
一、 简介
使用java调用ES主要由3种方式
- ES官方的Java API
- 参考文档：https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/7.17/introduction.html
- 快速开始：https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/7.17/connecting.html
- ES以前的官方Java API，当时使用的是HighLevelRestClient(已废弃，不推荐使用)
- Spring Data Elasticsearch (推荐使用这种，这里也主要介绍这种方式)
spring-data系列：spring提供操作数据的框架，比如常见的- spring-data-redis：操作redis
- spring-data-mongodb：操作mongodb
- spring-data-elasticsearch：操作elasticsearch
参考文档：https://docs.spring.io/spring-data/elasticsearch/docs/4.4.10/reference/html/
一定要注意版本，这个需要和elasticsearch相适应
二、 基本使用
2.1 环境搭建
- 引入依赖
<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-data-elasticsearchartifactId>
dependency>
- 修改配置文件
spring:
elasticsearch:
uris: 192.168.1.30:9200
2.2 实体类及其注解
@Document(indexName = "student")
public class Student {
// 代表这是主键 ，如果不指定，es会自动生成一个id
@Id
private Integer id;
@Field(type = FieldType.Keyword,index = true,store = false)
private String stuId;
@Field(type = FieldType.Text,analyzer = "ik_max_word")
private String name;
@Field(type = FieldType.Text,analyzer = "ik_max_word")
private String text;
@Field(type = FieldType.Integer)
private Integer age;
}
- Document：作用在类上，主要进行的是索引库的配置
- Id：作用在属性上，用于指定主键，在ES中，所有的数据都会有一个主键id，如果不指定，ES会默认创建
- Field：作用在属性上，主要是设置元素的类型，是否分词等
2.3 基本使用
这里介绍两种简单的方式实现数据的增删改查
2.3.1 ElasticsearchRepository实现CRUD
- 新建一个接口，继承ElasticsearchRepository即可
public interface StudentDao extends ElasticsearchRepository<Student,Integer> { }
- 测试新增和修改
public class esTest {
@Autowired
private StudentDao studentDao;
/**
* 如果id存在则是修改，如果不存在则新增
*/
@Test
public void addStudent(){
studentDao.save(new Student(1, "1001", "张三", "我叫张三，来自地球村", 24));
studentDao.save(new Student(2, "1001", "李四", "我叫李四，来自中国安徽", 12));
studentDao.save(new Student(3, "1002", "王五", "我叫张三，来自安徽合肥", 25));
studentDao.save(new Student(4, "1003", "管理员", "我叫管理员，来自中国合肥", 11));
studentDao.save(new Student(5, "1003", "王五", "我叫王五，来自中国合肥", 32));
}
- 删除数据
@Test
public void deleteStudent(){
studentDao.deleteById(5);
}
4.查询数据
@Test
public void queryAll(){
studentDao.findAll().forEach(System.out::println);
}
2.3.2 使用ElasticsearchRestTemplate实现CRUD
这一种是常用的方式，它提供了增删改查的功能，但是它的增删改查更改的灵活，适用于更复杂的操作，返回的结果集更加的全面，但是需要自己进行解析
- 注入ElasticsearchRestTemplate
@Autowired
private ElasticsearchRestTemplate elasticsearchRestTemplate;
- 测试新增和修改
@Test
public void addStudentTemplate(){
elasticsearchRestTemplate.save(new Student(6, "1007", "张三", "我叫张三，来自地球村", 24));
}
- 删除数据
@Test
public void deleteStudentTemplate(){
elasticsearchRestTemplate.delete( Student.builder().id(6).build());
}
- 查询数据
对于查询，主要是以下3个步骤
- 取参数
- 把参数组合为ES支持的搜索条件
- 从返回值中取结果
@Test
public void queryAllTemplate(){
// 定义一个查询条件
Query searchQuery = new NativeSearchQueryBuilder()
.withQuery(QueryBuilders.matchAllQuery())
.build();
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(searchQuery, Student.class);
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
2.4 详解查询
— 这里使用elasticsearchRestTemplate来实现基本的查询讲解，这是使用NativeSearchQueryBuilder来构建查询条件，这是spring提供的查询条件构造器，帮助构建json格式的请求体。
(这一节的例子都是由1-2 Elasticsearch基本介绍里面的3.4节DSL语法翻译过来的，建议对照)
匹配、词条这样的简单查询，使用QueryBuilders工具类来构建查询条件
2.4.1 匹配查询
- 单字段查询 多个词条之间是or的关系
@Test
public void testMatch(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询条件
MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("text", "来自安徽");
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(matchQueryBuilder).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
2.单字段查询 多个词条之间是and的关系
@Test
public void testMatch(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询条件
MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("text", "来自安徽").operator(Operator.AND);
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(matchQueryBuilder).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
- 如果不确定是and还是or的关系时，可以使用最小匹配参数
@Test
public void testMatchAndOr(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询条件
MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("text", "来自安徽").minimumShouldMatch("100%");
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(matchQueryBuilder).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
- 多字段查询
@Test
public void testMultiMatch(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询条件
MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("张三", "name", "text");
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(multiMatchQueryBuilder).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
2.4.2 词条查询
- 单词条查询
@Test
public void testTerm(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询条件
TermQueryBuilder stuId = QueryBuilders.termQuery("stuId", "1001");
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(stuId).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
- 多词条查询
@Test
public void testTerms(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询条件
TermsQueryBuilder stuId = QueryBuilders.termsQuery("stuId", "1001", "1002");
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(stuId).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
2.4.3 结果过滤
这里需要借助FetchSourceFilter这个工具类来实现
- 直接指定要显示的字段
@Test
public void testInclude(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 结果集过滤
FetchSourceFilter fetchSourceFilter = new FetchSourceFilter(new String[]{"stuId", "name"}, null);
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withSourceFilter(fetchSourceFilter).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
- 直接指定不需要显示的字段
@Test
public void testExcludes(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 结果集过滤
FetchSourceFilter fetchSourceFilter = new FetchSourceFilter(null, new String[]{"stuId"});
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withSourceFilter(fetchSourceFilter).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
2.4.4 高级查询
- 布尔组合
- 与的关系
查询出text包含安徽且name是李四的数据
@Test
public void tesBoolMust(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询条件
// 构建bool查询
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
// 构建必须的关系
boolQueryBuilder.must(QueryBuilders.matchQuery("text","安徽"));
boolQueryBuilder.must(QueryBuilders.matchQuery("name","李四"));
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(boolQueryBuilder).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
- 非的关系
查询出text不包含安徽和name不是李四的数据
@Test
public void tesBoolMustNot(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询条件
// 构建bool查询
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
// 构建必须的关系
boolQueryBuilder.mustNot(QueryBuilders.matchQuery("text","安徽"));
boolQueryBuilder.mustNot(QueryBuilders.matchQuery("name","李四"));
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(boolQueryBuilder).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
- should：或
查询出text包含安徽或者name是李四的数据
@Test
public void tesBoolShould(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询条件
// 构建bool查询
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
// 构建必须的关系
boolQueryBuilder.should(QueryBuilders.matchQuery("text","安徽"));
boolQueryBuilder.should(QueryBuilders.matchQuery("name","李四"));
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(boolQueryBuilder).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
- 范围查询
查询stuId在1001和1003之间的数据
@Test
public void testRange(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询条件
RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("stuId").gte(1001).lte(1003);
// 结果过滤
FetchSourceFilter fetchSourceFilter = new FetchSourceFilter(new String[]{"stuId", "name"}, null);
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(rangeQueryBuilder).withSourceFilter(fetchSourceFilter).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
- 模糊查询
@Test
public void testFuzzy(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询条件
FuzzyQueryBuilder fuzzyQuery = QueryBuilders.fuzzyQuery("stuId", "1007");
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(fuzzyQuery).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
2.4.5 过滤
@Test
public void testFilter(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 范围查询
RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("stuId").gte(1001).lte(1003);
// 词条查询
TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name", "李四");
// 组合上面的查询条件
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
boolQueryBuilder.must(rangeQueryBuilder);
boolQueryBuilder.filter(termQueryBuilder);
// 执行查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(boolQueryBuilder).build(),Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
2.4.6 排序
这里使用排序工具类SortBuilders
- 单字段排序
@Test
public void testSore(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询所有
MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();
// 排序
FieldSortBuilder fieldSortBuilder = SortBuilders.fieldSort("stuId").order(SortOrder.DESC);
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(matchAllQueryBuilder).withSort(fieldSortBuilder).build(), Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
- 多字段排序
@Test
public void testSores(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 查询所有
MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();
// 排序
FieldSortBuilder fieldSortBuilder1 = SortBuilders.fieldSort("stuId").order(SortOrder.DESC);
FieldSortBuilder fieldSortBuilder2 = SortBuilders.fieldSort("age").order(SortOrder.ASC);
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.withQuery(matchAllQueryBuilder).withSort(fieldSortBuilder1).withSort(fieldSortBuilder2).build(), Student.class);
// 解析结果
for (SearchHit<Student> studentSearchHit : search) {
System.out.println(studentSearchHit.getContent());
}
}
2.4.7 聚合
聚合工具类：AggregationBuilders
- 桶
@Test
public void testAggs(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 桶
// 添加一个新的桶，桶的类型为terms，桶的名称为name，桶的字段为stuId
TermsAggregationBuilder termsAggregationBuilder = AggregationBuilders.terms("name").field("stuId");
// 查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.addAggregation(termsAggregationBuilder).build(), Student.class);
// 封装结果
Aggregations aggregations = (Aggregations)search.getAggregations().aggregations();
// 根据名称获取桶，因为进行的是Terms聚合，需要将结果转化为Terms类型
Terms terms = aggregations.get("name");
// 遍历
for (Terms.Bucket bucket : terms.getBuckets()) {
System.out.println(bucket.getKeyAsString()+bucket.getDocCount());
}
}
- 度量
@Test
public void testAggsAvg(){
// 创建一个查询条件构造器
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
// 桶
// 添加一个新的桶，桶的类型为terms，桶的名称为name，桶的字段为stuId
TermsAggregationBuilder termsAggregationBuilder = AggregationBuilders.terms("name").field("stuId")
// 添加度量
.subAggregation(AggregationBuilders.sum("sumAge").field("age"));
// 查询
SearchHits<Student> search = elasticsearchRestTemplate.search(nativeSearchQueryBuilder.addAggregation(termsAggregationBuilder).build(), Student.class);
// 封装结果
Aggregations aggregations = (Aggregations)search.getAggregations().aggregations();
// 根据名称获取桶，因为进行的是Terms聚合，需要将结果转化为Terms类型
Terms terms = aggregations.get("name");
// 遍历
for (Terms.Bucket bucket : terms.getBuckets()) {
Aggregations childAggregations = bucket.getAggregations();
// 获取子聚合
ParsedSum sumAge = childAggregations.get("sumAge");
System.out.println(bucket.getKeyAsString()+"--"+bucket.getDocCount()+"--"+sumAge.getValue());
}
}