Elasticsearch（二）

Clasticsearch（二）

DSL查询语法

文档

文档：https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html

常见查询类型包括：

• 查询所有：查询出所有数据，一般测试用。如：match_all

• 全文检索查询：利用分词器对用户输入的内容分词，然后去倒排索引库中匹配。如

• • match_query
  • multi_match_query

• 精确查询：根据精确词条查找数据，一般是查找keyword、数值、日期、boolean等字符。如：

• • ids
  • range
  • term

• 地理查询：根据经纬度查询。例如：

• • geo_distance
  • geo_bounding_box

• 复合查询：复合查询可以将上述各种查询条件组合起来，合并查询条件。如

• • bool
  • function_score

基本语法

查询DSL基本语法

GET /indexName/_search
{
	"query":{
  	"查询类型":{
    	"查询条件":"条件值"
    }
  }
}

全文检索查询

GET /indexName/_search
{
  "query": {
    "match": {
      "FIELD": "TEXT"
    }
  }
}

精确查询

常用：

• term查询

GET /hotel/_search
{
  "query": {
    "term": {
      "city": {
        "value": "上海"
      }
    }
  }
}

• range查询

GET /hotel/_search
{
  "query": {
    "range": {
      "price": {
        "gte": 100,
        "lte": 300
      }
    }
  }
}

地理查询

常见场景包括：

• 携程：搜索我附近的酒店
• 滴滴：搜索我附近的出租车
• 微信：搜索我附近的人

根据经纬度查询：

• geo_bounding_box：查询geo_point值落在某个矩形范围的所有文档

• geo_distance：查询到指定中心小于某个距离值得所有文档

复合查询

复合查询：复合查询可以将其它简单查询组合起来，实现复杂得搜索逻辑，例如：

• function score：算分函数查询，可以控制文档相关性算分，控制文档排名

相关性算分

Function Score Query

案例：给“如家”这个品牌得酒店排名靠前一些

Boolean Query

布尔查询是一个或多个子句得组合。子查询得组合方式有：

• must：必须匹配每个子查询，类似“与”
• should：选择性匹配子查询，类似“或”
• must_not：必须不匹配，不参与算分，类似“非”
• filter：必须匹配，不参与算分

案例：利用bool查询实现功能

GET /hotel/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "name": "如家"
          }
        }
      ],
      "must_not": [
        {
          "range": {
            "price": {
              "gt": 400
            }
          }
        }
      ],
      "filter": [
        {
          "geo_distance": {
            "distance": "10km",
            "location": {
              "lat": 31.21,
              "lon": 121.5
            }
          }
        }
      ]
    }
  }
}

搜索结果处理

排序

elasticsearch支持搜索结果排序，默认是根据相关度算分来排序。可以排序类型有：keyword类型、数值类型、地理坐标类型、日期类型等。

分页

elasticsearch默认情况下只返回top10得数据。如果要查询更多数据就需要修改分页参数。

elasticsearch中通过修改from、size参数来控制要返回得分页结果：

深度分页问题

ES是分布式的，所以会面临深度分页问题。

深度分页解决方案

针对深度分页，ES提供了两种解决方案：

• search after：分页时需要排序，原理是从上一次的排序值开始，查询下一页数据。官方推荐使用的方式。
• scroll：原理将排序数据形成快照，保存在内存。官方已经不推荐使用。

高亮

RestClient查询文档

快速入门

    @Test
    void testInit() throws IOException {
        // 1.准备Request
        SearchRequest request = new SearchRequest("hotel");
        // 2.准备DSL
        request.source().query(QueryBuilders.matchAllQuery());
        // 3.发送请求
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        System.out.println(response);
    }

解析JSON

    @Test
    void testInit() throws IOException {
        // 1.准备Request
        SearchRequest request = new SearchRequest("hotel");
        // 2.准备DSL
        request.source().query(QueryBuilders.matchAllQuery());
        // 3.发送请求
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        // 4.解析响应
        SearchHits hits = response.getHits();
        // 4.1.获取总条数
        long total = hits.getTotalHits().value;
        System.out.println(total);
        // 4.2.遍历文档数组
        for (SearchHit hit : hits.getHits()) {
            // 4.3.获取json对象
            String json = hit.getSourceAsString();
            // 4.4.将json对象变成java对象
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            System.out.println(hotelDoc);
        }
    }

全文检索查询

精确查询

复合查询-boolean query

    @Test
    void testBool() throws IOException {
        // 1.准备Request
        SearchRequest request = new SearchRequest("hotel");
        // 2.准备DSL
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        boolQuery.must(QueryBuilders.termQuery("city","上海"));
        boolQuery.filter(QueryBuilders.rangeQuery("price").lte(250));
        request.source().query(boolQuery);
        // 3.发送请求
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        handleResponse(response);
    }

	/**
     * 解析响应
     * @param response
     */
    private static void handleResponse(SearchResponse response) {
        // 4.解析响应
        SearchHits hits = response.getHits();
        // 4.1.获取总条数
        long total = hits.getTotalHits().value;
        System.out.println(total);
        // 4.2.遍历文档数组
        for (SearchHit hit : hits.getHits()) {
            // 4.3.获取json对象
            String json = hit.getSourceAsString();
            // 4.4.将json对象变成java对象
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            System.out.println(hotelDoc);
        }
    }

排序和分页

    @Test
    void testPageAndSort() throws IOException {
        // 1.准备Request
        SearchRequest request = new SearchRequest("hotel");
        // 2.准备DSL
        request.source().query(QueryBuilders.matchAllQuery());
        // 排序
        request.source().sort("price", SortOrder.ASC);
        request.source().from(1).size(5);
        // 3.发送请求
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        handleResponse(response);
    }

高亮

    @Test
    void testHighlight() throws IOException {
        // 1.准备Request
        SearchRequest request = new SearchRequest("hotel");
        // 2.准备DSL
        request.source().query(QueryBuilders.matchQuery("all","如家"));
        request.source().highlighter(new HighlightBuilder().field("name").requireFieldMatch(false));
        // 3.发送请求
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        handleResponse(response);
    }

高亮的结果解析

/**
     * 解析响应
     * @param response
     */
    private static void handleResponse(SearchResponse response) {
        // 4.解析响应
        SearchHits hits = response.getHits();
        // 4.1.获取总条数
        long total = hits.getTotalHits().value;
        System.out.println(total);
        // 4.2.遍历文档数组
        for (SearchHit hit : hits.getHits()) {
            // 4.3.获取json对象
            String json = hit.getSourceAsString();
            // 4.4.将json对象变成java对象
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            // 获取高亮结果
            Map<String, HighlightField> highlightFields = hit.getHighlightFields();
            if (!CollectionUtils.isEmpty(highlightFields)) {
                // 根据字段名获取高亮结果
                HighlightField highlightField = highlightFields.get("name");
                if (highlightField != null) {
                    // 获取高亮值
                    String name = highlightField.getFragments()[0].string();
                    hotelDoc.setName(name);
                }
            }
            System.out.println(hotelDoc);
        }
    }

demo案例

实现搜索功能

1.定义实体类，接受前端请求

RequestParams

@Data
public class RequestParams {
    private String key;
    private Integer page;
    private Integer size;
    private String sortBy;
}

PageResult

@Data
public class PageResult {
    private Long total;
    
    private List<HotelDoc> hotels;
}

2.定义controller接口，接受页面请求，调用IHotelService的search方法

    @PostMapping("/list")
    public PageResult search(@RequestBody RequestParams params) {
        return hotelService.search(params);
    }

3.定义IHotelService中的search方法，利用match查询实现根据关键字搜索酒店信息

 @Override
    public PageResult search(RequestParams params) {
        try {
            // 1.准备Request
            SearchRequest request = new SearchRequest("hotel");
            // 2.准备DSL
            String key = params.getKey();
            if (StringUtils.isBlank(key)) {
                request.source().query(QueryBuilders.matchAllQuery());
            } else {
                request.source().query(QueryBuilders.matchQuery("all", key));
            }
            // 分页
            int page = params.getPage();
            int size = params.getSize();
            request.source().from((page - 1) * size).size(size);
            // 3.发送请求
            SearchResponse response = client.search(request, RequestOptions.DEFAULT);
            return handleResponse(response);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    /**
     * 解析响应
     *
     * @param response
     */
    private static PageResult handleResponse(SearchResponse response) {
        // 4.解析响应
        SearchHits hits = response.getHits();
        // 4.1.获取总条数
        long total = hits.getTotalHits().value;
        // 4.2.遍历文档数组
        List<HotelDoc> hotels = new ArrayList<>();
        for (SearchHit hit : hits.getHits()) {
            // 4.3.获取json对象
            String json = hit.getSourceAsString();
            // 4.4.将json对象变成java对象
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            hotels.add(hotelDoc);
        }
        PageResult pageResult = new PageResult();
        pageResult.setTotal(total);
        pageResult.setHotels(hotels);
        return pageResult;
    }

添加品牌、城市、星级、价格等过滤功能

1.修改RequestParams类，添加brand、city、starName、minPrice、maxPrice等参数

package cn.itcast.hotel.pojo;

import lombok.Data;

/**
 * @author xc
 * @date 2023/5/11 16:14
 */
@Data
public class RequestParams {
    private String key;
    private Integer page;
    private Integer size;
    private String sortBy;

    private String city;

    private String brand;

    private String starName;

    private Integer minPrice;

    private Integer maxPrice;
}

2.修改search方法的实现，在关键字搜索时，如果brand等参数存在，对其做过滤

@Override
    public PageResult search(RequestParams params) {
        try {
            // 1.准备Request
            SearchRequest request = new SearchRequest("hotel");
            // 2.准备DSL
            // 构建BooleanQuery
            BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
            // 关键字搜索
            String key = params.getKey();
            if (StringUtils.isBlank(key)) {
                boolQuery.must(QueryBuilders.matchAllQuery());
            } else {
                boolQuery.must(QueryBuilders.matchQuery("all", key));
            }
            // 城市条件
            if (params.getCity() != null && !params.getCity().equals("")) {
                boolQuery.filter(QueryBuilders.termQuery("city",params.getCity()));
            }
            // 品牌条件
            if (params.getBrand() != null && !params.getBrand().equals("")) {
                boolQuery.filter(QueryBuilders.termQuery("brand",params.getBrand()));
            }
            // 星级条件
            if (params.getStarName() != null && !params.getStarName().equals("")) {
                boolQuery.filter(QueryBuilders.termQuery("starName",params.getStarName()));
            }
            // 价格条件
            if (params.getMinPrice() != null && params.getMaxPrice() != null) {
                boolQuery.filter(QueryBuilders.rangeQuery("price").gte(params.getMinPrice()).lte(params.getMaxPrice()));
            }
            request.source().query(boolQuery);

            // 分页
            int page = params.getPage();
            int size = params.getSize();
            request.source().from((page - 1) * size).size(size);
            // 3.发送请求
            SearchResponse response = client.search(request, RequestOptions.DEFAULT);
            return handleResponse(response);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

附近的酒店

距离排序

    @Override
    public PageResult search(RequestParams params) {
        try {
            // 1.准备Request
            SearchRequest request = new SearchRequest("hotel");
            // 2.准备DSL
            // 构建BooleanQuery
            BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
            // 关键字搜索
            String key = params.getKey();
            if (StringUtils.isBlank(key)) {
                boolQuery.must(QueryBuilders.matchAllQuery());
            } else {
                boolQuery.must(QueryBuilders.matchQuery("all", key));
            }
            // 城市条件
            if (params.getCity() != null && !params.getCity().equals("")) {
                boolQuery.filter(QueryBuilders.termQuery("city",params.getCity()));
            }
            // 品牌条件
            if (params.getBrand() != null && !params.getBrand().equals("")) {
                boolQuery.filter(QueryBuilders.termQuery("brand",params.getBrand()));
            }
            // 星级条件
            if (params.getStarName() != null && !params.getStarName().equals("")) {
                boolQuery.filter(QueryBuilders.termQuery("starName",params.getStarName()));
            }
            // 价格条件
            if (params.getMinPrice() != null && params.getMaxPrice() != null) {
                boolQuery.filter(QueryBuilders.rangeQuery("price").gte(params.getMinPrice()).lte(params.getMaxPrice()));
            }
            request.source().query(boolQuery);

            // 分页
            int page = params.getPage();
            int size = params.getSize();
            request.source().from((page - 1) * size).size(size);

            // 2.3.排序
            String location = params.getLocation();
            if (location != null && !location.equals("")){
                request.source().sort(SortBuilders
                        .geoDistanceSort("location",new GeoPoint(location))
                        .order(SortOrder.ASC)
                        .unit(DistanceUnit.KILOMETERS)
                );
            }


            // 3.发送请求
            SearchResponse response = client.search(request, RequestOptions.DEFAULT);
            return handleResponse(response);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    /**
     * 解析响应
     *
     * @param response
     */
    private static PageResult handleResponse(SearchResponse response) {
        // 4.解析响应
        SearchHits hits = response.getHits();
        // 4.1.获取总条数
        long total = hits.getTotalHits().value;
        // 4.2.遍历文档数组
        List<HotelDoc> hotels = new ArrayList<>();
        for (SearchHit hit : hits.getHits()) {
            // 4.3.获取json对象
            String json = hit.getSourceAsString();
            // 4.4.将json对象变成java对象
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            // 获取排序值
            Object[] sortValues = hit.getSortValues();
            if (sortValues.length > 0) {
                Object sortValue = sortValues[0];
                hotelDoc.setDistance(sortValue);
            }
            hotels.add(hotelDoc);
        }
        PageResult pageResult = new PageResult();
        pageResult.setTotal(total);
        pageResult.setHotels(hotels);
        return pageResult;
    }

广告置顶

1.给HotelDoc类添加isAD字段，Boolean类型

package cn.itcast.hotel.pojo;

import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@NoArgsConstructor
public class HotelDoc {
    private Long id;
    private String name;
    private String address;
    private Integer price;
    private Integer score;
    private String brand;
    private String city;
    private String starName;
    private String business;
    private String location;
    private String pic;

    private Object distance;
    
    private Boolean isAD;

    public HotelDoc(Hotel hotel) {
        this.id = hotel.getId();
        this.name = hotel.getName();
        this.address = hotel.getAddress();
        this.price = hotel.getPrice();
        this.score = hotel.getScore();
        this.brand = hotel.getBrand();
        this.city = hotel.getCity();
        this.starName = hotel.getStarName();
        this.business = hotel.getBusiness();
        this.location = hotel.getLatitude() + ", " + hotel.getLongitude();
        this.pic = hotel.getPic();
    }
}

2.修改search方法，添加function score功能，给isAD值为true的酒店增加权重

    @Override
    public PageResult search(RequestParams params) {
        try {
            // 1.准备Request
            SearchRequest request = new SearchRequest("hotel");
            // 2.准备DSL
            // 构建BooleanQuery
            BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
            // 关键字搜索
            String key = params.getKey();
            if (StringUtils.isBlank(key)) {
                boolQuery.must(QueryBuilders.matchAllQuery());
            } else {
                boolQuery.must(QueryBuilders.matchQuery("all", key));
            }
            // 城市条件
            if (params.getCity() != null && !params.getCity().equals("")) {
                boolQuery.filter(QueryBuilders.termQuery("city",params.getCity()));
            }
            // 品牌条件
            if (params.getBrand() != null && !params.getBrand().equals("")) {
                boolQuery.filter(QueryBuilders.termQuery("brand",params.getBrand()));
            }
            // 星级条件
            if (params.getStarName() != null && !params.getStarName().equals("")) {
                boolQuery.filter(QueryBuilders.termQuery("starName",params.getStarName()));
            }
            // 价格条件
            if (params.getMinPrice() != null && params.getMaxPrice() != null) {
                boolQuery.filter(QueryBuilders.rangeQuery("price").gte(params.getMinPrice()).lte(params.getMaxPrice()));
            }
            // 2.算分
            FunctionScoreQueryBuilder functionScoreQueryBuilder = QueryBuilders
                    .functionScoreQuery(boolQuery, new FunctionScoreQueryBuilder.FilterFunctionBuilder[]{
                            new FunctionScoreQueryBuilder.FilterFunctionBuilder(
                                    QueryBuilders.termQuery("isAD",true),
                                    ScoreFunctionBuilders.weightFactorFunction(10)
                            )
                    });

            request.source().query(functionScoreQueryBuilder);

            // 分页
            int page = params.getPage();
            int size = params.getSize();
            request.source().from((page - 1) * size).size(size);

            // 2.3.排序
            String location = params.getLocation();
            if (location != null && !location.equals("")){
                request.source().sort(SortBuilders
                        .geoDistanceSort("location",new GeoPoint(location))
                        .order(SortOrder.ASC)
                        .unit(DistanceUnit.KILOMETERS)
                );
            }
            // 3.发送请求
            SearchResponse response = client.search(request, RequestOptions.DEFAULT);
            return handleResponse(response);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }
        int size = params.getSize();
            request.source().from((page - 1) * size).size(size);

            // 2.3.排序
            String location = params.getLocation();
            if (location != null && !location.equals("")){
                request.source().sort(SortBuilders
                        .geoDistanceSort("location",new GeoPoint(location))
                        .order(SortOrder.ASC)
                        .unit(DistanceUnit.KILOMETERS)
                );
            }
            // 3.发送请求
            SearchResponse response = client.search(request, RequestOptions.DEFAULT);
            return handleResponse(response);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }