Elasticsearch Java客户端和Spring data elasticsearch-Elasticsearch文章三

官网

https://www.elastic.co/cn/

整合springboot看上一篇文章

版本

一定要对应好版本，Elasticsearch 的不同版本变化是真大，

https://docs.spring.io/spring-data/elasticsearch/docs/4.4.10/reference/html/

组件版本说明

Springboot: 2.7.10
spring-data-elasticsearch: 4.4.10
spring-boot-starter-data-elasticsearch: 2.7.10
elasticsearch-java: 7.17.9

实现代码地址

https://github.com/OrderDong/microservice-boot
分支：microservice-boot-1.0.6-es

microservice-boot-eleastic模块

所有pom.xml引用

        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-data-elasticsearchartifactId>
        dependency>
        
        <dependency>
            <groupId>co.elastic.clientsgroupId>
            <artifactId>elasticsearch-javaartifactId>
            <version>7.17.9version>
        dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.coregroupId>
            <artifactId>jackson-databindartifactId>
            <version>2.12.3version>
        dependency>
        <dependency>
            <groupId>jakarta.jsongroupId>
            <artifactId>jakarta.json-apiartifactId>
            <version>2.0.1version>
        dependency>

        

        
        <dependency>
            <groupId>org.springframeworkgroupId>
            <artifactId>spring-webfluxartifactId>
        dependency>

        <dependency>
            <groupId>io.projectreactor.nettygroupId>
            <artifactId>reactor-netty-httpartifactId>
        dependency>

es Spring Data Repositories

类似于spring data封装的ORM框架，也可以理解为MVC的M层，当然，完全可以灵活应用，也支持rest层
https://docs.spring.io/spring-data/elasticsearch/docs/4.4.10/reference/html/#repositories

例子：

    //controller
    @PostMapping("/saveVisitLog")
    @ResponseBody
    public String saveVisitLog(@RequestBody VisitLog visitLog){
        VisitLog saveVisitLog = visitLogService.saveVisitLog(visitLog);
        log.info("----saveVisitLog----:{}", JSON.toJSONString(saveVisitLog));
        return JSON.toJSONString(saveVisitLog);
    }

    @GetMapping("/getVisitLogAll")
    @ResponseBody
    public String getVisitLogAll(){
        List<VisitLog> logList = visitLogService.findAll();
        log.info("----getVisitLogAll----:{}", JSON.toJSONString(logList));
        return JSON.toJSONString(logList);
    }


    //service
    @Override
    public VisitLog saveVisitLog(VisitLog visitLog) {
        return visitLogRepository.save(visitLog);
    }

    @Override
    public List<VisitLog> findAll() {
        Iterable<VisitLog> visitLogs = visitLogRepository.findAll();
        List<VisitLog> logs = new ArrayList<>();
        visitLogs.forEach(visitLog -> logs.add(visitLog));
        return logs;
    }

    //dao层
    @Repository
    public interface VisitLogRepository extends ElasticsearchRepository<VisitLog, String> {


    }

ElasticsearchRepository分析

CrudRepository最终继承了这个对象，这个对象被按照规则封装实现了具体的操作

我们可以看下ElasticsearchRepository实现了哪些方法：

@NoRepositoryBean
public interface CrudRepository<T, ID> extends Repository<T, ID> {
    <S extends T> S save(S entity);

    <S extends T> Iterable<S> saveAll(Iterable<S> entities);

    Optional<T> findById(ID id);

    boolean existsById(ID id);

    Iterable<T> findAll();

    Iterable<T> findAllById(Iterable<ID> ids);

    long count();

    void deleteById(ID id);

    void delete(T entity);

    void deleteAllById(Iterable<? extends ID> ids);

    void deleteAll(Iterable<? extends T> entities);

    void deleteAll();
}

es Spring Data Repositories 关键字

spring data封装的ORM框架，很多接口继承的方法，也可以自定义方法，需要按照封装的规则自定义
https://docs.spring.io/spring-data/elasticsearch/docs/4.4.10/reference/html/#repositories.definition

例子：

    //controller
    //自定义查询
    @GetMapping("/getVisitLogByUserLoginId")
    @ResponseBody
    public String getVisitLogByUserLoginId(Integer userLoginId){
        List<VisitLog> logList = visitLogService.findByUserLoginId(userLoginId);
        log.info("----getVisitLogAll----:{}", JSON.toJSONString(logList));
        return JSON.toJSONString(logList);
    }



    //service impl
        @Override
    public List<VisitLog> findByUserLoginId(Integer userLoginId) {
        return visitLogRepository.findByUserLoginId(userLoginId);
    }


    //
    @Repository
    public interface VisitLogRepository extends ElasticsearchRepository<VisitLog, String> {

        //自定义规则,By--关键字，UserLoginId字段，翻译下，就是查询-find 通过--By  字段--UserLoginId
        //spring data repository会去解析，同样的方式自行查reference文档
        List<VisitLog> findByUserLoginId(Integer userLoginId);
    }

es Spring Data Repositories client 加载

当然，一般使用封装好的也有一定的麻烦事情，也可以自己加载客户端，使用spring data repository 加载，两种方式，但是7.17的版本把这种方式去掉了，
https://docs.spring.io/spring-data/elasticsearch/docs/4.4.10/reference/html/#elasticsearch.clients

rest风格客户端

The Java High Level REST Clientis the default client of Elasticsearch,it is configured likes hown:

package org.lwd.microservice.boot.es.config;

import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.client.ClientConfiguration;
import org.springframework.data.elasticsearch.client.RestClients;
import org.springframework.data.elasticsearch.config.AbstractElasticsearchConfiguration;

/**
 * rest风格客户端
 * The Java High Level REST Client is the default client of Elasticsearch, it is configured like shown:
 *
 * @author weidong
 * @version V1.0.0
 * @since 2023/7/27
 */
@Configuration
public class EsRestHighLevelClientConfig extends AbstractElasticsearchConfiguration {

    @Value("${spring.elasticsearch.uris}")
    private String uris;

    @Override
    @Bean
    public RestHighLevelClient elasticsearchClient() {

        final ClientConfiguration clientConfiguration = ClientConfiguration.builder()
                .connectedTo(uris)
                .build();

        return RestClients.create(clientConfiguration).rest();
    }

}

我们看下，那个bean可用(猜测一定被封装好了)，AbstractElasticsearchConfiguration

public abstract class AbstractElasticsearchConfiguration extends ElasticsearchConfigurationSupport {

    /**
     * Return the {@link RestHighLevelClient} instance used to connect to the cluster. 

     *
     * @return never {@literal null}.
     */
    @Bean
    public abstract RestHighLevelClient elasticsearchClient();

    /**
     * Creates {@link ElasticsearchOperations}.
     *
     * @return never {@literal null}.
     */
    @Bean(name = { "elasticsearchOperations", "elasticsearchTemplate" })
    public ElasticsearchOperations elasticsearchOperations(ElasticsearchConverter elasticsearchConverter,
            RestHighLevelClient elasticsearchClient) {

        ElasticsearchRestTemplate template = new ElasticsearchRestTemplate(elasticsearchClient, elasticsearchConverter);
        template.setRefreshPolicy(refreshPolicy());

        return template;
    }
}

不知道各位发现没：@Bean(name = { “elasticsearchOperations”, “elasticsearchTemplate” })
用这个bean去操作

直接执行dsl例子

    //rest模版操作
    @Autowired
    ElasticsearchOperations elasticsearchOperations;


        //rest客户端模版
    @GetMapping("/getHighTempt")
    @ResponseBody
    public String getHighTempt(){
        Query query = new StringQuery("{ \"match\": { \"userLoginId\": 1 } } ");
        query.setPageable(PageRequest.of(0, 10));
        SearchHits<VisitLog>  searchHits = elasticsearchOperations.search(query,VisitLog.class);
        return JSON.toJSONString(searchHits);
    }

响应式客户端-ReactiveElasticsearchClient

• The ReactiveElasticsearchClient is a non official driver based on WebClient.
• It uses the request/response objects provided by the Elasticsearch core project.
• Calls are directly operated on the reactive stack, not wrapping async (thread pool bound) responses into reactive types.
  如果不知道什么是响应式，自行百度

pom.xml

        <dependency>
            <groupId>org.springframeworkgroupId>
            <artifactId>spring-webfluxartifactId>
        dependency>

        <dependency>
            <groupId>io.projectreactor.nettygroupId>
            <artifactId>reactor-netty-httpartifactId>
        dependency>

例子

package org.lwd.microservice.boot.es.config;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.client.ClientConfiguration;
import org.springframework.data.elasticsearch.client.reactive.ReactiveElasticsearchClient;
import org.springframework.data.elasticsearch.client.reactive.ReactiveRestClients;
import org.springframework.data.elasticsearch.config.AbstractReactiveElasticsearchConfiguration;

/**
 * 响应式客户端
 * The ReactiveElasticsearchClient is a non official driver based on WebClient.
 * It uses the request/response objects provided by the Elasticsearch core project.
 * Calls are directly operated on the reactive stack, not wrapping async (thread pool bound) responses into reactive types.
 *
 * @author weidong
 * @version V1.0.0
 * @since 2023/7/28
 */
@Configuration
public class EsReactiveRestClientConfig extends AbstractReactiveElasticsearchConfiguration {

    @Value("${spring.elasticsearch.uris}")
    private String uris;

    @Bean
    @Override
    public ReactiveElasticsearchClient reactiveElasticsearchClient() {
        final ClientConfiguration clientConfiguration = ClientConfiguration.builder()
                .connectedTo(uris) //
                .build();
        return ReactiveRestClients.create(clientConfiguration);
    }
}

    //响应式模版操作
    @Autowired
    ReactiveElasticsearchOperations reactiveElasticsearchOperations;


    //响应式客户端
    @GetMapping("/getReactiveTemp")
    @ResponseBody
    public String getReactiveTemp(){
        Query query = new StringQuery("{ \"match\": { \"userLoginId\": 1 } } ");
        query.setPageable(PageRequest.of(0, 10));
        Flux<SearchHit<VisitLog>> result = reactiveElasticsearchOperations.search(query,VisitLog.class);
        return JSON.toJSONString(result);
    }

ES 原生方式加载客户端

从 ElasticSearch7.17 这个版本开始，原先的 Java 高级客户端
Java High Level REST Client 废弃了，不支持了。老实说，ElasticSearch 算是我用过的所有 Java 工具中，更新最为激进的一个了，在 Es7 中废弃了 TransportClient，7.17 又废弃了 TransportClient，那么现在用啥呢？现在的客户端叫做 Elasticsearch Java API Client。

Elasticsearch Java API Client 具有如下特性：

为所有 Elasticsearch APIs 提供强类型的请求和响应。
所有 API 都有阻塞和异步版本。
使用构建器模式，在创建复杂的嵌套结构时，可以编写简洁而可读的代码。
通过使用对象映射器（如 Jackson 或任何实现了 JSON-B 的解析器），实现应用程序类的无缝集成。
将协议处理委托给一个 http 客户端，如 Java Low Level REST Client，它负责所有传输级的问题。HTTP 连接池、重试、节点发现等等由它去完成。
关于第三点，松哥吐槽一句，确实简洁，但是可读性一般般吧。

另外还有两点需要注意：

Elasticsearch Java 客户端是向前兼容的，即该客户端支持与 Elasticsearch 的更大或相等的次要版本进行通信。
Elasticsearch Java 客户端只向后兼容默认的发行版本，并且没有做出保证。

pom.xml

        <dependency>
            <groupId>co.elastic.clientsgroupId>
            <artifactId>elasticsearch-javaartifactId>
            <version>7.17.9version>
        dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.coregroupId>
            <artifactId>jackson-databindartifactId>
            <version>2.12.3version>
        dependency>
        <dependency>
            <groupId>jakarta.jsongroupId>
            <artifactId>jakarta.json-apiartifactId>
            <version>2.0.1version>
        dependency>

加载原生客户端

注意下，一定要用原生elasticsearch-java jar中的类

package org.lwd.microservice.boot.es.config;

import co.elastic.clients.elasticsearch.ElasticsearchClient;
import co.elastic.clients.json.jackson.JacksonJsonpMapper;
import co.elastic.clients.transport.ElasticsearchTransport;
import co.elastic.clients.transport.rest_client.RestClientTransport;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * 使用elasticsearch client 7.17.9
 *
 * @author weidong
 * @version V1.0.0
 * @since 2023/7/31
 */
@Configuration
public class EsRestClientConfig {
    @Value("${spring.elasticsearch.uris}")
    private String uris;

    @Bean
    public ElasticsearchClient initSyncRestClient() {
        RestClient restClient = RestClient.builder(HttpHost.create(uris)).build();
        ElasticsearchTransport transport = new RestClientTransport(
                restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        return client;
    }
}

controller使用

    @Autowired
    ElasticsearchClient elasticsearchClient;
    //elasticsearch client 7.17.9
    @GetMapping("/getRestTemp")
    @ResponseBody
    public String getRestTemp(){
        StringReader sr = new StringReader(
                "{\n" +
                        "  \"query\": { \"match\": {\n" +
                        "    \"userLoginId\": \"1\"\n" +
                        "  }}" +
                        "}"
        );
        //查询所有
       /* StringReader sr = new StringReader(
               "{\n" +
                       "  \"query\": {\n" +
                       "    \"match_all\" : {}\n" +
                       "  }\n" +
                       "}\n"
        );*/
        SearchRequest request = new SearchRequest.Builder().index(Arrays.asList(new String[]{"visit_log"}))
                .withJson(sr)
                .build();
        try {
            SearchResponse<VisitLog>  search = elasticsearchClient.search(request,VisitLog.class);
            System.out.println("search.toString() = " + search.toString());
            long took = search.took();
            System.out.println("took = " + took);
            boolean b = search.timedOut();
            System.out.println("b = " + b);
            ShardStatistics shards = search.shards();
            System.out.println("shards = " + shards);
            HitsMetadata<VisitLog> hits = search.hits();
            TotalHits total = hits.total();
            System.out.println("total = " + total);
            Double maxScore = hits.maxScore();
            System.out.println("maxScore = " + maxScore);
            List<Hit<VisitLog>> list = hits.hits();
            for (Hit<VisitLog> visitLogHit : list) {
                System.out.println("visitLogHit.source() = " + JSON.toJSONString(visitLogHit.source()));
                System.out.println("visitLogHit.score() = " + visitLogHit.score());
                System.out.println("visitLogHit.index() = " + visitLogHit.index());
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return "";
    }

可以看到，直接把查询的 JSON 参数传进来也是可以的。这样我们就可以先在 Kibana 中写好脚本，然后直接将脚本拷贝到 Java 代码中来执行就行了。

验证结果

request [POST http://10.255.20.231:9200/visit_log/_search?typed_keys=true] returned 1 warnings: [299 Elasticsearch-7.17.9-ef48222227ee6b9e70e502f0f0daa52435ee634d "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-minimal-setup.html to enable security."]
search.toString() = SearchResponse: {"took":0,"timed_out":false,"_shards":{"failed":0.0,"successful":1.0,"total":1.0,"skipped":0.0},"hits":{"total":{"relation":"eq","value":1},"hits":[{"_index":"visit_log","_id":"2","_score":0.9808291,"_type":"_doc","_source":"org.lwd.microservice.boot.es.entity.VisitLog@1bbbdce9"}],"max_score":0.9808291}}
took = 0
b = false
shards = ShardStatistics: {"failed":0.0,"successful":1.0,"total":1.0,"skipped":0.0}
total = TotalHits: {"relation":"eq","value":1}
maxScore = 0.9808291
visitLogHit.source() = {"_class":"org.lwd.microservice.boot.es.entity.VisitLog","createTime":"2023-07-27 16:34:36","id":2,"initialRequest":"http://localhost:8023","msgContent":"test es add2","serverHostName":"weidong","serverIpAddress":"127.0.0.1","tableName":"VisitLog","userLoginId":"1"}
visitLogHit.score() = 0.9808291
visitLogHit.index() = visit_log

测试请求http文件-TestEs.http

## 新增日志
POST http://localhost:8023/es/saveVisitLog
Content-Type: application/json

{
  "id": 3,
  "tableName": "VisitLog",
  "userLoginId": 3,
  "serverIpAddress": "127.0.0.1",
  "serverHostName": "weidong",
  "initialRequest": "http://localhost:8023",
  "msgContent": "test es add3",
  "createTime": "2023-07-27 16:34:36"
}


### 获取所有数据-不分页

GET http://localhost:8023/es/getVisitLogAll

### 获取所有数据-自定义查询规则

GET http://localhost:8023/es/getVisitLogByUserLoginId?userLoginId=1

### 获取所有数据-high rest

GET http://localhost:8023/es/getHighTempt

### 获取所有数据-reactive rest

GET http://localhost:8023/es/getReactiveTemp

### 获取所有数据-elasticsearch client 7.17.9

GET http://localhost:8023/es/getRestTemp

外传

 原创不易，如若本文能够帮助到您的同学
 支持我：关注我+点赞+收藏⭐️
 留言：探讨问题，看到立马回复
 格言：己所不欲勿施于人 扬帆起航、游历人生、永不言弃！