从源码中学习篇:Spring-Data-ES 同时连接多个ES集群。使JPA使用更顺畅

从源码中学习篇:Spring-Data-ES 同时连接多个ES集群。使JPA使用更顺畅

  • 一、背景
  • 二、创建原理
  • 三、核心类
  • 四、实际代码
    • 1、配置文件application.properties
    • 2、集群1:配置类C1ElasticsearchProperties
    • 3、集群1:创建ES连接的类C1ElasticsearchAutoConfiguration
    • 4、集群1:创建模版的类C1ElasticsearchDataAutoConfiguration
    • 5、集群2:配置类: C2ElasticsearchProperties
    • 6、集群2:创建ES连接的类: C2ElasticsearchAutoConfiguration
    • 7、集群2:创建模板的类: C2ElasticsearchDataAutoConfiguration

一、背景

由于 spring-data-elasticsearch 中默认只能连接一个ES集群,但是业务上需要在同一个工程中连接多个ES集群。
网上搜索了有一个可以用,但那是一个第三方ES的框架,我想保持spring-data-elasticsearch的纯度,于是就想到了看源码,并从源码中学习,从而获得修改思路。

二、创建原理

spring-data-elasticsearch 靠spring的自动配置机制,当检测到配置文件中定义了 spring.data.elasticsearch.cluster-nodes 这个属性的时候,就会触发创建ES集群连接,并且自动化的创建 Repositories

三、核心类

以下类来自包org.springframework.boot:spring-boot-autoconfigure:2.0.4.RELEASE
ElasticsearchProperties负责收集配置文件中的集群配置项,顶头上指定了配置项的前缀(默认为: spring.data.elasticsearch)
ElasticsearchAutoConfiguration负责创建 TransportClient, 同时还判断是否存在配置(spring.data.elasticsearch.c1.cluster-nodes),如果存在才创建该bean
ElasticsearchDataAutoConfiguration负责创建 ElasticsearchTemplate

以下类来自包org.springframework.data:spring-data-elasticsearch:3.0.9.RELEASE
@EnableElasticsearchRepositories负责指定扫描的包(basePackages)和 elasticsearchTemplateRef
application.properties负责配置ES集群的连接信息

四、实际代码

1、配置文件application.properties

# 集群1的配置
spring.data.elasticsearch.c1.cluster-name=es-c1
spring.data.elasticsearch.c1.cluster-nodes=127.0.0.1:9301
# 集群2的配置
spring.data.elasticsearch.c2.cluster-name=es-c2
spring.data.elasticsearch.c2.cluster-nodes=127.0.0.1:9302

2、集群1:配置类C1ElasticsearchProperties

该类拷贝自org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchProperties
其唯一的修改为@ConfigurationProperties(prefix = "spring.data.elasticsearch.c1"),将prefix修改自定义的配置前缀即可。
代码如下:

package com.xxx.search.es;

import org.springframework.boot.context.properties.ConfigurationProperties;

import java.util.HashMap;
import java.util.Map;

/**
 * Configuration properties for Elasticsearch.
 *
 * @author Artur Konczak
 * @author Mohsin Husen
 * @since 1.1.0
 */
@ConfigurationProperties(prefix = "spring.data.elasticsearch.c1")
public class C1ElasticsearchProperties {

	/** Elasticsearch cluster name. */
	private String clusterName = "elasticsearch";
	/** Comma-separated list of cluster node addresses.	 */
	private String clusterNodes;
	/** Additional properties used to configure the client.	 */
	private Map<String, String> properties = new HashMap<>();

	public String getClusterName() {
		return this.clusterName;
	}

	public void setClusterName(String clusterName) {
		this.clusterName = clusterName;
	}

	public String getClusterNodes() {
		return this.clusterNodes;
	}

	public void setClusterNodes(String clusterNodes) {
		this.clusterNodes = clusterNodes;
	}

	public Map<String, String> getProperties() {
		return this.properties;
	}

	public void setProperties(Map<String, String> properties) {
		this.properties = properties;
	}
}

3、集群1:创建ES连接的类C1ElasticsearchAutoConfiguration

该类拷贝自org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchAutoConfiguration
其中修改了3处:

  1. ConditionalOnProperty(prefix = "spring.data.elasticsearch.c1")
    将prefix修改自定义的配置前缀即可。
  2. 创建TransportClient@Bean增加参数,指定了自定义的bean名称为c1TransportClient
  3. 去掉了创建TransportClient@ConditionalOnMissingBean

代码如下:

package com.xxx.search.es;

import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.client.TransportClientFactoryBean;

import java.util.Properties;

@Configuration
@ConditionalOnClass({Client.class, TransportClientFactoryBean.class})
@ConditionalOnProperty(prefix = "spring.data.elasticsearch.c1", name = {"cluster-nodes"}, matchIfMissing = false)
@EnableConfigurationProperties({C1ElasticsearchProperties.class})
public class C1ElasticsearchAutoConfiguration {

    private final C1ElasticsearchProperties properties;

    public C1ElasticsearchAutoConfiguration(C1ElasticsearchProperties properties) {
        this.properties = properties;
    }

    @Bean(name = "c1TransportClient")
    public TransportClient elasticsearchClient() throws Exception {
        TransportClientFactoryBean factory = new TransportClientFactoryBean();
        factory.setClusterNodes(this.properties.getClusterNodes());
        factory.setProperties(this.createProperties());
        factory.afterPropertiesSet();
        TransportClient transportClient = factory.getObject();
        System.out.println("transportAddresses1: " + transportClient.transportAddresses());
        return transportClient;
    }

    private Properties createProperties() {
        Properties properties = new Properties();
        properties.put("cluster.name", this.properties.getClusterName());
        properties.putAll(this.properties.getProperties());
        return properties;
    }
}

4、集群1:创建模版的类C1ElasticsearchDataAutoConfiguration

该类拷贝自org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchDataAutoConfiguration
2个修改点:

  1. 创建模版bean的方法中,@Bean增加自定义名称:c1ElasticsearchTemplate;删除@ConditionalOnMissingBean;在参数client增加注解@Qualifier("c1TransportClient")
  2. 类上增加注解@EnableElasticsearchRepositories(basePackages = "com.xxx.search.es.c1", elasticsearchTemplateRef = "c1ElasticsearchTemplate")。这里须要指定当前ES模版要扫描的包,以及使用的模版bean名称(即当前类中创建的自定义模版bean)

代码如下:

package com.xxx.search.es;

import org.elasticsearch.client.Client;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.convert.ElasticsearchConverter;
import org.springframework.data.elasticsearch.core.convert.MappingElasticsearchConverter;
import org.springframework.data.elasticsearch.core.mapping.SimpleElasticsearchMappingContext;
import org.springframework.data.elasticsearch.repository.config.EnableElasticsearchRepositories;

/**
 * {@link EnableAutoConfiguration Auto-configuration} for Spring Data's Elasticsearch
 * support.
 * 

* Registers an {@link ElasticsearchTemplate} if no other bean of the same type is * configured. * * @author Artur Konczak * @author Mohsin Husen * @see EnableElasticsearchRepositories * @since 1.1.0 */ @Configuration @ConditionalOnClass({ Client.class, ElasticsearchTemplate.class }) @AutoConfigureAfter(C1ElasticsearchAutoConfiguration.class) @EnableElasticsearchRepositories(basePackages = "com.xxx.search.es.c1", elasticsearchTemplateRef = "c1ElasticsearchTemplate") public class C1ElasticsearchDataAutoConfiguration { @Bean("c1ElasticsearchTemplate") @ConditionalOnBean(Client.class) public ElasticsearchTemplate elasticsearchTemplate( @Qualifier("c1TransportClient") Client client, ElasticsearchConverter converter) { try { return new ElasticsearchTemplate(client, converter); } catch (Exception ex) { throw new IllegalStateException(ex); } } @Bean @ConditionalOnMissingBean public ElasticsearchConverter elasticsearchConverter(SimpleElasticsearchMappingContext mappingContext) { return new MappingElasticsearchConverter(mappingContext); } @Bean @ConditionalOnMissingBean public SimpleElasticsearchMappingContext mappingContext() { return new SimpleElasticsearchMappingContext(); } }

5、集群2:配置类: C2ElasticsearchProperties

修改内容同集群1,代码如下:

package com.xxx.search.es;

import org.springframework.boot.context.properties.ConfigurationProperties;

import java.util.HashMap;
import java.util.Map;

/**
 * Configuration properties for Elasticsearch.
 *
 * @author Artur Konczak
 * @author Mohsin Husen
 * @since 1.1.0
 */
@ConfigurationProperties(prefix = "spring.data.elasticsearch.c2")
public class C2ElasticsearchProperties {

	/** Elasticsearch cluster name.	 */
	private String clusterName = "elasticsearch";

	/** Comma-separated list of cluster node addresses.	 */
	private String clusterNodes;

	/**Additional properties used to configure the client.	 */
	private Map<String, String> properties = new HashMap<>();

	public String getClusterName() {
		return this.clusterName;
	}

	public void setClusterName(String clusterName) {
		this.clusterName = clusterName;
	}

	public String getClusterNodes() {
		return this.clusterNodes;
	}

	public void setClusterNodes(String clusterNodes) {
		this.clusterNodes = clusterNodes;
	}

	public Map<String, String> getProperties() {
		return this.properties;
	}

	public void setProperties(Map<String, String> properties) {
		this.properties = properties;
	}
}

6、集群2:创建ES连接的类: C2ElasticsearchAutoConfiguration

修改内容同集群1,代码如下:

package com.xxx.search.es;

import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.client.TransportClientFactoryBean;

import java.util.Properties;

@Configuration
@ConditionalOnClass({Client.class, TransportClientFactoryBean.class})
@ConditionalOnProperty(prefix = "spring.data.elasticsearch.c2", name = {"cluster-nodes"}, matchIfMissing = false)
@EnableConfigurationProperties({C2ElasticsearchProperties.class})
public class C2ElasticsearchAutoConfiguration {

    private final C2ElasticsearchProperties properties;

    public C2ElasticsearchAutoConfiguration(C2ElasticsearchProperties properties) {
        this.properties = properties;
    }

    @Bean(name = "c2TransportClient")
    public TransportClient elasticsearchClient() throws Exception {
        TransportClientFactoryBean factory = new TransportClientFactoryBean();
        factory.setClusterNodes(this.properties.getClusterNodes());
        factory.setProperties(this.createProperties());
        factory.afterPropertiesSet();
        TransportClient transportClient = factory.getObject();
        System.out.println("transportAddresses2: " + transportClient.transportAddresses());
        return transportClient;
    }

    private Properties createProperties() {
        Properties properties = new Properties();
        properties.put("cluster.name", this.properties.getClusterName());
        properties.putAll(this.properties.getProperties());
        return properties;
    }
}

7、集群2:创建模板的类: C2ElasticsearchDataAutoConfiguration

修改内容同集群1,代码如下:

package com.xxx.search.es;

import org.elasticsearch.client.Client;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.convert.ElasticsearchConverter;
import org.springframework.data.elasticsearch.core.convert.MappingElasticsearchConverter;
import org.springframework.data.elasticsearch.core.mapping.SimpleElasticsearchMappingContext;
import org.springframework.data.elasticsearch.repository.config.EnableElasticsearchRepositories;

/**
 * {@link EnableAutoConfiguration Auto-configuration} for Spring Data's Elasticsearch
 * support.
 * 

* Registers an {@link ElasticsearchTemplate} if no other bean of the same type is * configured. * * @author Artur Konczak * @author Mohsin Husen * @see EnableElasticsearchRepositories * @since 1.1.0 */ @Configuration @ConditionalOnClass({ Client.class, ElasticsearchTemplate.class }) @AutoConfigureAfter(C2ElasticsearchAutoConfiguration.class) @EnableElasticsearchRepositories(basePackages = "com.xxx.search.es.c2", elasticsearchTemplateRef = "c2ElasticsearchTemplate") public class C2ElasticsearchDataAutoConfiguration { @Bean("c2ElasticsearchTemplate") @ConditionalOnBean(Client.class) public ElasticsearchTemplate elasticsearchTemplate( @Qualifier("c2TransportClient") Client client, ElasticsearchConverter converter) { try { return new ElasticsearchTemplate(client, converter); } catch (Exception ex) { throw new IllegalStateException(ex); } } @Bean @ConditionalOnMissingBean public ElasticsearchConverter elasticsearchConverter(SimpleElasticsearchMappingContext mappingContext) { return new MappingElasticsearchConverter(mappingContext); } @Bean @ConditionalOnMissingBean public SimpleElasticsearchMappingContext mappingContext() { return new SimpleElasticsearchMappingContext(); } }

你可能感兴趣的:(Elasticsearch,多数据源,Spring-boot,spring,boot,java,es,elasticsearch)