从源码中学习篇：Spring-Data-ES 同时连接多个ES集群。使JPA使用更顺畅
从源码中学习篇：Spring-Data-ES 同时连接多个ES集群。使JPA使用更顺畅
- 一、背景
- 二、创建原理
- 三、核心类
- 四、实际代码
-
- 1、配置文件application.properties
- 2、集群1：配置类C1ElasticsearchProperties
- 3、集群1：创建ES连接的类C1ElasticsearchAutoConfiguration
- 4、集群1：创建模版的类C1ElasticsearchDataAutoConfiguration
- 5、集群2：配置类: C2ElasticsearchProperties
- 6、集群2：创建ES连接的类: C2ElasticsearchAutoConfiguration
- 7、集群2：创建模板的类: C2ElasticsearchDataAutoConfiguration
一、背景
由于 spring-data-elasticsearch 中默认只能连接一个ES集群，但是业务上需要在同一个工程中连接多个ES集群。
网上搜索了有一个可以用，但那是一个第三方ES的框架，我想保持spring-data-elasticsearch的纯度，于是就想到了看源码，并从源码中学习，从而获得修改思路。
二、创建原理
spring-data-elasticsearch 靠spring的自动配置机制，当检测到配置文件中定义了 spring.data.elasticsearch.cluster-nodes 这个属性的时候，就会触发创建ES集群连接，并且自动化的创建 Repositories
三、核心类
以下类来自包org.springframework.boot:spring-boot-autoconfigure:2.0.4.RELEASE
ElasticsearchProperties负责收集配置文件中的集群配置项，顶头上指定了配置项的前缀(默认为: spring.data.elasticsearch)
ElasticsearchAutoConfiguration负责创建 TransportClient, 同时还判断是否存在配置(spring.data.elasticsearch.c1.cluster-nodes)，如果存在才创建该bean
ElasticsearchDataAutoConfiguration负责创建 ElasticsearchTemplate
以下类来自包org.springframework.data:spring-data-elasticsearch:3.0.9.RELEASE
@EnableElasticsearchRepositories负责指定扫描的包(basePackages)和 elasticsearchTemplateRef
application.properties负责配置ES集群的连接信息
四、实际代码
1、配置文件application.properties
# 集群1的配置
spring.data.elasticsearch.c1.cluster-name=es-c1
spring.data.elasticsearch.c1.cluster-nodes=127.0.0.1:9301
# 集群2的配置
spring.data.elasticsearch.c2.cluster-name=es-c2
spring.data.elasticsearch.c2.cluster-nodes=127.0.0.1:9302
2、集群1：配置类C1ElasticsearchProperties
该类拷贝自org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchProperties
其唯一的修改为@ConfigurationProperties(prefix = "spring.data.elasticsearch.c1")，将prefix修改自定义的配置前缀即可。
代码如下：
package com.xxx.search.es;
import org.springframework.boot.context.properties.ConfigurationProperties;
import java.util.HashMap;
import java.util.Map;
/**
* Configuration properties for Elasticsearch.
*
* @author Artur Konczak
* @author Mohsin Husen
* @since 1.1.0
*/
@ConfigurationProperties(prefix = "spring.data.elasticsearch.c1")
public class C1ElasticsearchProperties {
/** Elasticsearch cluster name. */
private String clusterName = "elasticsearch";
/** Comma-separated list of cluster node addresses. */
private String clusterNodes;
/** Additional properties used to configure the client. */
private Map<String, String> properties = new HashMap<>();
public String getClusterName() {
return this.clusterName;
}
public void setClusterName(String clusterName) {
this.clusterName = clusterName;
}
public String getClusterNodes() {
return this.clusterNodes;
}
public void setClusterNodes(String clusterNodes) {
this.clusterNodes = clusterNodes;
}
public Map<String, String> getProperties() {
return this.properties;
}
public void setProperties(Map<String, String> properties) {
this.properties = properties;
}
}
3、集群1：创建ES连接的类C1ElasticsearchAutoConfiguration
该类拷贝自org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchAutoConfiguration
其中修改了3处：
- ConditionalOnProperty(prefix =
"spring.data.elasticsearch.c1")
将prefix修改自定义的配置前缀即可。 - 创建TransportClient的
@Bean增加参数，指定了自定义的bean名称为c1TransportClient - 去掉了创建TransportClient的
@ConditionalOnMissingBean
代码如下：
package com.xxx.search.es;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.client.TransportClientFactoryBean;
import java.util.Properties;
@Configuration
@ConditionalOnClass({Client.class, TransportClientFactoryBean.class})
@ConditionalOnProperty(prefix = "spring.data.elasticsearch.c1", name = {"cluster-nodes"}, matchIfMissing = false)
@EnableConfigurationProperties({C1ElasticsearchProperties.class})
public class C1ElasticsearchAutoConfiguration {
private final C1ElasticsearchProperties properties;
public C1ElasticsearchAutoConfiguration(C1ElasticsearchProperties properties) {
this.properties = properties;
}
@Bean(name = "c1TransportClient")
public TransportClient elasticsearchClient() throws Exception {
TransportClientFactoryBean factory = new TransportClientFactoryBean();
factory.setClusterNodes(this.properties.getClusterNodes());
factory.setProperties(this.createProperties());
factory.afterPropertiesSet();
TransportClient transportClient = factory.getObject();
System.out.println("transportAddresses1: " + transportClient.transportAddresses());
return transportClient;
}
private Properties createProperties() {
Properties properties = new Properties();
properties.put("cluster.name", this.properties.getClusterName());
properties.putAll(this.properties.getProperties());
return properties;
}
}
4、集群1：创建模版的类C1ElasticsearchDataAutoConfiguration
该类拷贝自org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchDataAutoConfiguration
2个修改点：
- 创建模版bean的方法中，@Bean增加自定义名称：
c1ElasticsearchTemplate；删除@ConditionalOnMissingBean；在参数client增加注解@Qualifier("c1TransportClient") - 类上增加注解
@EnableElasticsearchRepositories(basePackages = "com.xxx.search.es.c1", elasticsearchTemplateRef = "c1ElasticsearchTemplate")。这里须要指定当前ES模版要扫描的包，以及使用的模版bean名称(即当前类中创建的自定义模版bean)
代码如下：
package com.xxx.search.es;
import org.elasticsearch.client.Client;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.convert.ElasticsearchConverter;
import org.springframework.data.elasticsearch.core.convert.MappingElasticsearchConverter;
import org.springframework.data.elasticsearch.core.mapping.SimpleElasticsearchMappingContext;
import org.springframework.data.elasticsearch.repository.config.EnableElasticsearchRepositories;
/**
* {@link EnableAutoConfiguration Auto-configuration} for Spring Data's Elasticsearch
* support.
*
* Registers an {@link ElasticsearchTemplate} if no other bean of the same type is
* configured.
*
* @author Artur Konczak
* @author Mohsin Husen
* @see EnableElasticsearchRepositories
* @since 1.1.0
*/
@Configuration
@ConditionalOnClass({ Client.class, ElasticsearchTemplate.class })
@AutoConfigureAfter(C1ElasticsearchAutoConfiguration.class)
@EnableElasticsearchRepositories(basePackages = "com.xxx.search.es.c1", elasticsearchTemplateRef = "c1ElasticsearchTemplate")
public class C1ElasticsearchDataAutoConfiguration {
@Bean("c1ElasticsearchTemplate")
@ConditionalOnBean(Client.class)
public ElasticsearchTemplate elasticsearchTemplate(
@Qualifier("c1TransportClient") Client client,
ElasticsearchConverter converter) {
try {
return new ElasticsearchTemplate(client, converter);
}
catch (Exception ex) {
throw new IllegalStateException(ex);
}
}
@Bean
@ConditionalOnMissingBean
public ElasticsearchConverter elasticsearchConverter(SimpleElasticsearchMappingContext mappingContext) {
return new MappingElasticsearchConverter(mappingContext);
}
@Bean
@ConditionalOnMissingBean
public SimpleElasticsearchMappingContext mappingContext() {
return new SimpleElasticsearchMappingContext();
}
}
5、集群2：配置类: C2ElasticsearchProperties
修改内容同集群1，代码如下：
package com.xxx.search.es;
import org.springframework.boot.context.properties.ConfigurationProperties;
import java.util.HashMap;
import java.util.Map;
/**
* Configuration properties for Elasticsearch.
*
* @author Artur Konczak
* @author Mohsin Husen
* @since 1.1.0
*/
@ConfigurationProperties(prefix = "spring.data.elasticsearch.c2")
public class C2ElasticsearchProperties {
/** Elasticsearch cluster name. */
private String clusterName = "elasticsearch";
/** Comma-separated list of cluster node addresses. */
private String clusterNodes;
/**Additional properties used to configure the client. */
private Map<String, String> properties = new HashMap<>();
public String getClusterName() {
return this.clusterName;
}
public void setClusterName(String clusterName) {
this.clusterName = clusterName;
}
public String getClusterNodes() {
return this.clusterNodes;
}
public void setClusterNodes(String clusterNodes) {
this.clusterNodes = clusterNodes;
}
public Map<String, String> getProperties() {
return this.properties;
}
public void setProperties(Map<String, String> properties) {
this.properties = properties;
}
}
6、集群2：创建ES连接的类: C2ElasticsearchAutoConfiguration
修改内容同集群1，代码如下：
package com.xxx.search.es;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.client.TransportClientFactoryBean;
import java.util.Properties;
@Configuration
@ConditionalOnClass({Client.class, TransportClientFactoryBean.class})
@ConditionalOnProperty(prefix = "spring.data.elasticsearch.c2", name = {"cluster-nodes"}, matchIfMissing = false)
@EnableConfigurationProperties({C2ElasticsearchProperties.class})
public class C2ElasticsearchAutoConfiguration {
private final C2ElasticsearchProperties properties;
public C2ElasticsearchAutoConfiguration(C2ElasticsearchProperties properties) {
this.properties = properties;
}
@Bean(name = "c2TransportClient")
public TransportClient elasticsearchClient() throws Exception {
TransportClientFactoryBean factory = new TransportClientFactoryBean();
factory.setClusterNodes(this.properties.getClusterNodes());
factory.setProperties(this.createProperties());
factory.afterPropertiesSet();
TransportClient transportClient = factory.getObject();
System.out.println("transportAddresses2: " + transportClient.transportAddresses());
return transportClient;
}
private Properties createProperties() {
Properties properties = new Properties();
properties.put("cluster.name", this.properties.getClusterName());
properties.putAll(this.properties.getProperties());
return properties;
}
}
7、集群2：创建模板的类: C2ElasticsearchDataAutoConfiguration
修改内容同集群1，代码如下：
package com.xxx.search.es;
import org.elasticsearch.client.Client;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.convert.ElasticsearchConverter;
import org.springframework.data.elasticsearch.core.convert.MappingElasticsearchConverter;
import org.springframework.data.elasticsearch.core.mapping.SimpleElasticsearchMappingContext;
import org.springframework.data.elasticsearch.repository.config.EnableElasticsearchRepositories;
/**
* {@link EnableAutoConfiguration Auto-configuration} for Spring Data's Elasticsearch
* support.
*
* Registers an {@link ElasticsearchTemplate} if no other bean of the same type is
* configured.
*
* @author Artur Konczak
* @author Mohsin Husen
* @see EnableElasticsearchRepositories
* @since 1.1.0
*/
@Configuration
@ConditionalOnClass({ Client.class, ElasticsearchTemplate.class })
@AutoConfigureAfter(C2ElasticsearchAutoConfiguration.class)
@EnableElasticsearchRepositories(basePackages = "com.xxx.search.es.c2", elasticsearchTemplateRef = "c2ElasticsearchTemplate")
public class C2ElasticsearchDataAutoConfiguration {
@Bean("c2ElasticsearchTemplate")
@ConditionalOnBean(Client.class)
public ElasticsearchTemplate elasticsearchTemplate(
@Qualifier("c2TransportClient") Client client,
ElasticsearchConverter converter) {
try {
return new ElasticsearchTemplate(client, converter);
}
catch (Exception ex) {
throw new IllegalStateException(ex);
}
}
@Bean
@ConditionalOnMissingBean
public ElasticsearchConverter elasticsearchConverter(SimpleElasticsearchMappingContext mappingContext) {
return new MappingElasticsearchConverter(mappingContext);
}
@Bean
@ConditionalOnMissingBean
public SimpleElasticsearchMappingContext mappingContext() {
return new SimpleElasticsearchMappingContext();
}
}