springboot整合es启动报错的问题
今天打算用springboot整合es创建一个索引并往索引里面写数据的时候，项目启动的时候一直报下面的这个错误，错误大概如下，
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.elasticsearch.client.transport.TransportClient]: Factory method 'esClient' threw exception; nested exception is java.lang.IllegalStateException: availableProcessors is already set to [4], rejecting [4]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.0.5.RELEASE.jar:5.0.5.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:579) ~[spring-beans-5.0.5.RELEASE.jar:5.0.5.RELEASE]
... 44 common frames omitted
Caused by: java.lang.IllegalStateException: availableProcessors is already set to [4], rejecting [4]
at io.netty.util.NettyRuntime$AvailableProcessorsHolder.setAvailableProcessors(NettyRuntime.java:51) ~[netty-common-4.1.23.Final.jar:4.1.23.Final]
at io.netty.util.NettyRuntime.setAvailableProcessors(NettyRuntime.java:87) ~[netty-common-4.1.23.Final.jar:4.1.23.Final]
at org.elasticsearch.transport.netty4.Netty4Utils.setAvailableProcessors(Netty4Utils.java:87) ~[transport-netty4-client-6.1.1.jar:6.1.1]
at org.elasticsearch.transport.netty4.Netty4Transport.(Netty4Transport.java:115) ~[transport-netty4-client-6.1.1.jar:6.1.1]
at org.elasticsearch.transport.Netty4Plugin.lambda$getTransports$0(Netty4Plugin.java:84) ~[transport-netty4-client-6.1.1.jar:6.1.1]
at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:176) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.client.transport.TransportClient.(TransportClient.java:262) ~[elasticsearch-6.1.1.jar:6.1.1]
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:128) ~[transport-6.1.1.jar:6.1.1]
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:114) ~[transport-6.1.1.jar:6.1.1]
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:104) ~[transport-6.1.1.jar:6.1.1]
at com.congge.config.ElasticSearchConfig.esClient(ElasticSearchConfig.java:38) ~[classes/:na]
at com.congge.config.ElasticSearchConfig$$EnhancerBySpringCGLIB$$8b8b4508.CGLIB$esClient$0() ~[classes/:na]
at com.congge.config.ElasticSearchConfig$$EnhancerBySpringCGLIB$$8b8b4508$$FastClassBySpringCGLIB$$2307bde5.invoke() ~[classes/:na]
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) ~[spring-core-5.0.5.RELEASE.jar:5.0.5.RELEASE]
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:361) ~[spring-context-5.0.5.RELEASE.jar:5.0.5.RELEASE]
at com.congge.config.ElasticSearchConfig$$EnhancerBySpringCGLIB$$8b8b4508.esClient() ~[classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_171]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_171]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.0.5.RELEASE.jar:5.0.5.RELEASE]
... 45 common frames omitted
这个错误让人很恼火，因为之前在公司使用的时候，可能是运维的同事搭建好了自己没有尝试过直接使用，所以没有出现过这个问题，但是自己整合的时候出现了，网上查了一大堆资料，给了很多解释却没有一个用得上的，仔细对照了es的版本之后，发现5.X的版本使用的时候好像没有这个问题，5.X的版本使用的时候添加的依赖如下，即只需要添加这个依赖就可以启动了，
org.elasticsearch.client
transport
5.6.1
但是我使用的是6.X的版本，6.X的版本要添加的依赖要多两个，否则启动的时候直接报错，6.X我这里的依赖是，即这里面有一个整合netty的依赖包，我猜想6.X的底层通信是否在底层通信时依赖netty，源码可以研究下，
org.elasticsearch
elasticsearch
6.1.1
org.elasticsearch.client
transport
6.1.1
org.elasticsearch.plugin
transport-netty4-client
6.1.1
这时候启动工程就报了开篇的那个错误，后来总结了一下这个错误，大概意思就是项目初始化启动的时候esclient注入的时候和netty的相关配置参数冲突了，从问题描述大致也可以看出，解决办法如下，这里主要有两种，第一种是在es的配置类里面设置，即使用@PostConstruct注解，这个注解的含义不用多解释了吧？
/**
* es初始化加载全局配置类
* @author asus
*
*/
@Configuration
public class ElasticSearchConfig {
@PostConstruct
void init() {
System.setProperty("es.set.netty.runtime.available.processors", "false");
}
@Bean
public TransportClient esClient() throws UnknownHostException {
Settings settings = Settings.builder()
.put("cluster.name", "xunwu").put("client.transport.sniff", true).build();
@SuppressWarnings("resource")
TransportClient client = new PreBuiltTransportClient(settings)
.addTransportAddress(new TransportAddress(InetAddress.getByName("192.168.111.132"), 9300));
return client;
}
}
如果这样还是不行，就在启动main函数里面加，
@SpringBootApplication
public class App {
public static void main(String[] args) {
System.setProperty("es.set.netty.runtime.available.processors", "false");
SpringApplication.run(App.class, args);
}
}
然后我们再启动项目，这时候可以看到项目已经可以正常启动了，
我们往提前创建好的"xunwu"这个索引里面插入一条数据，可以看到创建成功，
再看看我们刚刚加入的那条数据，可以看到，数据已经取出来了，
对应的后台代码也一并附上，有需要的小伙伴可以参考，这里我简单写了两个测试接口，
@Controller
public class SearchController {
@Autowired
private SearchService searchService;
@Autowired
private TransportClient esClient;
//创建索引
@RequestMapping("/createIndexData")
@ResponseBody
public String createIndexData() {
Long houseId = 15L;
boolean success = searchService.index(houseId);
if (success) {
return "success";
}
return "fail";
}
//查询单个数据
@RequestMapping("/getIndexData")
@ResponseBody
public Map getIndexData() {
Long houseId = 15L;
Map resMap = searchService.printMessage(houseId);
return resMap;
}
//插叙所有
@RequestMapping("/getAllData")
@ResponseBody
public Object getAllData() {
SearchResponse searchResponse = esClient.prepareSearch("xunwu").setTypes("house")
.setSearchType(SearchType.QUERY_THEN_FETCH).setQuery(QueryBuilders.matchAllQuery()) // 查询所有
.setExplain(true).setFrom(0).setSize(10000)
.execute().actionGet();
SearchHits hits = searchResponse.getHits();
SearchHit[] searchHits = hits.getHits();
Map resMap = new HashMap<>();
for (SearchHit s : hits) {
Map single = s.getSourceAsMap();
resMap.put("houseId", single.get("houseId"));
resMap.put("cityEnName", single.get("cityEnName"));
resMap.put("subwayStationName", single.get("subwayStationName"));
resMap.put("street", single.get("street"));
resMap.put("title", single.get("title"));
resMap.put("tags", single.get("tags"));
}
return resMap;
}
}
service:
/**
* 打印创建成功的索引信息
* @param houseId
*/
@Override
public Map printMessage(Long houseId){
SearchRequestBuilder requestBuilder = this.esClient.prepareSearch(INDEX_NAME)
.setTypes(INDEX_TYPE)
.setQuery(
QueryBuilders.termQuery(HouseIndexKey.HOUSE_ID, houseId));
SearchResponse response = requestBuilder.get();
SearchHit[] hits = response.getHits().getHits();
Map resMap = new HashMap<>();
for(SearchHit s : hits){
Map single = s.getSourceAsMap();
resMap.put("houseId", single.get("houseId"));
resMap.put("cityEnName", single.get("cityEnName"));
resMap.put("subwayStationName", single.get("subwayStationName"));
resMap.put("street", single.get("street"));
resMap.put("title", single.get("title"));
resMap.put("tags", single.get("tags"));
}
return resMap;
}
@Override
public boolean index(Long houseId) {
Optional house = houseRepository.findById(houseId);
if (house.get() == null) {
logger.error("Index house {} is not exist!", houseId);
}
// 房屋基本信息
HouseIndexTemplate indexTemplate = new HouseIndexTemplate();
modelMapper.map(house.get(),indexTemplate);
// 房屋详情
Optional houseDetail = houseRepository.findById(houseId);
if (houseDetail.get() == null) {
// TODO 异常情况
}
modelMapper.map(houseDetail.get(),indexTemplate);
// 房屋标签信息
List tags = tagRepository.findAllByHouseId(houseId);
if (tags != null && tags.size() > 0) {
List tagsString = new ArrayList<>();
tags.forEach(tag -> {
tagsString.add(tag.getName());
});
indexTemplate.setTags(tagsString);
}
// 1、判断es中是否存在数据，决定是创建、更新还是删除
SearchRequestBuilder requestBuilder = this.esClient.prepareSearch(INDEX_NAME)
.setTypes(INDEX_TYPE)
.setQuery(
QueryBuilders.termQuery(HouseIndexKey.HOUSE_ID, houseId));
logger.debug(requestBuilder.toString());
SearchResponse searchResponse = requestBuilder.get();
boolean success;
long totalHits = searchResponse.getHits().getTotalHits();
if (totalHits == 0) { // 如果es没有查到数据，则为索引创建新数据
success = createIndex(indexTemplate);
}else if(totalHits == 1){ //每个houseId对应的数据只应该有一份，需要更新
String esId = searchResponse.getHits().getAt(0).getId();
success = updateIndex(esId, indexTemplate);
}else{
success = deleteAndCreate(indexTemplate, totalHits);
}
if(success){
logger.debug("Index success with house :" + houseId);
}
return success;
}
/**
* 创建索引
* @param template
* @return
*/
public boolean createIndex(HouseIndexTemplate template) {
try {
IndexResponse response = this.esClient.prepareIndex(INDEX_NAME, INDEX_TYPE)
.setSource(objectMapper.writeValueAsBytes(template),
XContentType.JSON).get();
logger.debug("Create index with house :" + template.getHouseId());
if(response.status() == RestStatus.CREATED){
return true;
}else{
return false;
}
} catch (JsonProcessingException e) {
logger.error("Error to create index :" + template.getHouseId(),e);
return false;
}
}
/**
* 更新索引
* @param esId
* @param template
* @return
*/
public boolean updateIndex(String esId,HouseIndexTemplate template) {
try {
UpdateResponse response = this.esClient.prepareUpdate(INDEX_NAME, INDEX_TYPE,esId)
.setDoc(objectMapper.writeValueAsBytes(template),
XContentType.JSON).get();
logger.debug("Update index with house :" + template.getHouseId());
if(response.status() == RestStatus.OK){
return true;
}else{
return false;
}
} catch (JsonProcessingException e) {
logger.error("Error to update index :" + template.getHouseId(),e);
return false;
}
}
/**
* 删除索引
* @param template
* @param totalHit
* @return
*/
public boolean deleteAndCreate(HouseIndexTemplate template,long totalHit) {
DeleteByQueryRequestBuilder builder = DeleteByQueryAction.INSTANCE
.newRequestBuilder(esClient)
.filter(QueryBuilders.termQuery(HouseIndexKey.HOUSE_ID, template.getHouseId()))
.source(INDEX_NAME);
logger.debug("delte by query for house :" + builder);
BulkByScrollResponse response = builder.get();
long deleteHit = response.getDeleted();
if(totalHit != deleteHit){
logger.warn("Need delete {},but {} was delete", totalHit,deleteHit);
return false;
}else{
//删除成功后直接创建新索引
return createIndex(template);
}
}
问题的描述和解决过程大致如上，有需要的同学可酌情参考