【1】添加Elasticsearch-starter
pom文件添加starter如下:
org.springframework.boot spring-boot-starter-data-elasticsearch
SpringBoot默认支持两种技术和Elasticsearch进行交互:Spring Data Elasticsearch和Jest。
Jest默认不生效,需要导入io.searchbox.client.JestClient。
maven依赖如下:
io.searchbox jest 5.3.3
Spring Data Elasticsearch主要作用如下:
① ElasticsearchAutoConfiguration中注册了client,属性有clusterNodes和clusterName。
② ElasticsearchDataAutoConfiguration注册了ElasticsearchTemplate来操作ES
@Configuration @ConditionalOnClass({ Client.class, ElasticsearchTemplate.class }) @AutoConfigureAfter(ElasticsearchAutoConfiguration.class) public class ElasticsearchDataAutoConfiguration { @Bean @ConditionalOnMissingBean @ConditionalOnBean(Client.class) public ElasticsearchTemplate elasticsearchTemplate(Client client, ElasticsearchConverter converter) { try { return new ElasticsearchTemplate(client, converter); } catch (Exception ex) { throw new IllegalStateException(ex); } } @Bean @ConditionalOnMissingBean public ElasticsearchConverter elasticsearchConverter( SimpleElasticsearchMappingContext mappingContext) { return new MappingElasticsearchConverter(mappingContext); } @Bean @ConditionalOnMissingBean public SimpleElasticsearchMappingContext mappingContext() { return new SimpleElasticsearchMappingContext(); } }
③ ElasticsearchRepositoriesAutoConfiguration 启用了ElasticsearchRepository
@Configuration @ConditionalOnClass({ Client.class, ElasticsearchRepository.class }) @ConditionalOnProperty(prefix = "spring.data.elasticsearch.repositories", name = "enabled", havingValue = "true", matchIfMissing = true) @ConditionalOnMissingBean(ElasticsearchRepositoryFactoryBean.class) @Import(ElasticsearchRepositoriesRegistrar.class) public class ElasticsearchRepositoriesAutoConfiguration { }
ElasticsearchRepository接口源码如下(类似于JPA中的接口):
@NoRepositoryBean public interface ElasticsearchRepositoryextends ElasticsearchCrudRepository { S index(S var1); Iterablesearch(QueryBuilder var1); Page search(QueryBuilder var1, Pageable var2); Page search(SearchQuery var1); Page searchSimilar(T var1, String[] var2, Pageable var3); void refresh(); Class getEntityClass(); }
【2】JestClient操作测试
application.properties配置如下:
# jest url配置 spring.elasticsearch.jest.uris=http://192.168.2.110:9200
测试类如下:
@RunWith(SpringRunner.class) @SpringBootTest public class SpringBootJestTest { @Autowired JestClient jestClient; @Test public void index(){ Article article = new Article(); article.setId(1); article.setAuthor("Tom"); article.setContent("hello world !"); article.setTitle("今日消息"); //构建一个索引功能,类型为news Index index = new Index.Builder(article).index("jest").type("news").build(); try { jestClient.execute(index); System.out.println("数据索引成功!"); } catch (IOException e) { e.printStackTrace(); } } @Test public void search(){ //查询表达式 String json = "{\n" + " \"query\" : {\n" + " \"match\" : {\n" + " \"content\" : \"hello\"\n" + " }\n" + " }\n" + "}"; //构建搜索功能 Search search = new Search.Builder(json).addIndex("jest").addType("news").build(); try { SearchResult result = jestClient.execute(search); System.out.println(result.getJsonString()); } catch (IOException e) { e.printStackTrace(); } } }
测试存储数据结果如下:
测试查询数据结果如下:
【3】 Elasticsearch版本调整
application.properties进行配置:
# Spring data elasticsearch配置 spring.data.elasticsearch.cluster-name=elasticsearch spring.data.elasticsearch.cluster-nodes=192.168.2.110:9300
这里节点名取自如下图:
启动主程序,可能报错如下(ES版本不合适):
查看Spring Data官网,其中spring data elasticsearch与elasticsearch适配表如下:
官网地址:https://github.com/spring-projects/spring-data-elasticsearch
我们在上篇博文中安装的ES版本为5.6.10,项目中SpringBoot版本为1.5.12,spring-boot-starter-data-elasticsearch为2.1.11,elasticsearch版本为2.4.6。
两种解决办法:① 升级SpringBoot版本;② 安装2.4.6版本的elasticsearch。
这里修改暴露的端口,重新使用docker安装2.4.6版本:
# 拉取2.4.6 镜像 docker pull registry.docker-cn.com/library/elasticsearch:2.4.6 # 启动容器 docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9201:9200 -p 9301:9300 --name ES02 bc337c8e4f
application.properties配置文件同步修改:
# jest url配置 spring.elasticsearch.jest.uris=http://192.168.2.110:9201 # Spring data elasticsearch配置 spring.data.elasticsearch.cluster-name=elasticsearch spring.data.elasticsearch.cluster-nodes=192.168.2.110:9301
此时再次启动程序:
【4】ElasticsearchRepository使用
类似于JPA,编写自定义Repository接口,继承自ElasticsearchRepository:
public interface BookRepository extends ElasticsearchRepository{ public List findByBookNameLike(String bookName); }
这里第一个参数为对象类型,第二个参数为对象的主键类型。
BookRepository 所拥有的方法如下图:
Book源码如下:
// 这里注意注解 @Document(indexName = "elastic",type = "book") public class Book { private Integer id; private String bookName; private String author; public Integer getId() { return id; } public void setId(Integer id) { this.id = id; } public String getBookName() { return bookName; } public void setBookName(String bookName) { this.bookName = bookName; } public String getAuthor() { return author; } public void setAuthor(String author) { this.author = author; } @Override public String toString() { return "Book{" + "id=" + id + ", bookName='" + bookName + '\'' + ", author='" + author + '\'' + '}'; } }
测试类如下:
@Autowired BookRepository bookRepository; @Test public void testRepository(){ Book book = new Book(); book.setAuthor("吴承恩"); book.setBookName("西游记"); book.setId(1); bookRepository.index(book); System.out.println("BookRepository 存入数据成功!"); }
测试结果如下图:
测试获取示例如下:
@Test public void testRepository2(){ for (Book book : bookRepository.findByBookNameLike("游")) { System.out.println("获取的book : "+book); } ; Book book = bookRepository.findOne(1); System.out.println("根据id查询 : "+book); }
测试结果如下图:
Elasticsearch支持方法关键字如下图所示
即,在BookRepository中使用上述关键字构造方法,即可使用,Elastic自行实现其功能!
支持@Query注解
如下所示,直接在方法上使用注解:
public interface BookRepository extends ElasticsearchRepository{ @Query("{"bool" : {"must" : {"field" : {"name" : "?0"}}}}") Page findByName(String name,Pageable pageable); }
【5】ElasticsearchTemplate使用
存入数据源码示例如下:
@Autowired ElasticsearchTemplate elasticsearchTemplate; @Test public void testTemplate01(){ Book book = new Book(); book.setAuthor("曹雪芹"); book.setBookName("红楼梦"); book.setId(2); IndexQuery indexQuery = new IndexQueryBuilder().withId(String.valueOf(book.getId())).withObject(book).build(); elasticsearchTemplate.index(indexQuery); }
测试结果如下:
查询数据示例如下:
@Test public void testTemplate02(){ QueryStringQueryBuilder stringQueryBuilder = new QueryStringQueryBuilder("楼"); stringQueryBuilder.field("bookName"); SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(stringQueryBuilder).build(); Pagebooks = elasticsearchTemplate.queryForPage(searchQuery,Book.class); Iterator iterator = books.iterator(); while(iterator.hasNext()){ Book book = iterator.next(); System.out.println("该次获取的book:"+book); } }
测试结果如下:
开源项目: https://github.com/spring-projects/spring-data-elasticsearch
https://github.com/searchbox-io/Jest/tree/master/jest
1.说明
本文主要讲解如何使用Spring Boot快速搭建Web框架,结合Spring Data 和 Jest 快速实现对阿里云ElasticSearch的全文检索功能。
主要使用组件:
Spring Boot Starter:可以帮助我们快速的搭建spring mvc 环境
Jest:一种rest访问es的客户端
elasticsearch:全文检索
spring data elasticsearch:结合spring data
thymeleaf:web前端模版框架
jquery:js框架
bootstrap:前端样式框架
2.项目Maven配置
以下为项目Maven配置,尤其需要注意各个组件的版本,以及注释部分。
各个组件的某些版本组合下回出现各种异常,以下maven为测试可通过的一个版本。
4.0.0 org.lewis esweb 0.1 org.springframework.boot spring-boot-starter-parent 2.0.0.M7 UTF-8 UTF-8 1.8 org.springframework.boot spring-boot-starter-web io.searchbox jest 5.3.2 org.elasticsearch elasticsearch 5.3.3 org.springframework.data spring-data-elasticsearch 3.0.0.RELEASE com.github.vanroy spring-boot-starter-data-jest 3.0.0.RELEASE org.springframework.boot spring-boot-devtools true org.springframework.boot spring-boot-starter-thymeleaf net.java.dev.jna jna 4.5.1 org.webjars jquery 3.3.0 org.webjars bootstrap 4.0.0 org.webjars webjars-locator 0.30 org.springframework.boot spring-boot-maven-plugin true
创建完成后,项目目录结构如下:
3.Spring Starter配置需使用SpringBootApplication启动需禁用ElasticsearchAutoConfiguration,ElasticsearchDataAutoConfiguration,否则会有异常HighLightJestSearchResultMapper Bean留待下面解释,主要为了解决spring data不支持elasticsearch检索highlight问题,此处为该Bean的注册
@SpringBootApplication @EnableAutoConfiguration(exclude = {ElasticsearchAutoConfiguration.class, ElasticsearchDataAutoConfiguration.class}) public class App { public static void main(String[] args) throws Exception { SpringApplication.run(App.class, args); } @Bean public HighLightJestSearchResultMapper highLightJestSearchResultMapper(){ return new HighLightJestSearchResultMapper(); } }
3.Entity配置
a) 歌曲Entity如下:
通过对Class进行Document注解,实现与ElasticSearch中的Index和Type一一对应。
该类在最终与ES返回结果映射时,仅映射其中_source部分。即如下图部分(highlight另说,后面单独处理了):
@Document(indexName = "songs",type = "sample",shards = 1, replicas = 0, refreshInterval = "-1") public class Song extends HighLightEntity{ @Id private Long id; private String name; private String href; private String lyric; private String singer; private String album; public Song(Long id, String name, String href, String lyric, String singer, String album, Map> highlight) { //省略 } public Song() { } //getter setter 省略... }
b) 为了解决Spring data elasticsearch问题,此处增加一个抽象类:HighLightEntity,其他Entity需要继承该类
package org.leiws.esweb.entity; import java.io.Serializable; import java.util.List; import java.util.Map; public abstract class HighLightEntity implements Serializable{ private Map> highlight; public Map > getHighlight() { return highlight; } public void setHighlight(Map > highlight) { this.highlight = highlight; } }
4.Repository配置
package org.leiws.esweb.repository; import org.leiws.esweb.entity.Song; import org.springframework.data.elasticsearch.repository.ElasticsearchRepository; public interface SongRepository extends ElasticsearchRepository{ }
5.Service配置
a) 接口
package org.leiws.esweb.service; import org.leiws.esweb.entity.Song; import org.springframework.data.domain.Page; import java.util.List; /** * The interface Song service. */ public interface SongService { /** * Search song list. * * @param pNum the p num * @param pSize the p size * @param keywords the keywords * @return the list */ public PagesearchSong(Integer pNum, Integer pSize, String keywords); }
b) 实现类
该类实现了具体如何分页,如何查询等
package org.leiws.esweb.service.impl; import com.github.vanroy.springdata.jest.JestElasticsearchTemplate; import org.apache.log4j.Logger; import org.elasticsearch.common.lucene.search.function.FiltersFunctionScoreQuery; import org.elasticsearch.index.query.MatchPhraseQueryBuilder; import org.elasticsearch.index.query.MatchQueryBuilder; import org.elasticsearch.index.query.QueryBuilders; import org.elasticsearch.index.query.functionscore.FunctionScoreQueryBuilder; import org.elasticsearch.index.query.functionscore.ScoreFunctionBuilders; import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder; import org.leiws.esweb.entity.Song; import org.leiws.esweb.repository.HighLightJestSearchResultMapper; import org.leiws.esweb.repository.SongRepository; import org.leiws.esweb.service.SongService; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.domain.Page; import org.springframework.data.domain.PageRequest; import org.springframework.data.domain.Pageable; import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder; import org.springframework.data.elasticsearch.core.query.SearchQuery; import org.springframework.stereotype.Service; import static org.elasticsearch.index.query.QueryBuilders.functionScoreQuery; import static org.elasticsearch.index.query.QueryBuilders.matchPhraseQuery; import java.util.List; @Service public class SongServiceImpl implements SongService{ private static final Logger LOGGER = Logger.getLogger(SongServiceImpl.class); /* 分页参数 */ private final static Integer PAGE_SIZE = 12; // 每页数量 private final static Integer DEFAULT_PAGE_NUMBER = 0; // 默认当前页码 /* 搜索模式 */ private final static String SCORE_MODE_SUM = "sum"; // 权重分求和模式 private final static Float MIN_SCORE = 10.0F; // 由于无相关性的分值默认为 1 ,设置权重分最小值为 10 @Autowired SongRepository songRepository; @Autowired JestElasticsearchTemplate jestElasticsearchTemplate; @Autowired HighLightJestSearchResultMapper jestSearchResultMapper; @Override public PagesearchSong(Integer pNum, Integer pSize, String keywords) { // 校验分页参数 if (pSize == null || pSize <= 0) { pSize = PAGE_SIZE; } if (pNum == null || pNum < DEFAULT_PAGE_NUMBER) { pNum = DEFAULT_PAGE_NUMBER; } LOGGER.info("\n searchCity: searchContent [" + keywords + "] \n "); // 构建搜索查询 SearchQuery searchQuery = getCitySearchQuery(pNum,pSize,keywords); LOGGER.info("\n searchCity: searchContent [" + keywords + "] \n DSL = \n " + searchQuery.getQuery().toString()); // Page cityPage = songRepository.search(searchQuery); Page cityPage = jestElasticsearchTemplate.queryForPage(searchQuery,Song.class,jestSearchResultMapper); return cityPage; } /** * 根据搜索词构造搜索查询语句 * * 代码流程: * - 权重分查询 * - 短语匹配 * - 设置权重分最小值 * - 设置分页参数 * * @param pNum 当前页码 * @param pSize 每页大小 * @param searchContent 搜索内容 * @return */ private SearchQuery getCitySearchQuery(Integer pNum, Integer pSize,String searchContent) { /* elasticsearch 2.4.6 版本写法 FunctionScoreQueryBuilder functionScoreQueryBuilder = QueryBuilders.functionScoreQuery() .add(QueryBuilders.boolQuery().should(QueryBuilders.matchQuery("lyric", searchContent)), ScoreFunctionBuilders.weightFactorFunction(1000)) .scoreMode(SCORE_MODE_SUM).setMinScore(MIN_SCORE); */ FunctionScoreQueryBuilder.FilterFunctionBuilder[] functions = { new FunctionScoreQueryBuilder.FilterFunctionBuilder( matchPhraseQuery("lyric", searchContent), ScoreFunctionBuilders.weightFactorFunction(1000)) }; FunctionScoreQueryBuilder functionScoreQueryBuilder = functionScoreQuery(functions).scoreMode(FiltersFunctionScoreQuery.ScoreMode.SUM).setMinScore(MIN_SCORE); // 分页参数 // Pageable pageable = new PageRequest(pNum, pSize); Pageable pageable = PageRequest.of(pNum, pSize); //高亮提示 HighlightBuilder.Field highlightField = new HighlightBuilder.Field("lyric") .preTags(new String[]{"", "", ""}) .postTags(new String[]{"", "", ""}) .fragmentSize(15) .numOfFragments(5) //highlightQuery必须单独设置,否则在使用FunctionScoreQuery时,highlight配置不生效,返回结果无highlight元素 //官方解释:Highlight matches for a query other than the search query. This is especially useful if you use a rescore query because those are not taken into account by highlighting by default. .highlightQuery(matchPhraseQuery("lyric", searchContent)); return new NativeSearchQueryBuilder() .withPageable(pageable) // .withSourceFilter(new FetchSourceFilter(new String[]{"name","singer","lyric"},new String[]{})) .withHighlightFields(highlightField) .withQuery(functionScoreQueryBuilder).build(); } }
c) 解决Spring Data ElasticSearch不支持Highlight的问题
通过自定义实现一个如下的JestSearchResultMapper,解决无法Highlight的问题
package org.leiws.esweb.repository; //import 省略 public class HighLightJestSearchResultMapper extends DefaultJestResultsMapper { private EntityMapper entityMapper; private MappingContext extends ElasticsearchPersistentEntity>, ElasticsearchPersistentProperty> mappingContext; public HighLightJestSearchResultMapper() { this.entityMapper = new DefaultEntityMapper(); this.mappingContext = new SimpleElasticsearchMappingContext(); } public HighLightJestSearchResultMapper(MappingContext extends ElasticsearchPersistentEntity>, ElasticsearchPersistentProperty> mappingContext, EntityMapper entityMapper) { this.entityMapper = entityMapper; this.mappingContext = mappingContext; } public EntityMapper getEntityMapper() { return entityMapper; } public void setEntityMapper(EntityMapper entityMapper) { this.entityMapper = entityMapper; } @Override publicAggregatedPage mapResults(SearchResult response, Class clazz) { return mapResults(response, clazz, null); } @Override public AggregatedPage mapResults(SearchResult response, Class clazz, List aggregations) { LinkedList results = new LinkedList<>(); for (SearchResult.Hit hit : response.getHits(JsonObject.class)) { if (hit != null) { T result = mapSource(hit.source, clazz); HighLightEntity highLightEntity = (HighLightEntity) result; highLightEntity.setHighlight(hit.highlight); results.add((T) highLightEntity); } } String scrollId = null; if (response instanceof ExtendedSearchResult) { scrollId = ((ExtendedSearchResult) response).getScrollId(); } return new AggregatedPageImpl<>(results, response.getTotal(), response.getAggregations(), scrollId); } private T mapSource(JsonObject source, Class clazz) { String sourceString = source.toString(); T result = null; if (!StringUtils.isEmpty(sourceString)) { result = mapEntity(sourceString, clazz); setPersistentEntityId(result, source.get(JestResult.ES_METADATA_ID).getAsString(), clazz); } else { //TODO(Fields results) : Map Fields results //result = mapEntity(hit.getFields().values(), clazz); } return result; } private T mapEntity(String source, Class clazz) { if (isBlank(source)) { return null; } try { return entityMapper.mapToObject(source, clazz); } catch (IOException e) { throw new ElasticsearchException("failed to map source [ " + source + "] to class " + clazz.getSimpleName(), e); } } private void setPersistentEntityId(Object entity, String id, Class clazz) { ElasticsearchPersistentEntity> persistentEntity = mappingContext.getRequiredPersistentEntity(clazz); ElasticsearchPersistentProperty idProperty = persistentEntity.getIdProperty(); // Only deal with text because ES generated Ids are strings ! if (idProperty != null) { if (idProperty.getType().isAssignableFrom(String.class)) { persistentEntity.getPropertyAccessor(entity).setProperty(idProperty, id); } } } }
上面类的大部分代码来源于:DefaultJestResultsMapper
重点修改部分为:
@Override publicAggregatedPage mapResults(SearchResult response, Class clazz, List aggregations) { LinkedList results = new LinkedList<>(); for (SearchResult.Hit hit : response.getHits(JsonObject.class)) { if (hit != null) { T result = mapSource(hit.source, clazz); HighLightEntity highLightEntity = (HighLightEntity) result; highLightEntity.setHighlight(hit.highlight); results.add((T) highLightEntity); } } String scrollId = null; if (response instanceof ExtendedSearchResult) { scrollId = ((ExtendedSearchResult) response).getScrollId(); } return new AggregatedPageImpl<>(results, response.getTotal(), response.getAggregations(), scrollId); }
6.Controller
相对简单,如普通的Spring Controller
@Controller @RequestMapping(value = "/search") public class SearchController { @Autowired SongService songService; /** * Song list string. * * @param map the map * @return the string */ @RequestMapping(method = RequestMethod.GET) public String songList(@RequestParam(value = "pNum") Integer pNum, @RequestParam(value = "pSize", required = false) Integer pSize, @RequestParam(value = "keywords") String keywords,ModelMap map){ map.addAttribute("pageSong",songService.searchSong(pNum,pSize,keywords)); return "songList"; } }
7.前端页面thymeleaf模版
存放目录为:resources/templates/songList.html
8.阿里云ElasticSearch连接配置
在resources/application.properties中配置如下:
spring.data.jest.uri=http://1xx.xxx.xxx.xxx:8080 spring.data.jest.username=username spring.data.jest.password=password spring.data.jest.maxTotalConnection=50 spring.data.jest.defaultMaxTotalConnectionPerRoute=50 spring.data.jest.readTimeout=5000
9.其他
a) thymeleaf 热启动配置,便于测试在resources/application.properties中配置如下:
spring.thymeleaf.cache=false
在pom.xml中增加:
org.springframework.boot spring-boot-devtools true org.springframework.boot spring-boot-maven-plugin true
3.每次还是需要重新compile后,修改的thymeleaf模版代码才会自动生效,因为spring boot启动是以target目录为准的
b) 阿里云elasticsearch在esc上配置ngnix代理,以支持本机可以公网访问,便于开发购买一台esc在ecs上安装ngnix,并配置代理信息server 部分如下:
server { listen 8080; #listen [::]:80 default_server; server_name {本机内网ip} {本机外网ip}; #root /usr/share/nginx/html; # Load configuration files for the default server block. #include /etc/nginx/default.d/*.conf; location / { proxy_pass http://{elasticsearch 内网 ip}:9200; } }
10. 最后,查询效果:
总结
以上所述是小编给大家介绍的SpringBoot 整合Jest实例代码,希望对大家有所帮助,如果大家有任何疑问请给我留言,小编会及时回复大家的。在此也非常感谢大家对脚本之家网站的支持!