ELK——ElasticSearch(ES) LogStash Kibana 范围查询 经纬度
Spring Boot 搭建 ELK，这才是正确看日志的方式！
一。快速入门实战
二。核心语法集群高可用实战演练
三。集群架构原理与搜索技术深入
四。底层原理与分组聚合
五。进阶与JavaAPI整合ES
六。API整合es/
什么是restful
restful是一种面向资源的架构风格，可以简单理解为使用URL定位资源，用HTTP动词描述操作
一。什么是ELK?ELK的下载安装启动(本次使用版本7.12.0，资料的是7.6.1)
1.什么是ELK?全文搜索引擎
ElasticSearch 检索，非关系型数据库
LogStash 日志收集
Kibana 可视化，相当于Navicat
IK 分词器
2.ElasticSearch和Lucene、solr的关系？
Lucene可以被认为是迄今为止最先进、性能最好、功能最全的搜索引擎库（框架）
但是想要使用Lucene，必须使用Java来作为开发语言并将其直接集成到你的应用中，并且Lucene的配置及使用非常复杂，你需要深入了解检索的相关知识来理解它是如何工作的
Lucene缺点：
只能在Java项目中使用，并且要以jar包的方式直接集成项目中
使用非常复杂-创建索引和搜索索引代码复杂
不支持集群环境-索引数据不同步（不支持大型项目）
索引数据如果太多就不行，索引库和应用所在同一个服务器，共同占用硬盘，共用空间少
Solr：
Solr利用Zookeeper进行分布式管理，而ES自身带有分布式协调管理功能
Solr支持更多格式的数据，比如JSON、XML、CSV，而ES仅支持json文件格式
Solr在传统的搜索应用中表现好于ES，但在处理实时搜索应用时效率明显低于ES
Solr是传统搜索应用的有力解决方案，但ES更适用于新兴的实时搜索应用
3.下载安装和启动ELK?
a.es不能使用root启动，必须使用普通用户来安装启动
创建用户组 groupadd elasticsearch
创建用户并设置密码 piaoransheng Huan421946786
用户添加到用户组 usermod -g elasticsearch piaoransheng
给用户授权es所在文件夹的权限， chown -R piaoransheng /usr/local/es
b.下载地址：https://www.elastic.co/cn/downloads， elasticsearch-7.6.1-linux-x86_64.tar.gz kibana-7.6.1-linux-x86_64.tar.gz
百度网盘有windows版本7.12.0
c.启动测试（直接解压tar.gz）
1.启动ES
cd elastibsearch/bin 输入./elasticsearch windows双击elasticsearch.bat
测试 curl http://localhost:9200，有返回内容就表示成功
2.启动kibana
先修改config里面的yml文件，
a.改server.host为服务器ip地址和server.port，
b.elasticsearch.host为http://服务器ip:9200
然后启动 cd /bin, ./kibana， 测试浏览器输入http://localhost:5601 windows双击kibana.bat
d.配置本地浏览器可以访问
1.更改配置文件config/ElasticSearch.yml
a.network.host取消注释，并且后面的ip地址改为服务器的地址
b.node.name取消注释
c.cluster.initial-master-nodes取消注释，并且数组里面只保留一个
2.重新运行 cd elasticsearch/bin,输入./elasticseach
3.浏览器重新访问es http://192.168.75.128:9200
二。分词器
1.ik分词器下载与安装
下载地址：https://github.com/medcl/elasticsearch-analysis-ik/releases
安装方式：下载的压缩包解压至es所在目录的/plugins/ik文件夹，然后重启es
2.测试是否安装成功
//ik分词器
POST _analyze
{
"analyzer": "ik_max_word", //ik_smart:最粗粒度 ik_max_word:最细粒度拆分
"text": "中华人民共和国"
}
//默认自带分词器
POST _analyze
{
"analyzer": "standard",
"text": "我爱你中国"
}
3.使用分词器
创建索引库的时候：
PUT /es_db02
{
"mappings": {
"properties": {
"id": {
"type": "long"
},
"subTitle": {
"type": "text"，
"analyzer":"ik_max_word"
}
}
}
}
三。基本语句使用
#查看索引结构
GET /product_db
#查看索引库全部数据
GET /product_db/_doc/_search #如果有指定类型，就是GET /product_db/product/_search
#删除索引库
DELETE /索引库名
#将旧的索引库数据复制到新的索引库
POST _reindex
{
"source": {
"index": "pile_base_op_location_index"
},
"dest": {
"index": "pile_base_op_location_index_new"
}
}
#删除文档
DELETE /pile_base_op_location_pile_evse_index/pile_base_op_location_pile_evse/159
#条件查询
GET /product_db/_doc/_search?q=id:26
POST /product_db/_doc/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"id": 26
}
}
]
}
}
}
模糊查询
POST /pile_base_op_location_pile_evse_index/_search
{
"query": {
"bool": {
"must": [
{
"wildcard": {
"pileSn": "*AE0022A1GM8CK00*"
}
}
]
}
}
}
in查询
#根据主键id批量查询 id字段后面加个s
GET /product_db/_doc/_mget?
{
"ids":["1","2","3"]
}
根据非主键id查询
POST /pile_base_op_location_pile_evse_index/_search
{
"query": {
"terms": {
"locationId": [
56,
133
]
}
}
}
索引 类型 文档 文档字段 映射
库 表 记录 列
1.索引
创建索引： PUT /es_db01 {结构映射，见下面}
查询索引结构 GET /es_db01 查看所有索引 http://localhost:9200/_cat/indices?v
删除索引 DELETE /es_db01
a、创建索引库时指定ik分词器，默认使用"standard",
PUT /es_db02
{
"settings": {
"index":{
"analysis.analyzer.default.type":"ik_max_word"
}
}
}
b、创建索引让查询时忽略大小写
@Data
@Document(indexName = "charge_detail_record_bill", type = "_doc", shards = 3)
@TypeAlias("charge_detail_record_bill")
@Setting(settingPath = "elastic/setting/ChargeDetailRecordBill.json")
public class ChargeDetailRecordBill {}
下面是json文件
{
"index": {
"max_ngram_diff": "30",
"refresh_interval": "1s",
"number_of_shards": "3",
"store": {
"type": "fs"
},
"analysis": {
"normalizer": {
"lowercase": {
"filter": [
"lowercase"
],
"type": "custom"
}
},
"analyzer": {
"index_analyzer": {
"filter": [
"lowercase"
],
"tokenizer": "my_tokenizer"
},
"search_analyzer": {
"filter": [
"lowercase"
],
"tokenizer": "whitespace"
}
},
"tokenizer": {
"my_tokenizer": {
"token_chars": [
"letter",
"digit",
"punctuation",
"symbol"
],
"min_gram": "1",
"type": "ngram",
"max_gram": "30"
}
}
},
"number_of_replicas": "1"
}
}
c、 创建索引映射 type有：long、text、date、keyword、boolean、nested（嵌套，对象的属性也是对象）
PUT /es_db02
{
"mappings": {
"properties": {
"id": {
"type": "long"
},
"subTitle": {
"type": "text"，
"analyzer":"ik_max_word"
},
"attrs": {
"type": "nested", //nested表示属性是对象类型
"properties": {
"attrId": {
"type": "long"
},
"attrName": {
"type": "keyword"
},
"attrValue": {
"type": "keyword"
}
}}}}}
2.文档(相当于行记录)
查询所有文档 GET /es_db01/_doc/_search hits里面的hits是个数组，存放多个文档
查询指定文档 GET /es_db01/_doc/1
删除文档 DELETE /es_db01/_doc/1
添加和修改文档 PUT(POST)/es_db01/_doc/1 索引/类型/id
{
"name":"张三",
"sex":1,
"age":25
}
put和post的异同点：
相同点：put和post都能起到创建更新的作用
不同点：
put要指定id，post可写可不写，如果不写id就由ES生成一个唯一id
put将所有数据都替换，post只会更新相同字段的值
条件查询文档，如查询age等于28岁 GET /es_db01/_doc/_search?q=age:28
查询年龄小于28的 GET /es_db01/_doc/_search?q=age:<28
范围查询文档，如查询age在25至26之间 GET /es_db01/_doc/_search?q=age[25 TO 28] to必须为大写
分页查询from=*&size=* GET /es_db01/_doc/_search?q=age[25 TO 28] &from=0&size=1
排序sort=字段:desc GET /es_db01/_doc/_search?q=age[25 TO 28]&sort=age:asc
只输出某些字段_source=字段,字段 GET /es_db01/_doc/_search?q=age[25 TO 28]&sort=age:asc&_source=name,age
多个id批量查询 GET /es_db01/_doc/_mget?
{
"ids":["1","2","3"]
}
批量创建文档：见下面截图
批量查询文档：见下面截图
批量删除文档:见下面截图
批量修改文档：见下面截图
3.映射：
获取映射：get enjoy-test/-mapping
四。DSL语言高级查询
1.DSL概述：领域专用语言，Domain Specific Language，DSL由叶子查询子句和复合查询子句两种子句组成
2.主要五个部分：查询 排序 分页 高亮 分组统计
query sort from&size， highlight， aggs nsted
SearchRequest
SearchSourceBuilder
BoolQueryBuilder
query下组合条件有四层： query——bool——must——match （简化版直接query——match）
must：所有条件都必须满足，即and
must_not：所有条件都不满足
should：有一个条件满足即可，即or
filter：不计算相关度评分： 功能和must类型
term：精确匹配'
terms in查询
match
wildcard:模糊匹配,前后加*
range：某个范围
exists:某个字段的值是否存在
ids：通过id批量查询
1.无条件、精确、模糊查询, GET和POST都可以
GET(POST) es_db01/_doc/_search
{
"query": {
"match_all": {}
}
}
GET /es_db01/_doc/_search
{
"query":{
"term":{
"name":"lhc"
}
}
}
POST /es_db01/_doc/_search
{
"query":{
"match":{
"name":"张三"
}
}
}
2.组合条件查询
POST product_db/_doc/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"name": "小米 11 手机"
}
}
]
}
}
}
3.分页 还有个深分页
POST /product_db/_doc/_search
{
"from":0,
"size":8,
"query":{
},
}
4.排序 sort
POST /product_db/_doc/_search
{
"query":{
},
"sort": [
{
"sale": { //需要属性值为整型的属性名称，不然报错。分组聚合也是
"order": "desc"
}
},
{
"salecount": {
"order": "desc"
}
}
]
}
5.高亮 结果显示在hits每一个对象的highlight属性里面
POST /product_db/_doc/_search
{
"query":{
},
"highlight": {
"pre_tags": [
""
],
"post_tags": [
""
],
"fields": [
{
"subTitle": {} ##这个subTitle要存在于query里面的match里面，结果才会出现高亮信息
}
]
}
}
6.聚合搜索(分组统计)，信息在外层hits的下面
bucket: 就是数据分组；如一个公司有A部门，B部门，那么根据部门分组就是两个bucket（A bucket和 B bucket）
metric: 是对分组bucekt的统计分析，如A bucket有多少人，最大值，最小值，平均值等
POST product_db/_doc/_search
{
"query": {
},
"aggs": { #//用aggregations也可以
"group_by_brandId": {
"terms": {
"field": "brandId", #//需要属性值为整型的属性名，排序也是
"size": 3,
"order":{
"doc_count":"asc"
}
}
}
}
}
#分组统计下再分组统计, 在terms的后面再加一个aggs,aggs里面可以有多个
POST product_db/_doc/_search
{
"query": {
},
"aggs": {
"group_by_brandId": {
"terms": {
"field": "brandId"
},
"aggs": {
"group_by_sale": {
"terms": {
"field": "sale"
}
},
"group_by_salecount": {
"terms": {
"field": "salecount"
}
}
}
}
}
}
7、查询字段为空字符串或者null的数据
boolQueryBuilder2.should(QueryBuilders.termQuery("state",""));
boolQueryBuilder2.should(QueryBuilders.boolQuery().mustNot(QueryBuilders.existsQuery("state")));
8.nested
五。Java API
1.项目依赖和主要对象
<!--es-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
<!--es另外一种-->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.12.0</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>7.12.0</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.12.0</version>
</dependency>
2.建立连接
private RestHighLevelClient restHighLevelClient;
public TestController() {
RestClientBuilder restClientBuilder = RestClient.builder(
new HttpHost("localhost", 9200, "http"),
new HttpHost("192.168.61.143", 9200, "http")
);
restHighLevelClient = new RestHighLevelClient(restClientBuilder);
}
或者
@Autowired
private RestHighLevelClient restHighLevelClient;
3.增删改查的参数对象
IndexRequest（增）
DeleteRequest（删）
UpdateRequest（改）
SearchRequest（查）
GetRequest（根据id查询）
/**
* 添加文档(行记录)
*/
@PostMapping("/add")
public void add(@RequestBody UserEntity userEntity) throws IOException {
//1.构建IndexRequest对象
IndexRequest indexRequest = new IndexRequest(user_db_name);
//2.设置文档id
indexRequest.id("3");
//3。设置文档数据，并设置请求的数据为json格式
String json1 = JSON.toJSONString(userEntity);
String json2 = JSONObject.toJSONString(userEntity);
System.out.println(json1);
System.out.println(json2);
indexRequest.source(json2, XContentType.JSON);
//4.发起请求添加数据到user索引库
restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
}
/**
* 查询所有文档（查询记录）
*/
@GetMapping("/search")
public void search(@RequestParam String dbName) throws IOException {
//1.构建IndexRequest对象
SearchRequest searchRequest = new SearchRequest(dbName);
//2.发送请求
SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
//3.解析返回对象
SearchHits hit = searchResponse.getInternalResponse().hits();
System.out.println(hit.toString());
}
SearchRequest： //声明索引库
SearchRequest searchRequest = new SearchRequest(new String[]{"product_db"}, searchSourceBuilder);
SearchSourceBuilder ：
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(boolQueryBuilder);
searchSourceBuilder.sort(sorts[0], SortOrder.ASC);
searchSourceBuilder.from(0).size(10);
searchSourceBuilder.highlighter(highlightBuilder);
searchSourceBuilder.aggregation(brandAgg);
a.查询query
BoolQueryBuilder
BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
boolQueryBuilder.filter(QueryBuilders.termQuery("hasStock", requestParamDTO.getHasStock()));
RangeQueryBuilder（范围）
RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("price");
rangeQueryBuilder.lte(prices[0]);
boolQueryBuilder.filter(rangeQueryBuilder);NestedQueryBuilder
NestedQueryBuilder
BoolQueryBuilder nestedBoolQueryBuilder = new BoolQueryBuilder();
NestedQueryBuilder nestedQueryBuilder = QueryBuilders.nestedQuery("attrs",nestedBoolQueryBuilder, ScoreMode.Avg);
boolQueryBuilder.filter(nestedQueryBuilder);
b.排序sort
searchSourceBuilder.sort(sorts[0], SortOrder.ASC);
c.分页 from/size
searchSourceBuilder.from((requestParamDTO.getPageNum()-1) * 10);
searchSourceBuilder.size(10);
d.高亮
if (!StringUtils.isEmpty(requestParamDTO.getKeyword())) {
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.field("name");
highlightBuilder.preTags("");
highlightBuilder.postTags("");
searchSourceBuilder.highlighter(highlightBuilder);
}
e.聚合 根据品牌、属性口模型看i就
//根据品牌聚合
TermsAggregationBuilder brandAgg = AggregationBuilders.terms("brand_agg");
brandAgg.field("brandId").size(10);
//品牌下再根据品牌名、品牌图片再次聚合
brandAgg.subAggregation(AggregationBuilders.terms("brand_name_agg")).field("brand_name").size(1);
brandAgg.subAggregation(AggregationBuilders.terms("brand_img_agg")).field("brandImg").size(1);
searchSourceBuilder.aggregation(brandAgg);
//根据属性信息进行聚合
NestedAggregationBuilder attrAgg = AggregationBuilders.nested("attr_agg", "attrs");
//根据属性id进行聚合
TermsAggregationBuilder attr_id_agg = AggregationBuilders.terms("arrt_id_agg").field("attrs.attrId");
attrAgg.subAggregation(attr_id_agg);
//属性id下再根据属性名称、属性值进行聚合
TermsAggregationBuilder attr_name_agg = AggregationBuilders.terms("attr_name_agg").field("attrs.attrName").size(1);
attr_id_agg.subAggregation(attr_name_agg);
TermsAggregationBuilder attr_value_agg = AggregationBuilders.terms("attr_value_agg").field("attrs.attrValue").size(50);
attr_id_agg.subAggregation(attr_value_agg);
searchSourceBuilder.aggregation(attrAgg);
2.案例 三个方法（test1 buildSearchRequest buildBoolQueryBuilder ）
private RestHighLevelClient restHighLevelClient;
//索引库名字
private static final String db_name = "product_db";
@PostMapping("/searchES")
public void test1(@RequestBody ESRequestParamDTO requestParamDTO) throws IOException {
//1.封装检索对象
SearchRequest searchRequest = buildSearchRequest(requestParamDTO);
//2.发送请求
SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
//3.封装结果集
System.out.println(searchResponse);
}
/**
* 构建es检索对象
*
* @return 前端请求参数
*/
private SearchRequest buildSearchRequest(ESRequestParamDTO requestParamDTO) {
//封装查询的DSL相关信息 1.query、2.排序、3.分页、4.高亮
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//一。query 关键字 品牌id 分类id 价格 是否有库存
BoolQueryBuilder boolQueryBuilder = this.buildBoolQueryBuilder(requestParamDTO);
searchSourceBuilder.query(boolQueryBuilder);
//二。排序 price_desc
if (!StringUtils.isEmpty(requestParamDTO.getSort())) {
String[] sorts = requestParamDTO.getSort().split("_");
SortOrder sortOrder = "asc".equalsIgnoreCase(sorts[1]) ? SortOrder.ASC : SortOrder.DESC;
searchSourceBuilder.sort(sorts[0], sortOrder);
}
//三。分页
searchSourceBuilder.from((requestParamDTO.getPageNum() - 1) * 10);
searchSourceBuilder.size(10);
//四。高亮
if (!StringUtils.isEmpty(requestParamDTO.getKeyword())) {
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.field("name");
highlightBuilder.preTags("");
highlightBuilder.postTags("");
searchSourceBuilder.highlighter(highlightBuilder);
}
//五。聚合
//根据品牌聚合
TermsAggregationBuilder brandAgg = AggregationBuilders.terms("brand_agg");
brandAgg.field("brandId").size(10);
//品牌下再根据品牌名、品牌图片再次聚合
brandAgg.subAggregation(AggregationBuilders.terms("brand_name_agg")).field("brand_name").size(1);
brandAgg.subAggregation(AggregationBuilders.terms("brand_img_agg")).field("brandImg").size(1);
searchSourceBuilder.aggregation(brandAgg);
//根据属性信息进行聚合
NestedAggregationBuilder attrAgg = AggregationBuilders.nested("attr_agg", "attrs");
//根据属性id进行聚合
TermsAggregationBuilder attr_id_agg = AggregationBuilders.terms("arrt_id_agg").field("attrs.attrId");
attrAgg.subAggregation(attr_id_agg);
//属性id下再根据属性名称、属性值进行聚合
TermsAggregationBuilder attr_name_agg = AggregationBuilders.terms("attr_name_agg").field("attrs.attrName").size(1);
attr_id_agg.subAggregation(attr_name_agg);
TermsAggregationBuilder attr_value_agg = AggregationBuilders.terms("attr_value_agg").field("attrs.attrValue").size(50);
attr_id_agg.subAggregation(attr_value_agg);
searchSourceBuilder.aggregation(attrAgg);
System.out.println("构建的DSL语句 {}:" + searchSourceBuilder.toString());
return new SearchRequest(new String[]{"product_db"}, searchSourceBuilder);
}
/**
* 构建BoolQueryBuilder
*
* @param requestParamDTO 请求对象
* @return BoolQueryBuilder
*/
private BoolQueryBuilder buildBoolQueryBuilder(ESRequestParamDTO requestParamDTO) {
//一。query 关键字 品牌id 分类id 价格 是否有库存
BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
//1.根据关键字keyword进行查询 query-bool-must
if (!StringUtils.isEmpty(requestParamDTO.getKeyword())) {
//单字段查询
boolQueryBuilder.must(QueryBuilders.matchQuery("name", requestParamDTO.getKeyword()));
//多字段查询
//boolQueryBuilder.must(QueryBuilders.multiMatchQuery(requestParamDTO.getKeyword(), "name", "keywords", "subTitle"));
}
//2.根据品牌id、类目id等进行过滤 query-bool-filter
if (!StringUtils.isEmpty(requestParamDTO.getBrandId())) {
boolQueryBuilder.filter(QueryBuilders.termQuery("brandId", requestParamDTO.getBrandId()));
}
if (!StringUtils.isEmpty(requestParamDTO.getCategoryId())) {
boolQueryBuilder.filter(QueryBuilders.termQuery("categoryId", requestParamDTO.getCategoryId()));
}
//3.判断是否有库存
if (null != requestParamDTO.getHasStock()) {
boolQueryBuilder.filter(QueryBuilders.termQuery("hasStock", requestParamDTO.getHasStock()));
}
//4.价格信息
if (!StringUtils.isEmpty(requestParamDTO.getPrice())) {
RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("price");
String[] prices = requestParamDTO.getPrice().split("_");
if (prices.length == 2) {
rangeQueryBuilder.gte(prices[0]).lte(prices[1]);
} else if (prices.length == 1) {
if (requestParamDTO.getPrice().startsWith("_")) {
rangeQueryBuilder.lte(prices[0]);
}
if (requestParamDTO.getPrice().endsWith("_")) {
rangeQueryBuilder.gte(prices[0]);
}
}
boolQueryBuilder.filter(rangeQueryBuilder);
}
//5.nested
// BoolQueryBuilder nestedBoolQueryBuilder = new BoolQueryBuilder();
// NestedQueryBuilder nestedQueryBuilder = QueryBuilders.nestedQuery("attrs", nestedBoolQueryBuilder, ScoreMode.Avg);
// boolQueryBuilder.filter(nestedQueryBuilder);
return boolQueryBuilder;
}
聚合查询
public Page<PilePageVO> stationPilePage2(PilePageDTO pilePageDTO) {
String indexName = "pile_base_op_location_evse_index";
String aggName = "aggName";
String field = "locationId";
//SearchSourceBuilder
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//检索条件
BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
searchSourceBuilder.query(boolQueryBuilder);
//去重聚合
CardinalityAggregationBuilder cardinality = AggregationBuilders.cardinality(field).field(field);//去重字段
//聚合Builder
TermsAggregationBuilder agg = AggregationBuilders
.terms(aggName)
.field(field)
.subAggregation(cardinality)
.order(BucketOrder.aggregation(field, true))
.size(pilePageDTO.getPageSize());
searchSourceBuilder.aggregation(agg);
//查询
try {
//查询请求对象
SearchRequest searchRequest = new SearchRequest(new String[]{indexName}, searchSourceBuilder);
SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
//获取聚合结果
Aggregations aggregations = searchResponse.getAggregations();
Terms terms = aggregations.get(aggName);
List<? extends Terms.Bucket> buckets = terms.getBuckets();
List<Object> locationIdList = buckets.stream().map(Terms.Bucket::getKey).collect(Collectors.toList());
log.info("聚合查询站点id集合：{}", JSONArray.toJSONString(locationIdList));
//获取常规结果
SearchHits searchHits = searchResponse.getHits();
SearchHit[] hitArray = searchHits.getHits();
List<OpLocationEvseElasticDTO> opLocationElasticDTOList = Lists.newArrayList();
for (SearchHit searchHit : hitArray) {
String sourceAsString = searchHit.getSourceAsString();
opLocationElasticDTOList.add(JSON.parseObject(sourceAsString, OpLocationEvseElasticDTO.class));
}
log.info("聚合查询常规结果枪数据：{} ", opLocationElasticDTOList.size());
if (CollectionUtils.isNotEmpty(opLocationElasticDTOList)) {
return null;
}
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
六。扩展
1.模板（相当于存储过程）
2.搜索建议，或者叫建议搜索（相当于百度搜索你好，下面出现好几个语句）
3.地图搜索（经纬度）
附录 索引库字段修改 数据不变
不能直接修改
## 1、创建到新索引
PUT /pile_base_op_location_index_new
{
"settings": {
"analysis": {
"normalizer": {
"lowercase": {
"type": "custom",
"filter": [
"lowercase"
]
}
}
}
},
"mappings": {
"properties": {
"address": {
"type": "keyword"
},
"announcement": {
"type": "keyword"
},
"announcementAt": {
"type": "long"
},
"appShow": {
"type": "boolean"
},
"billingRule": {
"type": "keyword"
},
"chargingWhenClosed": {
"type": "integer"
},
"city": {
"type": "keyword"
},
"country": {
"type": "keyword"
},
"createdAt": {
"type": "long"
},
"groupId": {
"type": "long"
},
"id": {
"type": "long"
},
"latitude": {
"type": "keyword"
},
"location": {
"type": "geo_point"
},
"longitude": {
"type": "keyword"
},
"name": {
"type": "keyword",
"normalizer": "lowercase"
},
"openType": {
"type": "integer"
},
"operationDate": {
"type": "long"
},
"operationType": {
"type": "keyword"
},
"operatorId": {
"type": "long"
},
"ownerId": {
"type": "long"
},
"postalCode": {
"type": "keyword"
},
"serviceTel": {
"type": "keyword"
},
"state": {
"type": "integer"
},
"status": {
"type": "integer"
},
"subOperatorId": {
"type": "long"
},
"timeZone": {
"type": "keyword"
},
"transPower": {
"type": "double"
},
"type": {
"type": "keyword"
},
"updatedAt": {
"type": "long"
}
}
}
}
## 2、将旧的索引库数据复制到新的索引库
POST _reindex
{
"source": {
"index": "pile_base_op_location_index"
},
"dest": {
"index": "pile_base_op_location_index_new"
}
}
## 3、删除旧索引
DELETE /pile_base_op_location_index
## 4、创建旧索引
PUT /pile_base_op_location_index
{
"settings": {
"analysis": {
"normalizer": {
"lowercase": {
"type": "custom",
"filter": [
"lowercase"
]
}
}
}
},
"mappings": {
"properties": {
"address": {
"type": "keyword"
},
"announcement": {
"type": "keyword"
},
"announcementAt": {
"type": "long"
},
"appShow": {
"type": "boolean"
},
"billingRule": {
"type": "keyword"
},
"chargingWhenClosed": {
"type": "integer"
},
"city": {
"type": "keyword"
},
"country": {
"type": "keyword"
},
"createdAt": {
"type": "long"
},
"groupId": {
"type": "long"
},
"id": {
"type": "long"
},
"latitude": {
"type": "keyword"
},
"location": {
"type": "geo_point"
},
"longitude": {
"type": "keyword"
},
"name": {
"type": "keyword",
"normalizer": "lowercase"
},
"openType": {
"type": "integer"
},
"operationDate": {
"type": "long"
},
"operationType": {
"type": "keyword"
},
"operatorId": {
"type": "long"
},
"ownerId": {
"type": "long"
},
"postalCode": {
"type": "keyword"
},
"serviceTel": {
"type": "keyword"
},
"state": {
"type": "integer"
},
"status": {
"type": "integer"
},
"subOperatorId": {
"type": "long"
},
"timeZone": {
"type": "keyword"
},
"transPower": {
"type": "double"
},
"type": {
"type": "keyword"
},
"updatedAt": {
"type": "long"
}
}
}
}
##5、将新的索引库数据复制到旧的索引库
POST _reindex
{
"source": {
"index": "pile_base_op_location_index_new"
},
"dest": {
"index": "pile_base_op_location_index"
}
}
附录商品搜索实战ES库
#4.实战
#4.1手机索引库
PUT product_db
{
"mappings":{
"properties":{
"id":{
"type":"long"
},
"name":{
"type":"text",
"analyzer":"ik_max_word"
},
"keywords":{
"type":"text",
"analyzer":"ik_max_word"
},
"subTitle":{
"type":"text",
"analyzer":"ik_max_word"
},
"salecount":{
"type":"long"
},
"putawayDate":{
"type":"date"
},
"price":{
"type":"double"
},
"promotionPrice":{
"type":"keyword"
},
"originalPrice":{
"type":"keyword"
},
"pic":{
"type":"keyword"
},
"sale":{
"type":"long"
},
"hasStock":{
"type":"boolean"
},
"brandId":{
"type":"long"
},
"brandName":{
"type":"keyword"
},
"brandImg":{
"type":"keyword"
},
"categoryId":{
"type":"long"
},
"categoryName":{
"type":"keyword"
},
"attrs":{
"type":"nested",
"properties":{
"attrId":{
"type":"long"
},
"attrName":{
"type":"keyword"
},
"attrValue":{
"type":"keyword"
}
}
}
}
}
}
#4.2手机数据
PUT /product_db/_doc/1
{
"id":"26",
"name":"小米 11 手机",
"keywords":"小米手机",
"subTitle":"AI智慧全面屏 6GB +64GB 亮黑色 全网通版 移动联通电信4G手机 双卡双待 双 卡双待",
"price":"3999",
"promotionPrice":"2999",
"originalPrice":"5999",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/xi aomi.jpg",
"sale":999,
"hasStock":true,
"salecount":999,
"putawayDate":"2021‐04‐01",
"brandId":6,
"brandName":"小米",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/1e34aef2a409119018a4c6258e39ecfb_222_222.png",
"categoryId":19,
"categoryName":"手机通讯",
"attrs":[
{
"attrId":1,
"attrName":"cpu",
"attrValue":"2核"
},
{
"attrId":2,
"attrName":"颜色",
"attrValue":"黑色"
}
]
}
PUT /product_db/_doc/2
{
"id":"27",
"name":"小米 10 手机",
"keywords":"小米手机",
"subTitle":"AI智慧全面屏 4GB +64GB 亮白色 全网通版 移动联通电信4G手机 双卡双待 双 卡双待",
"price":"2999",
"promotionPrice":"1999",
"originalPrice":"3999",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/xi aomi.jpg",
"sale":999,
"hasStock":false,
"salecount":99,
"putawayDate":"2021‐04‐02",
"brandId":6,
"brandName":"小米",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/1e34aef2a409119018a4c6258e39ecfb_222_222.png",
"categoryId":19,
"categoryName":"手机通讯",
"attrs":[
{
"attrId":1,
"attrName":"cpu",
"attrValue":"4核"
},
{
"attrId":2,
"attrName":"颜色",
"attrValue":"白色"
}
]
}
PUT /product_db/_doc/3
{
"id":"28",
"name":"小米 手机",
"keywords":"小米手机",
"subTitle":"AI智慧全面屏 4GB +64GB 亮蓝色 全网通版 移动联通电信4G手机 双卡双待 双 卡双待",
"price":"2999",
"promotionPrice":"1999",
"originalPrice":"3999",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/xi aomi.jpg",
"sale":999,
"hasStock":true,
"salecount":199,
"putawayDate":"2021‐04‐03",
"brandId":6,
"brandName":"小米",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/1e34aef2a409119018a4c6258e39ecfb_222_222.png",
"categoryId":19,
"categoryName":"手机通讯",
"attrs":[
{
"attrId":1,
"attrName":"cpu",
"attrValue":"2核"
},
{
"attrId":2,
"attrName":"颜色",
"attrValue":"蓝色"
}
]
}
PUT /product_db/_doc/4
{
"id":"29",
"name":"Apple iPhone 8 Plus 64GB 金色特别版 移动联通电信4G手机",
"keywords":"苹果手机",
"subTitle":"苹果手机 Apple产品年中狂欢节，好物尽享，美在智慧！速来 >> 勾选[保障服 务][原厂保2年]，获得AppleCare+全方位服务计划，原厂延保售后无忧。",
"price":"5999",
"promotionPrice":"4999",
"originalPrice":"7999",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/5a cc5248N6a5f81cd.jpg",
"sale":99,
"hasStock":true,
"salecount":1199,
"putawayDate":"2021‐04‐04",
"brandId":51,
"brandName":"苹果",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201806 07/timg.jpg",
"categoryId":19,
"categoryName":"手机通讯",
"attrs":[
{
"attrId":1,
"attrName":"cpu",
"attrValue":"4核"
},
{
"attrId":2,
"attrName":"颜色",
"attrValue":"金色"
}
]
}
PUT /product_db/_doc/5
{
"id":"30",
"name":"HLA海澜之家简约动物印花短袖T恤",
"keywords":"海澜之家衣服",
"subTitle":"HLA海澜之家短袖T恤",
"price":"199",
"promotionPrice":"99",
"originalPrice":"299",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/5a d83a4fN6ff67ecd.jpg!cc_350x449.jpg",
"sale":999,
"hasStock":true,
"salecount":19,
"putawayDate":"2021‐04‐05",
"brandId":50,
"brandName":"海澜之家",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/99d3279f1029d32b929343b09d3c72de_222_222.jpg",
"categoryId":8,
"categoryName":"T恤",
"attrs":[
{
"attrId":3,
"attrName":"尺寸",
"attrValue":"M"
},
{
"attrId":4,
"attrName":"颜色",
"attrValue":"黑色"
}
]
}
PUT /product_db/_doc/6
{
"id":"31",
"name":"HLA海澜之家蓝灰花纹圆领针织布短袖T恤",
"keywords":"海澜之家衣服",
"subTitle":"HLA海澜之家短袖T恤",
"price":"299",
"promotionPrice":"199",
"originalPrice":"299",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/5a c98b64N70acd82f.jpg!cc_350x449.jpg",
"sale":999,
"hasStock":true,
"salecount":399,
"putawayDate":"2021‐04‐06",
"brandId":50,
"brandName":"海澜之家",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/99d3279f1029d32b929343b09d3c72de_222_222.jpg",
"categoryId":8,
"categoryName":"T恤",
"attrs":[
{
"attrId":3,
"attrName":"尺寸",
"attrValue":"X"
},
{
"attrId":4,
"attrName":"颜色",
"attrValue":"蓝灰"
}
]
}
PUT /product_db/_doc/7
{
"id":"32",
"name":"HLA海澜之家短袖T恤男基础款",
"keywords":"海澜之家衣服",
"subTitle":"HLA海澜之家短袖T恤",
"price":"269",
"promotionPrice":"169",
"originalPrice":"399",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/5a 51eb88Na4797877.jpg",
"sale":999,
"hasStock":true,
"salecount":399,
"putawayDate":"2021‐04‐07",
"brandId":50,
"brandName":"海澜之家",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/99d3279f1029d32b929343b09d3c72de_222_222.jpg",
"categoryId":8,
"categoryName":"T恤",
"attrs":[
{
"attrId":3,
"attrName":"尺寸",
"attrValue":"L"
},
{
"attrId":4,
"attrName":"颜色",
"attrValue":"蓝色"
}
]
}
PUT /product_db/_doc/8
{
"id":"33",
"name":"小米（MI）小米电视4A ",
"keywords":"小米电视机家用电器",
"subTitle":"小米（MI）小米电视4A 55英寸 L55M5‐AZ/L55M5‐AD 2GB+8GB HDR 4K超高清 人工智能网络液晶平板电视",
"price":"2269",
"promotionPrice":"2169",
"originalPrice":"2399",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/5b 02804dN66004d73.jpg",
"sale":999,
"hasStock":true,
"salecount":132,
"putawayDate":"2021‐04‐09",
"brandId":6,
"brandName":"小米",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/1e34aef2a409119018a4c6258e39ecfb_222_222.png",
"categoryId":35,
"categoryName":"手机数码",
"attrs":[
{
"attrId":5,
"attrName":"屏幕尺寸",
"attrValue":"52"
},
{
"attrId":6,
"attrName":"机身颜色",
"attrValue":"黑色"
}
]
}
PUT /product_db/_doc/9
{
"id":"34",
"name":"小米（MI）小米电视4A 65英寸",
"keywords":"小米电视机家用电器",
"subTitle":"小米（MI）小米电视4A 65英寸 L55M5‐AZ/L55M5‐AD 2GB+8GB HDR 4K超高清 人工智能网络液晶平板电视",
"price":"3269",
"promotionPrice":"3169",
"originalPrice":"3399",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/5b 028530N51eee7d4.jpg",
"sale":999,
"hasStock":true,
"salecount":999,
"putawayDate":"2021‐04‐10",
"brandId":6,
"brandName":"小米",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/1e34aef2a409119018a4c6258e39ecfb_222_222.png",
"categoryId":35,
"categoryName":"手机数码",
"attrs":[
{
"attrId":5,
"attrName":"屏幕尺寸",
"attrValue":"65"
},
{
"attrId":6,
"attrName":"机身颜色",
"attrValue":"金色"
}
]
}
PUT /product_db/_doc/10
{
"id":"35",
"name":"耐克NIKE 男子 休闲鞋 ROSHE RUN 运动鞋 511881‐010黑色41码",
"keywords":"耐克运动鞋 鞋子",
"subTitle":"耐克NIKE 男子 休闲鞋 ROSHE RUN 运动鞋 511881‐010黑色41码",
"price":"569",
"promotionPrice":"369",
"originalPrice":"899",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/5b 235bb9Nf606460b.jpg",
"sale":999,
"hasStock":true,
"salecount":399,
"putawayDate":"2021‐04‐11",
"brandId":58,
"brandName":"NIKE",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201806 15/timg (51).jpg",
"categoryId":29,
"categoryName":"男鞋",
"attrs":[
{
"attrId":7,
"attrName":"尺码",
"attrValue":"42"
},
{
"attrId":8,
"attrName":"颜色",
"attrValue":"黑色"
}
]
}
PUT /product_db/_doc/11
{
"id":"36",
"name":"耐克NIKE 男子 气垫 休闲鞋 AIR MAX 90 ESSENTIAL 运动鞋 AJ1285‐101白色41 码",
"keywords":"耐克运动鞋 鞋子",
"subTitle":"AIR MAX 90 ESSENTIAL 运动鞋 AJ1285‐101白色",
"price":"769",
"promotionPrice":"469",
"originalPrice":"999",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/5b 19403eN9f0b3cb8.jpg",
"sale":999,
"hasStock":true,
"salecount":499,
"putawayDate":"2021‐04‐13",
"brandId":58,
"brandName":"NIKE",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201806 15/timg (51).jpg",
"categoryId":29,
"categoryName":"男鞋",
"attrs":[
{
"attrId":7,
"attrName":"尺码",
"attrValue":"44"
},
{
"attrId":8,
"attrName":"颜色",
"attrValue":"白色"
}
]
}
PUT /product_db/_doc/12
{
"id":"37",
"name":"(华为)HUAWEI MateBook X Pro 2019款 13.9英寸3K触控全面屏 轻薄笔记本",
"keywords":"轻薄笔记本华为 笔记本电脑",
"subTitle":"轻薄华为笔记本 电脑",
"price":"4769",
"promotionPrice":"4469",
"originalPrice":"4999",
"pic":"http://tuling‐mall.oss‐cn‐shenzhen.aliyuncs.com/tulingmall/images/202 00317/800_800_1555752016264mp.png",
"sale":999,
"hasStock":true,
"salecount":699,
"putawayDate":"2021‐04‐14",
"brandId":3,
"brandName":"华为",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/17f2dd9756d9d333bee8e60ce8c03e4c_222_222.jpg",
"categoryId":19,
"categoryName":"手机通讯",
"attrs":[
{
"attrId":9,
"attrName":"容量",
"attrValue":"16G"
},
{
"attrId":10,
"attrName":"网络",
"attrValue":"4G"
}
]
}
PUT /product_db/_doc/13
{
"id":"38",
"name":"华为nova6se 手机 绮境森林 全网通（8G+128G)",
"keywords":"轻薄笔记本华为 手机",
"subTitle":"华为nova6se 手机",
"price":"6769",
"promotionPrice":"6469",
"originalPrice":"6999",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180607/5a c1bf58Ndefaac16.jpg",
"sale":999,
"hasStock":true,
"salecount":899,
"putawayDate":"2021‐04‐15",
"brandId":3,
"brandName":"华为",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/17f2dd9756d9d333bee8e60ce8c03e4c_222_222.jpg",
"categoryId":19,
"categoryName":"手机通讯",
"attrs":[
{
"attrId":9,
"attrName":"容量",
"attrValue":"64G"
},
{
"attrId":10,
"attrName":"网络",
"attrValue":"5G"
}
]
}
PUT /product_db/_doc/14
{
"id":"39",
"name":"iPhone7/6s/8钢化膜苹果8Plus全屏复盖抗蓝光防窥防偷看手机膜",
"keywords":"手机膜",
"subTitle":"iPhone7/6s/8钢化膜苹果8Plus全屏复盖抗蓝光防窥防偷看手机膜",
"price":"29",
"promotionPrice":"39",
"originalPrice":"49",
"pic":"http://tuling‐mall.oss‐cn‐shenzhen.aliyuncs.com/tulingmall/images/202 00311/6df99dab78bb2014.jpg",
"sale":999,
"hasStock":true,
"salecount":799,
"putawayDate":"2021‐04‐16",
"brandId":51,
"brandName":"苹果",
"brandImg":"http://tuling‐mall.oss‐cn‐shenzhen.aliyuncs.com/tulingmall/image s/20200311/2b84746650fc122d67749a876c453619.png",
"categoryId":30,
"categoryName":"手机配件",
"attrs":[
{
"attrId":11,
"attrName":"手机膜‐材料",
"attrValue":"钢化"
},
{
"attrId":12,
"attrName":"手机膜‐颜色",
"attrValue":"白色"
}
]
}
PUT /product_db/_doc/15
{
"id":"40",
"name":"七匹狼短袖T恤男纯棉舒适春夏修身运动休闲短袖三条装 圆领3条装",
"keywords":"七匹狼服装 衣服",
"subTitle":"七匹狼短袖T恤男纯棉舒适春夏修身运动休闲短袖三条装 圆领3条装",
"price":"129",
"promotionPrice":"139",
"originalPrice":"149",
"pic":"http://tuling‐mall.oss‐cn‐shenzhen.aliyuncs.com/tulingmall/images/202 00311/19e846e727dff337.jpg",
"sale":999,
"hasStock":true,
"salecount":199,
"putawayDate":"2021‐04‐20",
"brandId":49,
"brandName":"七匹狼",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/18d8bc3eb13533fab466d702a0d3fd1f40345bcd.jpg",
"categoryId":8,
"categoryName":"T恤",
"attrs":[
{
"attrId":3,
"attrName":"尺寸",
"attrValue":"M"
},
{
"attrId":4,
"attrName":"颜色",
"attrValue":"白色"
}
]
}
PUT /product_db/_doc/16
{
"id":"41",
"name":"华为P40 Pro手机",
"keywords":"华为手机",
"subTitle":"华为P40 Pro手机",
"price":"2129",
"promotionPrice":"2139",
"originalPrice":"2149",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180607/5a c1bf58Ndefaac16.jpg",
"sale":999,
"hasStock":true,
"salecount":199,
"putawayDate":"2021‐05‐03",
"brandId":3,
"brandName":"华为",
"brandImg":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/201901 29/17f2dd9756d9d333bee8e60ce8c03e4c_222_222.jpg",
"categoryId":19,
"categoryName":"手机通讯",
"attrs":[
{
"attrId":9,
"attrName":"容量",
"attrValue":"128G"
},
{
"attrId":10,
"attrName":"网络",
"attrValue":"5G"
}
]
}
PUT /product_db/_doc/17
{
"id":"42",
"name":"朵唯智能手机 4G全网通 老人学生双卡双待手机",
"keywords":"朵唯手机",
"subTitle":"朵唯手机后置双摄，国产虎贲芯片！优化散热结构！浅薄机身！朵唯4月特 惠！",
"price":"3129",
"promotionPrice":"3139",
"originalPrice":"3249",
"pic":"http://macro‐oss.oss‐cn‐shenzhen.aliyuncs.com/mall/images/20180615/xi aomi.jpg",
"sale":999,
"hasStock":true,
"salecount":1199,
"putawayDate":"2021‐06‐01",
"brandId":59,
"brandName":"朵唯",
"brandImg":"http://tuling‐mall.oss‐cn‐shenzhen.aliyuncs.com/tulingmall/image s/20200311/2b84746650fc122d67749a876c453619.png",
"categoryId":19,
"categoryName":"手机通讯",
"attrs":[
{
"attrId":9,
"attrName":"容量",
"attrValue":"32G"
},
{
"attrId":10,
"attrName":"网络",
"attrValue":"4G"
}
]
}
七。类似JPA接口形式:ElasticsearchRepository
1、pom依赖
org.springframework.boot
spring-boot-starter-data-elasticsearch
2.2.6.RELEASE
2、配置文件es库
spring:
elasticsearch:
rest:
uris:
- ${ES_HTTP_SERVER:10.240.3.141:9200}
username: ${ES_USERNAME:app_writer}
password: ${ES_PASSWORD:YXBwX3dyaXRlcg==}
3、创建实体类
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;
import org.springframework.data.elasticsearch.annotations.GeoPointField;
import org.springframework.data.elasticsearch.core.geo.GeoPoint;
@Data
@Document(indexName = "user_index", type = "user") //第一个索引库名称_库 第二个类型_表(见es五个名字) 后面kibana查询的时候就是GET /user_index/user/_search 而不是 GET /user_index/_doc/_search
public class UserESEntity implements Serializable {
private static final long serialVersionUID = 1L;
@ApiModelProperty(value = "主键id")
@Id
private Long id;
@ApiModelProperty(value = "姓名")
@Field(type = FieldType.Text)
private String name;
@ApiModelProperty(value = "住址")
@Field(type = FieldType.Text)
private String address;
@ApiModelProperty(value = "住址")
@Field(type = FieldType.Integer)
private Integer age;
@ApiModelProperty(value = "地点")
@GeoPointField
private GeoPoint location;
//要有无参构造函数
public UserESEntity(){
}
}
4、创建repository
@Repository
public interface UserESRepository extends ElasticsearchRepository<UserESEntity, Long> {
}
5、注入repository并使用
a、注入
@Autowired
private OpLocationElastic opLocationElastic;
b、添加更新
GeoPoint geoPoint = new GeoPoint(Double.valueOf(opLocationElasticDTO.getLatitude()), Double.valueOf(opLocationElasticDTO.getLongitude()));
opLocationElasticDTO.setLocation(geoPoint);
opLocationElastic.save(opLocationElasticDTO);
userElasticsearchRepository.index(userElasticsearchEntity); 或者
userElasticsearchRepository.save(userElasticsearchEntity);
c、删除
opLocationElastic.deleteAll();
opLocationElastic.delete(opLocationElasticDTO);
d、查询
api方式查询
1、API条件查询
//总条件
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
//精确条件
if (StringUtils.isNotBlank(opLocationMapQueryDTO.getCode))) {
boolQueryBuilder.must(QueryBuilders.termQuery("code", "opLocationMapQueryDTO.getCode("));
}
//like模糊条件 前后加*
if (StringUtils.isNotBlank(opLocationMapQueryDTO.getName())) {
boolQueryBuilder.must(QueryBuilders.wildcardQuery("name", "*" + opLocationMapQueryDTO.getName() + "*"));
}
//in条件
boolQueryBuilder.must( QueryBuilders.termsQuery("id", Lists.newArrayList("54", "55")));
//大于或者小于
RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("price");
rangeQueryBuilder.lte(prices[0]);
//查询
Iterable<OpLocationElasticDTO> locationList = opLocationElastic.search(boolQueryBuilder);
2、or查询 商户id是and，场站id和场站名称是or关系
//条件1：商户id
BoolQueryBuilder boolQueryBuilder1 = QueryBuilders.boolQuery();
boolQueryBuilder1.must(QueryBuilders.termQuery("operatorId", sellerId));
//条件2：场站名称 站点id的组合
BoolQueryBuilder boolQueryBuilder2 = QueryBuilders.boolQuery();
//场站名称
boolQueryBuilder2.should(QueryBuilders.wildcardQuery("name", "*" + opLocationPageDTO.getStationNameOrPileSN() + "*");
//场站id
boolQueryBuilder2.should(QueryBuilders.termsQuery("id", stationIds));
//查询对象
SearchQuery searchQuery = new NativeSearchQuery(boolQueryBuilder1, boolQueryBuilder2)
//查询
stationESPageResult = opLocationElastic.search(searchQuery);
复杂查询看这个，上面那个可弃用，基本思路是看最外面是and还是or,然后一层一层往里面加QueryBuilders.boolQuery()
//根据场站id和状态查询，状态为或
BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
boolQueryBuilder.must(QueryBuilders.boolQuery()
.should(QueryBuilders.termsQuery("state", realTimeStatusNameList))
.should(QueryBuilders.termQuery("state", ""))
.should(QueryBuilders.boolQuery().mustNot(QueryBuilders.existsQuery("state"))));
boolQueryBuilder.must(QueryBuilders.termQuery("locationId", stationId));
log.info("状态满足条件的枪数据ES查询语言：{}", boolQueryBuilder);
Iterable<OpLocationEvseElasticDTO> opLocationEvseElasticDTOIterable = opLocationEvseElastic.search(boolQueryBuilder);
3、分页
//检索条件
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
//查询对象
NativeSearchQuery nativeSearchQuery = new NativeSearchQuery(boolQueryBuilder);//用NativeSearchQuery默认分页，分页size为10
//设置分页 没有设置默认(0,10)
PageRequest pageRequest = PageRequest.of(pilePageDTO.getPage() - 1, pilePageDTO.getPageSize());
nativeSearchQuery .setPageable(pageRequest);
//查询
org.springframework.data.domain.Page<OpLocationPileEvseElasticDTO> search = opLocationPileEvseElastic.search(nativeSearchQuery );
4、排序
//检索条件
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
//查询对象
NativeSearchQuery nativeSearchQuery = new NativeSearchQuery(boolQueryBuilder); //用NativeSearchQuery默认分页，分页size为10
//设置分页 没有设置默认(0,10)
PageRequest pageRequest = PageRequest.of(pilePageDTO.getPage() - 1, pilePageDTO.getPageSize());
nativeSearchQuery.setPageable(pageRequest);
//设置排序
Sort pileSNSort = Sort.by(Sort.Direction.DESC, "power");
nativeSearchQuery.addSort(pileSNSort);
//查询
org.springframework.data.domain.Page<OpLocationPileEvseElasticDTO> search = opLocationPileEvseElastic.search(nativeSearchQuery);
5、查询字段为空字符串或者nul的数据
boolQueryBuilder2.mustNot(QueryBuilders.regexpQuery("state", "[0-9]+"));
jpa方式查询
1、JPA查询所有
import com.google.common.collect.Lists;
Iterable<OpLocationElasticDTO> iterable = opLocationElastic.findAll();
for (OpLocationElasticDTO opLocationElasticDTO : locationIterable) {
log.info("es条件查询：{}", JSONArray.toJSONString(opLocationElasticDTO));
}
ArrayList<OpLocationElasticDTO> opLocationElasticDTOS = Lists.newArrayList(locationIterable);
2、JPA in查询、like查询、排序查询
a、repository写上接口方法
@Repository
public interface OpLocationElastic extends ElasticsearchRepository<OpLocationElasticDTO, Long> {
List<Long> findByIdInOrNameLikeOrderByIdAsc(String name,List<Long> stationIds);
}
b、service调用
repository.findByIdInOrNameLikeOrderByIdAsc("*" + name + "*",ids)
3、JPA分页(es分页的第一页是0,mybatis-plus是1)
a.repository写上接口方法(相比JPA的其他条件查询，只是参数加一个Pageable pageAble，返回类型改为Page就可以)
Page<UserESEntity> findByIdInOrNameLikeOrderByIdAsc(List<Long> ids, String name, Pageable pageable);
b.service调用
Pageable pageable = PageRequest.of(0, 3); //第一页是从0开始，所以正常dto.getPageIndex-1
Page<UserESEntity> name4 = userESRepository.findByIdInOrNameLikeOrderByIdAsc(ids, "*name*", pageable);
八。范围查询 经纬度
实体类属性
@ApiModelProperty(value = "地点")
@GeoPointField
private GeoPoint location;
条件查询
GeoDistanceQueryBuilder distanceQueryBuilder = new GeoDistanceQueryBuilder("location");
distanceQueryBuilder.point(opLocationMapQueryDTO.getLatitude(), opLocationMapQueryDTO.getLongitude());
distanceQueryBuilder.distance(range, DistanceUnit.KILOMETERS);
boolQueryBuilder.filter(distanceQueryBuilder);
Iterable<OpLocationGeoPointElasticDTO> locationGeoPointIterable = opLocationGeoPointElastic.search(boolQueryBuilder);
ArrayList<OpLocationGeoPointElasticDTO> opLocationGeoPointElasticDTOS = Lists.newArrayList(locationGeoPointIterable);
距离计算
double calculateDistance = GeoDistance.ARC.calculate(lat, lng, latitude, longitude, DistanceUnit.KILOMETERS);