ElasticSearch高级操作---javaAPI
批量操作
Bulk 批量操作是将文档的增删改查一些列操作，通过一次请求全都做完。减少网络传输次数。
脚本操作：
POST _bulk
{"delete":{"_index":"person","_id":"4"}}
{"create":{"_index":"person","_id":"4"}}
{"name":"五号","age":20,"address":"北京"}
{"update":{"_index":"person","_id":"2"}}
{"doc":{"name":"二号"}}
删除 索引为person的索引中id为4的文档
在索引为 person 的索引中创建id为4的文档，内容为下面一行的内容。
更新 索引为 person的索引中的id为2的文档，更新的内容在下面一行，文档类型为doc，将name对应的值修改为二号
一条json格式的脚本代码只能写在一行
每一条脚本代码相互之间是独立，互相不干扰。
基于SpringBoot的JavaAPI操作
/**
* 1. 批量操作 bulk
*/
@Test
public void testBulk() throws IOException {
//创建bulkrequest对象，整合所有操作
BulkRequest bulkRequest = new BulkRequest();
/*
# 1. 删除1号记录
# 2. 添加6号记录
# 3. 修改3号记录 名称为 “三号”
*/
//添加对应操作
//1. 删除1号记录
DeleteRequest deleteRequest = new DeleteRequest("person","1");
bulkRequest.add(deleteRequest);
//2. 添加6号记录
Map map = new HashMap();
map.put("name","六号");
IndexRequest indexRequest = new IndexRequest("person").id("6").source(map);
bulkRequest.add(indexRequest);
Map map2 = new HashMap();
map2.put("name","三号");
//3. 修改3号记录 名称为 “三号”
UpdateRequest updateReqeust = new UpdateRequest("person","3").doc(map2);
bulkRequest.add(updateReqeust);
//执行批量操作
BulkResponse response = client.bulk(bulkRequest, RequestOptions.DEFAULT);
RestStatus status = response.status();
System.out.println(status);
}
将MySql中的数据导入到Elastic Search中
/**
* 批量导入，数据库查询出来的spec字段是字符串格式的json数据，
*/
@Test
public void importData() throws IOException {
//1.查询所有数据，mysql
List<Goods> goodsList = goodsMapper.findAll();
//System.out.println(goodsList.size());
//2.bulk导入
BulkRequest bulkRequest = new BulkRequest();
//2.1 循环goodsList，创建IndexRequest添加数据
for (Goods goods : goodsList) {
//2.2 设置spec规格信息 Map的数据 specStr:{}
//goods.setSpec(JSON.parseObject(goods.getSpecStr(),Map.class));
String specStr = goods.getSpecStr();
//将json格式字符串转为Map集合
Map map = JSON.parseObject(specStr, Map.class);
//设置spec map
goods.setSpec(map);
//将goods对象转换为json字符串
String data = JSON.toJSONString(goods);//map --> {}
IndexRequest indexRequest = new IndexRequest("goods");
indexRequest.id(goods.getId()+"").source(data, XContentType.JSON);
bulkRequest.add(indexRequest);
}
BulkResponse response = client.bulk(bulkRequest, RequestOptions.DEFAULT);
System.out.println(response.status());
}
Goods类，省略getter和setter及其构造方法
public class Goods {
private int id;
private String title;
private double price;
private int stock;
private int saleNum;
private Date createTime;
private String categoryName;
private String brandName;
private Map spec;
@JSONField(serialize = false)//在转换JSON时，忽略该字段
private String specStr;//接收数据库的信息 "{}"
}
matchALl查询----查询所有文档
脚本操作
语法：
GET 索引名称/_search
{
"query":{
"match_all":{}
}
}
默认情况下，es一次展示十条数据
GET 索引名称/_search
{
"query":{
"match_all":{}
},
"form":0,
"size":100
}
通过 from 从哪儿开始 size 每页多少条
查询信息解读：
{
"took" : 1,// 耗时（毫秒）
"timed_out" : false, //是否超时
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {// 查询的结果集
"total" : {// 总数
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.0,//最大评分
"hits" : [// 数据列表
{
"_index" : "test2",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {// 它是数据document的json内容
"name" : "张三",
"age" : "1"
}
}
]
}
}
JavaAPI操作：
@Test
public void testMatchAll() throws IOException {
//2. 构建查询请求对象，指定查询的索引名称
SearchRequest searchRequest = new SearchRequest("goods");
//4. 创建查询条件构建器SearchSourceBuilder
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
//6. 查询条件
QueryBuilder query = QueryBuilders.matchAllQuery();//查询所有文档
//5. 指定查询条件
sourceBuilder.query(query);
//3. 添加查询条件构建器 SearchSourceBuilder
searchRequest.source(sourceBuilder);
// 8 . 添加分页信息
sourceBuilder.from(0);
sourceBuilder.size(100);
//1. 查询,获取查询结果
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
//7. 获取命中对象 SearchHits
SearchHits searchHits = searchResponse.getHits();
//7.1 获取总记录数
long value = searchHits.getTotalHits().value;
System.out.println("总记录数："+value);
List<Goods> goodsList = new ArrayList<>();
//7.2 获取Hits数据 数组
SearchHit[] hits = searchHits.getHits();
for (SearchHit hit : hits) {
//获取json字符串格式的数据
String sourceAsString = hit.getSourceAsString();
//转为java对象
Goods goods = JSON.parseObject(sourceAsString, Goods.class);
goodsList.add(goods);
}
for (Goods goods : goodsList) {
System.out.println(goods);
}
}
Term：精确查询
搜索关键字不分词与字段内容完全匹配，就是等号
GET _search
{
"query": {
"term": {
"name": {
"value": "张三"
}
}
}
}
后面的查询都只需要在上面matchAll的代码上更改query即可
/**
* termQuery:词条查询
*/
@Test
public void testTermQuery() throws IOException {
SearchRequest searchRequest = new SearchRequest("goods");
SearchSourceBuilder sourceBulider = new SearchSourceBuilder();
QueryBuilder query = QueryBuilders.termQuery("title","华为");//term词条查询
sourceBulider.query(query);
searchRequest.source(sourceBulider);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits searchHits = searchResponse.getHits();
//获取记录数
long value = searchHits.getTotalHits().value;
System.out.println("总记录数："+value);
List<Goods> goodsList = new ArrayList<>();
SearchHit[] hits = searchHits.getHits();
for (SearchHit hit : hits) {
String sourceAsString = hit.getSourceAsString();
//转为java
Goods goods = JSON.parseObject(sourceAsString, Goods.class);
goodsList.add(goods);
}
for (Goods goods : goodsList) {
System.out.println(goods);
}
}
Match：分词等值查询
分词之后进行等值匹配，可以求并集或者交集
// A:1、2、3 B:1、2、5
// operator:or并集=1、2、3、5
// operator:and交集=1、2
GET _search
{
"query": {
"match": {
"name": {
"query": "张三",
"operator": "or"
}
}
}
}
/**
* matchQuery:词条分词查询
*/
@Test
public void testMatchQuery() throws IOException {
SearchRequest searchRequest = new SearchRequest("goods");
SearchSourceBuilder sourceBulider = new SearchSourceBuilder();
MatchQueryBuilder query = QueryBuilders.matchQuery("title", "华为手机");
query.operator(Operator.AND);//求并集
sourceBulider.query(query);
searchRequest.source(sourceBulider);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits searchHits = searchResponse.getHits();
//获取记录数
long value = searchHits.getTotalHits().value;
System.out.println("总记录数："+value);
List<Goods> goodsList = new ArrayList<>();
SearchHit[] hits = searchHits.getHits();
for (SearchHit hit : hits) {
String sourceAsString = hit.getSourceAsString();
//转为java
Goods goods = JSON.parseObject(sourceAsString, Goods.class);
goodsList.add(goods);
}
for (Goods goods : goodsList) {
System.out.println(goods);
}
}
Wildcard：通配符查询
就是like查询，？是单个字符，*是任意字符。注意竟可能星号不要写在前面。（否则会进行全盘扫描，倒排索引失效） 分词后的模糊匹配，查询条件分词，模糊查询
GET _search
{
"query": {
"wildcard": {
"name": {
"value": "张?"
}
}
}
}
WildcardQueryBuilder query = QueryBuilders.wildcardQuery("name", "张?");
Regexp：正则查询
查询的字段使用正则表达式
GET _search
{
"query": {
"regexp": {
"name": "[张](.)*"
}
}
}
RegexpQueryBuilder query = QueryBuilders.regexpQuery("name", "张(.)*");
Prefix：前缀查询
查询以某个关键词开头的数据
GET _search
{
"query": {
"prefix": {
"name": "张"
}
}
}
PrefixQueryBuilder query = QueryBuilders.prefixQuery("name", "张");
Range：范围查询
GET _search
{
"query": {
"range": {
"age": {
"gte": 1, # 大于等于
"lte": 20 # 小于等于
}
}
},
"sort":[
{
"price":{
"order": "desc" # 指定price字段降序排列
}
}
]
}
RangeQueryBuilder query = QueryBuilders.rangeQuery("age").gte(1).lte(10);
sourceBuilder.sort("price","desc") // price 字段名
QueryString：表达式查询
- 会对查询条件进行分词
- 然后将分词后的查询条件和此条进行等值匹配
- 默认取并集or
- 可以指定多个查询字段
查询条件还可以去 AND
将“华为手机”进行分词然后进行等值匹配，去三个字段去查询，最后去并集。
省略写法
GET _search
{
"query": {
"query_string": {
"query": "name:张二 AND age:1"
}
}
}
// 两种写法
QueryStringQueryBuilder query = QueryBuilders.queryStringQuery("华为手机")
.field("title")
.field("categoryName")
.defaultOperator(Operator.AND);
QueryStringQueryBuilder query = QueryBuilders.queryStringQuery("name:张二 AND age:1");
SimpleQueryString：多字段条件查询，不支持 OR AND 连接符
- 对多个字段查询
- 对于输入的关键字不会进行分词
GET _search
{
"query": {
"simple_query_string": {
"fields": ["name","age"],
"query": "张三 1" # 张三查一次 1 查一次 最后取并集
}
}
}
Bool：组合条件查询
只组合其它搜索条件，无自己的特殊条件
- 组合方式
- must：执行查询，条件必须成立，并计算评分
- must_not 条件必须不成立，非
- should or 条件可以成立
- filter：条件必须称例，性能比must高，执行查询，不计算评分
GET _search
{
"query": {
"bool": {
"must": [
{
"term": {
"name": {
"value": "张三"
}
}
}
],
"filter": {
"range": {
"age": {
"gte": 1,
"lte": 20
}
}
}
}
}
}
javaAPI
TermQueryBuilder term = QueryBuilders.termQuery("name","张三");
WildcardQueryBuilder wildcard = QueryBuilders.wildcardQuery("name", "张?");
BoolQueryBuilder query = QueryBuilders.boolQuery();
query.must(term);
query.filter(wildcard);
builder.query(query);
聚合查询
- 指标聚合：相当于MySQL的聚合函数。max、min、avg、sum等
- 桶聚合：相当于MySQL的group By 操作，不要对text类型的数据进行分组，会失败
指标聚合
桶聚合
/**
* 聚合查询：桶聚合，分组查询
* 1. 查询title包含手机的数据
* 2. 查询品牌列表
*/
@Test
public void testAggQuery() throws IOException {
SearchRequest searchRequest = new SearchRequest("goods");
SearchSourceBuilder sourceBulider = new SearchSourceBuilder();
// 1. 查询title包含手机的数据
MatchQueryBuilder query = QueryBuilders.matchQuery("title", "手机");
sourceBulider.query(query);
// 2. 查询品牌列表
/*
参数：
1. 自定义的名称，将来用于获取数据
2. 分组的字段
*/
AggregationBuilder agg = AggregationBuilders.terms("goods_brands").field("brandName").size(100);
sourceBulider.aggregation(agg);
searchRequest.source(sourceBulider);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits searchHits = searchResponse.getHits();
//获取记录数
long value = searchHits.getTotalHits().value;
System.out.println("总记录数："+value);
List<Goods> goodsList = new ArrayList<>();
SearchHit[] hits = searchHits.getHits();
for (SearchHit hit : hits) {
String sourceAsString = hit.getSourceAsString();
//转为java
Goods goods = JSON.parseObject(sourceAsString, Goods.class);
goodsList.add(goods);
}
for (Goods goods : goodsList) {
System.out.println(goods);
}
// 获取聚合结果
Aggregations aggregations = searchResponse.getAggregations();
Map<String, Aggregation> aggregationMap = aggregations.asMap();
//System.out.println(aggregationMap);
Terms goods_brands = (Terms) aggregationMap.get("goods_brands");
List<? extends Terms.Bucket> buckets = goods_brands.getBuckets();
List brands = new ArrayList();
for (Terms.Bucket bucket : buckets) {
Object key = bucket.getKey();
brands.add(key);
}
for (Object brand : brands) {
System.out.println(brand);
}
}
高亮查询
- 高亮查询三要素： 高亮字段 前缀 后缀
javaAPI 操作
/**
*
* 高亮查询：
* 1. 设置高亮
* * 高亮字段
* * 前缀
* * 后缀
* 2. 将高亮了的字段数据，替换原有数据
*/
@Test
public void testHighLightQuery() throws IOException {
SearchRequest searchRequest = new SearchRequest("goods");
SearchSourceBuilder sourceBulider = new SearchSourceBuilder();
// 1. 查询title包含手机的数据
MatchQueryBuilder query = QueryBuilders.matchQuery("title", "手机");
sourceBulider.query(query);
//设置高亮
HighlightBuilder highlighter = new HighlightBuilder();
//设置三要素
highlighter.field("title");
highlighter.preTags("");
highlighter.postTags("");
sourceBulider.highlighter(highlighter);
// 2. 查询品牌列表
/*
参数：
1. 自定义的名称，将来用于获取数据
2. 分组的字段
*/
AggregationBuilder agg = AggregationBuilders.terms("goods_brands").field("brandName").size(100);
sourceBulider.aggregation(agg);
searchRequest.source(sourceBulider);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits searchHits = searchResponse.getHits();
//获取记录数
long value = searchHits.getTotalHits().value;
System.out.println("总记录数："+value);
List<Goods> goodsList = new ArrayList<>();
SearchHit[] hits = searchHits.getHits();
for (SearchHit hit : hits) {
String sourceAsString = hit.getSourceAsString();
//转为java
Goods goods = JSON.parseObject(sourceAsString, Goods.class);
// 获取高亮结果，替换goods中的title
Map<String, HighlightField> highlightFields = hit.getHighlightFields();
HighlightField HighlightField = highlightFields.get("title");
Text[] fragments = HighlightField.fragments();
//替换
goods.setTitle(fragments[0].toString());
goodsList.add(goods);
}
for (Goods goods : goodsList) {
System.out.println(goods);
}
// 获取聚合结果
Aggregations aggregations = searchResponse.getAggregations();
Map<String, Aggregation> aggregationMap = aggregations.asMap();
//System.out.println(aggregationMap);
Terms goods_brands = (Terms) aggregationMap.get("goods_brands");
List<? extends Terms.Bucket> buckets = goods_brands.getBuckets();
List brands = new ArrayList();
for (Terms.Bucket bucket : buckets) {
Object key = bucket.getKey();
brands.add(key);
}
for (Object brand : brands) {
System.out.println(brand);
}
}