Day143-145.尚品汇:商品上架、ES商品检索 | 动态DSL、搜索条件处理、logstash

目录

Day10

1. 商品上架

2. 根据用户检索的条件编写 dsl 语句

3. 更新商品热度

4. 动态生成 dsl 语句

4. JUC 、SpringCloud 回顾

Day11

1. JMM、CAS、Lock回顾

2. DSL 商品检索

3. 搜索条件处理 - url拼接

4. logstash: 日志收集框架:


Day10

回顾

1.    异步编排优化:多线程
2.    首页渲染:redis ;nginx --- 静态代理
3.    全文检索:goods 索引库;新的数据类型 nested - 允许数据彼此独立的检索和查询!

1. 商品上架

本质 将mysql数据存入ES

根据用户检索的条件编写dsl语句

es 6.8.1 索引库需要自己访问控制器才能生成

es 7.8.0 项目启动会自动创建索引库,前提:

operator : "and "   默认为or

// 商品上架 --- 将数据封装到Goods,并且Goods 保存到ES中,GoodsRepository来做这件事
    @Override
    public void upperGoods(Long skuId) {
        // 创建goods,给goods 赋值
        Goods goods = new Goods();
        // 远程调用获取对应的参数
        CompletableFuture skuInfoCompletableFuture = CompletableFuture.supplyAsync(() -> {
            SkuInfo skuInfo = productFeignClient.getSkuInfo(skuId);
            goods.setId(skuId);
            goods.setTitle(skuInfo.getSkuName());
            goods.setDefaultImg(skuInfo.getSkuDefaultImg());
            return skuInfo;
        },threadPoolExecutor);

        // skuInfo 可能是从缓存中获取数据; 延迟双删 --> 避免出现脏读,但还是有可能出现问题
        // 所以从数据库中重新获取
        CompletableFuture productCompletableFuture = CompletableFuture.runAsync(() -> {
            BigDecimal skuPrice = productFeignClient.getSkuPrice(skuId);
            goods.setPrice(skuPrice.doubleValue()); //价格为BigDecimal,需要转型
            //商品上架时间:方便按照时间排序
            goods.setCreateTime(new Date());
        },threadPoolExecutor);


        //赋值品牌数据
        CompletableFuture trademarkCompletableFuture = skuInfoCompletableFuture.thenAcceptAsync((skuInfo) -> {
            BaseTrademark trademark = productFeignClient.getTrademark(skuInfo.getTmId());
            goods.setTmId(trademark.getId());
            goods.setTmName(trademark.getTmName());
            goods.setTmLogoUrl(trademark.getLogoUrl());
        },threadPoolExecutor);

        //赋值分类数据
        CompletableFuture categoryCompletableFuture = skuInfoCompletableFuture.thenAcceptAsync((skuInfo) -> {
            BaseCategoryView categoryView = productFeignClient.getCategoryView(skuInfo.getCategory3Id());
            goods.setCategory1Id(categoryView.getCategory1Id());
            goods.setCategory2Id(categoryView.getCategory2Id());
            goods.setCategory3Id(categoryView.getCategory3Id());
            goods.setCategory1Name(categoryView.getCategory1Name());
            goods.setCategory2Name(categoryView.getCategory2Name());
            goods.setCategory3Name(categoryView.getCategory3Name());
        },threadPoolExecutor);

        //热度排名:hotScore默认值 0

        //赋值平台属性 (stream 在内部进行了优化和提升)
        CompletableFuture attrCompletableFuture = CompletableFuture.runAsync(() -> {
            List attrList = this.productFeignClient.getAttrList(skuId);
            List searchAttrList = attrList.stream().map(baseAttrInfo -> {
                SearchAttr searchAttr = new SearchAttr();
                searchAttr.setAttrId(baseAttrInfo.getId());
                searchAttr.setAttrName(baseAttrInfo.getAttrName());
                //skuId 对应的平台属性 -- 平台属性值只有一个
                searchAttr.setAttrValue(baseAttrInfo.getAttrValueList().get(0).getValueName());
                return searchAttr;
            }).collect(Collectors.toList());
            goods.setAttrs(searchAttrList);
        },threadPoolExecutor);

        // 存入ES
        CompletableFuture.allOf(
                skuInfoCompletableFuture,
                productCompletableFuture,
                trademarkCompletableFuture,
                categoryCompletableFuture,
                attrCompletableFuture
        ).join();

        goodsRepository.save(goods);
    }

2. 根据用户检索的条件编写 dsl 语句

# 分类ID
GET /goods/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "category3Id": "61"
          }
        }
      ]
    }
  }
}
#根据商品名称过滤
GET /goods/_search
{
  "query": {
    "match": {
      "title": {
        "query": "小米手机",
        "operator": "and"
      }
    }
  }
}

#品牌ID 现根据分类ID过滤,再根据品牌ID过滤
GET /goods/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "category3Id": "61"
          }
        },
        {
          "term": {
            "tmId": "1"
          }
        }
      ]
    }
  }
}
#平台属性 --- 数据类型是nested
GET /goods/_search
{
  "query": {
    "nested": {
      "path": "attrs",
      "query": {
        "bool": {
          "must": [
            {
              "match": {
                "attrs.attrId": "24"
              }
            },
            {
              "match": {
                "attrs.attrValue": "128G"
              }
            }
          ]
        }
      }
    }
  }
}
# 高亮
# Thymeleaf 标签th:utext 解析样式 小米
GET /goods/_search
{
  "query": {
    "match": {
      "title": "手机"
    }
  },
  "highlight": {
    "fields": {
      "title": {}
    },
    "pre_tags": [
      ""
    ],
    "post_tags": [
      ""
    ]
  }
}
#聚合
#目的:展示数据,进行过滤
GET /goods/_search
{
  "query": {
    "match": {
      "category3Id": "61"
    }
  },
  "aggs": {
    "tmIdAgg": {
      "terms": {
        "field": "tmId",
        "size": 10
      }
    }
  }
}
#品牌聚合+平台属性聚合 方便取数据
GET /goods/_search
{
  "query": {
    "match": {
      "category3Id": "61"
    }
  },
  "aggs": {
    "tmIdAgg": {
      "terms": {
        "field": "tmId",
        "size": 10
      },
      "aggs": {
        "tmNameAgg": {
          "terms": {
            "field": "tmName",
            "size": 10
          }
        },
        "tmLogoUrlAgg":{
          "terms": {
            "field": "tmLogoUrl",
            "size": 10
          }
        }
      }
    },
    "attrAgg":{
      "nested": {
        "path": "attrs"
      },
      "aggs": {
        "attrIdAgg": {
          "terms": {
            "field": "attrs.attrId",
            "size": 10
          },
          "aggs": {
            "attrNameAgg": {
              "terms": {
                "field": "attrs.attrName",
                "size": 10
              },
              "aggs": {
                "attValueAgg": {
                  "terms": {
                    "field": "attrs.attrValue",
                    "size": 10
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

3. 更新商品热度

考虑缓存穿透 缓存击穿 缓存雪崩,存储时的类型,value

    //更新ES商品热度
    @Override
    public void incrHotScore(Long skuId) {
        //被访问的次数 -- redis 考虑三个问题+数据类型
        //string incrby key 100 ; 
        //ZSet zincrby hotScore 100 21
        //详情string 热度zset  秒杀list
        String hotKey = "hotScore";
        Double count = redisTemplate.opsForZSet().incrementScore(hotKey, "skuId:" + skuId, 1);
        // 10 的倍数存入ES,减少IO
        if(count%10==0){
            // 更新ES
            Optional optional = goodsRepository.findById(skuId);
            Goods goods = optional.get();
            goods.setHotScore(count.longValue());
            goodsRepository.save(goods);
        }
    }

在商品详情item 远程调用ListApiController控制器方法即可 

4. 动态生成 dsl 语句

//商品检索
    @Override
    public SearchResponseVo search(SearchParam searchParam) {
        //1.生成dsl语句
        //2.执行dsl
        //3.将执行之后的结果进行封装
        //声明一个查询请求对象
        SearchRequest searchRequest = this.buildDsl(searchParam);
        //执行dsl语句
        SearchResponse searchResponse = null;
        try {
            searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        } catch (IOException e) {
            e.printStackTrace();
        }
        //将执行之后的结果进行封装 SearchResponseVo
        SearchResponseVo searchResponseVo = this.parseResult(searchResponse);
        /*
            private List trademarkList;
            private List attrsList = new ArrayList<>();
            private List goodsList = new ArrayList<>();
            private Long total;
             ------ 以上四个方法在parseResult中赋值
            private Integer pageSize;
            private Integer pageNo;
            private Long totalPages;
         */
        //设置每页默认显示条数:3
        searchResponseVo.setPageSize(searchParam.getPageSize());
        searchResponseVo.setPageNo(searchParam.getPageNo());
        /*Long totalPages = searchResponseVo.getTotal()%searchParam.getPageSize()==0?
                searchResponseVo.getTotal()%searchParam.getPageSize():
                searchResponseVo.getTotal()%searchParam.getPageSize()+1;*/
        // from = (pageNo-1)*PageSize
        Long totalPages = (searchResponseVo.getTotal()+searchParam.getPageSize()-1)/searchParam.getPageSize();
        searchResponseVo.setTotalPages(totalPages);

        return searchResponseVo;
    }


    //动态生成dsl语句
    private SearchRequest buildDsl(SearchParam searchParam) {

        // 构建查询器 {}
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        //{bool}
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();

        // 判断男用户是否根据三级分类id进行检索
        if(!StringUtils.isEmpty(searchParam.getCategory3Id())){
            //{filter term}
            boolQueryBuilder.filter(QueryBuilders.termQuery("category3Id",searchParam.getCategory3Id()));
        }
        if(!StringUtils.isEmpty(searchParam.getCategory2Id())){
            boolQueryBuilder.filter(QueryBuilders.termQuery("category2Id",searchParam.getCategory2Id()));
        }
        if(!StringUtils.isEmpty(searchParam.getCategory1Id())){
            boolQueryBuilder.filter(QueryBuilders.termQuery("category1Id",searchParam.getCategory1Id()));
        }
        //根据关键词检索
        if(!StringUtils.isEmpty(searchParam.getKeyword())){
            boolQueryBuilder.must(QueryBuilders.
                    matchQuery("title",searchParam.getKeyword()).operator(Operator.AND));
        }

        //根据平台属性值进行过滤、
        //先获取到用户点击的数据
        String[] props = searchParam.getProps();
        if(props!=null && props.length>0){
            for (String prop : props) {
                //  第一次遍历 prop=24:256G:机身内存  第二次遍历 prop=106:安卓手机:手机一级
                String[] split = prop.split(":");
                // 声明中间层的bool
                BoolQueryBuilder boolBuilder = QueryBuilders.boolQuery();
                // 声明内层的bool
                BoolQueryBuilder innerBoolBuilder = QueryBuilders.boolQuery();
                // 设置内层
                innerBoolBuilder.must(QueryBuilders.matchQuery("attrs.attrId",split[0]));
                innerBoolBuilder.must(QueryBuilders.matchQuery("attrs.attrValue",split[1]));
                // 设置中间层
                boolBuilder.must(QueryBuilders.nestedQuery("attrs",innerBoolBuilder, ScoreMode.None));
                boolQueryBuilder.filter(boolBuilder);
                //
            }
        }

        //根据品牌Id 进行检索: trademark=3:华为
        String trademark = searchParam.getTrademark();
        if (!StringUtils.isEmpty(trademark)) {
            //分割
            String[] split = trademark.split(":");
            //判断是否根据品牌检索
            if(split!=null && split.length==2){
                //{filter term}
                boolQueryBuilder.filter(QueryBuilders.termQuery("tmId",split[0]));
            }
        }

        //分页,高亮,排序

        //{query}
        searchSourceBuilder.query(boolQueryBuilder);
        //{query -- bool}

        return null;
    }

    private SearchResponseVo parseResult(SearchResponse searchResponse) {
        return null;
    }

4. JUC 、SpringCloud 回顾

下午:JUC

1. 线程 - 进程 (java默认两个线程 main、gc)

每一个程序都有一个进程,操作系统动态执行的基本单元;

一个进程中可以包含若干个线程;线程是资源调度的最小单位

2. 并发 - 并行

并行:同一时间多个线程在执行,同一时刻多个线程在访问同一个资源,  (烧水泡面)

并发:同一时间多个线程在做同一件事 (秒杀、春运抢票)

3. wait - sleep

wait 释放锁,sleep 不释放锁

wait是Object的方法,sleep是Thread

4. sync

同步方法锁  this

静态同步方法锁 Class.class

同步代码块 ()中对象

5. 线程间的通信

wait notify notifyAll

线程操作资源类,高内聚低耦合

判断 干活 通知

防止线程虚假唤醒 while

6. 定制化通信  JUC.Lock (一个线程打印五次,一个十次)

lock.lock;lock.tryLock;lock.unlock();

Object Condition

await

signal() signalAll();

7. Lock 和Sync区别

共同点:可重入、独占锁

不同点:Lock 程序员控制,不容易发生死锁;Sync jvm控制,容易发生死锁

recurrentLock:如何理解可重入、公平锁、尝试获取锁

重入前提:需要同一个线程否则重入不了

8. 创建线程的集中方式:Thread、Runnable、Callable、Pool

Runnable Callable区别:Callable可获取返回值、异常,重写call()

        需要借助FutureTask 未来任务:

        get():获取当前线程执行结果

        isDone():判断这个线程是否执行完成

9. 阻塞队列:控制线程什么时候挂起,什么时候运行

        阻塞队列满的时候,再向队列中添加数据,则会发生阻塞 put

        阻塞队列空的时候,再向队列中获取数据,则会发生阻塞 take

SpringCloud

1. 分布式:n个计算机组成的系统

2. 集群:多台服务器集中在一起,实现同一个业务

3. RPC 远程过程调用:需要经过网络传输;需要协议http、https,所以需要实现序列化

Day143-145.尚品汇:商品上架、ES商品检索 | 动态DSL、搜索条件处理、logstash_第1张图片

Day143-145.尚品汇:商品上架、ES商品检索 | 动态DSL、搜索条件处理、logstash_第2张图片

 ES 使用 net

ESC 弹性云:可扩容的服务器

spring组件:别说阿里巴巴的

3. SpringBoot核心注解:SpringBootApplication:自动注入和自动装配

SpringBootCOnfiguration、EnableAutoCOnfiguration

Resource:Javax的注解 默认按照ByName注入

Autoware:Spring 的注解 默认按照ByType注入

SpringCloud组件

Eureka 注册中心 服务治理 -- 了解,以后不会用
Ribbon 负载均衡Load Balancer(LB)  默认轮询

Ribbon VS Nginx 负载均衡的区别(Gateway)

Nginx 是服务器负载均衡,接收客户请求,可以直接配置服务器地址列表

Ribbon 是本地负载均衡,需要从注册中心获取到服务名,本质:找的接口 /*/product/**

请求 -》 nginx  -》多个微服务 (根据接口去找服务!!)

Gateway网关底层整合了Ribbon,配置是一样的

Day143-145.尚品汇:商品上架、ES商品检索 | 动态DSL、搜索条件处理、logstash_第3张图片

Feign 和 OpenFeign 的区别

Feign 是一个SpringCloud中一个轻量级的RestFul 的http客户端,能够进行远程调用并支持负载均衡

OpenFeign 在原生基础上支持了SpringMVC注解,通过动态代理的方式产生实现类,实现负载均衡。

Hystrix断路器 保护后台微服务,类似Sentinel

Day11

1. JMM、CAS、Lock回顾

JMM:用于定义数据读写的规则保证同一份代码在不同的环境中,能够正常运行。

每个线程都有一个主内存、工作内存,需要先将主内存的资源读取到工作内存进行操作。

主内存、工作内存。

三个特性:

原子性:int i = 1 原子性,i++ 非原子性;锁保证

可见性:在多线程中要保证这个变量可见 volatile,sync,finall

有序性:多线程中执行顺序有序 sync,valatile

valatile:可见性、有序性不具有原子性;修饰的资源多线程可见;弱同步机制

有序性原理:禁止指令重排,有个内存屏障,执行时能保证有序

CAS:比较交换 campareAndSwapInt

cas原理:CAS(this,便宜量,预期值,修改值) 

底层 do {} while()

缺点:开销大、ABA问题、不能保证代码块具有原子性

AtomicInteger 具有原子性操作的类

Lock 底层原理

lock 接口有ReentrantLock实现类:lock() unlock();

AQS:AbstractQueuedSynchronizer 抽象队列同步器

FIFO --> 队列 队列中存储很多线程,这些线程共同抢占一个资源(volatile 修饰的 state)

上锁本质:state+1 解锁本质:state-1

//	上锁 本质 获取state +1
final void lock() {
	//	比较交换
	if (compareAndSetState(0, 1))
		//	设置用户线程
		setExclusiveOwnerThread(Thread.currentThread());
	else
		acquire(1);
}

final boolean nonfairTryAcquire(int acquires) {
	final Thread current = Thread.currentThread();
	//	获取资源
	int c = getState();
	if (c == 0) {
		if (compareAndSetState(0, acquires)) {
			setExclusiveOwnerThread(current);
			return true;
		}
	}
	else if (current == getExclusiveOwnerThread()) {  // 可重入
		//	如果可重入在原理基础上+1
		int nextc = c + acquires;
		if (nextc < 0) // overflow
			throw new Error("Maximum lock count exceeded");
		setState(nextc);
		return true;
	}
	return false;
}

//	解锁本质:state - 1
public void unlock() {
	sync.release(1);
}
protected final boolean tryRelease(int releases) {
	int c = getState() - releases;
	if (Thread.currentThread() != getExclusiveOwnerThread())
		throw new IllegalMonitorStateException();
	boolean free = false;
	if (c == 0) {
		free = true;
		setExclusiveOwnerThread(null);
	}
	setState(c);
	return free;
}

Day143-145.尚品汇:商品上架、ES商品检索 | 动态DSL、搜索条件处理、logstash_第4张图片

2. DSL 商品检索

聚合的目的: 去重显示、根据聚合的数据进行过滤

    //商品检索
    @Override
    public SearchResponseVo search(SearchParam searchParam) throws IOException {
        //1.生成dsl语句
        //2.执行dsl
        //3.将执行之后的结果进行封装
        //声明一个查询请求对象
        SearchRequest searchRequest = this.buildDsl(searchParam);
        //执行dsl语句
        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);

        //将执行之后的结果进行封装 SearchResponseVo
        SearchResponseVo searchResponseVo = this.parseResult(searchResponse);
        /*
            private List trademarkList;
            private List attrsList = new ArrayList<>();
            private List goodsList = new ArrayList<>();
            private Long total;
             ------ 以上四个方法在parseResult中赋值
            private Integer pageSize;
            private Integer pageNo;
            private Long totalPages;
         */
        //设置每页默认显示条数:3
        searchResponseVo.setPageSize(searchParam.getPageSize());
        searchResponseVo.setPageNo(searchParam.getPageNo());
        /*Long totalPages = searchResponseVo.getTotal()%searchParam.getPageSize()==0?
                searchResponseVo.getTotal()%searchParam.getPageSize():
                searchResponseVo.getTotal()%searchParam.getPageSize()+1;*/
        // from = (pageNo-1)*PageSize
        Long totalPages = (searchResponseVo.getTotal()+searchParam.getPageSize()-1)/searchParam.getPageSize();
        searchResponseVo.setTotalPages(totalPages);

        return searchResponseVo;
    }

    //动态生成dsl语句
    private SearchRequest buildDsl(SearchParam searchParam) {

        // 构建查询器 {}
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        //{bool}
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();

        // 判断男用户是否根据三级分类id进行检索
        if(!StringUtils.isEmpty(searchParam.getCategory3Id())){
            //{filter term}
            boolQueryBuilder.filter(QueryBuilders.termQuery("category3Id",searchParam.getCategory3Id()));
        }
        if(!StringUtils.isEmpty(searchParam.getCategory2Id())){
            boolQueryBuilder.filter(QueryBuilders.termQuery("category2Id",searchParam.getCategory2Id()));
        }
        if(!StringUtils.isEmpty(searchParam.getCategory1Id())){
            boolQueryBuilder.filter(QueryBuilders.termQuery("category1Id",searchParam.getCategory1Id()));
        }
        //根据关键词检索
        if(!StringUtils.isEmpty(searchParam.getKeyword())){
            boolQueryBuilder.must(QueryBuilders.matchQuery("title",searchParam.getKeyword()).operator(Operator.AND));
            //高亮
            HighlightBuilder highlightBuilder = new HighlightBuilder();
            highlightBuilder.field("title").preTags("").postTags("");
            //HighlightBuilder highlightBuilder = searchSourceBuilder.highlighter()
            //        .field("title").preTags("").postTags("");
            searchSourceBuilder.highlighter(highlightBuilder);
        }

        //根据平台属性值进行过滤、
        //先获取到用户点击的数据
        String[] props = searchParam.getProps();
        if(props!=null && props.length>0){
            for (String prop : props) {
                //  第一次遍历 prop=24:256G:机身内存  第二次遍历 prop=106:安卓手机:手机一级
                String[] split = prop.split(":");
                // 声明中间层的bool
                BoolQueryBuilder boolBuilder = QueryBuilders.boolQuery();
                // 声明内层的bool
                BoolQueryBuilder innerBoolBuilder = QueryBuilders.boolQuery();
                // 设置内层
                innerBoolBuilder.must(QueryBuilders.matchQuery("attrs.attrId",split[0]));
                innerBoolBuilder.must(QueryBuilders.matchQuery("attrs.attrValue",split[1]));
                // 设置中间层
                boolBuilder.must(QueryBuilders.nestedQuery("attrs",innerBoolBuilder, ScoreMode.None));
                boolQueryBuilder.filter(boolBuilder);
                //
            }
        }

        //根据品牌Id 进行检索: trademark=3:华为
        String trademark = searchParam.getTrademark();
        if (!StringUtils.isEmpty(trademark)) {
            //分割
            String[] split = trademark.split(":");
            //判断是否根据品牌检索
            if(split!=null && split.length==2){
                //{filter term}
                boolQueryBuilder.filter(QueryBuilders.termQuery("tmId",split[0]));
            }
        }
        //{query}
        searchSourceBuilder.query(boolQueryBuilder);

        //分页
        //from 当前的起始条数
        int from = (searchParam.getPageNo()-1)*searchParam.getPageSize();
        searchSourceBuilder.from(from);
        //默认展示三条
        searchSourceBuilder.size(searchParam.getPageSize());

        //排序:规则表圛,是根据页面传递的数据决定的
        //order=1:asc order2:desc
        String order = searchParam.getOrder();
        if(!StringUtils.isEmpty(order)){
            String[] split = order.split(":");
            //需要知道排序的字段与规则 1=hostScore 2=price
            String field = "";
            if(split!=null && split.length==2){
                switch (split[0]){
                    case "1":
                        field = "hotScore";
                        //searchSourceBuilder.sort("hotScore","asc".equals(split[1])? SortOrder.ASC:SortOrder.DESC);
                        break;
                    case "2":
                        field = "price";
                        //searchSourceBuilder.sort("price","asc".equals(split[1])? SortOrder.ASC:SortOrder.DESC);
                        break;
                }
                searchSourceBuilder.sort(field,"asc".equals(split[1])? SortOrder.ASC:SortOrder.DESC);
            }
        } else {
            //默认热度 倒序
            searchSourceBuilder.sort("hotScore",SortOrder.DESC);
        }

        //聚合: 去重,显示,提供检索条件
        //第一部分:品牌  注意结构!并列关系!
        searchSourceBuilder.aggregation(AggregationBuilders.terms("tmIdAgg").field("tmId")
                .subAggregation(AggregationBuilders.terms("tmNameAgg").field("tmName"))
                .subAggregation(AggregationBuilders.terms("tmLogoUrlAgg").field("tmLogoUrl")));

        //第二部分:平台属性 -- 数据类型 nested
        searchSourceBuilder.aggregation(AggregationBuilders.nested("attrAgg","attrs")
                .subAggregation(AggregationBuilders.terms("attrIdAgg").field("attrs.attrId")
                        .subAggregation(AggregationBuilders.terms("attrNameAgg").field("attrs.attrName"))
                        .subAggregation(AggregationBuilders.terms("attrValueAgg").field("attrs.attrValue"))));

        //声明一个请求对象 GET /goods/_search 在哪个索引库查询
        SearchRequest searchRequest = new SearchRequest("goods");
        //将dsl语句赋值给查询对象
        searchRequest.source(searchSourceBuilder);
        //看到dsl语句
        System.out.println("DSL:\t"+searchSourceBuilder.toString());
        //哪些field需要展示数据,哪些不需要展示数据
        searchSourceBuilder.fetchSource(new String[]{"id","defaultImg","title","price","createTime"},null);
        return searchRequest;
    }

    //进行结果封装 返回数据
    private SearchResponseVo parseResult(SearchResponse searchResponse) {
        SearchResponseVo searchResponseVo = new SearchResponseVo();
        SearchHits hits = searchResponse.getHits();
        //设置总记录数
        //private Long total;
        searchResponseVo.setTotal(hits.getTotalHits().value);

        //设置商品的集合
        //private List goodsList = new ArrayList<>();
        ArrayList goodsList = new ArrayList<>();
        SearchHit[] subHits = hits.getHits();
        if(subHits!=null && subHits.length>0){
            for (SearchHit subHit : subHits) {
                //获取goods字符串
                String sourceAsString = subHit.getSourceAsString();
                //将其转换为goods
                Goods goods = JSON.parseObject(sourceAsString, Goods.class);
                //判断用户是否根据关键词进行检索,如果是则获取高亮字段
                if(subHit.getHighlightFields().get("title")!=null){
                    Text title = subHit.getHighlightFields().get("title").getFragments()[0];
                    goods.setTitle(title.toString());
                }
                goodsList.add(goods);
            }
        }
        searchResponseVo.setGoodsList(goodsList);

        //设置品牌集合
        //private List trademarkList;
        Map aggregationMap = searchResponse.getAggregations().asMap();
        // 应获取桶里的数据! 多态 根据key类型决定ParsedLongTerms
        ParsedLongTerms tmIdAgg = (ParsedLongTerms) aggregationMap.get("tmIdAgg");
        //循环遍历时获取品牌ID给searchResponseVo,再将对象添加到集合中
        List trademarkList = tmIdAgg.getBuckets().stream().map((bucket) -> {
            SearchResponseTmVo searchResponseTmVo = new SearchResponseTmVo();
            //获取品牌ID
            String tmId = ((Terms.Bucket) bucket).getKeyAsString();
            searchResponseTmVo.setTmId(Long.parseLong(tmId));

            //获取品牌Name
            ParsedStringTerms tmNameAgg = bucket.getAggregations().get("tmNameAgg");
            String tmName = tmNameAgg.getBuckets().get(0).getKeyAsString();
            searchResponseTmVo.setTmName(tmName);

            //获取品牌LogoUrl
            ParsedStringTerms tmLogoUrlAgg = bucket.getAggregations().get("tmLogoUrlAgg");
            String tmLogoUrl = tmLogoUrlAgg.getBuckets().get(0).getKeyAsString();
            searchResponseTmVo.setTmLogoUrl(tmLogoUrl);
            return searchResponseTmVo;
        }).collect(Collectors.toList());
        searchResponseVo.setTrademarkList(trademarkList);

        //平台属性值: 数据类型nested
        //private List attrsList = new ArrayList<>();
        ParsedNested attrAgg = (ParsedNested) aggregationMap.get("attrAgg");
        ParsedLongTerms attrIdAgg = attrAgg.getAggregations().get("attrIdAgg");
        //循环遍历获取平台属性ID、平台属性名、平台属性值集合
        List attrsList = attrIdAgg.getBuckets().stream().map(bucket -> {
            //声明一个平台属性对象
            SearchResponseAttrVo searchResponseAttrVo = new SearchResponseAttrVo();
            //获取平台属性ID
            String attrId = bucket.getKeyAsString();
            searchResponseAttrVo.setAttrId(Long.parseLong(attrId));

            //获取平台属性名
            ParsedStringTerms attrNameAgg = bucket.getAggregations().get("attrNameAgg");
            String attrName = attrNameAgg.getBuckets().get(0).getKeyAsString();
            searchResponseAttrVo.setAttrName(attrName);

            //获取平台属性值集合
            ParsedStringTerms attrValueAgg = bucket.getAggregations().get("attrValueAgg");

            /*List buckets = attrValueAgg.getBuckets();
            ArrayList list = new ArrayList<>();
            buckets.forEach(value->{
                String ValueName = value.getKeyAsString();
                list.add(ValueName);
            });*/

            List valueList = attrValueAgg.getBuckets().stream().map(Terms.Bucket::getKeyAsString).collect(Collectors.toList());
            searchResponseAttrVo.setAttrValueList(valueList);

            return searchResponseAttrVo;
        }).collect(Collectors.toList());

        searchResponseVo.setAttrsList(attrsList);

        return searchResponseVo;
    }

}

3. 搜索条件处理 - url拼接

网关 -》 域名解析 Gateway-》OpenFeign service-inem

//商品检索
@Controller
public class ListController {

    @Qualifier("com.atguigu.gmall.list.client.ListFeignClient")
    @Autowired
    private ListFeignClient listFeignClient;

    //通过浏览器会将用户输入的检索条件,直接封装到这个实体类!  --- 为什么能封装进去? 因为参数名一样
    @GetMapping("list.html")
    public String search(SearchParam searchParam, Model model) {
        //  urlParam searchParam trademarkParam propsParamList orderMap 需要自己组装!
        //  trademarkList attrsList goodsList pageNo totalPages  这些数据都是实体类的属性 SearchResponseVo
        //  面包屑:trademarkParam propsParamList
        Result result = listFeignClient.list(searchParam);

        // urlParam 表示点击平台拼接属性值之前的url路径
        String urlParam = this.makeUrlParam(searchParam);
        //品牌面包屑
        String trademarkParam = this.makeTradeMarkParam(searchParam.getTrademark());
        //平台属性面包屑:
        List searchAttrList = this.makeSearchAttr(searchParam.getProps());
        //List searchAttrList = this.makeSearchAttr(searchParam.getProps());

        //排序规则
        Map orderMap = this.makeOrderMap(searchParam.getOrder());

        //map 可以充当对象,poenFeign远程调用时
        model.addAllAttributes(result.getData());
        model.addAttribute("searchParam",searchParam);
        model.addAttribute("urlParam",urlParam);
        model.addAttribute("trademarkParam",trademarkParam);
        model.addAttribute("propsParamList",searchAttrList);
        model.addAttribute("orderMap",orderMap);

        return "list/index";
    }
    //制作排序
    private Map makeOrderMap(String order) {
        //  order=1:asc  order=1:desc   order=2:asc order=2:desc
        HashMap map = new HashMap<>();
        if(!StringUtils.isEmpty(order)){
            String[] split = order.split(":");
            //用户点击了排序
            if(split!=null && split.length==2){
                //按照哪种方式排序
                map.put("type",split[0]);
                //排序规则 asc 或 desc
                map.put("sort",split[1]);
            }
        }else {
            map.put("type",1);
            //排序规则 asc 或 desc
            map.put("sort","desc");
        }
        return map;
    }

    //平台属性面包屑集合
    private List makeSearchAttr(String[] props) {
        ArrayList searchAttrList = new ArrayList<>();
        //  props=24:256G:机身内存&props=23:8G:运行内存
        if(props!=null && props.length>0){
            for (String prop : props) {
                String[] split = prop.split(":");
                if(split!=null && split.length==3){
                    SearchAttr searchAttr = new SearchAttr();
                    searchAttr.setAttrId(Long.parseLong(split[0]));
                    searchAttr.setAttrValue(split[1]);
                    searchAttr.setAttrName(split[2]);
                    searchAttrList.add(searchAttr);
                }
            }
            return searchAttrList;
        }
        return null;
    }

    //品牌面包屑
    private String makeTradeMarkParam(String trademark) {
        // trademark=1:小米
        if(!StringUtils.isEmpty(trademark)){
            String[] split = trademark.split(":");
            if(split!=null && split.length==2){
                return "品牌:"+trademark.split(":")[1];
            }
        }
        return null;
    }

    //记录用户通过哪些条件进行了检索
    private String makeUrlParam(SearchParam searchParam) {
        // 字符串拼接
        StringBuilder stringBuilder = new StringBuilder();
        // 判断用户根据哪些条件进行了检索
        //全文检索
        if(!StringUtils.isEmpty(searchParam.getKeyword())){
            stringBuilder.append("keyword=").append(searchParam.getKeyword());
        }
        //分类ID
        if(!StringUtils.isEmpty(searchParam.getCategory3Id())){
            stringBuilder.append("category3Id=").append(searchParam.getCategory3Id());
        }
        if(!StringUtils.isEmpty(searchParam.getCategory2Id())){
            stringBuilder.append("category2Id=").append(searchParam.getCategory2Id());
        }
        if(!StringUtils.isEmpty(searchParam.getCategory1Id())){
            stringBuilder.append("category1Id=").append(searchParam.getCategory1Id());
        }
        //品牌
        if(!StringUtils.isEmpty(searchParam.getTrademark())){
            if(stringBuilder.length()>0){
                stringBuilder.append("&trademark=").append(searchParam.getTrademark());
            }
        }
        //平台属性值
        String[] props = searchParam.getProps();
        if (props != null && props.length > 0) {
            // 循环遍历
            for (String prop : props) {
                if(stringBuilder.length()>0){
                    stringBuilder.append("&props=").append(prop);
                }
            }
        }
        System.out.println("list.html?"+stringBuilder.toString());
        return "list.html?"+stringBuilder.toString();
    }
}

4. logstash: 日志收集框架:


        日志需要有固定存储!
    ELK:    
        安装:
        添加配置文件:
        添加logstash 的依赖jar包。

Day143-145.尚品汇:商品上架、ES商品检索 | 动态DSL、搜索条件处理、logstash_第5张图片

 Day143-145.尚品汇:商品上架、ES商品检索 | 动态DSL、搜索条件处理、logstash_第6张图片

你可能感兴趣的:(尚品汇,java,数据库,服务器)