mongodb的多表联查与后续的数据处理

背景与简述

1. 背景

   使用nosql作多表操作时很麻烦的,所以平时都没使用过多表,但最近遇到一个项目必须使用多表,没法,就研究了一下mongodb的多表联查功能.
   mongodb的多表联查主要通过聚合完场,使用的是关键子$lookup,而后续处理中$unwind则是关键的一环.以下是这次的记录:

2. 版本

   mongodb:3.6
   spring:5.0.7
   spring-data:2.0.7
   mongo-java-driver:3.6.3
   版本问题需要注意一下,如果版本不兼容会出现:The ‘cursor’ option is required, except for aggregate…的问题,解决办法是升级版本,spring-data2.x的运行环境是spring5.x以及jdk8+,mongodb-java-driver也应该升级到3.6以上

数据

1. user表

{
    "_id" : ObjectId("5b69062240a6d80a6cece003"),
    "name" : "小明",
    "age" : 28,
    "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
    "_class" : "com.xiangpeng.bo.UserBo"
}

2. orders表

/* 1 */
{
    "_id" : ObjectId("5b69062240a6d80a6cece004"),
    "uid" : ObjectId("5b69062240a6d80a6cece003"),
    "money" : 10.0,
    "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
    "produce" : "产品1",
    "_class" : "com.xiangpeng.bo.OrderBo"
}

/* 2 */
{
    "_id" : ObjectId("5b6a5711c2eee4295c63768e"),
    "uid" : ObjectId("5b69062240a6d80a6cece003"),
    "money" : 20.0,
    "produce" : "产品2",
    "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
    "_class" : "com.xiangpeng.bo.OrderBo"
}

查询

1. mongodb的多表查询比较简单,使用$lookup关键字即可:

db.user.aggregate([{$lookup:{from:"orders",localField:"_id",foreignField:"uid",as:"orders"}}])

结果:

{
    "_id" : ObjectId("5b69062240a6d80a6cece003"),
    "name" : "小明",
    "age" : 28,
    "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
    "_class" : "com.xiangpeng.bo.UserBo",
    "orders" : [ 
        {
            "_id" : ObjectId("5b69062240a6d80a6cece004"),
            "uid" : ObjectId("5b69062240a6d80a6cece003"),
            "money" : 10.0,
            "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
            "produce" : "产品1",
            "_class" : "com.xiangpeng.bo.OrderBo"
        }, 
        {
            "_id" : ObjectId("5b6a5711c2eee4295c63768e"),
            "uid" : ObjectId("5b69062240a6d80a6cece003"),
            "money" : 20.0,
            "produce" : "产品2",
            "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
            "_class" : "com.xiangpeng.bo.OrderBo"
        }
    ]
}

---

参数解释:
form:需要关联的外表名,$lookup的多变查询使用的是左外连接
localField:本表的外表关联字段;
foreignField:外表的关联字段;
as:参考查询结果,使用$lookup进行查询后会将所有符合条件的文档封装为一个list,as参数定义这个list的名字;

数据处理

• 使用$unwind将数据打散:

db.user.aggregate([
		{$lookup:{from:"orders",localField:"_id",foreignField:"uid",as:"orders"},
		{$unwind:"$orders"}
])

$unwind的作用是将文档中的数组拆分为多条,拆分结果为:

/* 1 */
{
    "_id" : ObjectId("5b69062240a6d80a6cece003"),
    "name" : "小明",
    "age" : 28,
    "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
    "_class" : "com.xiangpeng.bo.UserBo",
    "orders" : {
        "_id" : ObjectId("5b69062240a6d80a6cece004"),
        "uid" : ObjectId("5b69062240a6d80a6cece003"),
        "money" : 10.0,
        "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
        "produce" : "产品1",
        "_class" : "com.xiangpeng.bo.OrderBo"
    }
}

/* 2 */
{
    "_id" : ObjectId("5b69062240a6d80a6cece003"),
    "name" : "小明",
    "age" : 28,
    "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
    "_class" : "com.xiangpeng.bo.UserBo",
    "orders" : {
        "_id" : ObjectId("5b6a5711c2eee4295c63768e"),
        "uid" : ObjectId("5b69062240a6d80a6cece003"),
        "money" : 20.0,
        "produce" : "产品2",
        "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
        "_class" : "com.xiangpeng.bo.OrderBo"
    }
}

• 数据过滤

  现在可以对数据进行过滤,数据过滤的步骤应该尽可能提前,但如果过滤条件中也需要筛选外表条件的话就没办法放前面了,过滤在聚合中使用$match

db.user.aggregate([
	{$lookup:{from:"orders",localField:"_id",foreignField:"uid",as:"orders"}},
	{$unwind:"$orders"},
	{$match:{name:"小明","orders.produce":"产品2"}}
])

查出小明买的产品2订单
结果展示

/* 1 */
{
    "_id" : ObjectId("5b69062240a6d80a6cece003"),
    "name" : "小明",
    "age" : 28,
    "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
    "_class" : "com.xiangpeng.bo.UserBo",
    "orders" : {
        "_id" : ObjectId("5b6a5711c2eee4295c63768e"),
        "uid" : ObjectId("5b69062240a6d80a6cece003"),
        "money" : 20.0,
        "produce" : "产品2",
        "createtime" : ISODate("2018-08-07T02:38:26.601Z"),
        "_class" : "com.xiangpeng.bo.OrderBo"
    }
}

• 如果对字段结果有要求可以使用$project进行字段筛选:

db.user.aggregate([
	{$lookup:{from:"orders",localField:"_id",foreignField:"uid",as:"orders"}},
	{$unwind:"$orders"},
	{$match:{name:"小明","orders.produce":"产品2"}},
	{$project:{name:"$name",age:"$age",produce:"$orders.produce",money:"$orders.money"}}])

再聚合中$可以用作引用相应字段的值
结果为:

/* 1 */
{
    "_id" : ObjectId("5b69062240a6d80a6cece003"),
    "name" : "小明",
    "age" : 28,
    "produce" : "产品2",
    "money" : 20.0
}

使用spring-data-mongodb在dao层的代码

@Repository("aggregateDao")
public class AggregateDaoImpl implements IAggregateDao {
	@Autowired
	private MongoTemplate mongoTemplate;

	public AggregationResults aggregateLookup() {
		// 创建条件
		AggregationOperation lookup = Aggregation.lookup("orders", "_id", "uid", "orders");
		AggregationOperation unwind = Aggregation.unwind("orders");
		AggregationOperation match = Aggregation.match(Criteria.where("name").is("小明").and("orders.produce").is("产品2"));
		AggregationOperation project = Aggregation.project("name", "age", "orders.produce", "orders.money");
		
		// 将条件封装到Aggregate管道
		Aggregation aggregation = Aggregation.newAggregation(lookup, unwind, match, project);
		
		// 查询
		AggregationResults aggregate = mongoTemplate.aggregate(aggregation, "user", Document.class);
		
		return aggregate;
	}
}

ps:一些小坑

1. $lookup是如果涉及关联"_id",注意两个字段的类型,用string类型匹配ObjectId类型是没有结果的~~
2. _class字段也是有些作用的,比如说使用$sum时用作分组
3. 数据处理后续:Document返回的值,如果用作前端返回,ObjectId是会被当成BSON解析的~