nosql介绍
nosql,全称是 not only sql, 即“不仅于sql”,相较于关系型数据库,nosql更加灵活,无需去维护复杂的数据关系。数据是json格式,更加直观易读。
mongodb是应用最广泛的一种nosql数据库
安装mongodb
以mac为例:
☁ ~ brew install mongodb
开启服务
☁ ~ sudo brew services start mongo
Password:
==> Successfully started `mongodb` (label: homebrew.mxcl.mongodb)
连续服务
☁ ~ mongo
MongoDB shell version v3.4.2
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 4.0.2
WARNING: shell and server versions do not match
Server has startup warnings:
2018-09-08T10:56:57.451+0800 I CONTROL [initandlisten]
2018-09-08T10:56:57.451+0800 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2018-09-08T10:56:57.451+0800 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2018-09-08T10:56:57.451+0800 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2018-09-08T10:56:57.451+0800 I CONTROL [initandlisten]
配置文件
☁ mysite cat /usr/local/etc/mongod.conf
systemLog:
destination: file
path: /usr/local/var/log/mongodb/mongo.log
logAppend: true
storage:
dbPath: /usr/local/var/mongodb
net:
bindIp: 127.0.0.1
库操作
查看所有库
> show dbs;
admin 0.000GB
config 0.000GB
local 0.000GB
使用库
> use test; # 如果库不存在,会自动创建该库
switched to db test
> show dbs; # 新建的库没有数据,是不会显示在库列表中的
admin 0.000GB
config 0.000GB
local 0.000GB
删除库
> use test
switched to db test
> db.dropDatabase()
{ "ok" : 1 }
集合操作
创建集合
> db.createCollection('col1')
{ "ok" : 1 }
显示集合
> show collections;
col1
删除集合
> db.col1.drop()
true
文档操作
插入文档
> db.col.insert({ # 创建文档时,如果集合不存在,会自动创建集合
... 'name': '郭靖',
... 'gender': true,
... 'age': 20
... });
WriteResult({ "nInserted" : 1 })
查询文档
> db.col.find() # 查出所有文档
{ "_id" : ObjectId("5b933ce95032d051d0771953") }
{ "_id" : ObjectId("5b933ced5032d051d0771954") }
{ "_id" : ObjectId("5b933d1f565bbd5e857dbd89"), "name" : "郭靖", "gender" : true, "age" : 20 }
> db.col.find({name: '郭靖'}) # 根据条件筛选文档
{ "_id" : ObjectId("5b933d1f565bbd5e857dbd89"), "name" : "郭靖", "gender" : true, "age" : 20 }
更新文档
> db.col.update({'name': '郭靖'}, {$set: {'age': 25}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.col.find({name: '郭靖'})
{ "_id" : ObjectId("5b933d1f565bbd5e857dbd89"), "name" : "郭靖", "gender" : true, "age" : 25 }
删除文档
> db.col.remove({'name': '郭靖'})
WriteResult({ "nRemoved" : 1 })
数据类型
名称 | 释义 |
---|---|
Object ID | 文档ID |
String | 字符串,最常用,必须是有效的UTF-8 |
Boolean | 存储一个布尔值,true或false |
Integer | 整数可以是32位或64位,这取决于服务器 |
Double | 存储浮点值 |
Arrays | 数组或列表,多个值存储到一个键 |
Object | 用于嵌入式的文档,即一个值为一个文档 |
Null | 存储Null值 |
Timestamp | 时间戳 |
Date | 存储当前日期或时间的UNIX时间格式 |
object id 每个文档都有一个属性,为_id,保证每个文档的唯一性, objectID是一个12字节的十六进制数
前4个字节为当前时间戳
接下来3个字节的机器ID
接下来的2个字节中MongoDB的服务进程id
最后3个字节是简单的增量值
查询进阶
数据初始化
let data = [
{'name':'郭靖', 'age':20, 'skill':'降龙十八掌', 'gender':true},
{'name':'黄蓉', 'age':18, 'skill':'桃花岛武功', 'gender':false},
{'name':'黄药师', 'age':58, 'skill':'碧海潮生曲', 'gender':true},
{'name':'一灯大师', 'age':60, 'skill':'一阳指', 'gender':true},
{'name':'小龙女', 'age':25, 'skill':'玉女心经', 'gender':false},
{'name':'李莫愁', 'age':30, 'skill':'赤练神掌', 'gender':false},
{'name':'乔峰', 'age':35, 'skill':'降龙十八掌', 'gender':true},
{'name':'王语嫣', 'age':22, 'skill':'懂得各派武功', 'gender':false},
]
for (i in data) {
data[i]
db.person.insert(data[i])
}
- findOne
> db.person.findOne(); // 只查询一条
{
"_id" : ObjectId("5b9363badfee996b08be20af"),
"name" : "郭靖",
"age" : 20,
"skill" : "降龙十八掌",
"gender" : true
}
> db.person.findOne({'name': '小龙女'});
{
"_id" : ObjectId("5b9363badfee996b08be20b3"),
"name" : "小龙女",
"age" : 25,
"skill" : "玉女心经",
"gender" : false
}
- 比较运算
> db.person.find({age: {$gt: 50}}) // 大于
{ "_id" : ObjectId("5b9363badfee996b08be20b1"), "name" : "黄药师", "age" : 58, "skill" : "碧海潮生曲", "gender" : true }
{ "_id" : ObjectId("5b9363badfee996b08be20b2"), "name" : "一灯大师", "age" : 60, "skill" : "一阳指", "gender" : true }
> db.person.find({age: {$gte: 60}}) // 大于等于
{ "_id" : ObjectId("5b9363badfee996b08be20b2"), "name" : "一灯大师", "age" : 60, "skill" : "一阳指", "gender" : true }
> db.person.find({age: {$lt: 20}}) // 小于
{ "_id" : ObjectId("5b9363badfee996b08be20b0"), "name" : "黄蓉", "age" : 18, "skill" : "桃花岛武功", "gender" : false }
> db.person.find({age: {$lte: 20}}) // 小于等于
{ "_id" : ObjectId("5b9363badfee996b08be20af"), "name" : "郭靖", "age" : 20, "skill" : "降龙十八掌", "gender" : true }
{ "_id" : ObjectId("5b9363badfee996b08be20b0"), "name" : "黄蓉", "age" : 18, "skill" : "桃花岛武功", "gender" : false }
- 逻辑运算
> db.person.find({$or:[{name:'郭靖'}, {age: {$gt: 50}}]}) // or查询
{ "_id" : ObjectId("5b9363badfee996b08be20af"), "name" : "郭靖", "age" : 20, "skill" : "降龙十八掌", "gender" : true }
{ "_id" : ObjectId("5b9363badfee996b08be20b1"), "name" : "黄药师", "age" : 58, "skill" : "碧海潮生曲", "gender" : true }
{ "_id" : ObjectId("5b9363badfee996b08be20b2"), "name" : "一灯大师", "age" : 60, "skill" : "一阳指", "gender" : true }
> db.person.find({age: {$in: [18, 20]}}) // in查询
{ "_id" : ObjectId("5b9363badfee996b08be20af"), "name" : "郭靖", "age" : 20, "skill" : "降龙十八掌", "gender" : true }
{ "_id" : ObjectId("5b9363badfee996b08be20b0"), "name" : "黄蓉", "age" : 18, "skill" : "桃花岛武功", "gender" : false }
- 正则查询
> db.person.find({name: /^小/})
{ "_id" : ObjectId("5b9363badfee996b08be20b3"), "name" : "小龙女", "age" : 25, "skill" : "玉女心经", "gender" : false }
- js函数查询
> db.person.find({$where:function(){return this.age>40}})
{ "_id" : ObjectId("5b9363badfee996b08be20b1"), "name" : "黄药师", "age" : 58, "skill" : "碧海潮生曲", "gender" : true }
{ "_id" : ObjectId("5b9363badfee996b08be20b2"), "name" : "一灯大师", "age" : 60, "skill" : "一阳指", "gender" : true }
> db.person.find({$where:function(){return this.skill.indexOf('掌')>=0}})
{ "_id" : ObjectId("5b9363badfee996b08be20af"), "name" : "郭靖", "age" : 20, "skill" : "降龙十八掌", "gender" : true }
{ "_id" : ObjectId("5b9363badfee996b08be20b4"), "name" : "李莫愁", "age" : 30, "skill" : "赤练神掌", "gender" : false }
{ "_id" : ObjectId("5b9363badfee996b08be20b5"), "name" : "乔峰", "age" : 35, "skill" : "降龙十八掌", "gender" : true }
- skip与limit
> db.person.find().skip(1).limit(1)
{ "_id" : ObjectId("5b9363badfee996b08be20b0"), "name" : "黄蓉", "age" : 18, "skill" : "桃花岛武功", "gender" : false }
> db.person.find().limit(1).skip(1)
{ "_id" : ObjectId("5b9363badfee996b08be20b0"), "name" : "黄蓉", "age" : 18, "skill" : "桃花岛武功", "gender" : false }
skip与limit可单独,也可组合使用。虽然在这种查询条件下,两者的顺序不会影响结果。但推荐使用skip().limit()的顺序。因为在聚合查询时两者的顺序不同会导致结果不同。
- 显示字段
> db.person.find({}, {name: 1}); // 查询条件为空时,也要有{}空json,_id默认是显示的
{ "_id" : ObjectId("5b9363badfee996b08be20af"), "name" : "郭靖" }
{ "_id" : ObjectId("5b9363badfee996b08be20b0"), "name" : "黄蓉" }
{ "_id" : ObjectId("5b9363badfee996b08be20b1"), "name" : "黄药师" }
{ "_id" : ObjectId("5b9363badfee996b08be20b2"), "name" : "一灯大师" }
{ "_id" : ObjectId("5b9363badfee996b08be20b3"), "name" : "小龙女" }
{ "_id" : ObjectId("5b9363badfee996b08be20b4"), "name" : "李莫愁" }
{ "_id" : ObjectId("5b9363badfee996b08be20b5"), "name" : "乔峰" }
{ "_id" : ObjectId("5b9363badfee996b08be20b6"), "name" : "王语嫣" }
> db.person.find({}, {name: 1, _id: 0}); // 不显示_id
{ "name" : "郭靖" }
{ "name" : "黄蓉" }
{ "name" : "黄药师" }
{ "name" : "一灯大师" }
{ "name" : "小龙女" }
{ "name" : "李莫愁" }
{ "name" : "乔峰" }
{ "name" : "王语嫣" }
- sort排序
> db.person.find({}, {_id:0, name:1, age:1}).sort({age: 1}); // 正序
{ "name" : "黄蓉", "age" : 18 }
{ "name" : "郭靖", "age" : 20 }
{ "name" : "王语嫣", "age" : 22 }
{ "name" : "小龙女", "age" : 25 }
{ "name" : "李莫愁", "age" : 30 }
{ "name" : "乔峰", "age" : 35 }
{ "name" : "黄药师", "age" : 58 }
{ "name" : "一灯大师", "age" : 60 }
> db.person.find({}, {_id:0, name:1, age:1}).sort({age: -1}); // 倒序
{ "name" : "一灯大师", "age" : 60 }
{ "name" : "黄药师", "age" : 58 }
{ "name" : "乔峰", "age" : 35 }
{ "name" : "李莫愁", "age" : 30 }
{ "name" : "小龙女", "age" : 25 }
{ "name" : "王语嫣", "age" : 22 }
{ "name" : "郭靖", "age" : 20 }
{ "name" : "黄蓉", "age" : 18 }
- count统计
> db.person.count()
8
> db.person.count({age: {$gt: 20}})
6
- distinct过滤重复
> db.person.find({age: {$gt: 20}}, {_id:0, gender:1})
{ "gender" : true }
{ "gender" : true }
{ "gender" : false }
{ "gender" : false }
{ "gender" : true }
{ "gender" : false }
> db.person.distinct('gender', {age: {$gt: 20}})
[ true, false ]
聚合
$group分组
// 按gender字段进行分组
db.person.aggregate([
{$group:{_id:'$gender'}}
]);
{ "_id" : false }
{ "_id" : true }
// 分组后再进行求和统计:求各个组的记录条数
db.person.aggregate([
{$group:{_id:'$gender', counter:{$sum: 1}}}
]);
{ "_id" : false, "counter" : 4 }
{ "_id" : true, "counter" : 4 }
// 指定某个字段求和
db.person.aggregate([
{$group:{_id:'$gender', counter:{$sum: '$age'}}}
]);
{ "_id" : false, "counter" : 95 }
{ "_id" : true, "counter" : 173 }
// 将分组中指定字段的值归为一个数组
db.person.aggregate([
{$group:{_id:'$gender', counter:{$push: '$name'}}}
]);
{ "_id" : false, "counter" : [ "黄蓉", "小龙女", "李莫愁", "王语嫣" ] }
{ "_id" : true, "counter" : [ "郭靖", "黄药师", "一灯大师", "乔峰" ] }
// 将分组中包括的文档归为一个数组
db.person.aggregate([
{$group:{_id:'$gender', counter:{$push: '$$ROOT'}}}
]);
{ "_id" : false, "counter" : [ { "_id" : ObjectId("5b9363badfee996b08be20b0"), "name" : "黄蓉", "age" : 18, "skill" : "桃花岛武功", "gender" : false }, { "_id" : ObjectId("5b9363badfee996b08be20b3"), "name" : "小龙女", "age" : 25, "skill" : "玉女心经", "gender" : false }, { "_id" : ObjectId("5b9363badfee996b08be20b4"), "name" : "李莫愁", "age" : 30, "skill" : "赤练神掌", "gender" : false }, { "_id" : ObjectId("5b9363badfee996b08be20b6"), "name" : "王语嫣", "age" : 22, "skill" : "懂得各派武功", "gender" : false } ] }
{ "_id" : true, "counter" : [ { "_id" : ObjectId("5b9363badfee996b08be20af"), "name" : "郭靖", "age" : 20, "skill" : "降龙十八掌", "gender" : true }, { "_id" : ObjectId("5b9363badfee996b08be20b1"), "name" : "黄药师", "age" : 58, "skill" : "碧海潮生曲", "gender" : true }, { "_id" : ObjectId("5b9363badfee996b08be20b2"), "name" : "一灯大师", "age" : 60, "skill" : "一阳指", "gender" : true }, { "_id" : ObjectId("5b9363badfee996b08be20b5"), "name" : "乔峰", "age" : 35, "skill" : "降龙十八掌", "gender" : true } ] }
$match匹配
// 匹配age大于20的文档
db.person.aggregate([
{$match: {age:{$gt:20}}},
]);
{ "_id" : ObjectId("5b9363badfee996b08be20b1"), "name" : "黄药师", "age" : 58, "skill" : "碧海潮生曲", "gender" : true }
{ "_id" : ObjectId("5b9363badfee996b08be20b2"), "name" : "一灯大师", "age" : 60, "skill" : "一阳指", "gender" : true }
{ "_id" : ObjectId("5b9363badfee996b08be20b3"), "name" : "小龙女", "age" : 25, "skill" : "玉女心经", "gender" : false }
{ "_id" : ObjectId("5b9363badfee996b08be20b4"), "name" : "李莫愁", "age" : 30, "skill" : "赤练神掌", "gender" : false }
{ "_id" : ObjectId("5b9363badfee996b08be20b5"), "name" : "乔峰", "age" : 35, "skill" : "降龙十八掌", "gender" : true }
{ "_id" : ObjectId("5b9363badfee996b08be20b6"), "name" : "王语嫣", "age" : 22, "skill" : "懂得各派武功", "gender" : false }
// 在匹配的基础上再进行分组统计
db.person.aggregate([
{$match: {age:{$gt:20}}},
{$group: {_id:'$gender', counter:{$sum:1}}}
]);
{ "_id" : false, "counter" : 3 }
{ "_id" : true, "counter" : 3 }
$project显示字段
db.person.aggregate([
{$match: {age:{$gt:20}}},
{$group: {_id:'$gender', counter:{$sum:1}}},
{$project: {_id:0, counter:1}}
]);
{ "counter" : 3 }
{ "counter" : 3 }
$sort排序
db.person.aggregate([
{$match: {age: {$gt: 20}}},
{$group: {_id: '$gender', counter: {$sum:1}}},
{$project: {counter: 1}},
{$sort: {_id: -1}}
]);
{ "_id" : true, "counter" : 3 }
{ "_id" : false, "counter" : 3 }
limit
// skip 和 limit 在聚合时有顺序区分的。开发时养成先skip,再limit的习惯
db.person.aggregate([
{$match: {age: {$gt: 20}}},
{$group: {_id: '$gender', counter: {$sum:1}}},
{$project: {counter: 1}},
{$sort: {_id: -1}},
{$skip: 1},
{$limit: 1}
]);
{ "_id" : false, "counter" : 3 }
db.person.aggregate([
{$match: {age: {$gt: 20}}},
{$group: {_id: '$gender', counter: {$sum:1}}},
{$project: {counter: 1}},
{$sort: {_id: -1}},
{$limit: 1},
{$skip: 1}
]);
// 结果为空
$unwind
将文档中的数组解开
db.shirt.insert({_id:1, title:'t-shirt', size:['M', 'L', 'S']});
db.shirt.aggregate([
{$unwind: '$size'}
]);
{ "_id" : 1, "title" : "t-shirt", "size" : "M" }
{ "_id" : 1, "title" : "t-shirt", "size" : "L" }
{ "_id" : 1, "title" : "t-shirt", "size" : "S" }
// unwind作用的字段为不同值时的情况
db.shirt.insert({_id: 2, title: 't2', size:[]});
db.shirt.insert({_id: 3, title: 't3'});
db.shirt.insert({_id: 4, title: 't4', size:null});
db.shirt.insert({_id: 5, title: 't5', size:'M'});
// 空值,没有size的数据丢了
db.shirt.aggregate([
{$unwind: '$size'}
]);
{ "_id" : 1, "title" : "t-shirt", "size" : "M" }
{ "_id" : 1, "title" : "t-shirt", "size" : "L" }
{ "_id" : 1, "title" : "t-shirt", "size" : "S" }
{ "_id" : 5, "title" : "t5", "size" : "M" }
// 阻止空值丢失
db.shirt.aggregate([
{$unwind: {path: '$size', preserveNullAndEmptyArrays:true}}
]);
{ "_id" : 1, "title" : "t-shirt", "size" : "M" }
{ "_id" : 1, "title" : "t-shirt", "size" : "L" }
{ "_id" : 1, "title" : "t-shirt", "size" : "S" }
{ "_id" : 2, "title" : "t2" }
{ "_id" : 3, "title" : "t3" }
{ "_id" : 4, "title" : "t4", "size" : null }
{ "_id" : 5, "title" : "t5", "size" : "M" }
索引
创建一个一百万文档的集合
for(i=0; i<1000000; i++){
db.test_index.insert({name:'test'+i, rank:i});
};
WriteResult({ "nInserted" : 1 })
在没有索引的情况下查找数据
db.test_index.find({name: 'test10000'});
{ "_id" : ObjectId("5b937532dfee996b08be47c7"), "name" : "test10000", "rank" : 10000 }
db.test_index.find({name: 'test10000'}).explain('executionStats'); // executionTimeMillis 值为 416 表示执行 416 毫秒
创建索引
db.test_index.ensureIndex({name: 1});
{
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"ok" : 1
}
再次分析性能
db.test_index.find({name: 'test10000'}).explain('executionStats'); // executionTimeMillis 降为 8 毫秒
用户权限管理
- 创建超级管理员
- 修改配置文件,启用身份验证
- 重启服务
- 使用超级管理员登录
- 创建普通用户
- 使用普通用户登录
创建用户
db.createUser({
user:'admin',
pwd:'admin123',
roles:[{role:'root', db:'admin'}],
passwordDigestor: 'server'
})
修改配置
security:
authorization: enabled
重启服务
sudo brew services restart mongo
直接用mongo
登录,无法使用show dbs
等命令
☁ ~ mongo
MongoDB shell version v3.4.2
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 4.0.2
WARNING: shell and server versions do not match
> db
test
> show dbs;
2018-09-08T09:51:53.531+0800 E QUERY [thread1] Error: listDatabases failed:{
"ok" : 0,
"errmsg" : "command listDatabases requires authentication",
"code" : 13,
"codeName" : "Unauthorized"
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:62:1
shellHelper.show@src/mongo/shell/utils.js:755:19
shellHelper@src/mongo/shell/utils.js:645:15
@(shellhelp2):1:1
使用超级管理员登录
☁ ~ mongo -u admin -p --authenticationDatabase admin
MongoDB shell version v3.4.2
Enter password:
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 4.0.2
WARNING: shell and server versions do not match
Server has startup warnings:
2018-09-08T09:51:35.534+0800 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2018-09-08T09:51:35.534+0800 I CONTROL [initandlisten]
> show dbs;
admin 0.000GB
config 0.000GB
local 0.000GB
py3 0.004GB
创建普通用户
db.createUser({
user:'py3',
pwd:'admin123',
roles:[{role:'readWrite', db:'py3'}],
passwordDigestor: 'server'
})
使用普通管理员登录
☁ ~ mongo -u py3 -p --authenticationDatabase py3
MongoDB shell version v3.4.2
Enter password:
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 4.0.2
WARNING: shell and server versions do not match
> show dbs;
py3 0.004GB
普通管理员只能在指定的数据库,无法登录其他数据库
☁ ~ mongo -u py3 -p --authenticationDatabase admin
MongoDB shell version v3.4.2
Enter password:
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 4.0.2
WARNING: shell and server versions do not match
2018-09-08T10:14:02.358+0800 E QUERY [thread1] Error: Authentication failed. :
DB.prototype._authOrThrow@src/mongo/shell/db.js:1459:20
@(auth):6:1
@(auth):1:2
exception: login failed