Go实战--golang中使用MongoDB(mgo)
生命不止，继续 go go go !!!
昨天分享了golang如何操作redis数据库，那今天就介绍一下golang中如何使用mongodb数据库。
何为MongoDB？
简介
MongoDB 是由C++语言编写的，是一个基于分布式文件存储的开源数据库系统。
在高负载的情况下，添加更多的节点，可以保证服务器性能。
MongoDB 旨在为WEB应用提供可扩展的高性能数据存储解决方案。
MongoDB 将数据存储为一个文档，数据结构由键值(key=>value)对组成。MongoDB 文档类似于 JSON 对象
MongoDB (from humongous) is a free and open-source cross-platform document-oriented database program. Classified as a NoSQL database program, MongoDB uses JSON-like documents with schemas. MongoDB is developed by MongoDB Inc. and is free and open-source, published under a combination of the GNU Affero General Public License and the Apache License.
特点
MongoDB的提供了一个面向文档存储，操作起来比较简单和容易。
你可以在MongoDB记录中设置任何属性的索引 (如：FirstName=”Sameer”,Address=”8 Gandhi Road”)来实现更快的排序。
你可以通过本地或者网络创建数据镜像，这使得MongoDB有更强的扩展性。
如果负载的增加（需要更多的存储空间和更强的处理能力） ，它可以分布在计算机网络中的其他节点上这就是所谓的分片。
Mongo支持丰富的查询表达式。查询指令使用JSON形式的标记，可轻易查询文档中内嵌的对象及数组。
MongoDb 使用update()命令可以实现替换完成的文档（数据）或者一些指定的数据字段 。
Mongodb中的Map/reduce主要是用来对数据进行批量处理和聚合操作。
Map和Reduce。Map函数调用emit(key,value)遍历集合中所有的记录，将key与value传给Reduce函数进行处理。
Map函数和Reduce函数是使用Javascript编写的，并可以通过db.runCommand或mapreduce命令来执行MapReduce操作。
GridFS是MongoDB中的一个内置功能，可以用于存放大量小文件。
MongoDB允许在服务端执行脚本，可以用Javascript编写某个函数，直接在服务端执行，也可以把函数的定义存储在服务端，下次直接调用即可。
MongoDB支持各种编程语言:RUBY，PYTHON，JAVA，C++，PHP，C#等多种语言。
MongoDB安装简单。
MongoDB与redis的对比
MongoDB和Redis都是NoSQL，采用结构型数据存储。二者在使用场景中，存在一定的区别，这也主要由于二者在内存映射的处理过程，持久化的处理方法不同。
MongoDB建议集群部署，更多的考虑到集群方案，Redis更偏重于进程顺序写入，虽然支持集群，也仅限于主-从模式。
When to use Redis?
Caching
Caching using MongoDB simply doesn’t make a lot of sense. It would be too slow.
If you have enough time to think about your DB design.
You can’t simply throw in your documents into Redis. You have to think of the way you in which you want to store and organize your data. One example are hashes in Redis. They are quite different from “traditional”, nested objects, which means you’ll have to rethink the way you store nested documents. One solution would be to store a reference inside the hash to another hash (something like key: [id of second hash]). Another idea would be to store it as JSON, which seems counter-intuitive to most people with a *SQL-background.
If you need really high performance.
Beating the performance Redis provides is nearly impossible. Imagine you database being as fast as your cache. That’s what it feels like using Redis as a real database.
If you don’t care that much about scaling.
Scaling Redis is not as hard as it used to be. For instance, you could use a kind of proxy server in order to distribute the data among multiple Redis instances. Master-slave replication is not that complicated, but distributing you keys among multiple Redis-instances needs to be done on the application site (e.g. using a hash-function, Modulo etc.). Scaling MongoDB by comparison is much simpler.
When to use MongoDB
Prototyping, Startups, Hackathons
MongoDB is perfectly suited for rapid prototyping. Nevertheless, performance isn’t that good. Also keep in mind that you’ll most likely have to define some sort of schema in your application.
When you need to change your schema quickly.
Because there is no schema! Altering tables in traditional, relational DBMS is painfully expensive and slow. MongoDB solves this problem by not making a lot of assumptions on your underlying data. Nevertheless, it tries to optimize as far as possible without requiring you to define a schema.
Windows下安装MongoDB
官网：
https://www.mongodb.com/
下载地址：
https://www.mongodb.com/download-center#community
把安装位置加入到环境变量中，例如：
D:\Program Files\MongoDB\Server\3.4\bin
命令行键入：
C:\Users\wangs>mongo.exe --version
MongoDB shell version v3.4.6
git version: c55eb86ef46ee7aede3b1e2a5d184a7df4bfb5b5
OpenSSL version: OpenSSL 1.0.1u-fips 22 Sep 2016
allocator: tcmalloc
modules: none
build environment:
distmod: 2008plus-ssl
distarch: x86_64
target_arch: x86_64启动mongodb服务
创建一个目录，然后设置为mongodb的data目录：
例如：D:\mongodb_data\db
命令行键入：
C:\Users\wangs>mongod.exe --dbpath d:\mongodb_data\db
2017-07-13T20:10:37.851-0700 I CONTROL [initandlisten] MongoDB starting : pid=5524 port=27017 dbpath=d:\mongodb_data\db 64-bit host=LAPTOP-MNU6522J
2017-07-13T20:10:37.851-0700 I CONTROL [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
2017-07-13T20:10:37.852-0700 I CONTROL [initandlisten] db version v3.4.6
2017-07-13T20:10:37.853-0700 I CONTROL [initandlisten] git version: c55eb86ef46ee7aede3b1e2a5d184a7df4bfb5b5
2017-07-13T20:10:37.853-0700 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1u-fips 22 Sep 2016
2017-07-13T20:10:37.853-0700 I CONTROL [initandlisten] allocator: tcmalloc
2017-07-13T20:10:37.853-0700 I CONTROL [initandlisten] modules: none
2017-07-13T20:10:37.853-0700 I CONTROL [initandlisten] build environment:
2017-07-13T20:10:37.853-0700 I CONTROL [initandlisten] distmod: 2008plus-ssl
2017-07-13T20:10:37.853-0700 I CONTROL [initandlisten] distarch: x86_64
2017-07-13T20:10:37.853-0700 I CONTROL [initandlisten] target_arch: x86_64
2017-07-13T20:10:37.853-0700 I CONTROL [initandlisten] options: { storage: { dbPath: "d:\mongodb_data\db" } }
2017-07-13T20:10:37.948-0700 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=3540M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2017-07-13T20:10:38.768-0700 I CONTROL [initandlisten]
2017-07-13T20:10:38.768-0700 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-07-13T20:10:38.770-0700 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2017-07-13T20:10:38.772-0700 I CONTROL [initandlisten]
2017-07-14T11:10:39.846+0800 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory 'd:/mongodb_data/db/diagnostic.data'
2017-07-14T11:10:40.094+0800 I INDEX [initandlisten] build index on: admin.system.version properties: { v: 2, key: { version: 1 }, name: "incompatible_with_version_32", ns: "admin.system.version" }
2017-07-14T11:10:40.094+0800 I INDEX [initandlisten] building index using bulk method; build may temporarily use up to 500 megabytes of RAM
2017-07-14T11:10:40.219+0800 I INDEX [initandlisten] build index done. scanned 0 total records. 0 secs
2017-07-14T11:10:40.221+0800 I COMMAND [initandlisten] setting featureCompatibilityVersion to 3.4
2017-07-14T11:10:40.282+0800 I NETWORK [thread1] waiting for connections on port 27017客户端连接mongodb
C:\Users\wangs>mongo.exe
MongoDB shell version v3.4.6
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.6
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
Server has startup warnings:
2017-07-13T20:13:28.423-0700 I CONTROL [initandlisten]
2017-07-13T20:13:28.457-0700 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-07-13T20:13:28.459-0700 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2017-07-13T20:13:28.486-0700 I CONTROL [initandlisten]创建数据库
> use superWANG
switched to db superWANG
> db
superWANG向数据库插入数据
> db.superWang.insert({"name":"golang"})
WriteResult({ "nInserted" : 1 })查看所有数据库
> show dbs
admin 0.000GB
local 0.000GB
superWANG 0.000GB删除数据库
> db.dropDatabase()
{ "dropped" : "superWANG", "ok" : 1 }
> show dbs
admin 0.000GB
local 0.000GBgolang中使用MongoDB
mgo（音mango）是MongoDB的Go语言驱动，它用基于Go语法的简单API实现了丰富的特性，并经过良好测试。
文档
http://godoc.org/labix.org/v2/mgo
获取
go get gopkg.in/mgo.v2连接
session, err := mgo.Dial(url)切换数据库
db := session.DB("test")切换集合
通过Database.C()方法切换集合（Collection）。
func (db Database) C(name string) *Collection插入
func (c *Collection) Insert(docs ...interface{}) errorc := session.DB("store").C("books")
err = c.Insert(book) 查询
func (c Collection) Find(query interface{}) Query更新
c := session.DB("store").C("books")
err = c.Update(bson.M{"isbn": isbn}, &book)查询所有
c := session.DB("store").C("books")
var books []Book
err := c.Find(bson.M{}).All(&books)删除
c := session.DB("store").C("books")
err := c.Remove(bson.M{"isbn": isbn})应用
package main
import (
"fmt"
"log"
"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
type Person struct {
Name string
Phone string
}
func main() {
session, err := mgo.Dial("localhost:27017")
if err != nil {
panic(err)
}
defer session.Close()
// Optional. Switch the session to a monotonic behavior.
session.SetMode(mgo.Monotonic, true)
c := session.DB("test").C("people")
err = c.Insert(&Person{"superWang", "13478808311"},
&Person{"David", "15040268074"})
if err != nil {
log.Fatal(err)
}
result := Person{}
err = c.Find(bson.M{"name": "superWang"}).One(&result)
if err != nil {
log.Fatal(err)
}
fmt.Println("Name:", result.Name)
fmt.Println("Phone:", result.Phone)
}
输出：
Name: superWang
Phone: 13478808311
Microservice with MongoDB in Go
再来点实战的，使用mongodb做一个微服务, 这里使用Goji.
什么是Goji?
Goji is a HTTP request multiplexer, similar to net/http.ServeMux. It compares incoming requests to a list of registered Patterns, and dispatches to the Handler that corresponds to the first matching Pattern. Goji also supports Middleware (composable shared functionality applied to every request) and uses the standard context to store request-scoped values.
获取
go get goji.io使用
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
"goji.io"
"goji.io/pat"
"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
func ErrorWithJSON(w http.ResponseWriter, message string, code int) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.WriteHeader(code)
fmt.Fprintf(w, "{message: %q}", message)
}
func ResponseWithJSON(w http.ResponseWriter, json []byte, code int) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.WriteHeader(code)
w.Write(json)
}
type Book struct {
ISBN string `json:"isbn"`
Title string `json:"title"`
Authors []string `json:"authors"`
Price string `json:"price"`
}
func main() {
session, err := mgo.Dial("localhost:27017")
if err != nil {
panic(err)
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
ensureIndex(session)
mux := goji.NewMux()
mux.HandleFunc(pat.Get("/books"), allBooks(session))
mux.HandleFunc(pat.Post("/books"), addBook(session))
mux.HandleFunc(pat.Get("/books/:isbn"), bookByISBN(session))
mux.HandleFunc(pat.Put("/books/:isbn"), updateBook(session))
mux.HandleFunc(pat.Delete("/books/:isbn"), deleteBook(session))
http.ListenAndServe("localhost:8080", mux)
}
func ensureIndex(s *mgo.Session) {
session := s.Copy()
defer session.Close()
c := session.DB("store").C("books")
index := mgo.Index{
Key: []string{"isbn"},
Unique: true,
DropDups: true,
Background: true,
Sparse: true,
}
err := c.EnsureIndex(index)
if err != nil {
panic(err)
}
}
func allBooks(s *mgo.Session) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
session := s.Copy()
defer session.Close()
c := session.DB("store").C("books")
var books []Book
err := c.Find(bson.M{}).All(&books)
if err != nil {
ErrorWithJSON(w, "Database error", http.StatusInternalServerError)
log.Println("Failed get all books: ", err)
return
}
respBody, err := json.MarshalIndent(books, "", " ")
if err != nil {
log.Fatal(err)
}
ResponseWithJSON(w, respBody, http.StatusOK)
}
}
func addBook(s *mgo.Session) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
session := s.Copy()
defer session.Close()
var book Book
decoder := json.NewDecoder(r.Body)
err := decoder.Decode(&book)
if err != nil {
ErrorWithJSON(w, "Incorrect body", http.StatusBadRequest)
return
}
c := session.DB("store").C("books")
err = c.Insert(book)
if err != nil {
if mgo.IsDup(err) {
ErrorWithJSON(w, "Book with this ISBN already exists", http.StatusBadRequest)
return
}
ErrorWithJSON(w, "Database error", http.StatusInternalServerError)
log.Println("Failed insert book: ", err)
return
}
w.Header().Set("Content-Type", "application/json")
w.Header().Set("Location", r.URL.Path+"/"+book.ISBN)
w.WriteHeader(http.StatusCreated)
}
}
func bookByISBN(s *mgo.Session) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
session := s.Copy()
defer session.Close()
isbn := pat.Param(r, "isbn")
c := session.DB("store").C("books")
var book Book
err := c.Find(bson.M{"isbn": isbn}).One(&book)
if err != nil {
ErrorWithJSON(w, "Database error", http.StatusInternalServerError)
log.Println("Failed find book: ", err)
return
}
if book.ISBN == "" {
ErrorWithJSON(w, "Book not found", http.StatusNotFound)
return
}
respBody, err := json.MarshalIndent(book, "", " ")
if err != nil {
log.Fatal(err)
}
ResponseWithJSON(w, respBody, http.StatusOK)
}
}
func updateBook(s *mgo.Session) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
session := s.Copy()
defer session.Close()
isbn := pat.Param(r, "isbn")
var book Book
decoder := json.NewDecoder(r.Body)
err := decoder.Decode(&book)
if err != nil {
ErrorWithJSON(w, "Incorrect body", http.StatusBadRequest)
return
}
c := session.DB("store").C("books")
err = c.Update(bson.M{"isbn": isbn}, &book)
if err != nil {
switch err {
default:
ErrorWithJSON(w, "Database error", http.StatusInternalServerError)
log.Println("Failed update book: ", err)
return
case mgo.ErrNotFound:
ErrorWithJSON(w, "Book not found", http.StatusNotFound)
return
}
}
w.WriteHeader(http.StatusNoContent)
}
}
func deleteBook(s *mgo.Session) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
session := s.Copy()
defer session.Close()
isbn := pat.Param(r, "isbn")
c := session.DB("store").C("books")
err := c.Remove(bson.M{"isbn": isbn})
if err != nil {
switch err {
default:
ErrorWithJSON(w, "Database error", http.StatusInternalServerError)
log.Println("Failed delete book: ", err)
return
case mgo.ErrNotFound:
ErrorWithJSON(w, "Book not found", http.StatusNotFound)
return
}
}
w.WriteHeader(http.StatusNoContent)
}
}
测试结果
POST
GET 所有书
GET 某本书
PUT 更新
DELETE 删除
