在开发大型项目时,将日志进行结构化以提高可读性、可查询性和速度是非常重要的。
为什么你选择不使用其他结构化日志库,如logrus或zap?
Zerolog 是一款高性能且极易使用的日志库,zerolog 只专注于记录 JSON 格式的日志,号称 0 内存分配。
除了其卓越的性能外,Zerolog 还提供了许多有用的工具。
Github 官方地址:https://github.com/rs/zerolog
官方文档:https://pkg.go.dev/github.com/rs/zerolog
go get -u github.com/rs/zerolog/log
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
func main() {
// UNIX Time is faster and smaller than most timestamps
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
// {"level":"debug","time":1686128038,"message":"hello world"}
log.Print("hello world")
}
常规使用与标准库 log 非常相似,只不过输出的是 JSON 格式的日志。
zerolog 允许以键值对的形式将数据添加到日志消息中,添加到消息中的数据添加了关于日志事件的上下文,这对
于调试和问题追踪都是至关重要的。与 zap 一样, zerolog 也区分字段类型,不同的是 zerolog 采用链式调用的
方式:
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
func main() {
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
// {"level":"debug","Scale":"833 cents","Interval":833.09,"time":1686130340,"message":"Fibonacci is everywhere"}
log.Debug().
Str("Scale", "833 cents").
Float64("Interval", 833.09).
Msg("Fibonacci is everywhere")
// {"level":"debug","Name":"Tom","time":1686130340}
log.Debug().
Str("Name", "Tom").
Send()
}
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
func main() {
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
ll := log.With().Caller().Str("Author", "Tom").Logger()
ll.Debug().Str("Key1", "Value1").Send()
ll.Debug().Str("Key2", "Value2").Send()
ll.Debug().Str("Key3", "Value3").Msg("hello")
ll.Debug().Str("Key4", "Value4").Msg("world")
}
# 输出
{"level":"debug","Author":"Tom","Key1":"Value1","time":1686130826,"caller":"....../zerolog/go-zerolog/003.go:11"}
{"level":"debug","Author":"Tom","Key2":"Value2","time":1686130826,"caller":"....../zerolog/go-zerolog/003.go:12"}
{"level":"debug","Author":"Tom","Key3":"Value3","time":1686130826,"caller":"....../zerolog/go-zerolog/003.go:13","message":"hello"}
{"level":"debug","Author":"Tom","Key4":"Value4","time":1686130826,"caller":"....../zerolog/go-zerolog/003.go:14","message":"world"}
与 zap 相同的是,都定义了强类型字段。
与 zap 不同的是,zerolog 采用链式调用。
zerolog 提供了从 Trace 到 Panic 七个级别:
panic (zerolog.PanicLevel, 5)
fatal (zerolog.FatalLevel, 4)
error (zerolog.ErrorLevel, 3)
warn (zerolog.WarnLevel, 2)
info (zerolog.InfoLevel, 1)
debug (zerolog.DebugLevel, 0)
trace (zerolog.TraceLevel, -1)
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
func main() {
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
// 没有级别
log.Log().Msg("hello world")
log.Trace().Msg("hello world")
log.Debug().Msg("hello world")
log.Info().Msg("hello world")
log.Warn().Msg("hello world")
log.Error().Msg("hello world")
log.Fatal().Msg("hello world")
log.Panic().Msg("hello world")
}
# 输出
{"time":1686131728,"message":"hello world"}
{"level":"trace","time":1686131728,"message":"hello world"}
{"level":"debug","time":1686131728,"message":"hello world"}
{"level":"info","time":1686131728,"message":"hello world"}
{"level":"warn","time":1686131728,"message":"hello world"}
{"level":"error","time":1686131728,"message":"hello world"}
{"level":"fatal","time":1686131728,"message":"hello world"}
可以调用 SetGlobalLevel() 设置全局 Logger 的日志级别。
设置日志等级:
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
func main() {
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
zerolog.SetGlobalLevel(zerolog.ErrorLevel)
// 没有级别
log.Log().Msg("hello world")
log.Trace().Msg("hello world")
log.Debug().Msg("hello world")
log.Info().Msg("hello world")
log.Warn().Msg("hello world")
log.Error().Msg("hello world")
log.Fatal().Msg("hello world")
log.Panic().Msg("hello world")
}
# 输出
{"time":1686132006,"message":"hello world"}
{"level":"error","time":1686132006,"message":"hello world"}
{"level":"fatal","time":1686132006,"message":"hello world"}
不带级别和消息的日志记录:
您可以选择使用log方法在没有特定级别的情况下进行日志记录,也可以通过在msg方法的msg字符串参数中设置
一个空字符串来编写不带消息的内容。
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
func main() {
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
// {"foo":"bar","time":1686132216}
log.Log().
Str("foo", "bar").
Msg("")
}
您可以使用Err方法记录错误:
package main
import (
"errors"
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
func main() {
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
err := errors.New("seems we have an error here")
// {"level":"error","error":"seems we have an error here","time":1686132556}
log.Error().Err(err).Msg("")
}
errors 的默认字段名称是 error,您可以通过设置zerolog.ErrorFieldName来更改此名称以满足您的需要。
package main
import (
"github.com/pkg/errors"
"github.com/rs/zerolog/pkgerrors"
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
func main() {
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
zerolog.ErrorStackMarshaler = pkgerrors.MarshalStack
err := outer()
log.Error().Stack().Err(err).Msg("")
}
func inner() error {
return errors.New("seems we have an error here")
}
func middle() error {
err := inner()
if err != nil {
return err
}
return nil
}
func outer() error {
err := middle()
if err != nil {
return err
}
return nil
}
# 输出
{"level":"error","stack":[{"func":"inner","line":"18","source":"009.go"},{"func":"middle","line":"22","source":"009.go"},{"func":"outer","line":"30","source":"009.go"},{"func":"main","line":"13","source":"009.go"},{"func":"main","line":"250","source":"proc.go"},{"func":"goexit","line":"1571","source":"asm_amd64.s"}],"error":"seems we have an error here","time":1686133719}
必须设置 zerolog.ErrorStackMarshaller,堆栈才能输出任何内容。
package main
import (
"errors"
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
func main() {
err := errors.New("A repo man spends his life getting into tense situations")
service := "myservice"
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
// {"level":"fatal","error":"A repo man spends his life getting into tense situations","service":"myservice","time":1686185774,"message":"Cannot start myservice"}
log.Fatal().
Err(err).
Str("service", service).
Msgf("Cannot start %s", service)
}
注意:使用Msgf会生成一个分配,即使logger 被禁用。
全局 Logger:上面我们使用 log.Debug()、log.Info() 调用的是全局的 Logger 。全局的 Logger 使用比较简单,不
需要额外创建。
package main
import "github.com/rs/zerolog/log"
func main(){
log.Logger = log.With().Str("foo", "bar").Logger()
// {"level":"info","foo":"bar","time":"2023-06-08T10:00:40+08:00","message":"Hello World!"}
log.Info().Msg("Hello World!")
}
全局的 Logger ,这种方式有一个明显的缺点:如果在某个地方修改了设置,将影响全局的日志记录。为了消除这
种影响,我们需要创建新的 Logger。
package main
import (
"github.com/rs/zerolog"
"os"
)
func main() {
logger := zerolog.New(os.Stderr).With().Timestamp().Logger()
// {"level":"info","foo":"bar","time":"2023-06-08T09:04:46+08:00","message":"hello world"}
logger.Info().Str("foo", "bar").Msg("hello world")
}
调用 zerlog.New() 传入一个 io.Writer 作为日志写入器即可。
基于当前的 Logger 可以创建一个子 Logger,子 Logger 可以在父 Logger 上附加一些额外的字段。调用
logger.With() 创建一个上下文,然后为它添加字段,最后调用 Logger() 返回一个新的 Logger:
package main
import (
"github.com/rs/zerolog"
"os"
)
func main() {
logger := zerolog.New(os.Stderr)
sublogger := logger.With().
Str("component", "foo").
Logger()
// {"level":"info","component":"foo","time":"2023-06-08T09:11:14+08:00","message":" hello world"}
sublogger.Info().Msg("hello world")
}
sublogger 会额外输出 “component”:“foo” 这个字段。
zerolog 提供了多种选项定制输入日志的行为。
zerolog 提供了一个 ConsoleWriter 可输出便于我们阅读的,带颜色的日志。
调用 zerolog.Output() 来启用 ConsoleWriter。
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
"os"
)
func main(){
log.Logger = log.Output(zerolog.ConsoleWriter{Out: os.Stderr})
// 9:17AM INF Hello world foo=bar
log.Info().Str("foo", "bar").Msg("Hello world")
}
我们还能进一步对 ConsoleWriter 进行配置,定制输出的级别、信息、字段名、字段值的格式:
package main
import (
"fmt"
"github.com/rs/zerolog"
"os"
"strings"
"time"
)
func main() {
output := zerolog.ConsoleWriter{Out: os.Stdout, TimeFormat: time.RFC3339}
output.FormatLevel = func(i interface{}) string {
return strings.ToUpper(fmt.Sprintf("| %-6s|", i))
}
output.FormatMessage = func(i interface{}) string {
return fmt.Sprintf("***%s****", i)
}
output.FormatFieldName = func(i interface{}) string {
return fmt.Sprintf("%s:", i)
}
output.FormatFieldValue = func(i interface{}) string {
return strings.ToUpper(fmt.Sprintf("%s", i))
}
log := zerolog.New(output).With().Timestamp().Logger()
// 2023-06-08T09:20:22+08:00 | INFO | ***Hello World**** foo:BAR
log.Info().Str("foo", "bar").Msg("Hello World")
}
ConsoleWriter的性能不够理想,建议只在开发环境中使用!
记录的字段可以任意嵌套,这通过 Dict() 来实现。
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
func main() {
// {"level":"info","foo":"bar","dict":{"bar":"baz","n":1},"time":"2023-06-08T09:47:55+08:00","message":"hello world"}
log.Info().
Str("foo", "bar").
Dict("dict", zerolog.Dict().
Str("bar", "baz").
Int("n", 1),
).Msg("hello world")
}
输出的日志中级别默认的字段名为 level,信息默认为 message,时间默认为 time。可以通过 zerolog 中
LevelFieldName/MessageFieldName/TimestampFieldName 来设置:
package main
import (
"github.com/rs/zerolog"
"os"
)
func main() {
zerolog.TimestampFieldName = "t"
zerolog.LevelFieldName = "l"
zerolog.MessageFieldName = "m"
logger := zerolog.New(os.Stderr).With().Timestamp().Logger()
// {"l":"info","t":"2023-06-08T09:52:34+08:00","m":"hello world"}
logger.Info().Msg("hello world")
}
注意,这个设置是全局的。
有时我们需要输出文件名和行号,以便能很快定位代码位置,方便找出问题。这可以通过在创建子 Logger 时带
入 Caller()选项完成:
package main
import "github.com/rs/zerolog/log"
func main(){
log.Logger = log.With().Caller().Logger()
// {"level":"info","time":"2023-06-08T10:04:40+08:00","caller":"....../go-zerolog/017.go:7","message":"hello world"}
log.Info().Msg("hello world")
}
自己定义:
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
"strconv"
)
func main(){
zerolog.CallerMarshalFunc = func(pc uintptr, file string, line int) string {
short := file
for i := len(file) - 1; i > 0; i-- {
if file[i] == '/' {
short = file[i+1:]
break
}
}
file = short
return file + ":" + strconv.Itoa(line)
}
log.Logger = log.With().Caller().Logger()
// {"level":"info","time":"2023-06-08T10:06:09+08:00","caller":"018.go:22","message":"hello world"}
log.Info().Msg("hello world")
}
如果您的编写器可能很慢或不是线程安全的,并且您需要日志生成器永远不会被慢的编写器拖慢,那么您可以使用
diode.Writer,如下所示:
package main
import (
"fmt"
"github.com/rs/zerolog"
"github.com/rs/zerolog/diode"
"os"
"time"
)
func main(){
wr := diode.NewWriter(os.Stdout, 1000, 10*time.Millisecond, func(missed int) {
fmt.Printf("Logger Dropped %d messages", missed)
})
log := zerolog.New(wr)
// {"level":"debug","message":"test"}
log.Print("test")
}
有时候日志太多了反而对我们排查问题造成干扰,zerolog 支持日志采样的功能,可以每隔多少条日志输出一
次,其他日志丢弃:
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
func main() {
sampled := log.Sample(&zerolog.BasicSampler{N: 10})
for i := 0; i < 20; i++ {
sampled.Info().Msg("will be logged every 10 message")
}
}
# 输出
{"level":"info","time":"2023-06-08T10:25:37+08:00","message":"will be logged every 10 message"}
{"level":"info","time":"2023-06-08T10:25:37+08:00","message":"will be logged every 10 message"}
更高级的采样:
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
"time"
)
func main(){
// 只采样Debug日志,在1s内最多输出5条日志,超过5条时,每隔100条输出一条
sampled := log.Sample(zerolog.LevelSampler{
DebugSampler: &zerolog.BurstSampler{
Burst: 5,
Period: 1*time.Second,
NextSampler: &zerolog.BasicSampler{N: 100},
},
})
// {"level":"debug","time":"2023-06-08T10:55:59+08:00","message":"hello world"}
// {"level":"debug","time":"2023-06-08T10:55:59+08:00","message":"hello world"}
// {"level":"debug","time":"2023-06-08T10:55:59+08:00","message":"hello world"}
// {"level":"debug","time":"2023-06-08T10:55:59+08:00","message":"hello world"}
// {"level":"debug","time":"2023-06-08T10:55:59+08:00","message":"hello world"}
// {"level":"debug","time":"2023-06-08T10:55:59+08:00","message":"hello world"}
for i := 0; i < 50; i++ {
sampled.Debug().Msg("hello world")
}
}
zerolog 支持钩子,我们可以针对不同的日志级别添加一些额外的字段或进行其他的操作:
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
)
type SeverityHook struct{}
func (h SeverityHook) Run(e *zerolog.Event, level zerolog.Level, msg string) {
if level != zerolog.NoLevel {
e.Str("severity", level.String())
}
}
func main(){
hooked := log.Hook(SeverityHook{})
// {"level":"warn","time":"2023-06-08T10:59:31+08:00","severity":"warn"}
hooked.Warn().Msg("")
}
package main
import (
"context"
"github.com/rs/zerolog/log"
)
func main(){
ctx := log.With().Str("component", "module").Logger().WithContext(context.Background())
// {"level":"info","component":"module","time":"2023-06-08T11:04:28+08:00","message":"hello world"}
log.Ctx(ctx).Info().Msg("hello world")
}
package main
import (
"github.com/rs/zerolog"
stdlog "log"
"os"
)
func main(){
log := zerolog.New(os.Stdout).With().
Str("foo", "bar").
Logger()
stdlog.SetFlags(0)
stdlog.SetOutput(log)
// {"foo":"bar","message":"hello world"}
stdlog.Print("hello world")
}
github.com/rs/zerolog/hlog 包提供了一些帮助程序来将 zerolog 与 http.Handler 集成。
package main
import (
"github.com/justinas/alice"
"github.com/rs/zerolog"
"github.com/rs/zerolog/hlog"
"net/http"
"os"
"time"
)
func main() {
log := zerolog.New(os.Stdout).With().
Timestamp().
Str("role", "my-service").
Str("host", "127.0.0.1").
Logger()
c := alice.New()
// Install the logger handler with default output on the console
c = c.Append(hlog.NewHandler(log))
// Install some provided extra handler to set some request's context fields.
// Thanks to that handler, all our logs will come with some prepopulated fields.
c = c.Append(hlog.AccessHandler(func(r *http.Request, status, size int, duration time.Duration) {
hlog.FromRequest(r).Info().
Str("method", r.Method).
Stringer("url", r.URL).
Int("status", status).
Int("size", size).
Dur("duration", duration).
Msg("")
}))
c = c.Append(hlog.RemoteAddrHandler("ip"))
c = c.Append(hlog.UserAgentHandler("user_agent"))
c = c.Append(hlog.RefererHandler("referer"))
c = c.Append(hlog.RequestIDHandler("req_id", "Request-Id"))
// Here is your final handler
h := c.Then(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Get the logger from the request's context. You can safely assume it
// will be always there: if the handler is removed, hlog.FromRequest
// will return a no-op logger.
hlog.FromRequest(r).Info().
Str("user", "current user").
Str("status", "ok").
Msg("Something happened")
}))
http.Handle("/", h)
if err := http.ListenAndServe(":8080", nil); err != nil {
log.Fatal().Err(err).Msg("Startup failed")
}
}
# 输出
{"level":"info","role":"my-service","host":"127.0.0.1","ip":"127.0.0.1:50189","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/114.0.0.0 Safari/537.36 Edg/114.0.1823.37","req_id":"ci0kg1n2tm03o212g3kg","user":"current user","status":"ok","time":"2023-06-08T11:16:22+08:00","messa
ge":"Something happened"}
{"level":"info","role":"my-service","host":"127.0.0.1","ip":"127.0.0.1:50189","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/114.0.0.0 Safari/537.36 Edg/114.0.1823.37","req_id":"ci0kg1n2tm03o212g3kg","method":"GET","url":"/","status":0,"size":0,"duration":16.0218,"time":"2023-
06-08T11:16:22+08:00"}
{"level":"info","role":"my-service","host":"127.0.0.1","ip":"127.0.0.1:50189","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/114.0.0.0 Safari/537.36 Edg/114.0.1823.37","referer":"http://127.0.0.1:8080/","req_id":"ci0kg1n2tm03o212g3l0","user":"current user","status":"ok","time"
:"2023-06-08T11:16:22+08:00","message":"Something happened"}
{"level":"info","role":"my-service","host":"127.0.0.1","ip":"127.0.0.1:50189","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/114.0.0.0 Safari/537.36 Edg/114.0.1823.37","referer":"http://127.0.0.1:8080/","req_id":"ci0kg1n2tm03o212g3l0","method":"GET","url":"/favicon.ico","statu
s":0,"size":0,"duration":0.997,"time":"2023-06-08T11:16:22+08:00"}
zerolog.MultiLevelWriter 可用于将日志消息发送到多个输出, 在本例中,我们将日志消息发送到os.Stdout和内
置的ConsoleWriter。
package main
import (
"github.com/rs/zerolog"
"os"
)
func main() {
consoleWriter := zerolog.ConsoleWriter{Out: os.Stdout}
multi := zerolog.MultiLevelWriter(consoleWriter, os.Stdout)
logger := zerolog.New(multi).With().Timestamp().Logger()
// 11:26AM INF Hello World!
// {"level":"info","time":"2023-06-08T11:26:21+08:00","message":"Hello World!"}
logger.Info().Msg("Hello World!")
}
package main
import (
"fmt"
"github.com/rs/zerolog"
"os"
"strings"
"time"
)
var Logger zerolog.Logger
func init() {
timeFormat := "2006-01-02 15:04:05"
zerolog.TimeFieldFormat = timeFormat
// 创建log目录
logDir := "./run_log/"
err := os.MkdirAll(logDir, os.ModePerm)
if err != nil {
fmt.Println("Mkdir failed, err:", err)
return
}
// 把日志同时往控制台和日志文件里输出,日志文件用日期每日分拆
fileName := logDir + time.Now().Format("2006-01-02") + ".log"
logFile, _ := os.OpenFile(fileName, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
consoleWriter := zerolog.ConsoleWriter{Out: os.Stdout, TimeFormat: timeFormat}
consoleWriter.FormatLevel = func(i interface{}) string {
return strings.ToUpper(fmt.Sprintf("| %-6s|", i))
}
consoleWriter.FormatMessage = func(i interface{}) string {
return fmt.Sprintf("%s", i)
}
consoleWriter.FormatFieldName = func(i interface{}) string {
return fmt.Sprintf("%s:", i)
}
consoleWriter.FormatFieldValue = func(i interface{}) string {
return fmt.Sprintf("%s;", i)
}
multi := zerolog.MultiLevelWriter(consoleWriter, logFile)
Logger = zerolog.New(multi).With().Timestamp().Logger()
}
func main(){
// 2023-06-08 14:33:44 | INFO | 开始登录... account:sdhhk; website:xx;
Logger.Info().
Str("website", "xx").
Str("account", "sdhhk").
Msg("开始登录...")
}
某些设置可以更改并将应用于所有loggers:
log.Logger
zerolog.SetGlobalLevel
zerolog.DisableSampling
zerolog.TimestampFieldName
zerolog.LevelFieldName
zerolog.MessageFieldName
zerolog.ErrorFieldName
zerolog.TimeFieldFormat:
zerolog.TimeFormatUnix,zerolog.TimeFormatUnixMs,zerolog.TimeFormatUnixMicro
zerolog.DurationFieldUnit
zerolog.DurationFieldInteger
zerolog.ErrorHandler
Str
Bool
Int, Int8, Int16, Int32, Int64
Uint, Uint8, Uint16, Uint32, Uint64
Float32, Float64
Err
Func
Timestamp
Time
Dur
Dict
RawJSON
Hex
Interface
大多数字段也可以使用切片格式:Strs for []string, Errs for []error。
1、zerolog 不会对重复的字段删除
package main
import (
"github.com/rs/zerolog"
"os"
)
func main(){
logger := zerolog.New(os.Stderr).With().Timestamp().Logger()
// {"level":"info","time":"2023-06-08T14:08:35+08:00","time":"2023-06-08T14:08:35+08:00","message":"dup"}
logger.Info().
Timestamp().
Msg("dup")
}
2、链式调用必须调用 Msg、Msgf、Send才能输出日志,Send 相当于调用 Msg(“”)。
3、一旦调用 Msg,Event 将会被处理(放回池中或丢掉),不允许二次调用。