[源码分析]go-graceful如何shutdown gracefully

仓库地址

https://github.com/tylerstillwater/graceful

概述

服务的优雅关闭是指在关闭服务进程时, 不影响还在处理中的逻辑.

总体上的要点只有两个:

1. 服务处于关闭中状态, 不再接收新的请求.
2. 等待还在处理中的请求, 可设置超时机制.

经典应用场景

• 进程接收到突发的中断信号
• 平滑升级

源码解析

//graceful.go
func (srv *Server) Serve(listener net.Listener) error {

	if srv.ListenLimit != 0 {
		listener = LimitListener(listener, srv.ListenLimit)
	}

	// Make our stopchan
	srv.StopChan()

	// Track connection state
	add := make(chan net.Conn)
	idle := make(chan net.Conn)
	active := make(chan net.Conn)
	remove := make(chan net.Conn)

	// 注册回调函数, 当一个conn发生变化时的操作
	// 到这一层即可 net会完成通知调用
	srv.Server.ConnState = func(conn net.Conn, state http.ConnState) {
		switch state {
		case http.StateNew:
			add <- conn
		case http.StateActive:
			active <- conn
		case http.StateIdle:
			idle <- conn
		case http.StateClosed, http.StateHijacked:
			remove <- conn
		}

		srv.stopLock.Lock()
		defer srv.stopLock.Unlock()

		// 正常启动 srv.ConnState == nil
		if srv.ConnState != nil {
			srv.ConnState(conn, state)
		}
	}

	// Manage open connections
	shutdown := make(chan chan struct{})
	kill := make(chan struct{})
	// 将所有chan传入函数中, 监听变化
	go srv.manageConnections(add, idle, active, remove, shutdown, kill)

	interrupt := srv.interruptChan()
	// Set up the interrupt handler
	if !srv.NoSignalHandling {
		signalNotify(interrupt)
	}
	quitting := make(chan struct{})
	go srv.handleInterrupt(interrupt, quitting, listener)

	// Serve with graceful listener.
	// Execution blocks here until listener.Close() is called, above.
	// 得到新conn 会通过http.go调用connstat() 通知
	err := srv.Server.Serve(listener)
	if err != nil {
		// If the underlying listening is closed, Serve returns an error
		// complaining about listening on a closed socket. This is expected, so
		// let's ignore the error if we are the ones who explicitly closed the
		// socket.
		select {
		case <-quitting:
			err = nil
		default:
		}
	}

	srv.shutdown(shutdown, kill)

	return err
}

解析:

1. 通过重写http.Server.ConnState函数, 使http conn在状态发生变化时, 通知到manageConnections 函数, 通过多个chan传递信息.

这是标准库中对ConnState的注释, 这可选的函数对graceful的实现提供了很大的便利, 意味着我们不用去跟踪一个conn的状态变化, 只需要被通知即可.

// net/http/server.go
type Server struct {
    ...
	// ConnState specifies an optional callback function that is
	// called when a client connection changes state. See the
	// ConnState type and associated constants for details.
	ConnState func(net.Conn, ConnState)
}	

2. 在获取到conn的变化信号后, 另起一个协程manageConnections, 对所有conn进行管理, 以下继续分析.

func (srv *Server) manageConnections(add, idle, active, remove chan net.Conn, shutdown chan chan struct{}, kill chan struct{}) {
	var done chan struct{}
	srv.connections = map[net.Conn]struct{}{}
	srv.idleConnections = map[net.Conn]struct{}{}
	for {
		select {
		case conn := <-add:
			srv.connections[conn] = struct{}{}
			srv.idleConnections[conn] = struct{}{} // Newly-added connections are considered idle until they become active.
		case conn := <-idle:
			srv.idleConnections[conn] = struct{}{}
		case conn := <-active:
			delete(srv.idleConnections, conn)
		case conn := <-remove:
			delete(srv.connections, conn)
			delete(srv.idleConnections, conn)
			if done != nil && len(srv.connections) == 0 {
				done <- struct{}{}
				return
			}
		case done = <-shutdown:
			if len(srv.connections) == 0 && len(srv.idleConnections) == 0 {
				done <- struct{}{}
				return
			}
			// shutdown 关闭所有的空闲连接 防止空闲连接再次接受请求
			// a shutdown request has been received. if we have open idle
			// connections, we must close all of them now. this prevents idle
			// connections from holding the server open while waiting for them to
			// hit their idle timeout.
			for k := range srv.idleConnections {
				if err := k.Close(); err != nil {
					srv.logf("[ERROR] %s", err)
				}
			}
			// kill 关闭当前的所有连接
		case <-kill:
			srv.stopLock.Lock()
			defer srv.stopLock.Unlock()

			srv.Server.ConnState = nil
			for k := range srv.connections {
				if err := k.Close(); err != nil {
					srv.logf("[ERROR] %s", err)
				}
			}
			return
		}
	}
}

解析:

1. 把idleConnections内的所有空闲连接关闭, 来防止继续接收新的连接.
2. 于此同时shutdown(shutdown, kill)函数处理kill这个chan, 受到<-kill的信号之后, for…range关闭所有的还存在连接.

func (srv *Server) shutdown(shutdown chan chan struct{}, kill chan struct{}) {
	// Request done notification
	...
	/// 等到设定的超时时间后发送kill信号
	if srv.Timeout > 0 {
		select {
		case <-done:
		case <-time.After(srv.Timeout):
			close(kill)
		}
	} else {
		<-done
	}
	// Close the stopChan to wake up any blocked goroutines.
	...

总结

gracefully shutdown至此结束, 可以看到graceful并不能保证所有请求处理完毕再推出, 只是在shutdown之前管理好连接并等待一个超时时间.