Token bucket
The token bucket algorithm can be conceptually understood as follows:
- A token is added to the bucket every
1/rseconds. r是每秒生产的token总数 - The bucket can hold at the most
btokens. If a token arrives when the bucket is full, it is discarded. bucket初始时就是满的，并且满了就塞不进tokens了 - When a packet (network layer PDU) of n bytes arrives,
- if at least n tokens are in the bucket, n tokens are removed from the bucket, and the packet is sent to the network.
- if fewer than n tokens are available, no tokens are removed from the bucket, and the packet is considered to be non-conformant.
简单来说，有个bucket，容量为b；每秒我们将生产r个tokens，尝试往bucket里面放，当bucket满时则无法继续放入
当外界发送的请求到来时，系统先从bucket里面取出若干数量的token，若成功取出，则该请求被confirmed
这样便达到限流目的（如：一个请求需要一个token，那就是每秒r个请求的限流）
golang golang.org/x/time/rate 对token bucket算法的实现
先总结
私以为，golang.org/x/time/rate在实现上有2处精巧的地方
- 把离散的token连续化。每秒生成r个token，很容易得出，每1/r秒生成1个token；但time/rate打破了常规，它将token连续化了，认为每时每刻都在产生token，token从0...->0.01，0.011，0.012...->1...->r。把离散的token连续化，在time/rate代码上的表现就是token的类型是float64
- token是必要时通过实时计算得到的。token bucket本质上是一个生产者-消费者模型。如果按照每秒生成r个token，每1/r秒生成1个token这个思路，普通的实现方式一般是【实现一个token生成器作为生产者，一直按固定频率生成token】。但是time/rate并不是这样做，它是在需要取token的时刻，才通过计算去获得那一时刻的token数量，通过实时计算替代了生产
看源码
核心部分包括：
- advance方法。计算前进到t时刻时的token数量
- reserveN方法。获取token，底层都是调用了reserveN
完整源码如下：
// Package rate provides a rate limiter.
package rate
import (
"context"
"fmt"
"math"
"sync"
"time"
)
// Limit defines the maximum frequency of some events.
// Limit is represented as number of events per second.
// A zero Limit allows no events.
// 【Limit 每秒允许事件发生的次数，也就是每秒产生的token数，也就是上文提到的r. 注意，类型是float64，token被连续化了】
type Limit float64
// Inf is the infinite rate limit; it allows all events (even if burst is zero).
const Inf = Limit(math.MaxFloat64)
// Every converts a minimum time interval between events to a Limit.
// 【Every 是一个帮助函数，通过两次事件的间隔时间，计算每秒允许事件发生的次数】
func Every(interval time.Duration) Limit {
if interval <= 0 {
return Inf
}
return 1 / Limit(interval.Seconds())
}
// A Limiter controls how frequently events are allowed to happen.
// It implements a "token bucket" of size b, initially full and refilled
// at rate r tokens per second.
// Informally, in any large enough time interval, the Limiter limits the
// rate to r tokens per second, with a maximum burst size of b events.
// As a special case, if r == Inf (the infinite rate), b is ignored.
// See https://en.wikipedia.org/wiki/Token_bucket for more about token buckets.
//
// The zero value is a valid Limiter, but it will reject all events.
// Use NewLimiter to create non-zero Limiters.
//
// Limiter has three main methods, Allow, Reserve, and Wait.
// Most callers should use Wait.
//
// Each of the three methods consumes a single token.
// They differ in their behavior when no token is available.
// If no token is available, Allow returns false.
// If no token is available, Reserve returns a reservation for a future token
// and the amount of time the caller must wait before using it.
// If no token is available, Wait blocks until one can be obtained
// or its associated context.Context is canceled.
//
// The methods AllowN, ReserveN, and WaitN consume n tokens.
type Limiter struct {
mu sync.Mutex
limit Limit
burst int
tokens float64 // 推论：Limiter中的tokens永远小于burst
// last is the last time the limiter's tokens field was updated
// 【某一时刻的tokens是靠计算得到的，因此Limiter本身就像tokens的db，对于当前记录的tokens值，需要一个time version，就是last】
last time.Time
// lastEvent is the latest time of a rate-limited event (past or future)
lastEvent time.Time
}
// Limit returns the maximum overall event rate.
func (lim *Limiter) Limit() Limit {
lim.mu.Lock()
defer lim.mu.Unlock()
return lim.limit
}
// Burst returns the maximum burst size. Burst is the maximum number of tokens
// that can be consumed in a single call to Allow, Reserve, or Wait, so higher
// Burst values allow more events to happen at once.
// A zero Burst allows no events, unless limit == Inf.
func (lim *Limiter) Burst() int {
lim.mu.Lock()
defer lim.mu.Unlock()
return lim.burst
}
// TokensAt returns the number of tokens available at time t.
func (lim *Limiter) TokensAt(t time.Time) float64 {
lim.mu.Lock()
_, tokens := lim.advance(t) // does not mutate lim
lim.mu.Unlock()
return tokens
}
// Tokens returns the number of tokens available now.
func (lim *Limiter) Tokens() float64 {
return lim.TokensAt(time.Now())
}
// NewLimiter returns a new Limiter that allows events up to rate r and permits
// bursts of at most b tokens.
func NewLimiter(r Limit, b int) *Limiter {
return &Limiter{
limit: r,
burst: b,
}
}
// Allow reports whether an event may happen now.
func (lim *Limiter) Allow() bool {
return lim.AllowN(time.Now(), 1)
}
// AllowN reports whether n events may happen at time t.
// Use this method if you intend to drop / skip events that exceed the rate limit.
// Otherwise use Reserve or Wait.
func (lim *Limiter) AllowN(t time.Time, n int) bool {
return lim.reserveN(t, n, 0).ok
}
// A Reservation holds information about events that are permitted by a Limiter to happen after a delay.
// A Reservation may be canceled, which may enable the Limiter to permit additional events.
type Reservation struct {
ok bool // 【表示这个预约是否成功】
lim *Limiter
tokens int // 【预约了几个token, 如果预约取消，则返还给lim】
timeToAct time.Time // 【事件允许发生的时间】
// This is the Limit at reservation time, it can change later.
limit Limit
}
// OK returns whether the limiter can provide the requested number of tokens
// within the maximum wait time. If OK is false, Delay returns InfDuration, and
// Cancel does nothing.
func (r *Reservation) OK() bool {
return r.ok
}
// Delay is shorthand for DelayFrom(time.Now()).
func (r *Reservation) Delay() time.Duration {
return r.DelayFrom(time.Now())
}
// InfDuration is the duration returned by Delay when a Reservation is not OK.
const InfDuration = time.Duration(math.MaxInt64)
// DelayFrom returns the duration for which the reservation holder must wait
// before taking the reserved action. Zero duration means act immediately.
// InfDuration means the limiter cannot grant the tokens requested in this
// Reservation within the maximum wait time.
func (r *Reservation) DelayFrom(t time.Time) time.Duration {
if !r.ok {
return InfDuration
}
delay := r.timeToAct.Sub(t)
if delay < 0 {
return 0
}
return delay
}
// Cancel is shorthand for CancelAt(time.Now()).
func (r *Reservation) Cancel() {
r.CancelAt(time.Now())
}
// CancelAt indicates that the reservation holder will not perform the reserved action
// and reverses the effects of this Reservation on the rate limit as much as possible,
// considering that other reservations may have already been made.
func (r *Reservation) CancelAt(t time.Time) {
if !r.ok {
return
}
r.lim.mu.Lock()
defer r.lim.mu.Unlock()
if r.lim.limit == Inf || r.tokens == 0 || r.timeToAct.Before(t) {
return
}
// calculate tokens to restore
// The duration between lim.lastEvent and r.timeToAct tells us how many tokens were reserved
// after r was obtained. These tokens should not be restored.
restoreTokens := float64(r.tokens) - r.limit.tokensFromDuration(r.lim.lastEvent.Sub(r.timeToAct))
if restoreTokens <= 0 {
return
}
// advance time to now
t, tokens := r.lim.advance(t)
// calculate new number of tokens
tokens += restoreTokens
if burst := float64(r.lim.burst); tokens > burst {
tokens = burst
}
// update state
r.lim.last = t
r.lim.tokens = tokens
if r.timeToAct == r.lim.lastEvent {
prevEvent := r.timeToAct.Add(r.limit.durationFromTokens(float64(-r.tokens)))
if !prevEvent.Before(t) {
r.lim.lastEvent = prevEvent
}
}
}
// Reserve is shorthand for ReserveN(time.Now(), 1).
func (lim *Limiter) Reserve() *Reservation {
return lim.ReserveN(time.Now(), 1)
}
// ReserveN returns a Reservation that indicates how long the caller must wait before n events happen.
// The Limiter takes this Reservation into account when allowing future events.
// The returned Reservation’s OK() method returns false if n exceeds the Limiter's burst size.
// Usage example:
//
// r := lim.ReserveN(time.Now(), 1)
// if !r.OK() {
// // Not allowed to act! Did you remember to set lim.burst to be > 0 ?
// return
// }
// time.Sleep(r.Delay())
// Act()
//
// Use this method if you wish to wait and slow down in accordance with the rate limit without dropping events.
// If you need to respect a deadline or cancel the delay, use Wait instead.
// To drop or skip events exceeding rate limit, use Allow instead.
func (lim *Limiter) ReserveN(t time.Time, n int) *Reservation {
r := lim.reserveN(t, n, InfDuration)
return &r
}
// Wait is shorthand for WaitN(ctx, 1).
func (lim *Limiter) Wait(ctx context.Context) (err error) {
return lim.WaitN(ctx, 1)
}
// WaitN blocks until lim permits n events to happen.
// It returns an error if n exceeds the Limiter's burst size, the Context is
// canceled, or the expected wait time exceeds the Context's Deadline.
// The burst limit is ignored if the rate limit is Inf.
func (lim *Limiter) WaitN(ctx context.Context, n int) (err error) {
// The test code calls lim.wait with a fake timer generator.
// This is the real timer generator.
newTimer := func(d time.Duration) (<-chan time.Time, func() bool, func()) {
timer := time.NewTimer(d)
return timer.C, timer.Stop, func() {}
}
return lim.wait(ctx, n, time.Now(), newTimer)
}
// wait is the internal implementation of WaitN.
func (lim *Limiter) wait(ctx context.Context, n int, t time.Time, newTimer func(d time.Duration) (<-chan time.Time, func() bool, func())) error {
lim.mu.Lock()
burst := lim.burst
limit := lim.limit
lim.mu.Unlock()
if n > burst && limit != Inf {
return fmt.Errorf("rate: Wait(n=%d) exceeds limiter's burst %d", n, burst)
}
// Check if ctx is already cancelled
select {
case <-ctx.Done():
return ctx.Err()
default:
}
// Determine wait limit
waitLimit := InfDuration
if deadline, ok := ctx.Deadline(); ok {
waitLimit = deadline.Sub(t)
}
// Reserve
r := lim.reserveN(t, n, waitLimit)
if !r.ok {
return fmt.Errorf("rate: Wait(n=%d) would exceed context deadline", n)
}
// Wait if necessary
delay := r.DelayFrom(t)
if delay == 0 {
return nil
}
ch, stop, advance := newTimer(delay)
defer stop()
advance() // only has an effect when testing
select {
case <-ch:
// We can proceed.
return nil
case <-ctx.Done():
// Context was canceled before we could proceed. Cancel the
// reservation, which may permit other events to proceed sooner.
r.Cancel()
return ctx.Err()
}
}
// SetLimit is shorthand for SetLimitAt(time.Now(), newLimit).
func (lim *Limiter) SetLimit(newLimit Limit) {
lim.SetLimitAt(time.Now(), newLimit)
}
// SetLimitAt sets a new Limit for the limiter. The new Limit, and Burst, may be violated
// or underutilized by those which reserved (using Reserve or Wait) but did not yet act
// before SetLimitAt was called.
func (lim *Limiter) SetLimitAt(t time.Time, newLimit Limit) {
lim.mu.Lock()
defer lim.mu.Unlock()
t, tokens := lim.advance(t)
lim.last = t
lim.tokens = tokens
lim.limit = newLimit
}
// SetBurst is shorthand for SetBurstAt(time.Now(), newBurst).
func (lim *Limiter) SetBurst(newBurst int) {
lim.SetBurstAt(time.Now(), newBurst)
}
// SetBurstAt sets a new burst size for the limiter.
func (lim *Limiter) SetBurstAt(t time.Time, newBurst int) {
lim.mu.Lock()
defer lim.mu.Unlock()
t, tokens := lim.advance(t)
lim.last = t
lim.tokens = tokens
lim.burst = newBurst
}
// reserveN is a helper method for AllowN, ReserveN, and WaitN.
// maxFutureReserve specifies the maximum reservation wait duration allowed.
// reserveN returns Reservation, not *Reservation, to avoid allocation in AllowN and WaitN.
// 【在t时刻预约n个token，如果不行，可以多等maxFutureReserve】
func (lim *Limiter) reserveN(t time.Time, n int, maxFutureReserve time.Duration) Reservation {
lim.mu.Lock()
defer lim.mu.Unlock()
if lim.limit == Inf {
return Reservation{
ok: true,
lim: lim,
tokens: n,
timeToAct: t,
}
} else if lim.limit == 0 {
var ok bool
if lim.burst >= n {
ok = true
lim.burst -= n
}
return Reservation{
ok: ok,
lim: lim,
tokens: lim.burst, // 这里tokens的value不是n，是为了cancel的时候恢复burst？
timeToAct: t,
}
}
t, tokens := lim.advance(t)
// Calculate the remaining number of tokens resulting from the request.
tokens -= float64(n)
// Calculate the wait duration
var waitDuration time.Duration
if tokens < 0 {
waitDuration = lim.limit.durationFromTokens(-tokens)
}
// Decide result
ok := n <= lim.burst && waitDuration <= maxFutureReserve
// Prepare reservation
r := Reservation{
ok: ok,
lim: lim,
limit: lim.limit,
}
if ok {
r.tokens = n
r.timeToAct = t.Add(waitDuration)
// Update state
lim.last = t
lim.tokens = tokens
lim.lastEvent = r.timeToAct
}
return r
}
// advance calculates and returns an updated state for lim resulting from the passage of time.
// lim is not changed.
// advance requires that lim.mu is held.
// 【算出到t时刻时的token数量，显然t是不能向前回溯的】
func (lim *Limiter) advance(t time.Time) (newT time.Time, newTokens float64) {
last := lim.last
if t.Before(last) {
last = t // 向前回溯是unexpected的，以此trick防止向前回溯。如果t是过去，最终会返回t和当前的tokens
}
// Calculate the new number of tokens, due to time that passed.
elapsed := t.Sub(last)
delta := lim.limit.tokensFromDuration(elapsed)
tokens := lim.tokens + delta
if burst := float64(lim.burst); tokens > burst {
tokens = burst
}
return t, tokens
}
// durationFromTokens is a unit conversion function from the number of tokens to the duration
// of time it takes to accumulate them at a rate of limit tokens per second.
func (limit Limit) durationFromTokens(tokens float64) time.Duration {
if limit <= 0 {
return InfDuration
}
seconds := tokens / float64(limit)
return time.Duration(float64(time.Second) * seconds)
}
// tokensFromDuration is a unit conversion function from a time duration to the number of tokens
// which could be accumulated during that duration at a rate of limit tokens per second.
func (limit Limit) tokensFromDuration(d time.Duration) float64 {
if limit <= 0 {
return 0
}
return d.Seconds() * float64(limit)
}
只能计算未来token，不能计算历史token
注意，阅读源码advance方法发现，通过【计算】实现的token bucket算法，只能计算未来token，不能计算历史token
即只能计算未来某个时刻的token数量，而不能计算过去某个时刻的token数量
因此，一旦一个Reservation预约的是遥远未来的token，就会把Limiter.last直接提到遥远的未来，但Limiter无法计算过去某个时刻的token（Limiter其实简单粗暴地用Limiter.tokens作为过去某个时刻的可用token），这很可能影响更近期的Reservation
when target time t is before lim.last, lim.advance(t) will return t itself and lim.tokens (lim.tokens实际上是在lim.last时的token数量，被用来作为t时可用的token量)
so, if we make reservations in reverse time order, the tokens kept by lim will go all the way down and will never increase so that finally no more reservations can be made
reservations in reverse time order example:
https://go.dev/play/p/SzOI9oT8Ipc
// You can edit this code!
// Click here and start typing.
package main
import (
"fmt"
"time"
"golang.org/x/time/rate"
)
func main() {
limiter := rate.NewLimiter(rate.Every(1*time.Second/20), 6)
for i := 10; i > 0; i-- {
t := time.Date(2022, 12, 12, 0, i, 0, 0, time.Local)
ok := limiter.AllowN(t, 3)
fmt.Println("allow: ", ok, "reservation time: ", t, "tokens:", limiter.TokensAt(t))
}
}
// 以下reservation是倒序的：
// allow: true reservation time: 2022-12-12 00:10:00 +0000 UTC tokens: 3 ->满足该预约后，limiter.last = 2022-12-12 00:10:00, limiter.tokens = 3
// allow: true reservation time: 2022-12-12 00:09:00 +0000 UTC tokens: 0 ->满足该预约后，limiter.last = 2022-12-12 00:09:00, limiter.tokens = 0
// allow: false reservation time: 2022-12-12 00:08:00 +0000 UTC tokens: 0
// allow: false reservation time: 2022-12-12 00:07:00 +0000 UTC tokens: 0
// allow: false reservation time: 2022-12-12 00:06:00 +0000 UTC tokens: 0
// allow: false reservation time: 2022-12-12 00:05:00 +0000 UTC tokens: 0
// allow: false reservation time: 2022-12-12 00:04:00 +0000 UTC tokens: 0
// allow: false reservation time: 2022-12-12 00:03:00 +0000 UTC tokens: 0
// allow: false reservation time: 2022-12-12 00:02:00 +0000 UTC tokens: 0
// allow: false reservation time: 2022-12-12 00:01:00 +0000 UTC tokens: 0
// ------ 以下摘自package rate-----
func (lim *Limiter) AllowN(t time.Time, n int) bool {
r := lim.reserveN(t, n, 0)
fmt.Println("after reserve, lim's tokens, last, lastevent:", lim.tokens, lim.last, lim.lastEvent)
return r.ok
}
// reserveN is a helper method for AllowN, ReserveN, and WaitN.
// maxFutureReserve specifies the maximum reservation wait duration allowed.
// reserveN returns Reservation, not *Reservation, to avoid allocation in AllowN and WaitN.
func (lim *Limiter) reserveN(t time.Time, n int, maxFutureReserve time.Duration) Reservation {
lim.mu.Lock()
defer lim.mu.Unlock()
if lim.limit == Inf {
return Reservation{
ok: true,
lim: lim,
tokens: n,
timeToAct: t,
}
} else if lim.limit == 0 {
var ok bool
if lim.burst >= n {
ok = true
lim.burst -= n
}
return Reservation{
ok: ok,
lim: lim,
tokens: lim.burst,
timeToAct: t,
}
}
t, tokens := lim.advance(t)
// Calculate the remaining number of tokens resulting from the request.
tokens -= float64(n)
// Calculate the wait duration
var waitDuration time.Duration
if tokens < 0 {
waitDuration = lim.limit.durationFromTokens(-tokens)
}
// Decide result
ok := n <= lim.burst && waitDuration <= maxFutureReserve
// Prepare reservation
r := Reservation{
ok: ok,
lim: lim,
limit: lim.limit,
}
if ok {
r.tokens = n
r.timeToAct = t.Add(waitDuration)
// Update state
lim.last = t
lim.tokens = tokens
lim.lastEvent = r.timeToAct
}
return r
}
// advance calculates and returns an updated state for lim resulting from the passage of time.
// lim is not changed.
// advance requires that lim.mu is held. if t.Before(last)的设计告诉我们，无法计算历史token
func (lim *Limiter) advance(t time.Time) (newT time.Time, newTokens float64) {
last := lim.last
if t.Before(last) {
last = t
}
// Calculate the new number of tokens, due to time that passed.
elapsed := t.Sub(last)
delta := lim.limit.tokensFromDuration(elapsed)
tokens := lim.tokens + delta
if burst := float64(lim.burst); tokens > burst {
tokens = burst
}
return t, tokens
}
因此，严格来说，rate实际上只适配reservation time不断向前的情况
关于reservation的cancel
直观理解上：
每一个event都占用一定时间跨度内产生的token，并且各个event占用的时间跨度彼此间无重叠部分，即各event在串行占有时间段以获得token
具体到代码上看：
一个reservation占用的token为
发起reserve时刻 的 lim.lastEvent -> reservation.timeToAct 这个时间段内产生的token总数
那么当这个reservation取消的时候，是否释放掉这个时间段内的token总数呢？
rate.go中的代码并不是直接释放掉这个时间段内的token总数
我仍然无法理解以下这一段代码关于restoreTokens的计算
// func (r *Reservation) CancelAt(t time.Time)
// calculate tokens to restore
// The duration between lim.lastEvent and r.timeToAct tells us how many tokens were reserved
// after r was obtained. These tokens should not be restored.
restoreTokens := float64(r.tokens) - r.limit.tokensFromDuration(r.lim.lastEvent.Sub(r.timeToAct))
if restoreTokens <= 0 {
return
}
总的来看，rate这个实现还有不少难以理解之处，也无法计算历史token，因此不必过于深究。体会其【计算】思想即可