代码位于:order/common/blockcutter目录,该模块用于将Envelope消息进行打包切分区块
// Ordered should be invoked sequentially as messages are ordered
//
// messageBatches length: 0, pending: false
// - impossible, as we have just received a message
// messageBatches length: 0, pending: true
// - no batch is cut and there are messages pending
// messageBatches length: 1, pending: false
// - the message count reaches BatchSize.MaxMessageCount
// messageBatches length: 1, pending: true
// - the current message will cause the pending batch size in bytes to exceed BatchSize.PreferredMaxBytes.
// messageBatches length: 2, pending: false
// - the current message size in bytes exceeds BatchSize.PreferredMaxBytes, therefore isolated in its own batch.
// messageBatches length: 2, pending: true
// - impossible
从源码注释中可以看出,最多会有2个批次的数据,打包区块基于以下几个因素考虑:
1. BatchTimeout 一个批次最大等待的时长,默认为2s,超过时长立即Cut 形成一个batch
# Batch Timeout: The amount of time to wait before creating a batch.
2. BatchSize.MaxMessageCount 批次最大消息数量,默认值为500,超过500立即Cut
# Absolute Max Bytes: The absolute maximum number of bytes allowed for
# the serialized messages in a batch. The maximum block size is this value
# plus the size of the associated metadata (usually a few KB depending
# upon the size of the signing identities). Any transaction larger than
# this value will be rejected by ordering. If the "kafka" OrdererType is
# selected, set 'message.max.bytes' and 'replica.fetch.max.bytes' on
# the Kafka brokers to a value that is larger than this one.
3.BatchSize.AbsoluteMaxBytes 批次绝对最大Bytes大小,默认配置10MB,注意:如果使用Kafka作为OrderType时,必须设置 message.max.bytes 以及replica.fetch.max.bytes大小超过该值,否则会出现异常
# Absolute Max Bytes: The absolute maximum number of bytes allowed for
# the serialized messages in a batch. The maximum block size is this value
# plus the size of the associated metadata (usually a few KB depending
# upon the size of the signing identities). Any transaction larger than
# this value will be rejected by ordering. If the "kafka" OrdererType is
# selected, set 'message.max.bytes' and 'replica.fetch.max.bytes' on
# the Kafka brokers to a value that is larger than this one.
4.BatchSize.PreferredMaxBytes Block最佳最大值,默认值为2MB,当一个Envelope消息大小大于2MB时立即Cut形成只有一个消息的区块,如果之前Batch有限消息那么BlockCutter将返回2个区块
# Preferred Max Bytes: The preferred maximum number of bytes allowed
# for the serialized messages in a batch. Roughly, this field may be considered
# the best effort maximum size of a batch. A batch will fill with messages
# until this size is reached (or the max message count, or batch timeout is
# exceeded). If adding a new message to the batch would cause the batch to
# exceed the preferred max bytes, then the current batch is closed and written
# to a block, and a new batch containing the new message is created. If a
# message larger than the preferred max bytes is received, then its batch
# will contain only that message. Because messages may be larger than
# preferred max bytes (up to AbsoluteMaxBytes), some batches may exceed
# the preferred max bytes, but will always contain exactly one transaction.
PreferredMaxBytes: 2 MB
blockcutter的核心实现为Ordered方法,具体实现如下:
func (r *receiver) Ordered(msg *cb.Envelope) (messageBatches [][]*cb.Envelope, pending bool) {
//判断当前批次是否为空,如果为空标记区块形成时间
if len(r.pendingBatch) == 0 {
// We are beginning a new batch, mark the time
r.PendingBatchStartTime = time.Now()
}
//获取order配置,主要是获取BatchTimeout,MaxMessageCount, AbsoluteMaxBytes等上述三个参数用于判定形成区块的规则
ordererConfig, ok := r.sharedConfigFetcher.OrdererConfig()
if !ok {
logger.Panicf("Could not retrieve orderer config to query batch parameters, block cutting is not possible")
}
batchSize := ordererConfig.BatchSize()
//获取当前消息大小
messageSizeBytes := messageSizeBytes(msg)
//当当前消息大于PreferredMaxBytes时,直接Cut形成区块Batch
if messageSizeBytes > batchSize.PreferredMaxBytes {
logger.Debugf("The current message, with %v bytes, is larger than the preferred batch size of %v bytes and will be isolated.", messageSizeBytes, batchSize.PreferredMaxBytes)
// cut pending batch, if it has any messages
//该条件就是最多形成2个区块Batch的代码
if len(r.pendingBatch) > 0 {
messageBatch := r.Cut()
messageBatches = append(messageBatches, messageBatch)
}
// create new batch with single message
//创建一个新的Batch只有该消息(大于PreferredMaxBytes值)
messageBatches = append(messageBatches, []*cb.Envelope{msg})
// Record that this batch took no time to fill
r.Metrics.BlockFillDuration.With("channel", r.ChannelID).Observe(0)
return
}
//当消息不大于PreferredMaxBytes时,判断消息该消息与之前的消息大小是否大于PreferredMaxBytes
messageWillOverflowBatchSizeBytes := r.pendingBatchSizeBytes+messageSizeBytes > batchSize.PreferredMaxBytes
//如果大于2MB则形成区块
if messageWillOverflowBatchSizeBytes {
logger.Debugf("The current message, with %v bytes, will overflow the pending batch of %v bytes.", messageSizeBytes, r.pendingBatchSizeBytes)
logger.Debugf("Pending batch would overflow if current message is added, cutting batch now.")
messageBatch := r.Cut()
r.PendingBatchStartTime = time.Now()
messageBatches = append(messageBatches, messageBatch)
}
logger.Debugf("Enqueuing message into batch")
r.pendingBatch = append(r.pendingBatch, msg)
r.pendingBatchSizeBytes += messageSizeBytes
pending = true
//判断Batch消息是否大于MaxMessageCount,如果是则形成区块Batch
if uint32(len(r.pendingBatch)) >= batchSize.MaxMessageCount {
logger.Debugf("Batch size met, cutting batch")
messageBatch := r.Cut()
messageBatches = append(messageBatches, messageBatch)
pending = false
}
return
}
至于参数BatchTimeout最大区块切分超时,主要实现分散到具体共识代码中,如kafka共识中(consensus/kafka/chain.go)的
chain.processMessageToBlocks中处理BatchTimeout定时器,当超时时会发送KafkaMessage_TimeToCut到Kafka中,进行区块切分,具体代码如下:
case <-chain.timer:
if err := sendTimeToCut(chain.producer, chain.channel, chain.lastCutBlockNumber+1, &chain.timer); err != nil {
logger.Errorf("[channel: %s] cannot post time-to-cut message = %s", chain.ChainID(), err)
// Do not return though
counts[indexSendTimeToCutError]++
} else {
counts[indexSendTimeToCutPass]++
}
}
其他blockcutter方法:
// Cut returns the current batch and starts a new one
func (r *receiver) Cut() []*cb.Envelope {
r.Metrics.BlockFillDuration.With("channel", r.ChannelID).Observe(time.Since(r.PendingBatchStartTime).Seconds())
r.PendingBatchStartTime = time.Time{}
batch := r.pendingBatch
r.pendingBatch = nil
r.pendingBatchSizeBytes = 0
return batch
}
用于Batch具体切分,主要是记录切分Batch的时间,设置pendingBatchSizeBytes ,以及Metrics等参数。
作为Order公共模块blockcutter,在solo/kafka/etcdraft 共识模块实现中至关重要,batch切分完成后一般会调用ChainSupport.CreateNextBlock创建区块,调用ChainSupport.writeBlock写入到具体的Ledger中。