TFLite: 模型文件的结构和解析器
TFLite Model file
Model
table Model {
// Version of the schema.
version:uint;
// A list of all operator codes used in this model. This is
// kept in order because operators carry an index into this
// vector.
operator_codes:[OperatorCode];
// All the subgraphs of the model. The 0th is assumed to be the main
// model.
subgraphs:[SubGraph];
// A description of the model.
description:string;
// Buffers of the model.
// NOTE: It is required that the first entry in here is always an empty
// buffer. This is so that the default buffer index of zero in Tensor
// will always refer to a valid empty buffer.
buffers:[Buffer];
}
OperatorCode
// An OperatorCode can be an enum value (BuiltinOperator) if the operator is a
// builtin, or a string if the operator is custom.
table OperatorCode {
builtin_code:BuiltinOperator;
custom_code:string;
}
// A list of builtin operators. Builtin operators are slightly faster than custom
// ones, but not by much. Moreover, while custom operators accept an opaque
// object containing configuration parameters, builtins have a predetermined
// set of acceptable options.
enum BuiltinOperator : byte {
ADD = 0,
AVERAGE_POOL_2D = 1,
CONCATENATION = 2,
CONV_2D = 3,
DEPTHWISE_CONV_2D = 4,
// DEPTH_TO_SPACE = 5,
// DEQUANTIZE = 6,
EMBEDDING_LOOKUP = 7,
}
其中的Op Code(每个op code的Indice),注意这个和Op(具体的运算:input/output tensor,具体的操作)区别开
TFLite模型文件格式：主结构体是Modle，其中的operator_codes定义了该模型用到的算子Op Code;
subgraphs定义了Model中的子图，其中第一个子图是主图；
buffers:是数据存储区域，主要保存是模型的权重
SubGraph
Model中最主要的结构体，定义了图的具体结构
// The root type, defining a model.
table SubGraph {
// A list of all tensors used in this model.//all tensors中又分为inputs/output?
tensors:[Tensor];
// Indices of the input tensors.
inputs:[int];
// Indices of the output tensors.
outputs:[int];
// All operators, in execution order.
operators:[Operator];
// Name of subgraph (used for debugging).
name:string;
}
tensors:定义了子图中的所有tensor
inputs/outputs通过索引的形式定义了整个图的输入、输出tensor
operators定义了子图中的各个算子
Tensor
table Tensor {
// The tensor shape. The meaning of each entry is operator-specific but
// builtin ops use: [batch size, number of channels, height, width] (That's
// Tensorflow's NCHW).
shape:[int];
type:TensorType;
// An index that refers to the buffers table at the root of the model. Or,
// if there is no data buffer associated (i.e. intermediate results), then
// this is 0 (which refers to an always existent empty buffer).
//
// The data_buffer itself is an opaque container, with the assumption that the
// target device is little-endian. In addition, all builtin operators assume
// the memory is ordered such that if `shape` is [4, 3, 2], then index
// [i, j, k] maps to data_buffer[i*3*2 + j*3 + k].
buffer:uint;
name:string; // For debugging and importing back into tensorflow.
quantization:QuantizationParameters; // Optional.
}
Tensor定义了形状、类型和对应的buffer索引值
Operator
// An operator takes tensors as inputs and outputs. The type of operation being
// performed is determined by an index into the list of valid OperatorCodes,
// while the specifics of each operations is configured using builtin_options
// or custom_options.
table Operator {
// Index into the operator_codes array. Using an integer here avoids
// complicate map lookups.
opcode_index:uint;
inputs:[int];
outputs:[int];
builtin_options:BuiltinOptions;
custom_options:[ubyte];
}
operator是subGraph中的最重要结构体，定义了图的形状
opcode_index定义了算子的索引
inputs/outputs定义了operator输入、输出的tensor的索引值.
TFlLite解析器
数据流图用边和节点来表示，边就是tensor，在节点进行运算得到另一个tensor.
解析器怎么解析模型文件：通过mmap的形式把model文件加载到内存，这样得到了Ops, Tensors和buffers, 模型的buffer是只读的用于保存权重信息，可数值可变的Tensor生成可读写的buffer,解析器还包含执行具体计算的代码称之为kernel.
模型文件中的tensor被加载为TfLiteTensor格式，并集中存放于TfLiteContext中。所有TfLiteTensor都有指针指向只读内存区域或可读写内存区域。
模型中的operator被加载为TfliteNode包含了输入输出tensor的索引值
ops对应的操作符存储与TfLiteRegistration它包含了指向kernel opetator的函数指针
OpResolver负责维护函数指针对应关系
在加载模型时，会确定op的执行顺序，并依次执行。
模型文件结构对应的文件是contrib/lite/schema/schema_v3.fbs
解析器对应的数据结构文件是：
lite/context.h; lite/model.h; lite/interpreter.h; lite/kernels/register.h
视频见：https://www.bilibili.com/video/av24219725/