ONNX动态输入尺寸的问题

ONNX模型导出动态尺寸的问题

具体可以看一下这个回答

This is a very good question and it’s a topic we have been discussing repeatedly recently. The answer has three parts:

1. whether onnx supports representing models with dynamic shape
2. whether frontends (like pytorch) supports exporting models with dynamic shape
3. whether backends (like caffe2) supports importing models with dynamic shape

For 1, in serialization format’s level, onnx supports representing models with dynamic shape. If you look at TensorShapeProto which is used to describe the shape of the inputs and outputs, it has dim_param to represent symbolic/dynamic shape. However, at this point, all the tooling (checker, shape inference, etc) basically don’t handle symbolic shape but simply assume them to be static. There are ongoing work to update them, e.g. #632 is going to address this issue in shape inference. The work should not be too much and we can expect it happening soon.

For 2, depends on the type of frontends:

• For tensorflow->onnx and caffe2->onnx, these frontends are doing static graph to static graph conversion, they normally support dynamic shape out of the box. But I do have seen few cases where due to different semantics of some operators between these frameworks and onnx, even the static conversion needs some shapes information, but these can be solved relatively easy (e.g. by adding support of the onnx semantics in the framework).
• For pytorch->onnx or other similar frontends that use tracing (on limited set of inputs sample inputs), dynamic shape is a natural limitation but not technically impossible. By rewriting few places in the model that operates directly on concrete sizes to a form that can let the tracer be aware of these operations (see here for a pytorch example, exported models will be compatible with dynamic shapes. Recently we have scanned through some real world pytorch models (e.g.) and found actually all the size operations are pretty simple and thus easily rewritable.

For 3, the technical difficulties are similar to frontends that do static conversions, and thus should be relatively easy to tackle.

So my conclusion is, yes, dynamic shape is definitely something we should and will support in the near future, just it’s not ready yet at this point.

具体的这个issue下面也有给出了一些简单的策略：

model = onnx.load('model.onnx')
model.graph.input[0].type.tensor_type.shape.dim[0].dim_param = '?'
onnx.save(model, 'dynamic_model.onnx')