1、主要错误显示"The 'zeros_like_leaf' operation does not support the type [Slice[Int64 : Int64 : Int64]]"我在analyze_fail.dat文件中确实发现存在该算子和输入类型,但是不知道具体代码位置,以及如何修改,请求帮助。
【截图信息】
【日志信息】(可选,上传日志内容或者附件)
ssh://[email protected]:22/home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/bin/python -u /home/data3/yy/wydt/maskrcnn_gpu_v2/train.py
{'enable_modelarts': 'Whether training on modelarts, default: False', 'data_url': 'Dataset url for obs', 'train_url': 'Training output url for obs', 'data_path': 'Dataset path for local', 'output_path': 'Training output path for local', 'ann_file': 'Ann file, default is val.json.', 'device_target': 'Target device type', 'enable_profiling': 'Whether enable profiling while training,
[WARNING] OPTIMIZER(230183,7fad004dd700,python):2021-11-19-22:07:25.609.085 [mindspore/ccsrc/frontend/optimizer/ad/dfunctor.cc:803] GetPrimalUser] J operation has no relevant primal call in the same graph. Func graph: 12842_1_construct_wrapper.1485, J user: 12842_1_construct_wrapper.1485:construct{[0]: [CNode]1486, [1]: 1487, [2]: 1488, [3]: 1489, [4]: 1490, [5]: gt_num, [6]: gt_mask, [7]: [CNode]1491, [8]: [CNode]1492}
[EXCEPTION] ANALYZER(230183,7fad004dd700,python):2021-11-19-22:07:32.794.195 [mindspore/ccsrc/frontend/operator/composite/multitype_funcgraph.cc:161] GenerateFromTypes] The 'zeros_like_leaf' operation does not support the type [Slice[Int64 : Int64 : Int64]]
There are 11 prototypes for overload function `zeros_like_leaf`, corresponding location info:
1. [IOMonad]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(138)
def _zeros_like_io_monad(x):
^
2. [Dictionary]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(105)
def _zeros_like_dict(x):
3. [unknown]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(91)
def _zeros_like_abstract_error(x):
^
4. [Object:kObjectTypeRefKey]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(77)
def _zeros_like_refkey_type(x):
^
5. [kMetaTypeNone]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(71)
def _zeros_like_type_none(x):
^
6. [Func]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(44)
def _zeros_like_func(x):
^
7. [kMetaTypeTypeType]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(65)
def _zeros_like_type_type(x):
^
8. [Tensor]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(59)
def _zeros_like_tensor(x):
^
9. [Bool]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(35)
def _zeros_like_bool(x):
^
10. [UMonad]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(124)
def _zeros_like_u_monad(x):
^
11. [Number]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(29)
def _zeros_like_scalar(x):
^
Traceback (most recent call last):
File "/home/data3/yy/wydt/maskrcnn_gpu_v2/train.py", line 216, in
train_maskrcnn()
File "/home/data3/yy/wydt/maskrcnn_gpu_v2/train.py", line 211, in train_maskrcnn
model.train(config.epoch_size, dataset, callbacks=cb)
File "/home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/train/model.py", line 726, in train
sink_size=sink_size)
File "/home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/train/model.py", line 504, in _train
self._train_dataset_sink_process(epoch, train_dataset, list_callback, cb_params, sink_size)
File "/home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/train/model.py", line 566, in _train_dataset_sink_process
outputs = self._train_network(*inputs)
File "/home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/nn/cell.py", line 404, in __call__
out = self.compile_and_run(*inputs)
File "/home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/nn/cell.py", line 682, in compile_and_run
self.compile(*inputs)
File "/home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/nn/cell.py", line 669, in compile
_cell_graph_executor.compile(self, *inputs, phase=self.phase, auto_parallel_mode=self._auto_parallel_mode)
File "/home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/common/api.py", line 548, in compile
result = self._graph_executor.compile(obj, args_list, phase, use_vm, self.queue_name)
RuntimeError: mindspore/ccsrc/frontend/operator/composite/multitype_funcgraph.cc:161 GenerateFromTypes] The 'zeros_like_leaf' operation does not support the type [Slice[Int64 : Int64 : Int64]]
There are 11 prototypes for overload function `zeros_like_leaf`, corresponding location info:
1. [IOMonad]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(138)
def _zeros_like_io_monad(x):
^
2. [Dictionary]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(105)
def _zeros_like_dict(x):
3. [unknown]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(91)
def _zeros_like_abstract_error(x):
^
4. [Object:kObjectTypeRefKey]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(77)
def _zeros_like_refkey_type(x):
^
5. [kMetaTypeNone]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(71)
def _zeros_like_type_none(x):
^
6. [Func]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(44)
def _zeros_like_func(x):
^
7. [kMetaTypeTypeType]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(65)
def _zeros_like_type_type(x):
^
8. [Tensor]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(59)
def _zeros_like_tensor(x):
^
9. [Bool]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(35)
def _zeros_like_bool(x):
^
10. [UMonad]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(124)
def _zeros_like_u_monad(x):
^
11. [Number]
In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/composite/multitype_ops/zeros_like_impl.py(29)
def _zeros_like_scalar(x):
^
The function call stack (See file '/home/data3/yy/wydt/maskrcnn_gpu_v2/rank_0/om/analyze_fail.dat' for more details):
# 0 In file /home/data3/yy/wydt/maskrcnn_gpu_v2/src/network_define.py(109)
def construct(self, x, img_shape, gt_bboxe, gt_label, gt_num, gt_mask):
# 1 In file /home/data2/yt_data/anaconda3/envs/pointrend_msp1.5/lib/python3.7/site-packages/mindspore/ops/_grad/grad_implementations.py(139)
return F.tuple_setitem(C.zeros_like(data), idx, dout), C.zeros_like(idx)
^
[WARNING] MD(230183,7fad004dd700,python):2021-11-19-22:07:37.555.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/device_queue_op.cc:74] ~DeviceQueueOp] preprocess_batch: 3; batch_queue: 0, 0, 0, 1, 1, 0, 16; push_start_time: 2021-11-19-22:04:54.028.574, 2021-11-19-22:04:55.511.627, 2021-11-19-22:04:55.511.793; push_end_time: 2021-11-19-22:04:54.028.986, 2021-11-19-22:04:55.511.780, 2021-11-19-22:07:37.087.323.
Process finished with exit code 1
需要检查1运行后保存所有的IR图,可以在运行的时候加上context.set_context(save_graphs=True, save_graphs_path="....") 之后把运行保存下来的ir图