【opencv】Layout of the output array img is incompatible with cv::Mat

找了一上午的问题！可算找到解决办法了哈哈哈哈，赶紧记录一下！

---

出错的程序是下面这段，是目标跟踪的demo程序，在给定视频上测试，标注真值bbox与预测bbox。

def main():
    # load config
    cfg.merge_from_file(args.config)
    cfg.CUDA = torch.cuda.is_available()
    device = torch.device('cuda' if cfg.CUDA else 'cpu')

    # create model
    model = ModelBuilder('test')

    # load model
    model = load_pretrain(model, args.snapshot).eval().to(device)

    # build tracker
    tracker = TCTrackTracker(model)

    # 低光照增强
    with torch.no_grad():
        scale_factor = 12
        DCE_net = DCEnet.enhance_net_nopool(scale_factor).cuda()
        DCE_net.eval()
        DCE_net.load_state_dict(torch.load('../preprocessing/DCE/Epoch99.pth'))

    first_frame = True
    if args.video_name:
        video_name = args.video_name.split('/')[-1].split('.')[0]
    else:
        video_name = 'webcam'
    cv2.namedWindow(video_name, cv2.WND_PROP_FULLSCREEN)
    for frame in get_frames(args.video_name):  # frame: ndarray:(720,1280,3)
        with torch.no_grad():
            frame = (np.asarray(frame) / 255.0)
            data_lowlight = torch.from_numpy(frame).float()
            h = (data_lowlight.shape[0] // scale_factor) * scale_factor
            w = (data_lowlight.shape[1] // scale_factor) * scale_factor
            data_lowlight = data_lowlight[0:h, 0:w, :]
            data_lowlight = data_lowlight.permute(2, 0, 1)
            data_lowlight = data_lowlight.cuda().unsqueeze(0)
            frame, _ = DCE_net(data_lowlight)  # torch.Size([1, 3, 720, 1272])
            frame = np.array(255*frame.cpu()).astype(np.uint8).squeeze(0).transpose(1, 2, 0)  # frame: ndarray:(720,1272,3)
        if first_frame:
            try:
                if args.anno_name == '':
                    init_rect = cv2.selectROI(video_name, frame, False, False)
                else:
                    with open(args.anno_name, 'r') as f:
                        anno = f.read().strip()
                    anno = anno.split('\n')
                    anno = [c.split(',') for c in anno]
                    init_rect = [int(i) for i in anno[0]]
                    index = 0
                    cv2.rectangle(frame, (init_rect[0], init_rect[1]),
                                  (init_rect[0] + init_rect[2], init_rect[1] + init_rect[3]),
                                  (0, 0, 255), 2)
            except:
                exit()
            tracker.init(frame, init_rect)
            first_frame = False
        else:
            outputs = tracker.track(frame)
            bbox = list(map(int, outputs['bbox']))
            a = np.concatenate((frame, np.random.randint(0, 255, (720, 200, 3), np.uint8)), axis=1)
            cv2.rectangle(frame, (bbox[0], bbox[1]),
                          (bbox[0]+bbox[2], bbox[1]+bbox[3]),
                          (0, 0, 255), 2)
            if args.anno_name != '':
                index += 1
                gt_bbox = [int(i) for i in anno[index]]
                cv2.rectangle(frame, (gt_bbox[0], gt_bbox[1]),
                              (gt_bbox[0] + gt_bbox[2], gt_bbox[1] + gt_bbox[3]),
                              (0, 255, 0), 2)
            cv2.imshow(video_name, frame)
            cv2.waitKey(40)

我在tracker前加了个低光照增强的操作，所以变动了 cv2.rectangle 输入的图像(frame)，然后程序就无法运行了，但是不会报错，在debug的时候发现程序是在运行第51行的 cv2.rectangle 出错，所以自动跳到了except 里，然后 exit() 了。

写博客写到这的时候我才反应过来为什么第一帧错误的时候（if first_frame部分）不会显示错误信息而后续帧的时候（else部分）才会显示错误信息，要是早点发现就不会因为没有报错信息而折腾这么久了，直接把try-except去掉->错误信息->搜索

我以为是标注的框的坐标超出图像范围了（实际并不是），所以在第51行改成了：

a = np.concatenate((frame, np.random.randint(0, 255, (720, 100, 3), np.uint8)), axis=1)
cv2.rectangle(a, (init_rect[0], init_rect[1]),
              (init_rect[0] + init_rect[2], init_rect[1] + init_rect[3]),
              (0, 0, 255), 2)
# cv2.rectangle(frame, (init_rect[0], init_rect[1]),
#               (init_rect[0] + init_rect[2], init_rect[1] + init_rect[3]),
#               (0, 0, 255), 2)

结果这几行程序竟然可以正常运行了。但这不是个好办法。

整个程序共有三处 cv2.rectangle，我只修改了第一处（第51行），所以第二处（第62行）出错，错误如下：

cv2.error: OpenCV(4.5.5) :-1: error: (-5:Bad argument) in function 'rectangle'
> Overload resolution failed:
>  - Layout of the output array img is incompatible with cv::Mat
>  - Expected Ptr<cv::UMat> for argument 'img'
>  - Layout of the output array img is incompatible with cv::Mat
>  - Expected Ptr<cv::UMat> for argument 'img'

终于拿到错误信息啦，然后根据 Layout of the output array img is incompatible with cv::Mat 搜索。

解决方法直通车

https://stackoverflow.com/questions/23830618/python-opencv-typeerror-layout-of-the-output-array-incompatible-with-cvmat

把输入的frame copy一下，三处都修改（第51、62、68行）。

cv2.rectangle(frame.copy(), (init_rect[0], init_rect[1]),
              (init_rect[0] + init_rect[2], init_rect[1] + init_rect[3]),
              (0, 0, 255), 2)

问题解决。

god & gary bradski knows why…