目标跟踪、目标检测、前景分割不分家,如SiamMask、SiamR-CNN
这篇文章针对目标框可起到很好地分割效果。
注:原代码的运行环境为Ubuntu,本文在Windows10系统下完成配置。
1、论文下载:
Interactive Object Segmentation with Inside-Outside Guidance. [paper][code]
2、代码下载:
https://github.com/shiyinzhang/Inside-Outside-Guidance
3、建立且激活虚拟环境IOG
conda create -n IOG python=3.7.0
activate IOG
4、安装torch和torchvision
pip install torch===1.4.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install torchvision===0.5.0 -f https://download.pytorch.org/whl/torch_stable.html
5、安装pycocotools和opencv-python
pip install pycocotools opencv-python
6、下载预训练模型放到工程路径
链接:https://pan.baidu.com/s/1-BViduSNAd-lwe81tMv-2A
提取码:72sj
7、下载PascalVOC2012数据集
Pascal VOC Dataset Mirror
只下载图中标注的数据集,然后解压即可。
8、打开mypath.py设置路径
9、打开test.py设置预训练模型路径
10、运行python test.py遇到错误
Traceback (most recent call last):
File "test.py", line 2, in
import scipy.misc as sm
ModuleNotFoundError: No module named 'scipy'
错误原因是: 没有安装scipy,解决方法:pip install scipy
11、再次运行遇到错误
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.BrokenPipeError: [Errno 32] Broken pipe
错误原因:线程冲突,解决办法:把test.py的71行num_workers=1去掉。
11、再次运行,出现错误
Traceback (most recent call last):
File "test.py", line 91, in
sm.imsave(os.path.join(save_dir_res_list[0], metas['image'][0] + '-' + metas['object'][0] + '.png'), result)
AttributeError: module 'scipy.misc' has no attribute 'imsave'
错误原因:scipy版本不兼容问题,解决方法:将scipy降级到1.2.1版本之下即可,pip install scipy==1.2.1
12、再次运行成功。
结果存放在run_0/Results路径下
13、测试单张图片
1)新建 inference.py
from datetime import datetime
import scipy.misc as sm
from collections import OrderedDict
import glob
import numpy as np
import socket
# PyTorch includes
import torch
import torch.optim as optim
from torchvision import transforms
from torch.utils.data import DataLoader
# Custom includes
from dataloaders.combine_dbs import CombineDBs as combine_dbs
import dataloaders.pascal as pascal
import dataloaders.sbd as sbd
from dataloaders import custom_transforms as tr
from networks.loss import class_cross_entropy_loss
from dataloaders.helpers import *
from networks.mainnetwork import *
import matplotlib.pyplot as plt
from PIL import Image
import cv2
import argparse
def process(image_name):
# Set gpu_id to -1 to run in CPU mode, otherwise set the id of the corresponding gpu
gpu_id = 0
device = torch.device("cuda:"+str(gpu_id) if torch.cuda.is_available() else "cpu")
if torch.cuda.is_available():
print('Using GPU: {} '.format(gpu_id))
# Setting parameters
resume_epoch = 100 # test epoch
nInputChannels = 5 # Number of input channels (RGB + heatmap of IOG points)
# Network definition
modelName = 'IOG_pascal'
net = Network(nInputChannels=nInputChannels,
num_classes=1,
backbone='resnet101',
output_stride=16,
sync_bn=None,
freeze_bn=False)
# load pretrain_dict
pretrain_dict = torch.load('IOG_PASCAL_SBD.pth')
net.load_state_dict(pretrain_dict)
# net.to(device)
# Generate result of the validation images
net.eval()
image = np.array(Image.open(image_name).convert('RGB'))
im_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
roi = cv2.selectROI(im_rgb)
image = image.astype(np.float32)
bbox = np.zeros_like(image[..., 0])
# bbox[0: 130, 220: 320] = 1 # for ponny
# bbox[220: 390, 370: 570] = 1
bbox[int(roi[1]):int(roi[1]+roi[3]), int(roi[0]):int(roi[0]+roi[2])] = 1
void_pixels = 1 - bbox
sample = {'image': image, 'gt': bbox, 'void_pixels': void_pixels}
trns = transforms.Compose([
tr.CropFromMask(crop_elems=('image', 'gt','void_pixels'), relax=30, zero_pad=True),
tr.FixedResize(resolutions={'gt': None, 'crop_image': (512, 512), 'crop_gt': (512, 512), 'crop_void_pixels': (512, 512)},flagvals={'gt':cv2.INTER_LINEAR,'crop_image':cv2.INTER_LINEAR,'crop_gt':cv2.INTER_LINEAR,'crop_void_pixels': cv2.INTER_LINEAR}),
tr.IOGPoints(sigma=10, elem='crop_gt',pad_pixel=10),
tr.ToImage(norm_elem='IOG_points'),
tr.ConcatInputs(elems=('crop_image', 'IOG_points')),
tr.ToTensor()])
tr_sample = trns(sample)
inputs = tr_sample['concat'][None]
# inputs = inputs.to(device)
outputs = net.forward(inputs)[-1]
# outputs = fine_out.to(torch.device('cpu'))
pred = np.transpose(outputs.data.numpy()[0, :, :, :], (1, 2, 0))
pred = 1 / (1 + np.exp(-pred))
pred = np.squeeze(pred)
gt = tens2image(tr_sample['gt'])
bbox = get_bbox(gt, pad=30, zero_pad=True)
result = crop2fullmask(pred, bbox, gt, zero_pad=True, relax=0,mask_relax=False)
light = np.zeros_like(image)
light[:, :, 2] = 255.
alpha = 0.8
blending = (alpha * light + (1 - alpha) * image) * result[..., None] + (1 - result[..., None]) * image
blending[blending > 255.] = 255
cv2.imshow('resulting segmentation', cv2.cvtColor(blending.astype(np.uint8), cv2.COLOR_RGB2BGR))
cv2.waitKey(0)
cv2.destroyAllWindows()
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Run class agnostic segmentation')
parser.add_argument('--image_name', type=str, default='samples/IMG-20201203-WA0023.jpg',
help='path to target image')
args = parser.parse_args()
process(args.image_name)
2)运行python inference.py --image_name E:/Datasets/OTB2015/Lemming/img/0001.jpg
3)选取目标框后,回车
4)分割结果