Labelme与FastDeploy相结合，辅助分割标注
1.项目背景
大家可能遇到这样的情况(可能只有我遇到)。已经有100个已经标注好的分割数据，也训练好一个模型。但是突然觉得样本数量不够，想增加到500个。但是如果完全手动标注500个，估计会很累。然后现在你已经训练好了一个模型，能不能利用这个训练好的模型对500个数据进行推理，然后对结果进行精调。但是一般推理结果mask都是一个数组或者图片，不好对其进行标注微调。
现在想到一个方法就是：使用FastDeploy高效推理工具(在cup下推理都好快),加载训练好的模型，得到预测的mask结果，然后转换成json图片，再用labelme进行读取，进行手动微调。把推理这部分代码写入labelme中，增加一些按钮，就有如下的工具。
魔改的Labelme的Github地址:https://github.com/richarddddd198/Labelme-auto-seg
方法使用如下
2.FastDeploy
FastDeploy是一款全场景、易用灵活、极致高效的AI推理部署工具。提供开箱即用的云边端部署体验, 支持超过150+ Text, Vision, Speech和跨模态模型，并实现端到端的推理性能优化。包括图像分类、物体检测、图像分割、人脸检测、人脸识别、关键点检测、抠图、OCR、NLP、TTS等任务，满足开发者多场景、多硬件、多平台的产业部署需求。
Github仓库地址：https://github.com/PaddlePaddle/FastDeploy
#clone paddleseg套件，训练一个语义分割模型
!git clone https://gitee.com/paddlepaddle/PaddleSeg.git
#安装paddleseg 和fastdeploy
!pip install paddleseg fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
3.数据
任务类型：语义分割
数据格式：2D jpg
分割目标：头部磁共振的胼胝体，
分割类别：1
样本数量：120张
#解压数据
!unzip -o /home/aistudio/data/data91411/callosum.zip -d /home/aistudio/work
# 生成文件列表文件
import os
import numpy as np
DATA_ROOT_DIR = '/home/aistudio/work/callosum'
def make_list():
img_list = [img for img in os.listdir(os.path.join(DATA_ROOT_DIR, 'origin'))]
data_path_list = []
for image_id in img_list:
image_path = os.path.join(DATA_ROOT_DIR, 'origin',image_id)
label_path = os.path.join(DATA_ROOT_DIR, 'mask',image_id.split('.')[0]+'.png')
data_path_list.append((image_path, label_path))
np.random.seed(5)
np.random.shuffle(data_path_list)
total_len = len(data_path_list)
train_data_len = int(total_len*0.8)
train_data = data_path_list[0 : train_data_len]
val_data = data_path_list[train_data_len : ]
with open(os.path.join(DATA_ROOT_DIR, 'train_list.txt'), "w") as f:
for image, label in train_data:
f.write(f"{image} {label}\n")
with open(os.path.join(DATA_ROOT_DIR, 'val_list.txt'), "w") as f:
for image, label in val_data:
f.write(f"{image} {label}\n")
if __name__ == '__main__':
make_list()
4.配置训练文件并开始训练
batch_size: 6
iters: 5000
train_dataset:
type: Dataset
dataset_root: /home/aistudio/
train_path: /home/aistudio/work/callosum/train_list.txt
num_classes: 2
transforms:
- type: RandomHorizontalFlip
- type: RandomRotation
max_rotation: 15
- type: RandomDistort
brightness_range: 0.2
contrast_range: 0.2
saturation_range: 0.2
- type: Normalize
- type: Resize
target_size: [256, 256]
mode: train
val_dataset:
type: Dataset
dataset_root: /home/aistudio/
val_path: /home/aistudio/work/callosum/val_list.txt
num_classes: 2
transforms:
- type: Resize
target_size: [256, 256]
- type: Normalize
mode: val
optimizer:
type: sgd
momentum: 0.9
weight_decay: 4.0e-5
lr_scheduler:
type: PolynomialDecay
learning_rate: 0.02
end_lr: 0
power: 0.9
loss:
types:
- type: CrossEntropyLoss
coef: [1]
model:
type: UNet
num_classes: 2
use_deconv: False
pretrained: Null
#训练
%cd ~/PaddleSeg/
!python train.py --config /home/aistudio/configcallosum.yml --do_eval --use_vdl --save_interval 48 --save_dir output_callosum
/home/aistudio/PaddleSeg
#验证
!python tools/val.py \
--config /home/aistudio/configcallosum.yml \
--model_path output_callosum/best_model/model.pdparams
#5000轮后分割精度如下
"""
2022-12-03 09:44:01 [INFO] [EVAL] #Images: 24 mIoU: 0.9327 Acc: 0.9983 Kappa: 0.9280 Dice: 0.9640
2022-12-03 09:44:01 [INFO] [EVAL] Class IoU:
[0.9983 0.8671]
2022-12-03 09:44:01 [INFO] [EVAL] Class Precision:
[0.9989 0.9462]
2022-12-03 09:44:01 [INFO] [EVAL] Class Recall:
[0.9994 0.9121]
"""
#导出模型，好让FastDeploy加载推理
#--input_shape 的 256 256 就是训练是图片出入模型的尺寸。
!python tools/export.py --config /home/aistudio/configcallosum.yml \
--model_path output_callosum/best_model/model.pdparams \
--save_dir output_callosum/inference_model_callosum \
--input_shape 1 3 256 256
W1203 09:45:02.714906 6309 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W1203 09:45:02.719321 6309 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
2022-12-03 09:45:04 [INFO] Loaded trained params of model successfully.
2022-12-03 09:45:06 [INFO] The inference model is saved in output_callosum/inference_model_callosum
5.使用fastdeploy加载模型推理
1.先用vision.segmentation.PaddleSegModel加载刚才Paddleseg导出训练好的模型
2.使用model.predict进行推理。
3.result.label_map得到对应的预测值，reshape与输入图片尺寸一致即可得到预测的mask
import numpy as np
import cv2
import fastdeploy.vision as vision
import matplotlib.pyplot as plt
model = vision.segmentation.PaddleSegModel('/home/aistudio/PaddleSeg/output_callosum/inference_model_callosum/model.pdmodel',
'/home/aistudio/PaddleSeg/output_callosum/inference_model_callosum/model.pdiparams',
'/home/aistudio/PaddleSeg/output_callosum/inference_model_callosum/deploy.yaml')
im = cv2.imread("/home/aistudio/work/callosum/origin/176.jpg")
result = model.predict(im.copy())
mask = np.array(result.label_map).reshape(256,256).astype(np.uint8)
plt.imshow(mask,'gray')
plt.show()
[INFO] fastdeploy/vision/common/processors/transform.cc(93)::FuseNormalizeHWC2CHW Normalize and HWC2CHW are fused to NormalizeAndPermute in preprocessing pipeline.
[INFO] fastdeploy/vision/common/processors/transform.cc(159)::FuseNormalizeColorConvert BGR2RGB and NormalizeAndPermute are fused to NormalizeAndPermute with swap_rb=1
[INFO] fastdeploy/backends/openvino/ov_backend.cc(199)::InitFromPaddle Compile OpenVINO model on device_name:CPU.
[INFO] fastdeploy/runtime.cc(532)::Init Runtime initialized with Backend::OPENVINO in Device::CPU.
ce::CPU.
6.把mask图转换成json文件
对预测的mask 转换成labelme格式的json文件，让labelme可以识别。
下图是labelme读取重新生成的json文件。
import base64
import os
import json
# 图片转换成base64
def image_to_base64(path):
with open(path, 'rb') as img:
b64encode = base64.b64encode(img.read())
s = b64encode.decode()
b64_encode = 'data:image/jpeg;base64,%s' % s
return b64_encode
def get_points(contour,isRemoveSamlleTarget=True):
"""对轮廓点做适当的处理，例如点太小的目标去掉，或者点太多的，间隔取点"""
num = len(contour[:, 0, 0])
if isRemoveSamlleTarget:
if num < 10: #可以适当去除小目标
return contour[:, 0], 0
if num > 200: #点太多，可以适当减少点
hundred = num // 30 # 步长
tem = contour[:, 0][::hundred]
return tem, 1
else:
return contour[:, 0], 1
def generate_json(name, h, w, shapes,imageData):
#按格式要求生成字典，再换成json
dict = {}
dict["version"] = "5.1.0"
dict["flags"] = {}
dict["shapes"] = shapes
dict["imagePath"] = name
dict["imageData"] = imageData
dict["imageHeight"] = h
dict["imageWidth"] = w
return json.dumps(dict, ensure_ascii=False,indent=4)
def generateJosn(img_path,mask,label_name):
"""对mask找到轮廓，生成坐标点"""
img_base = os.path.basename(img_path)
shapeslist = list()
h, w = mask.shape
for label in label_name.keys():
temp = mask.copy()
temp[temp == label] = 255
temp[temp!= 255] = 0
ret, binary = cv2.threshold(temp, 0, 255, cv2.THRESH_BINARY )
binary = np.uint8(binary)
contours, heriachy = cv2.findContours(binary, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
for contour in contours:
shapesdict = {"label":'', "points":'', "group_id":"null", "shape_type":"polygon", "flags":{}}
points, flag = get_points(contour)
points = points.tolist()
if flag ==1:
shapesdict['label'] = label_name[label]
shapesdict['points'] = points
shapeslist.append(shapesdict)
imageData = image_to_base64(img_path).split(',')[1]
json_content = generate_json(img_base,h,w,shapeslist,imageData)
return json_content
im_path="/home/aistudio/work/callosum/origin/176.jpg"
label_name = {1:"callosum"}#这里需要手动新建标签，让labelme知道对应的类别的标签名
save_jons_path = os.path.join('/home/aistudio/',os.path.basename(im_path).split('.')[0]+'.json')
with open(save_jons_path,'w',encoding='utf8') as f:
json_content = generateJosn(im_path,mask,label_name)
f.write(json_content)
此文章为搬运
原项目链接