halcon脚本-深度学习【语义分割】

---

前言

本文讲解使用halcon的语义分割是使用步骤，标注工具不使用halcon提供的标注工具，而是使用各个深度学习框架都使用的labelMe工具，然后使用hde脚本以及python脚本转化为标准的halcon训练及文件
本文涉及数据标注、数据转化、训练、评估、预测几个模块。

---

一、数据标注

语义分割数据标注主要使用labelImg工具，python安装只需要：pip install labelme 即可，然后在命令提示符输入：labelme即可，如图：


在这里只需要修改“OpenDir“，“OpenDir“主要是存放图片需要标注的路径
选择好路径之后即可开始绘制：

我在平时标注的时候快捷键一般只用到：
createpolygons：（ctrl+N）开始绘制
a：上一张
d：下一张
绘制过程如图：

就只需要一次把目标绘制完成即可。
标注完的文件如图：

---

二、halcon训练预测流程步骤

本模块主要是做数据转换，转化为标准的halcon所需的训练文件

1.Json文件转label图片

本功能主要是解析所有的json文件，然后分别生成对应的label图片，classes.txt的类别文件

1.解析json

读取本地json获取本次标注的所有类别，以及得到各个类别的样本数：

All_label_name = ['_background_']
label_name_dict = {}
for each_img in ImgFile:
    if each_img.split('.')[0]+'.json' in JsonFile:
        ParseDataList.append([each_img,each_img.split('.')[0]+'.json'])
        data = json.load(open(os.path.join(JsonDir,each_img.split('.')[0]+'.json'),encoding='gbk'))
        for shape in sorted(data['shapes'], key=lambda x: x['label']):
            label_name = shape['label']
            if label_name not in All_label_name:
                All_label_name.append(label_name)
            if label_name not in label_name_dict:
                label_name_dict[label_name]=0
            label_name_dict[label_name]+=1
    else:
        ParseDataList.append([each_img, ''])

All_label_name：得到的类别列表
label_name_dict：得到类别样本数字典

2.生成label图片

imageData = data.get('imageData')
if not imageData:
    with open(ImageFile, 'rb') as f:
        imageData = f.read()
        imageData = base64.b64encode(imageData).decode('utf-8')
label_name_to_value = {'_background_': 0}
for shape in sorted(data['shapes'], key=lambda x: x['label']):
    label_name = shape['label']
    if label_name in label_name_to_value:
        label_value = label_name_to_value[label_name]
    else:
        label_value = len(label_name_to_value)
        label_name_to_value[label_name] = label_value
img = utils.img_b64_to_arr(imageData)
lbl, _ = utils.shapes_to_label(
    img.shape, data['shapes'], label_name_to_value
)

lbl：为图片文件，此脚本最后得到的效果图如图：

label的图像解析为：像素值为0的代表背景，像素值为1的代表类别1,像素值代表2的为类别2,以此类推

---

2.转化halcon训练所需的hdict

1.定义输入的文件路径以及输出的路径

* 存放图片的文件夹路径
imageDir:='../DataSet/DataImage'
* 存放上边python生成的label文件夹路径
LabelDir:='../DataSet/Temp_SegDataSet/Labels'
* 存放类别的classes.txt路径
classFile:='../DataSet/Temp_SegDataSet/classes.txt'
* 生成的halcon训练所需的hdict
DataList:='../DataSet/Temp_SegDataSet/dataset.hdict'

备注：此处的hdict其实只是存放各种路径便于halcon对模型进行输入图片

2.读取classes.txt文件

open_file (classFile, 'input', FileHandle)
repeat
    fread_line(FileHandle, oneline, IsEOF)
    if(IsEOF == 1)
        break        
    endif
    if(oneline == ' ' or oneline=='\n')
        continue
    endif
    tuple_regexp_replace (oneline, '\n', '', oneline)
    tuple_length (ClassIndex, Length)
    ClassIndex[Length]:=Length
    if(Length==0)
        oneline:='background'
    endif
    tuple_concat (ClassID, oneline, ClassID)
until (IsEOF)
close_file (FileHandle)

3.设置halcon字典内容

tuple_remove (LabelFiles, Index1, LabelFiles)
tuple_length (ImageDict, Length)
create_dict (tempImgDist)
set_dict_tuple (tempImgDist, 'image_id', Length)
set_dict_tuple (tempImgDist, 'image_file_name', BaseName_Image + '.' + Extension_Image)
set_dict_tuple (tempImgDist, 'segmentation_file_name', BaseName_Label + '.' + Extension_Label)
tuple_concat (ImageDict, tempImgDist, ImageDict)

4.hdict效果展示

使用hdevelop工具可以清楚看到hdict的文件内容如图：

---

3.halcon脚本-模型训练

1.定义变量

这里主要是输入数据的变量路径

* 模型选择，此处是halcon的自带模型，halcon目前此版本无法自定义模型结构
Base_ModelFile:=['./DL_BaseModel/BaseModel_seg_compact.dat','./DL_BaseModel/BaseModel_seg_enhanced.dat']
* 上边使用脚本的生成的hdict文件路径
TrainDataSetDict:='./DataSet/Temp_SegDataSet/dataset.hdict'
* 原始图片的文件存放路径
ImageDir:='./DataSet/DataImage'
* 上边生成的Label的文件存放路径
LabelDir:='./DataSet/Temp_SegDataSet/Labels'
* 训练过程中生成的文件都存放到此文件夹中，后期训练完会自动删除
OutputDir:='./DataSet/Temp_SegDataSet/Output'
* 模型最终保存文件
ModelFile:='./best_Seg.dat'

2.模型参数定义

此变量主要是模型训练的参数定义，可自行修改

* halcon提供的模型，这里选择第0个模型
ModelType:=0
* 是否使用GPU
UseGpu:=true
* 每次迭代的batch_size,-1代表程序自己判断，以最大的可能去设置，否这可自行设置，
BatchSize := -1
* 学习率
lr := 0.0008
* 优化器的参数，可设置0.99
Momentum := 0.9
* 训练次数
NumEpochs := 120
* 学习率在本次训练中修改的次数
LRChangeNum:=5
* 验证集合每经过1次大迭代进行一次验证
EvaluationIntervalEpochs := 1
* 随机数种子
SeedRand := 42
* 设置模型的输入图像的宽度和高度以及通道数
ImageWidth := 512
ImageHeight := 512
ImageNumChannels := 3
* 数据像素值预处理参数
ImageRangeMin := -127
ImageRangeMax := 128
* 归一化
NormalizationType := 'true'
* 巻积操作前的处理方式
DomainHandling := 'full_domain'

3.读取数据集

这里主要是读取数据集然后分割数据集，按85%训练集和15%的验证集划分

read_dict (TrainDataSetDict, [], [], DLDataset)
get_dict_tuple (DLDataset, 'class_ids', ClassIDs)
get_dict_tuple (DLDataset, 'class_names', ClassNames)
set_dict_tuple (DLDataset, 'image_dir', ImageDir)
set_dict_tuple (DLDataset, 'segmentation_dir', LabelDir)
split_dl_dataset (DLDataset, 85, 15, [])
create_dl_preprocess_param ('segmentation', ImageWidth, ImageHeight, ImageNumChannels, ImageRangeMin, ImageRangeMax, NormalizationType, DomainHandling, [], [], [], [], DLPreprocessParam)

4.预现实标注效果

此功能不是必须的

if(ShowExample)
    get_dict_tuple (DLDataset, 'samples', DatasetSamples)
    find_dl_samples (DatasetSamples, 'split', 'train', 'match', SampleIndices)
    tuple_shuffle (SampleIndices, ShuffledIndices)
    read_dl_samples (DLDataset, ShuffledIndices[0:9], DLSampleBatchDisplay)
    create_dict (WindowHandleDict)
    for Index := 0 to |DLSampleBatchDisplay| - 1 by 1
        dev_display_dl_data (DLSampleBatchDisplay[Index], [], DLDataset, ['image','segmentation_image_ground_truth'], [], WindowHandleDict)
        get_dict_tuple (WindowHandleDict, 'segmentation_image_ground_truth', WindowHandleImage)
        dev_set_window (WindowHandleImage[1])
        Text := 'Press Run (F5) to continue'
        dev_disp_text (Text, 'window', 400, 40, 'black', [], [])
        stop ()
    endfor
    dev_display_dl_data_close_windows (WindowHandleDict)
endif

5.学习率修改

此方式不是必须的，不过有会更好

if (|ChangeLearningRateEpochs| > 0)
    create_dict (ChangeStrategy)
    set_dict_tuple (ChangeStrategy, 'model_param', 'learning_rate')
    set_dict_tuple (ChangeStrategy, 'initial_value', lr)
    set_dict_tuple (ChangeStrategy, 'epochs', ChangeLearningRateEpochs)
    set_dict_tuple (ChangeStrategy, 'values', ChangeLearningRateValues)
    GenParamName := [GenParamName,'change']
    GenParamValue := [GenParamValue,ChangeStrategy]
endif

6.读取模型文件

read_dict (DLDatasetFileName, [], [], DLDataset)
open_file (Base_ModelFile[ModelType], 'input_binary', FileHandle)
fread_serialized_item (FileHandle, SerializedItemHandle)
close_file (FileHandle)
deserialize_dl_model (SerializedItemHandle, DLModelHandle)

7.设置模型参数

get_dict_tuple (DLDataset, 'preprocess_param', DLPreprocessParam)
get_dict_tuple (DLDataset, 'class_ids', ClassIDs)
set_dl_model_param_based_on_preprocessing (DLModelHandle, DLPreprocessParam, ClassIDs)
set_dl_model_param (DLModelHandle, 'learning_rate', lr)
set_dl_model_param (DLModelHandle, 'momentum', Momentum)
if (BatchSize == -1)
    set_dl_model_param_max_gpu_batch_size (DLModelHandle, 100)
else
    set_dl_model_param (DLModelHandle, 'batch_size', BatchSize)
endif
if (|WeightPrior| > 0)
    set_dl_model_param (DLModelHandle, 'weight_prior', WeightPrior)
endif
set_dl_model_param (DLModelHandle, 'runtime_init', 'immediately')

8.训练

create_dl_train_param (DLModelHandle, NumEpochs, EvaluationIntervalEpochs, EnableDisplay, SeedRand, GenParamName, GenParamValue, TrainParam)
train_dl_model (DLDataset, DLModelHandle, TrainParam, 0.0, TrainResults, TrainInfos, EvaluationInfos)

训练效果图如图：

4.halcon脚本-模型评估

1.参数定义

RetrainedModelFileName := DataDirectory + '/best_model.hdl'
DLDatasetFileName := DataDirectory + '/dl_dataset.hdict'
* 评估指标
SegmentationMeasures := ['mean_iou','pixel_accuracy','class_pixel_accuracy','pixel_confusion_matrix']
BatchSize := 1
UseGPU := true
* 评估验证集，可视化10张图
NumDisplay := 10

2.模型读取及设置

read_dl_model (RetrainedModelFileName, DLModelHandle)
set_dl_model_param (DLModelHandle, 'batch_size', BatchSize)
if (not UseGPU)
    set_dl_model_param (DLModelHandle, 'runtime', 'cpu')
endif
set_dl_model_param (DLModelHandle, 'runtime_init', 'immediately')

read_dict (DLDatasetFileName, [], [], DLDataset)
create_dict (GenParamEval)
set_dict_tuple (GenParamEval, 'measures', SegmentationMeasures)
set_dict_tuple (GenParamEval, 'show_progress', 'true')

3.模型评估及验证

evaluate_dl_model (DLDataset, DLModelHandle, 'split', 'validation', GenParamEval, EvaluationResult, EvalParams)
create_dict (WindowHandleDict)
create_dict (GenParamEvalDisplay)
set_dict_tuple (GenParamEvalDisplay, 'display_mode', ['measures','absolute_confusion_matrix'])
dev_display_segmentation_evaluation (EvaluationResult, EvalParams, GenParamEvalDisplay, WindowHandleDict)

效果如图：

4.模型保存

此处模型保存使用了halcon自带的序列化方式

serialize_dl_model (DLModelHandle, SerializedItemHandle)
open_file (ModelFile, 'output_binary', FileHandle)
fwrite_serialized_item (FileHandle, SerializedItemHandle)
close_file (FileHandle)

5.halcon脚本-模型预测

1.参数变量设置

testDir:='./DataSet/DataImage'
ModelFile:='./best_Seg.dat'
ClassTxt:='./DataSet/SegDataSet/classes.txt'
UseGpu:=true
ImageWidth :=512
ImageHeight := 512
ImageNumChannels := 3

2.模型参数设置

open_file (ModelFile, 'input_binary', FileHandle)
fread_serialized_item (FileHandle, SerializedItemHandle)
close_file (FileHandle)
deserialize_dl_model (SerializedItemHandle, DLModelHandle)
set_dl_model_param (DLModelHandle, 'batch_size', BatchSizeInference)
set_dl_model_param (DLModelHandle, 'runtime', 'cpu')
set_dl_model_param (DLModelHandle, 'runtime_init', 'immediately')
get_dl_model_param (DLModelHandle, 'class_ids', ClassIDs)

3.模型预测

read_image (ImageBatch, ImageFiles[Index1])
    gen_dl_samples_from_images (ImageBatch, DLSampleBatch)
    preprocess_dl_samples (DLSampleBatch, DLPreprocessParam)
    apply_dl_model (DLModelHandle, DLSampleBatch, ['segmentation_image','segmentation_confidence'], DLResultBatch)
    get_dict_object (SegmentationImage, DLResultBatch, 'segmentation_image')
    threshold (SegmentationImage, ClassRegions, ClassIDs, ClassIDs)
    dev_display_dl_data (DLSampleBatch, DLResultBatch, DatasetInfo, 'segmentation_image_result', GenParamDisplay, WindowHandleDict)
    

效果如图：
以往训练模型已删除，重新训练后上传效果图

---

总结

源码私聊