openVINO_training_extensions之一：Ubuntu16.04环境下使用tensorflow训练车牌识别

一、环境安装与配置

1. OpenVINO环境安装与配置
   安装openVINO环境请参照官网：
   https://docs.openvinotoolkit.org/2019_R2/_docs_install_guides_installing_openvino_linux.html

下载openvino_training_extensions

• 使用git clone下载到本地
  git clone https://github.com/opencv/openvino_training_extensions.git
  进入下载目录cd openvino_training_extensions
• 安装依赖环境
  sudo apt-get install libturbojpeg python3-tk python3-pip virtualenv
  
  到此openVINO前置环境安装好了

二、 配置安装训练环境

开始安装训练模型的环境

这里选择安装模型车牌识别License Plate Recognition
先决条件：

• Ubuntu * 16.04
• Python * 3.6
• TensorFlow * 1.13.1
• 带有Python API的OpenVINO™2019 R1以上

1. 按照如下命令依次执行创建并激活虚拟环境：
   cd $(git rev-parse --show-toplevel)/tensorflow_toolkit/lpr
   virtualenv venv -p python3 --prompt = “(lpr)”
   echo “. /opt/intel/openvino/bin/ setupvars.sh” >> venv/bin/activate
   . venv/bin/activate
2. 安装依赖模块
   pip3 install -e .
   pip3 install -e …/utils
   如果没有GPU, 请使用CPU_ONLY=true环境变量：
   CPU_ONLY=true pip3 install -e .
   pip3 install -e …/utils
3. 下载并准备所需的子模块
   bash …/prepare_modules.sh
4. 准备数据集
   下载培训数据并将其解压缩到data/synthetic_chinese_license_plates文件夹中。数据集必须由命名为训练图像的文件夹和命名为的crops注释的文本文件组成annotation。使用以下命令：
   • cd $(git rev-parse --show-toplevel)/data/synthetic_chinese_license_plates
   • wget https://download.01.org/opencv/openvino_training_extensions/datasets/license_plate_recognition/Synthetic_Chinese_License_Plates.tar.gz
   • tar xf Synthetic_Chinese_License_Plates.tar.gz
5. 提取训练数据存档后，从中运行Python脚本
   data/synthetic_chinese_license_plates/make_train_val_split.py将注释拆分为train并val通过将路径data/synthetic_chinese_license_plates/annotation 从存档传递到文件作为输入。该脚本会输出data/synthetic_chinese_license_plates/train和 data/synthetic_chinese_license_plates/val注释文件，并提供包含提取数据的文件夹中图像和标签的完整路径。使用以下命令：

• python3 make_train_val_split.py Synthetic_Chinese_License_Plates/annotation
• 运行完之后的文件夹结构如下：
  

6. 训练和评估
7. 配置训练的配置文件
   配置文件：tensorflow_toolkit / lpr / chinese_lp / config.py
   配置文件主要分为三个部分，这里主要配置一下训练的参数：
   class train:训练部分
   class eval:测试验证部分
   class infer:推理部分
   具体代码如下：

import os
from lpr.trainer import LPRVocab

input_shape = (24, 94, 3)  # (height, width, channels)
use_h_concat = False
use_oi_concat = False
max_lp_length = 20
rnn_cells_num = 128

# Licens plate patterns
lpr_patterns = [
  '^<[^>]*>[A-Z][0-9A-Z]{5}$',
  '^<[^>]*>[A-Z][0-9A-Z][0-9]{3}$',
  '^<[^>]*>[A-Z][0-9A-Z]{4}<[^>]*>$',  # , 
  '^WJ<[^>]*>[0-9]{4}[0-9A-Z]$',
]

# Path to the folder where all training and evaluation artifacts will be located
model_dir = os.path.realpath(os.path.join(os.path.dirname(os.path.abspath(__file__)), '..', 'model'))
if not os.path.exists(model_dir):
  os.makedirs(model_dir)


class train:
  # Path to annotation file with training data in per line format: 
  file_list_path = '../../data/synthetic_chinese_license_plates/Synthetic_Chinese_License_Plates/train'#训练的数据集目录

  batch_size = 32 #表示一次训练图片的批次
  steps = 250000 #总共迭代训练步数
  learning_rate = 0.001 #学习率
  grad_noise_scale = 0.001
  opt_type = 'Adam'

  save_checkpoints_steps = 1000      # Number of training steps when checkpoint should be saved
  display_iter = 100

  apply_basic_aug = False
  apply_stn_aug = True
  apply_blur_aug = False

  need_to_save_weights = True
  need_to_save_log = True

  class execution:
    CUDA_VISIBLE_DEVICES = "0"  # Environment variable to control CUDA device used for training
    per_process_gpu_memory_fraction = 0.8  # Fix extra memory allocation issue
    allow_growth = True  # Option which attempts to allocate only as much GPU memory based on runtime allocations


class eval:
  # Path to annotation file with validation data in per line format: 
  file_list_path = '../../data/synthetic_chinese_license_plates/Synthetic_Chinese_License_Plates/val'
  checkpoint = ''
  batch_size = 1

  class execution:
    CUDA_VISIBLE_DEVICES = "0"  # Environment variable to control CUDA device used for training
    per_process_gpu_memory_fraction = 0.8  # Fix extra memory allocation issue
    allow_growth = True  # Option which attempts to allocate only as much GPU memory based on runtime allocations


class infer:
  # Path to text file with list of images in per line format: 
  file_list_path = '../../data/synthetic_chinese_license_plates/Synthetic_Chinese_License_Plates/test_infer'
  checkpoint = ''
  batch_size = 1

  class execution:
    CUDA_VISIBLE_DEVICES = "0"  # Environment variable to control CUDA device used for training
    per_process_gpu_memory_fraction = 0.8  # Fix extra memory allocation issue
    allow_growth = True  # Option which attempts to allocate only as much GPU memory based on runtime allocations


vocab, r_vocab, num_classes = LPRVocab.create_vocab(train.file_list_path,
                                                    eval.file_list_path,
                                                    use_h_concat,
                                                    use_oi_concat)

8. 开始训练：python3 tools/train.py chinese_lp/config.py
   
   
9. 训练和评估工件默认存储在中lpr/chinese_lp/model，可以通过tensorboard来可视化查看训练状态
   运行命令 tensorboard --logdir=./model ，然后在浏览器中打开 http://localhost:6006.即可查看
   

三、导出模型并使用openVINO模型优化器装化成IR

导出训练的模型并通过openVINO模型优化器转换成IR中间表示文件

1. 执行命令
   python3 tools/export.py --data_type FP32 --output_dir model/export chinese_lp/config.py
   
   
   导出后的文件结构：
   
2. 使用转换后的模型IR文件来测试识别
   准备好一张车牌的图片，执行如下命令进行推理识别：
   python3 tools/infer_ie.py --model model/export/IR/FP32/lpr.xml --device=CPU --cpu_extension="${INTEL_OPENVINO_DIR}/deployment_tools/inference_engine/lib/intel64/libcpu_extension_avx2.so" --config chinese_lp/config.py