基于华为atlas的烟火检测实战
1、下载官方yolov5的v6.1版本
git clone https://github.com/ultralytics/yolov5.git
git checkout v6.1
2、烟火数据集准备：
tree -d
Images/train/目录下图片
Labels/train/目录下标签
3、数据格式转化：
数据集采用labelimg标注，xml文件转化为txt文件的代码如下，
import os.path
import xml.etree.ElementTree as ET
# 类别
class_names = ["fire"]
# voc数据集路径
vocPath = './'
# xml文件路径
xmlPath = vocPath + r'\Annotations'
# xml转换后txt文件存放路径
txtPath = vocPath + r'\txts'
files = []
if not os.path.exists(txtPath):
os.makedirs(txtPath)
for root, dirs, files in os.walk(xmlPath):
None
number = len(files)
print(number)
i = 0
while i < number:
name = files[i][0:-4]
xml_name = name + ".xml"
txt_name = name + ".txt"
xml_file_name = os.path.join(xmlPath, xml_name)
txt_file_name = os.path.join(txtPath, txt_name)
xml_file = open(xml_file_name, encoding='gb18030',errors='ignore')
tree = ET.parse(xml_file)
root = tree.getroot()
w = int(root.find('size').find('width').text)
h = int(root.find('size').find('height').text)
f_txt = open(txt_file_name, 'w+')
content = ""
first = True
for obj in root.iter('object'):
name = obj.find('name').text
# 若只有一类 ,即 class_num = 0
class_num = class_names.index(name)
xmlbox = obj.find('bndbox')
x1 = int(xmlbox.find('xmin').text)
x2 = int(xmlbox.find('xmax').text)
y1 = int(xmlbox.find('ymin').text)
y2 = int(xmlbox.find('ymax').text)
if first:
content += str(class_num) + " " + \
str((x1 + x2) / 2 / w) + " " + str((y1 + y2) / 2 / h) + " " + \
str((x2 - x1) / w) + " " + str((y2 - y1) / h)
first = False
else:
content += "\n" + \
str(class_num) + " " + \
str((x1 + x2) / 2 / w) + " " + str((y1 + y2) / 2 / h) + " " + \
str((x2 - x1) / w) + " " + str((y2 - y1) / h)
print(content)
f_txt.write(content)
f_txt.close()
xml_file.close()
i += 1
4、配置yaml文件：
data/fire_smoke.yaml
5、启动训练：
python train.py --img 640 --epochs 100 --data ./data/fire_smoke.yaml --weights yolov5s.pt
6、Pt模型转化为onnx模型
python export.py --weights best.pt --simplify
7、模型转化为atlas模型：
mkdir -p models/yolov5_fire_smoke
新建insert_op.cfg
aipp_op {
aipp_mode : static
related_input_rank : 0
input_format : YUV420SP_U8
src_image_size_w : 640
src_image_size_h : 640
crop : false
csc_switch : true
rbuv_swap_switch : false
matrix_r0c0 : 256
matrix_r0c1 : 0
matrix_r0c2 : 359
matrix_r1c0 : 256
matrix_r1c1 : -88
matrix_r1c2 : -183
matrix_r2c0 : 256
matrix_r2c1 : 454
matrix_r2c2 : 0
input_bias_0 : 0
input_bias_1 : 128
input_bias_2 : 128
var_reci_chn_0 : 0.0039216
var_reci_chn_1 : 0.0039216
var_reci_chn_2 : 0.0039216
}
新建yolov5_add_bs1_fp16.cfg
CLASS_NUM=2
BIASES_NUM=18
BIASES=10,13,16,30,33,23,30,61,62,45,59,119,116,90,156,198,373,326
SCORE_THRESH=0.25
#SEPARATE_SCORE_THRESH=0.001,0.001,0.001,0.001,0.001,0.001,0.001,0.001,0.001,0.001
OBJECTNESS_THRESH=0.0
IOU_THRESH=0.5
YOLO_TYPE=3
ANCHOR_DIM=3
MODEL_TYPE=2
RESIZE_FLAG=0
YOLO_VERSION=5
新建fire_smoke.names
fire
smoke
将yolov5的best.onnx模型拷贝到当前目录，进行onnx转化为om，输出yolov5_add_bs1_fp16.om
输入npu-smi info
atc --input_shape="images:1,3,640,640" --out_nodes="/model.24/Transpose:0;/model.24/Transpose_1:0;/model.24/Transpose_2:0" --output_type=FP32 --input_format=NCHW --output="./yolov5_add_bs1_fp16" --soc_version=Ascend310P3 --framework=5 --model="./best.onnx" --insert_op_conf=./insert_op.cfg
8、修改华为atlas推理的pipeline文件
修改pipeline/fire_smoke.pipeline文件
9、基于pipenine实现推理代码
实现简单的yolov5的推理函数yolov5.py，并运行
python3 yolov5.py
10、流媒体引擎ZLMediaKit搭建：
编译库
git clone https://github.com/ZLMediaKit/ZLMediaKit.git
cd ZLMediaKit/
git submodule update --init
mkdir build
cd build
cmake ..
make -j4
11、运行流媒体引擎库：
cd ZLMediaKit/release/linux/Debug
#通过-h可以了解启动参数
./MediaServer -h
#以守护进程模式启动
./MediaServer -d &
12、运行算法服务：
python3 server.py >&/dev/null&13、运行视频处理业务：
python3 push_stream.py在VLC中进行播放，
rtmp://10.100.1.1:19350/live/test
http://10.100.1.1:19350/live/test.live.flv
14、信创化容器制作：
实现信创化的docker file用于生成docker image，初始系统选择openeuler-20.09系统，docker file文件内容如下，
FROM opstool/openeuler:20.09
RUN mv /usr/bin/sh /usr/bin/sh.bak && ln -s /usr/bin/bash /usr/bin/sh
RUN sed -i 's/http:\/\/repo.openeuler.org/https:\/\/repo.huaweicloud.com\/openeuler/g' /etc/yum.repos.d/openEuler.repo
RUN yum install -y gcc cmake make
RUN yum install -y wget tar zlib-devel.aarch64
RUN yum install -y mesa-libGL.aarch64 openssl-devel
RUN yum install -y libffi-devel
RUN wget https://www.python.org/ftp/python/3.9.12/Python-3.9.12.tgz
RUN tar -xzvf Python-3.9.12.tgz
RUN cd Python-3.9.12 &&./configure --prefix=/usr/local/python3.9.12 --enable-shared && make -j8 && make install
RUN cp /usr/local/python3.9.12/lib/libpython3.9.so.1.0 /usr/lib
#RUN export LD_LIBRARY_PATH=/usr/local/python3.9.12/lib:$LD_LIBRARY_PATH
#RUN export PATH=/usr/local/python3.9.12/bin:$PATH
RUN echo "export LD_LIBRARY_PATH=/usr/local/python3.9.12/lib:$LD_LIBRARY_PATH" >> ~/.bashrc
RUN echo "export PATH=/usr/local/python3.9.12/bin:$PATH" >> ~/.bashrc
RUN source ~/.bashrc
RUN yum install -y python3-pip
RUN echo "source /data/ai_install_packages/MindX_SDK/mxVision/set_env.sh" >> ~/.bashrc
RUN echo "source /usr/local/Ascend/ascend-toolkit/set_env.sh" >> ~/.bashrc
RUN /usr/local/python3.9.12/bin/pip3 install opencv-python opencv-python-headless Pillow -i https://pypi.tuna.tsinghua.edu.cn/simple/
RUN /usr/local/python3.9.12/bin/pip3 install attrs cloudpickle decorator psutil scipy synr==0.5.0 tornado absl-py -i https://pypi.tuna.tsinghua.edu.cn/simple/
RUN /usr/local/python3.9.12/bin/pip3 install absl-py Flask gunicorn tqdm requests -i https://pypi.tuna.tsinghua.edu.cn/simple/
15、docker环境部署：
docker build . -t sitri/openeuler-20.09-ai:1.0.0
docker run --restart=always -itd -u root \
--network host \
--device=/dev/davinci0 \
--device=/dev/davinci_manager \
--device=/dev/devmm_svm \
--device=/dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /var/log/npu:/var/log/npu \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /usr/slog:/usr/slog \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/tools/:/usr/local/Ascend/driver/tools/ \
-v /usr/local/Ascend/add-ons/:/usr/local/Ascend/add-ons/ \
-v /usr/local/Ascend/ascend-toolkit/:/usr/local/Ascend/ascend-toolkit/ \
-v /data:/data \
--name="firesmoke" \
-w /data/ai_install_packages/fire_smoke \
sitri/openeuler-20.09-ai:1.0.0 \
/bin/bash \
-c "source ~/.bashrc && gunicorn -c gunicorn_config.py server:app"
16、整体效果
基于flask实现烟火检测算法的http服务，然后实现视频解码-AI识别-结果绘制于视频上进行视频编码的业务代码。
最终效果如下，上边为业务代码、左下角为流媒体引擎代码、右下角为AI服务代码、中间为AI实时视频识别效果。
references:
文档：
昇腾社区-官网丨昇腾万里 让智能无所不及
案例：
昇腾社区-官网丨昇腾万里 让智能无所不及
github：
ascend_community_projects: MindX边缘开发套件社区代码仓库
samples: CANN Samples
容器镜像：
AscendHub