树梅派上搭建tensorflow+opencv+pi camera的物体识别

前言

此教程参考自https://www.jianshu.com/p/ea5abe01aaf1 略有改动。

硬件及软件版本

1. 树梅派3b+
2. 树梅派系统：2018-04-18-raspbian-stretch(自带python 3.5 和 2.7)
3. 此处使用3.5版本
4. tensorflow 1.13.1
5. cv2 3.3.0
6. pi camera
7. libprotoc 3.8.0

安装及环境配置

可以安装docker通过pull别人打包好的镜像使用docker容器来运行

以下为自行配置：
-安装 opencv-python包

sudo pip3 install opencv-python

此处注意安装之后，检查安装

python3
>>> import cv2 

如果出现报错：ImportError: libjasper.so.1: cannot open shared object file: No such file or directory

那么请添加依赖包，详情：import cv2出错

• 安装 matplotlib包

sudo pip3 install matplotlib

• 安装 protobuf

请参考简书的：安装protobuf
此安装过程时间有点长，请耐心等待

• 安装 tensorflow

下载：
tensorflow下载
选择tensorflow-1.13.1-cp35-none-linux_armv7l.whl版本

wget https://github.com/lhelontra/tensorflow-on-arm/releases/download/v1.13.1/tensorflow-1.13.1-cp35-none-linux_armv7l.whl

执行：

sudo pip3 install tensorflow-1.13.1-cp35-none-linux_armv7l.whl

模型配置

• 下载tensorflow model API
  model API下载：model API

• 下载COCO数据集训练好的判别模型
  下载训练好的模型并放到上一步models下的object_detection/models目录
  模型下载地址
  模型下载：

wget download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_2018_01_28.tar.gz

解压出文件夹ssd_mobilenet_v1_coco_2018_01_28

tar -xzvf ssd_mobilenet_v1_coco_2018_01_28.tar.gz

• proto数据转为python文件

进入目录models/research，运行：

protoc object_detection/protos/*.proto --python_out=.

可以看见原来的proto文件转为了.py文件

连接摄像头

1. 将pi camera 插入开发板上(注意摄像头并不支持热插拔)
2. 开启vnc，激活摄像头功能，运行：

sudo raspi-config

树莓派显示

1. hdmi 或者 3.5寸小屏幕都可以
2. 远程桌面连接(不推荐)
3. vnc连接：
   windows下载ultraVNC viewer
   在树莓派上安装Tight VNC 包

   sudo apt-get install tightvncserver
   

   树莓派中执行：

   vncserver :1
   

   使用ultraVNC viewer连接
   

显示器连接成功后，执行：

import numpy as np
import os
import sys
import tarfile
import tensorflow as tf
import cv2
import time
from collections import defaultdict

sys.path.append("../..")

from object_detection.utils import label_map_util
from object_detection.utils import visualization_utils as vis_util


MODEL_NAME = 'ssd_mobilenet_v1_coco_2018_01_28'

PATH_TO_CKPT = MODEL_NAME + '/frozen_inference_graph.pb'

PATH_TO_LABELS = os.path.join('/home/pi/models/research/object_detection/data', 'mscoco_label_map.pbtxt')

model_path = "/home/pi/models/research/object_detection/models/ssd_mobilenet_v1_coco_2018_01_28/model.ckpt"

start = time.clock()
NUM_CLASSES = 90

end= time.clock()
print('load the model' ,(end -start))
detection_graph = tf.Graph()
with detection_graph.as_default():
    od_graph_def = tf.GraphDef()
with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:
    serialized_graph = fid.read()
    od_graph_def.ParseFromString(serialized_graph)
    tf.import_graph_def(od_graph_def, name='')

label_map = label_map_util.load_labelmap(PATH_TO_LABELS)

categories = label_map_util.convert_label_map_to_categories(label_map, max_num_classes=NUM_CLASSES, use_display_name=True)
category_index = label_map_util.create_category_index(categories)

cap = cv2.VideoCapture(0)
with detection_graph.as_default():
    with tf.Session(graph=detection_graph) as sess:
        writer = tf.summary.FileWriter("logs/", sess.graph)
        sess.run(tf.global_variables_initializer())

        loader = tf.train.import_meta_graph(model_path + '.meta')
        loader.restore(sess, model_path)
        while(1):
            start = time.clock()
            ret, frame = cap.read()
            if cv2.waitKey(1) & 0xFF == ord('q'):
                break
            image_np =frame

            image_np_expanded = np.expand_dims(image_np, axis=0)
            image_tensor = detection_graph.get_tensor_by_name('image_tensor:0')
            boxes = detection_graph.get_tensor_by_name('detection_boxes:0')
            scores = detection_graph.get_tensor_by_name('detection_scores:0')
            classes = detection_graph.get_tensor_by_name('detection_classes:0')
            num_detections = detection_graph.get_tensor_by_name('num_detections:0')

            (boxes, scores, classes, num_detections) = sess.run(
                [boxes, scores, classes, num_detections],
                feed_dict={image_tensor: image_np_expanded})

            vis_util.visualize_boxes_and_labels_on_image_array(
                image_np, np.squeeze(boxes),
                np.squeeze(classes).astype(np.int32),
                np.squeeze(scores),
                category_index,
                use_normalized_coordinates=True,
                line_thickness=6)
            end = time.clock()
            print ('One frame detect take time:' ,end - start)
            cv2.imshow("capture", image_np)
            print('after cv2 show')
            cv2.waitKey(1)
cap.release()
cv2.destroyAllWindows()

将代码保存为test_my.py保存在models/research/object_detection/models目录下
进入目录models/research/object_detection/models，运行：

sudo chmod 666 /dev/video0
python3 test_my.py

识别截图