Inference Engine Samples - Security Barrier Camera Demo

Inference Engine Samples - Security Barrier Camera Demo

https://software.intel.com/en-us/articles/OpenVINO-IE-Samples

Intel® Distribution of OpenVINO™ Toolkit - Code Samples
https://software.intel.com/en-us/openvino-toolkit/documentation/code-samples
This toolkit features numerous code examples that help you develop and optimize computer vision and image processing pipelines for Intel® processors.

Intel® Distribution of OpenVINO™ Toolkit - Documentation - Inference Engine Samples
https://software.intel.com/en-us/articles/OpenVINO-IE-Samples

barrier ['bærɪə]：n. 障碍物，屏障，界线 vt. 把...关入栅栏

Security Barrier Camera Demo

Description

This demo showcases Vehicle and License Plate Detection network followed by the Vehicle Attributes and License Plate Recognition applied on top of Vehicle Detection results. The corresponding topologies are shipped with the product:

• vehicle-license-plate-detection-barrier-0106, which is a primary detection network to find the vehicles and license plates
• vehicle-attributes-recognition-barrier-0039, which is executed on top of the results from the first network and reports general vehicle attributes, for example, vehicle type (car/van/bus/track) and color
• license-plate-recognition-barrier-0001, which is executed on top of the results from the first network and reports a string per recognized license plate

For more details on the topologies, please refer to their descriptions in the deployment_tools/intel_models folder of the Intel® Distribution of the OpenVINO™ toolkit installation directory.

Other demo objectives are:

• Video/Camera as inputs, via OpenCV*
• Example of a simple asynchronous networks pipelining: Vehicle Attributes and License Plate Recognition networks are executed on top of the Vehicle Detection results
• Visualization of Vehicle Attributes and License Plate information for each detected object

topology [tə'pɒlədʒɪ]：n. 拓扑学
asynchronous [ə'sɪŋkrənəs; eɪ-]：adj. 异步的，不同时的，不同期的
hetero ['het(ə)rəʊ]：adj. 异性恋的，非同性恋的 n. 异性恋者

How It Works

On the start-up, the application reads command line parameters and loads the specified networks. The Vehicle and License-Plate Detection network is required, and the other two are optional.

Upon getting a frame from the OpenCV VideoCapture, the application performs inference of Vehicles and License-Plates Detection network, then performs another two inferences using Vehicle Attributes Detection and License Plate Recognition networks (if those specified in command line) and displays the results.

Running

Running the application with the -h option yields the following usage message:

./security_barrier_camera_demo -h
InferenceEngine:
    API version ............ 
    Build .................. 

interactive_vehicle_detection [OPTION]
Options:

    -h                         Print a usage message.
    -i "" ""     Required. Path to video or image files. Default value is "cam" to work with cameras.
    -m ""                Required. Path to the Vehicle and License Plate Detection model .xml file.
    -m_va ""             Optional. Path to the Vehicle Attributes model .xml file.
    -m_lpr ""            Optional. Path to the License Plate Recognition model .xml file.
      -l ""     Optional. For CPU custom layers, if any. Absolute path to a shared library with the kernels implementation.
          Or
      -c ""     Optional. For GPU custom kernels, if any. Absolute path to an .xml file with the kernels description.
    -d ""              Optional. Specify the target device for Vehicle Detection (CPU, GPU, FPGA, MYRIAD, or HETERO).
    -d_va ""           Optional. Specify the target device for Vehicle Attributes (CPU, GPU, FPGA, MYRIAD, or HETERO).
    -d_lpr ""          Optional. Specify the target device for License Plate Recognition (CPU, GPU, FPGA, MYRIAD, or HETERO).
    -pc                        Optional. Enable per-layer performance statistics.
    -r                         Optional. Output inference results as raw values.
    -t                         Optional. Probability threshold for vehicle and license plate detections.
    -no_show                   Optional. Do not show processed video.
    -auto_resize               Optional. Enable resizable input with support of ROI crop and auto resize.
    -nireq                     Optional. Number of infer request for pipelined mode (default value is 1)
    -nc                        Optional. Number of processed cameras (default value is 1) if the input (-i) is specified as camera.

/home/strong/inference_engine_samples/intel64/Release/

[setupvars.sh] OpenVINO environment initialized
strong@foreverstrong:~/inference_engine_samples/intel64/Release$ ./security_barrier_camera_demo -h
[ INFO ] InferenceEngine: 
	API version ............ 1.4
	Build .................. 17328

interactive_vehicle_detection [OPTION]
Options:

    -h                         Print a usage message.
    -i "" ""     Required. Path to video or image files. Default value is "cam" to work with cameras.
    -m ""                Required. Path to the Vehicle and License Plate Detection model .xml file.
    -m_va ""             Optional. Path to the Vehicle Attributes model .xml file.
    -m_lpr ""            Optional. Path to the License Plate Recognition model .xml file.
      -l ""     Optional. For CPU custom layers, if any. Absolute path to a shared library with the kernels implementation.
          Or
      -c ""     Optional. For GPU custom kernels, if any. Absolute path to an .xml file with the kernels description.
    -d ""              Optional. Specify the target device for Vehicle Detection (CPU, GPU, FPGA, MYRIAD, or HETERO).
    -d_va ""           Optional. Specify the target device for Vehicle Attributes (CPU, GPU, FPGA, MYRIAD, or HETERO).
    -d_lpr ""          Optional. Specify the target device for License Plate Recognition (CPU, GPU, FPGA, MYRIAD, or HETERO).
    -pc                        Optional. Enable per-layer performance statistics.
    -r                         Optional. Output inference results as raw values.
    -t                         Optional. Probability threshold for vehicle and license plate detections.
    -no_show                   Optional. Do not show processed video.
    -auto_resize               Optional. Enable resizable input with support of ROI crop and auto resize.
    -nireq                     Optional. Number of infer request for pipelined mode (default value is 1).
    -nc                        Optional. Number of processed cameras (default value is 1) if the input (-i) is specified as camera.
strong@foreverstrong:~/inference_engine_samples/intel64/Release$ 

Running the application with an empty list of options yields the usage message given above and an error message.

To run the demo, you can use public models or a set of pre-trained and optimized models delivered with the package:

• /deployment_tools/intel_models/vehicle-license-plate-detection-barrier-0106
• /deployment_tools/intel_models/vehicle-attributes-recognition-barrier-0039
• /deployment_tools/intel_models/license-plate-recognition-barrier-0001

For example, to do inference on a GPU with the Intel Distribution of OpenVINO toolkit pre-trained models, run the following command:

./security_barrier_camera_demo -i /inputVideo.mp4 -m vehicle-license-plate-detection-barrier-0106.xml -m_va vehicle-attributes-recognition-barrier-0039.xml -m_lpr license-plate-recognition-barrier-0001.xml -d GPU

To do inference for two video inputs using two asynchronous infer request on FPGA with the Intel Distribution of OpenVINO toolkit pre-trained models, run the following command:

./security_barrier_camera_demo -i /inputVideo_0.mp4 /inputVideo_1.mp4 -m vehicle-license-plate-detection-barrier-0106.xml -m_va vehicle-attributes-recognition-barrier-0039.xml -m_lpr license-plate-recognition-barrier-0001.xml -d HETERO:FPGA,CPU -d_va HETERO:FPGA,CPU -d_lpr HETERO:FPGA,CPU -nireq 2

NOTE: Public models must be first converted to the Inference Engine format (.xml + .bin) using the Model Optimizer tool.
https://software.intel.com/en-us/articles/OpenVINO-ModelOptimizer

./security_barrier_camera_demo -i /opt/intel/computer_vision_sdk/deployment_tools/demo/car_1.bmp -d MYRIAD -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/vehicle-license-plate-detection-barrier-0106/FP16/vehicle-license-plate-detection-barrier-0106.xml -d_va MYRIAD -m_va /opt/intel/computer_vision_sdk/deployment_tools/intel_models/vehicle-attributes-recognition-barrier-0039/FP16/vehicle-attributes-recognition-barrier-0039.xml -d_lpr MYRIAD -m_lpr /opt/intel/computer_vision_sdk/deployment_tools/intel_models/license-plate-recognition-barrier-0001/FP16/license-plate-recognition-barrier-0001.xml

strong@foreverstrong:~/inference_engine_samples/intel64/Release$ ./security_barrier_camera_demo -i /opt/intel/computer_vision_sdk/deployment_tools/demo/car_1.bmp -d MYRIAD -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/vehicle-license-plate-detection-barrier-0106/FP16/vehicle-license-plate-detection-barrier-0106.xml -d_va MYRIAD -m_va /opt/intel/computer_vision_sdk/deployment_tools/intel_models/vehicle-attributes-recognition-barrier-0039/FP16/vehicle-attributes-recognition-barrier-0039.xml -d_lpr MYRIAD -m_lpr /opt/intel/computer_vision_sdk/deployment_tools/intel_models/license-plate-recognition-barrier-0001/FP16/license-plate-recognition-barrier-0001.xml
[ INFO ] InferenceEngine: 
	API version ............ 1.4
	Build .................. 17328
[ INFO ] Parsing input parameters
[ INFO ] Capturing video streams from the video files or loading images
[ INFO ] Files were added: 1
[ INFO ]     /opt/intel/computer_vision_sdk/deployment_tools/demo/car_1.bmp
[ INFO ] Number of input image files: 1
[ INFO ] Number of input video files: 0
[ INFO ] Number of input channels:    1
[ INFO ] Loading plugin MYRIAD

	API version ............ 1.4
	Build .................. 17328
	Description ....... myriadPlugin
[ INFO ] Loading network files for VehicleDetection
[ INFO ] Batch size is forced to  1
[ INFO ] Checking Vehicle Detection inputs
[ INFO ] Checking Vehicle Detection outputs
[ INFO ] Loading Vehicle Detection model to the MYRIAD plugin
[ INFO ] Loading network files for VehicleAttribs
[ INFO ] Batch size is forced to 1 for Vehicle Attribs
[ INFO ] Checking VehicleAttribs inputs
[ INFO ] Checking Vehicle Attribs outputs
[ INFO ] Loading Vehicle Attribs model to the MYRIAD plugin
[ INFO ] Loading network files for Licence Plate Recognition (LPR)
[ INFO ] Batch size is forced to  1 for LPR Network
[ INFO ] Checking LPR Network inputs
[ INFO ] Checking LPR Network outputs
[ INFO ] Loading LPR model to the MYRIAD plugin
[ INFO ] Start inference 

Avarage inference time: 94.9329 ms (10.5338 fps)

Total execution time: 51430.5

[ INFO ] Execution successful
strong@foreverstrong:~/inference_engine_samples/intel64/Release$ 

Optimization Hints for Heterogeneous Scenarios with FPGA

• OMP_NUM_THREADS: Specifies number of threads to use. For heterogeneous scenarios with FPGA, when several inference requests are used asynchronously, limiting the number of CPU threads with OMP_NUM_THREADS allows to avoid competing for resources between threads. For the Security Barrier Camera Demo, recommended value is OMP_NUM_THREADS=1.
• KMP_BLOCKTIME: Sets the time, in milliseconds, that a thread should wait, after completing the execution of a parallel region, before sleeping. The default value is 200ms, which is not optimal for the demo. Recommended value is KMP_BLOCKTIME=1.

Demo Output

The demo uses OpenCV to display the resulting frame with detections rendered as bounding boxes and text:

hint [hɪnt]：n. 暗示，线索 vt. 暗示，示意 vi. 示意
heterogeneous：[,het(ə)rə(ʊ)'dʒiːnɪəs; -'dʒen-]：adj. 多相的，异种的，不均匀的，由不同成分形成的
render ['rendə]：vt. 致使，提出，实施，着色，以...回报 vi. 给予补偿 n. 打底，交纳，粉刷

/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/security_barrier_camera_demo/

/home/strong/inference_engine_samples/security_barrier_camera_demo/

/home/strong/inference_engine_samples/intel64/Release/

security_barrier_camera_demo

/opt/intel/computer_vision_sdk/deployment_tools/demo/