NNStreamer简单使用

• https://github.com/nnstreamer/nnstreamer

在Ubuntu x86上简单使用nnstreamer 使用tensorflow lite 去推理yolov5

# 安装
$ sudo add-apt-repository ppa:nnstreamer/ppa
$ sudo apt-get update
$ sudo apt-get install nnstreamer

$ sudo add-apt-repository ppa:nnstreamer-example/ppa
$ sudo apt-get update
$ sudo apt-get install nnstreamer-example

sudo apt install nnstreamer-tensorflow2-lite
$ cd /usr/lib/nnstreamer/bin

./nnstreamer_example_pose_estimation_tflite 
python3 nnstreamer_example_object_detection_tflite.py 

• 使用 rtsp 流去做推理 nnstreamer_example_object_detection_tflite.py

        self.pipeline = Gst.parse_launch(
            'rtspsrc location=rtsp://admin:[email protected]:554/client1x ! rtph264depay ! h264parse ! decodebin ! videoconvert ! videoscale ! '
            'video/x-raw,width=640,height=480,format=RGB ! tee name=t_raw '
            't_raw. ! queue leaky=2 max-size-buffers=2 ! videoscale ! video/x-raw,width=300,height=300 ! tensor_converter ! '
            'tensor_transform mode=arithmetic option=typecast:float32,add:-127.5,div:127.5 ! '
            'tensor_filter framework=tensorflow-lite model=' + self.tflite_model + ' ! '
            'tensor_decoder mode=bounding_boxes option1=mobilenet-ssd option2='
            + self.tflite_label + ' option3=' + self.tflite_box_prior + ' option4=640:480 option5=300:300 !'
            'compositor name=mix sink_0::zorder=2 sink_1::zorder=1 ! videoconvert ! ximagesink '
            't_raw. ! queue leaky=2 max-size-buffers=10 ! mix. '
        )

• yolov5

gst-launch-1.0 v4l2src ! videoscale ! videoconvert ! video/x-raw,width=320,height=320,format=RGB,framerate=30/1,pixel-aspect-ratio=1/1 ! tee name=t   t. ! queue ! tensor_converter ! other/tensors,num_tensors=1,types=uint8,format=static,dimensions=3:320:320:1 !     tensor_transform mode=arithmetic option=typecast:float32,add:0.0,div:255.0 ! queue leaky=2 max-size-buffers=2 !     tensor_filter framework=tensorflow2-lite model=./yolov5s-fp16-320.tflite !     other/tensors,num_tensors=1,types=float32,format=static,dimensions=85:6300:1:1 !     tensor_decoder mode=bounding_boxes option1=yolov5 option2=./coco.txt option4=320:320 option5=320:320 !     video/x-raw,width=320,height=320,format=RGBA ! mix.sink_0   t. ! queue ! mix.sink_1 compositor name=mix sink_0::zorder=2 sink_1::zorder=1 ! videoconvert ! ximagesink

• 使用rtsp流作为输入源

gst-launch-1.0 rtspsrc location=rtsp://admin:[email protected]:554/client1x ! rtph264depay ! h264parse ! decodebin ! videoconvert ! video/x-raw,width=320,height=320,format=RGB,framerate=30/1,pixel-aspect-ratio=1/1 ! tee name=t   t. ! queue ! tensor_converter ! other/tensors,num_tensors=1,types=uint8,format=static,dimensions=3:320:320:1 !     tensor_transform mode=arithmetic option=typecast:float32,add:0.0,div:255.0 ! queue leaky=2 max-size-buffers=2 !     tensor_filter framework=tensorflow2-lite model=./yolov5s-fp16-320.tflite !     other/tensors,num_tensors=1,types=float32,format=static,dimensions=85:6300:1:1 !     tensor_decoder mode=bounding_boxes option1=yolov5 option2=./coco.txt option4=320:320 option5=320:320 !     video/x-raw,width=320,height=320,format=RGBA ! mix.sink_0   t. ! queue ! mix.sink_1 compositor name=mix sink_0::zorder=2 sink_1::zorder=1 ! videoconvert ! ximagesink