[Nvidia Jetson Xavier Developer Kit] deepstream6.0 上手任务3

test任务要求：

使用基于python的pipeline进行图片检测，官方demo跑通。

1. 寻找参考资料

任务要求中的官方demo没有明确指明在何处获取，参考英伟达官方deepStream的python apps部分内容，如图。地址：Python Sample Apps and Bindings Source Details — DeepStream 6.1 Release documentation

可以使用Gst Python构建deepstream管道（Gst Python是GStreamer框架的Python绑定文件）。有关管道构造示例，请参见示例应用程序主函数。

2.寻找demo文件

在官网教程中没有提到demo的任何信息，猜测demo在python bindings（binding意思应该是一个东西搭配的一组东西吧，好比说核酸采样样本管的binding是棉签和一个人的样本）。在之前下载好的deepStream_python_apps文件夹（这个就是binding）中搜索demo，果然收到了，右键打开文件所在位置，找到了接下来应该学习的文件。

注：文件我在windows上也下载了一遍，就在自己电脑上搜索的。

观察demo文件的父文件夹，可确定整个需要学习的内容即为该binding下的tests文件夹。

3.tests文件夹内容说明

2个文件夹+1个py文件。打开py文件是空的，应该是供运行后生成内容的。

integration中有README文件，按此学习。

4.README文件

就按着它的步骤三步走，装一个pytest包，完了就运行test.py。可以复制test完了自己编辑。

之后又发现了这个大包里的bindings文件夹，就是任务二应该做的事。

其中的none-*中的*换成自己电脑下载的名称，在build文件夹中可以看到这个whl文件。

### step1
```
mkdir -p bindings/build
cd bindings/build
```
### step2
#### Python 3.6
```
cmake ..  -DPYTHON_MINOR_VERSION=6
make
python3.6 -m venv env
```
#### Python 3.8
```
cmake ..  -DPYTHON_MINOR_VERSION=8
make
python3.8 -m venv env
```
### step3
```
. env/bin/activate
pip install pyds-1.1.3-py3-none-*.whl
pip install pytest
cd ../../tests/integration
pytest test.py
```

重点在step2 ，当cmake时，可能提示没有cmake文件，到大包里找到了bindings文件夹，把内容复制到step1新建的文件夹中，运行。报错如下。

报错缺少一个so文件，查看路径，发现是6.1版本ds，但安装的是6.0版本，应是cmake的内容版本不对，打开cmakelist文件，查找6.1，替换为6.0。删除bindings/build中构建的不完整文件，重新运行。ok

运行make，报错

Scanning dependencies of target pyds
[  6%] Building CXX object CMakeFiles/pyds.dir/src/pyds.cpp.o
In file included from /home/dongyutian/bindings/include/bind/bindanalyticsmeta.hpp:20:0,
                 from /home/dongyutian/bindings/src/pyds.cpp:19:
/home/dongyutian/bindings/include/bind/bind_string_property_definitions.h:18:10: fatal error: pybind11/pybind11.h: No such file or directory
 #include "pybind11/pybind11.h"
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
CMakeFiles/pyds.dir/build.make:62: recipe for target 'CMakeFiles/pyds.dir/src/pyds.cpp.o' failed
make[2]: *** [CMakeFiles/pyds.dir/src/pyds.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/pyds.dir/all' failed
make[1]: *** [CMakeFiles/pyds.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

发现找不到这个，但下载的包中有这个，为什么找不到，因为路径有问题，怀疑新创建的文件夹有问题。于是乎，运行以下代码，让指令在deepstream——python——apps中运行，然后再从step1开始。

cd /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_python_apps

执行创建python3.8环境时报错

The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt-get install python3-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_python_apps/bindings/build/env/bin/python3.8', '-Im', 'ensurepip', '--upgrade', '--default-pip']

执行上述中的，不过要换成3.8，要不还是一样的错误。

apt-get install python3-venv

执行第三步安装pyds，这个build文件夹中可以看到文件，我的ubuntu系统上却下载了linux_x86_64，本应是linux_aarch64。因为之前有得到aarch64的whl，所以我就弄过来了。

sudo pip3 install ./pyds-1.1.3-py3-none-linux_aarch64.whl -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com

如果之前弄过，会提示要求以满足，这步就相当于完成了。不行。回顾这个文件的产生，应当是cmake文件中的问题，找到linux_x86_64，改为linux_aarch64。重新开始所有步骤。回顾为什么出了问题，可能是因为自己没有看到bindings里的readme，太长了。

重新走后，能安装，但会有如下错误产生（不影响本命令的执行）。

Building wheels for collected packages: pgi, PyGObject, pycairo
  Running setup.py bdist_wheel for pgi ... error
  Complete output from command /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_python_apps/bindings/build/env/bin/python3.8 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-3srjfd08/pgi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp_lci1z3_pip-wheel- --python-tag cp38:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help
  
  error: invalid command 'bdist_wheel'
  
  ----------------------------------------
  Failed building wheel for pgi
  Running setup.py clean for pgi
  Running setup.py bdist_wheel for PyGObject ... error
  Complete output from command /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_python_apps/bindings/build/env/bin/python3.8 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-3srjfd08/PyGObject/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpgyfk77jfpip-wheel- --python-tag cp38:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help
  
  error: invalid command 'bdist_wheel'
  
  ----------------------------------------
  Failed building wheel for PyGObject
  Running setup.py clean for PyGObject
  Running setup.py bdist_wheel for pycairo ... error
  Complete output from command /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_python_apps/bindings/build/env/bin/python3.8 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-3srjfd08/pycairo/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpdqej050_pip-wheel- --python-tag cp38:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help
  
  error: invalid command 'bdist_wheel'
  
  ----------------------------------------
  Failed building wheel for pycairo
  Running setup.py clean for pycairo
Failed to build pgi PyGObject pycairo
Installing collected packages: pgi, pycairo, PyGObject, pyds
  Running setup.py install for pgi ... done
  Running setup.py install for pycairo ... done
  Running setup.py install for PyGObject ... done
Successfully installed PyGObject-3.42.2 pgi-0.0.11.2 pycairo-1.21.0 pyds-1.1.3

deepstream_demo.py

#!/usr/bin/env python3

# SPDX-FileCopyrightText: Copyright (c) 2021 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import os
import sys

sys.path.append('../..')

from tests.common.utils import is_aarch64, load_deepstream_libs
from tests.common.pipeline_filesink import PipelineFileSink
from tests.common.frame_iterator import FrameIterator

import pyds


def make_text_display(display_meta, frame_number, num_rects, obj_counter):
    display_meta.num_labels = 1
    py_nvosd_text_params = display_meta.text_params[0]
    disp_txt = f"Frame Number={frame_number} " \
               f"Number of Objects={num_rects} " \
               f"Vehicle_count={obj_counter['vehicle']} " \
               f"Person_count={obj_counter['person']}"
    py_nvosd_text_params.display_text = disp_txt

    # Now set the offsets where the string should appear
    py_nvosd_text_params.x_offset = 10
    py_nvosd_text_params.y_offset = 12

    # Font , font-color and font-size
    py_nvosd_text_params.font_params.font_name = "Consolas"
    py_nvosd_text_params.font_params.font_size = 10
    # set(red, green, blue, alpha); set to White
    py_nvosd_text_params.font_params.font_color.set(1.0, 1.0, 1.0, 1.0)

    # Text background color
    py_nvosd_text_params.set_bg_clr = 1
    # set(red, green, blue, alpha); set to Black
    py_nvosd_text_params.text_bg_clr.set(0.0, 0.0, 0.0, 1.0)


def frame_function(batch_meta, frame_meta, dict_data):
    obj_counter = dict_data["obj_counter"]
    pgie_class_id = dict_data["pgie_class_id"]
    frame_number = frame_meta.frame_num
    num_rects = frame_meta.num_obj_meta
    display_meta = pyds.nvds_acquire_display_meta_from_pool(
        batch_meta)
    make_text_display(display_meta, frame_number, num_rects, obj_counter)
    # Using pyds.get_string() to get display_text as string
    py_nvosd_text_params = display_meta.text_params[0]
    print(pyds.get_string(py_nvosd_text_params.display_text))
    pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)


def box_function(batch_meta, frame_meta, obj_meta, dict_data):
    obj_counter = dict_data["obj_counter"]
    pgie_class_id = dict_data["pgie_class_id"]
    obj_counter[pgie_class_id[obj_meta.class_id]] += 1
    obj_meta.rect_params.border_color.set(0.0, 0.0, 1.0, 0.0)


def parse_input(args):
    # Check input arguments
    if len(args) != 3:
        sys.stderr.write(f"usage: {args[0]}   \n")
        sys.exit(1)

    video_path = args[1]
    if not os.path.isfile(video_path):
        raise Exception(f"{video_path} is not a file")
    config_file = args[2]
    if not os.path.isfile(config_file):
        raise Exception(f"{config_file} is not a file")

    return config_file, video_path


def make_and_run_pipeline(config_file, video_path):
    properties = {
        "file-source": {
            "location": video_path
        },
        "Stream-muxer": {
            "width": 1280,
            "height": 720,
            "batch-size": 1,
            "batched-push-timeout": 4000000
        },
        "primary-inference": {
            "config-file-path": config_file
        },
        "filesink": {
            "location": "./out.mp4",
            "sync": 0,
            "async": 0,
        },
        "encoder": {
            "bitrate": 2000000
        }

    }

    pgie_class_id = {
        0: "vehicle",
        1: "person",
        2: "bicycle",
        3: "roadsign",
    }
    data_probe = {
        "pgie_class_id": pgie_class_id,
        "obj_counter": {
            pgie_class_id[0]: 0,
            pgie_class_id[1]: 0,
            pgie_class_id[2]: 0,
            pgie_class_id[3]: 0
        }
    }
    probe_function = FrameIterator(frame_function, box_function, data_probe)

    sp = PipelineFileSink(properties, is_aarch64())
    sp.set_probe(probe_function)
    sp.run()


def main(args):
    config_file, video_path = parse_input(args)
    load_deepstream_libs()
    make_and_run_pipeline(config_file, video_path)


if __name__ == '__main__':
    sys.exit(main(sys.argv))

test.py 

#!/usr/bin/env python3

# SPDX-FileCopyrightText: Copyright (c) 2021 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import pytest
import pyds

from tests.common.frame_iterator import FrameIterator
from tests.common.pipeline_fakesink import PipelineFakesink
from tests.common.pipeline_fakesink_tracker import PipelineFakesinkTracker
from tests.common.tracker_utils import get_tracker_properties_from_config
from tests.common.utils import is_aarch64

VIDEO_PATH1 = "/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264"
STANDARD_PROPERTIES1 = {
    "file-source": {
        "location": VIDEO_PATH1
    },
    "Stream-muxer": {
        "width": 1280,
        "height": 720,
        "batch-size": 1,
        "batched-push-timeout": 4000000
    },
    "primary-inference": {
        "config-file-path": "./ds_base_config.txt"
    }
}
STANDARD_PROPERTIES_TRACKER1 = {
    "file-source": {
        "location": VIDEO_PATH1
    },
    "Stream-muxer": {
        "width": 1280,
        "height": 720,
        "batch-size": 1,
        "batched-push-timeout": 4000000
    },
    "primary-inference": {
        "config-file-path": "ds_pgie_config.txt"
    },
    "secondary1-nvinference-engine": {
        "config-file-path", "ds_sgie1_config.txt"
    },
    "secondary2-nvinference-engine": {
        "config-file-path", "ds_sgie2_config.txt"
    },
    "secondary3-nvinference-engine": {
        "config-file-path", "ds_sgie3_config.txt"
    }
}
STANDARD_CLASS_ID1 = {
    0: "vehicle",
    1: "person",
    2: "bicycle",
    3: "roadsign",
}


def test_pipeline1():
    ### INIT DATA

    # defining the function to be called at each frame
    def frame_function(batch_meta, frame_meta, dict_data):
        pass

    # defining the function to be called at each object
    def box_function(batch_meta, frame_meta, obj_meta, dict_data):
        obj_counter = dict_data["obj_counter"]
        pgie_class_id = dict_data["pgie_class_id"]
        obj_counter[pgie_class_id[obj_meta.class_id]] += 1

    # defining a shared data dictionary
    data_probe = {
        "pgie_class_id": STANDARD_CLASS_ID1,
        "obj_counter": {
            STANDARD_CLASS_ID1[0]: 0,
            STANDARD_CLASS_ID1[1]: 0,
            STANDARD_CLASS_ID1[2]: 0,
            STANDARD_CLASS_ID1[3]: 0
        }
    }
    # Creating the probe function
    probe_function = FrameIterator(frame_function, box_function, data_probe)

    # Creating the pipeline
    sp = PipelineFakesink(STANDARD_PROPERTIES1, is_aarch64())
    # registering the probe function
    sp.set_probe(probe_function)

    ### LAUNCH BEHAVIOR
    # Running the pipeline
    sp.run()

    ### CHECK OUTPUT
    assert data_probe["obj_counter"]["person"] > 0
    assert data_probe["obj_counter"]["vehicle"] > 0
    assert data_probe["obj_counter"]["bicycle"] > 0


def test_pipeline2():
    ### INIT DATA

    # defining the function to be called at each frame
    def frame_function(batch_meta, frame_meta, dict_data):
        pass

    tracker_qty_expected = {
        "stream_id" : 1, # only one video
        "surface_stream_id" : 1,
        "numobj" : 22,
        "unique_id" : 317,
        "class_id": 3,
        "obj_label" : 3,
        "frame_num": 896,
        "tbox_left": 1340,
        "tbox_width" : 650,
        "tbox_top" : 1075,
        "tbox_right" : 488,
        "confidence": 464,
        "age": 464}

    def user_function(batch_meta, user_meta, dict_data):
        if not user_meta:
            return
        if not user_meta.base_meta.meta_type == pyds.NvDsMetaType.NVDS_TRACKER_PAST_FRAME_META:
            return
        pPastFrameObjBatch = pyds.NvDsPastFrameObjBatch.cast(user_meta.user_meta_data)
        tracker_data = dict_data["tracker_data"]
        for trackobj in pyds.NvDsPastFrameObjBatch.list(pPastFrameObjBatch):
            tracker_data["stream_id"].add(trackobj.streamID)
            tracker_data["surface_stream_id"].add(trackobj.surfaceStreamID)
            for pastframeobj in pyds.NvDsPastFrameObjStream.list(trackobj):
                tracker_data["numobj"].add(pastframeobj.numObj)
                tracker_data["unique_id"].add(pastframeobj.uniqueId)
                tracker_data["class_id"].add(pastframeobj.classId)
                tracker_data["obj_label"].add(pastframeobj.objLabel)
                for objlist in pyds.NvDsPastFrameObjList.list(pastframeobj):
                    tracker_data["frame_num"].add(objlist.frameNum)
                    tracker_data["tbox_left"].add(objlist.tBbox.left)
                    tracker_data["tbox_width"].add(objlist.tBbox.width)
                    tracker_data["tbox_top"].add(objlist.tBbox.top)
                    tracker_data["tbox_right"].add(objlist.tBbox.height)
                    tracker_data["confidence"].add(objlist.confidence)
                    tracker_data["age"].add(objlist.confidence)

    # defining the function to be called at each object
    def box_function(batch_meta, frame_meta, obj_meta, dict_data):
        obj_counter = dict_data["obj_counter"]
        pgie_class_id = dict_data["pgie_class_id"]
        obj_counter[pgie_class_id[obj_meta.class_id]] += 1

    tracker_cfg = get_tracker_properties_from_config("ds_tracker_config.txt")
    properties = {
        "file-source": {
            "location": VIDEO_PATH1
        },
        "Stream-muxer": {
            "width": 1280,
            "height": 720,
            "batch-size": 1,
            "batched-push-timeout": 4000000
        },
        "primary-inference": {
            "config-file-path": "ds_pgie_config.txt"
        },
        "secondary1-nvinference-engine": {
            "config-file-path": "ds_sgie1_config.txt"
        },
        "secondary2-nvinference-engine": {
            "config-file-path": "ds_sgie2_config.txt"
        },
        "secondary3-nvinference-engine": {
            "config-file-path": "ds_sgie3_config.txt"
        },
        "tracker": tracker_cfg
    }
    print(properties)
    # defining a shared data dictionary
    tracker_data = {}
    for elm in tracker_qty_expected.keys():
        tracker_data[elm] = set()
    data_probe = {
        "pgie_class_id": STANDARD_CLASS_ID1,
        "obj_counter": {
            STANDARD_CLASS_ID1[0]: 0,
            STANDARD_CLASS_ID1[1]: 0,
            STANDARD_CLASS_ID1[2]: 0,
            STANDARD_CLASS_ID1[3]: 0
        },
        "tracker_data": tracker_data
    }
    # Creating the probe function
    probe_function = FrameIterator(frame_function, box_function, data_probe,
                                   user_function)

    # Creating the pipeline
    sp = PipelineFakesinkTracker(properties, is_aarch64())
    # registering the probe function
    sp.set_probe(probe_function)

    ### LAUNCH BEHAVIOR
    # Running the pipeline
    sp.run()

    ### CHECK OUTPUT
    for key, expected_qty in tracker_qty_expected.items():
        qty = len(data_probe["tracker_data"][key])
        assert qty > 0