高通msm-V4L2-Camera驱动浅析4-stream

一、stream(流)的设计思路

为什么要使用stream这个设计呢？
如果你做为开发者，现在你有个任务：假设camera像素为500W，你需要把每一帧500w的图像传输给用户使用，怎么办？

• 方法1：通过【帧IO】访问方式
  使用read和write的方式，通过read读取每一帧数据，数据需要在内核和用户之间拷贝。但问题是：这种方式访问速度可能会非常慢。
  一帧500W的数据量，一般场景下，我们的帧率都至少有20多帧，这种访问速度肯定不能接受。

  那怎么办呢？可不可以这样，我在内核申请一片内存(buf),然后把它映射(mmap)到用户空间，这样用户可以像直接读内存一样，访问到我们内核空间的数据。

• 方法2：通过【流IO】访问方式：

  • 内存映射缓冲区(V4L2_MEMORY_MMAP)：在内核空间开辟缓冲区，应用通过mmap()系统调用映射到用户地址空间
  • 用户空间缓冲区(V4L2_MEMORY_USERPTR)：在用户空间的应用中开辟缓冲区，用户与内核空间之间交换缓冲区指针。

因此stream(流)的概念就诞生了!
stream又涉及到图像的vb2_buffer，vb2_buffer又由vb2_queue 队列管理

关于vb2_buffer与vb2_queue在下一篇文章在进行讨论。

二、stream(流)的创建

2.1 stream(流)数据结构

struct msm_stream {
    struct list_head list;//链表头

    /* stream index per session, same
     * as stream_id but set through s_parm
     */
    unsigned int stream_id;//流id
    /* vb2 buffer handling */
    struct vb2_queue *vb2_q;//buf队列
    spinlock_t stream_lock;
    struct list_head queued_list;
};

2.2 session、stream、vb2_buffer的关系

上一篇文章里面有讲到，session中包含了stream的队列：

struct msm_session {
    struct msm_queue_head stream_q;
}

stream里面又嵌入了struct vb2_queue *vb2_q队列;
而vb2_queue里面又包含用来保存和传输图像的buf——struct vb2_buffer *bufs[VB2_MAX_FRAME];

• session管理stream
• steam管理vb2_queue
• vb2_queue管理vb2_buffer

2.3 流的创建

int msm_create_stream(unsigned int session_id,
    unsigned int stream_id, struct vb2_queue *q)
{
    struct msm_session *session;
    struct msm_stream  *stream;
    //根据session_id判断会话是否已经创建
    session = msm_queue_find(msm_session_q, struct msm_session,
        list, __msm_queue_find_session, &session_id);
    if (!session)
        return -EINVAL;
    //申请内存
    stream = kzalloc(sizeof(*stream), GFP_KERNEL);
    if (!stream)
        return -ENOMEM;
    //赋值stream_id
    stream->stream_id = stream_id;
    //赋值vb2_queue队列
    stream->vb2_q = q;
    //初始化自旋锁
    spin_lock_init(&stream->stream_lock);
    //将新建立的流加入session管理的stream队列中
    msm_enqueue(&session->stream_q, &stream->list);
    //流的队列长度+1
    session->stream_q.len++;

    INIT_LIST_HEAD(&stream->queued_list);

    return 0;

流包括很多，比如预览流，拍照流，元数据流等待，这些流都由统一的stream_q队列管理，每创建一个流，
就会入队。

2.4 流的创建流程-从hal到kernel

细致的调用流程，参考【Camera专题】HAL层-addChannel和startChannel简析
这里写一下关键的接口

用户空间：HAL层
hardware/qcom/camera/QCamera2/HAL/QCamera2HWI.cpp

int32_t QCamera2HardwareInterface::addPreviewChannel()
{
`
    CDBG_HIGH("%s :zcf camera_handle=%d ops=%p", __func__,mCameraHandle->camera_handle,mCameraHandle->ops);
    pChannel = new QCameraChannel(mCameraHandle->camera_handle,
                                  mCameraHandle->ops);
···

    // preview only channel, don't need bundle attr and cb
    rc = pChannel->init(NULL, NULL, NULL);
··· 

    // meta data stream always coexists with preview if applicable
    rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_METADATA,
                            metadata_stream_cb_routine, this);
···

    if (isNoDisplayMode()) {
        rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_PREVIEW,
                                nodisplay_preview_stream_cb_routine, this);
    } else {
        rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_PREVIEW,
                                preview_stream_cb_routine, this);
    }
···
    if(···)
         rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_ANALYSIS,
                NULL, this);

    m_channels[QCAMERA_CH_TYPE_PREVIEW] = pChannel;
    return rc;
}

在启动预览的时候，添加了3路数据流，分别是：

• CAM_STREAM_TYPE_METADATA ： 元数据流
• CAM_STREAM_TYPE_PREVIEW：预览流
• CAM_STREAM_TYPE_ANALYSIS：人脸分析流

用户空间：HAL-mm-camera-interface
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_stream.c

调用流程：
mm_channel_init
mm_camera_intf_add_channel
mm_camera_intf_add_stream
mm_channel_fsm_fn
mm_channel_fsm_fn_stopped
mm_channel_add_stream
mm_stream_fsm_fn/*获取流 */
mm_stream_fsm_inited
mm_stream_init
mm_stream_set_ext_mode

最终是调用到mm_stream_set_ext_mode

int32_t mm_stream_set_ext_mode(mm_stream_t * my_obj)
{

    s_parm.type =  V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
    //通过IOCTL的方式，调用到内核。
    rc = ioctl(my_obj->fd, VIDIOC_S_PARM, &s_parm);

    return rc;
}

通过IOCTL的方式，调用到内核。
ioctl(my_obj->fd, VIDIOC_S_PARM, &s_parm);

内核空间

kernel/msm-4.9/drivers/media/v4l2-core/v4l2-ioctl.c

static struct v4l2_ioctl_info v4l2_ioctls[] = {
···
IOCTL_INFO_FNC(VIDIOC_S_PARM, v4l_s_parm, v4l_print_streamparm, INFO_FL_PRIO),
···
}

static int v4l_s_parm(const struct v4l2_ioctl_ops *ops,
                struct file *file, void *fh, void *arg)
{
    //调用vidioc_s_parm
    return ops->vidioc_s_parm(file, fh, p);
}

kernel/msm-4.9/drivers/media/platform/msm/camera_v2/camera/camera.c

static const struct v4l2_ioctl_ops camera_v4l2_ioctl_ops = {
···
    /* Stream type-dependent parameter ioctls */
    .vidioc_g_parm = camera_v4l2_g_parm,
    .vidioc_s_parm = camera_v4l2_s_parm,
···
};

在前面的文章，有说到ioctl_ops这个操作集合：
其中.vidioc_s_parm = camera_v4l2_s_parm。

static int camera_v4l2_s_parm(struct file *filep, void *fh,
    struct v4l2_streamparm *parm)
{
    //创建流
    rc = msm_create_stream(event_data->session_id,
        event_data->stream_id, &sp->vb2_q);

    return rc;
}

三、log

HAL层关于数据流的定义：

typedef enum {
    CAM_STREAM_TYPE_DEFAULT,       /* default stream type */
    CAM_STREAM_TYPE_PREVIEW,       /* preview */
    CAM_STREAM_TYPE_POSTVIEW,      /* postview */
    CAM_STREAM_TYPE_SNAPSHOT,      /* snapshot */
    CAM_STREAM_TYPE_VIDEO,         /* video */
    CAM_STREAM_TYPE_CALLBACK,      /* app requested callback */
    CAM_STREAM_TYPE_IMPL_DEFINED, /* opaque format: could be display, video enc, ZSL YUV */
    CAM_STREAM_TYPE_METADATA,      /* meta data */
    CAM_STREAM_TYPE_RAW,           /* raw dump from camif */
    CAM_STREAM_TYPE_OFFLINE_PROC,  /* offline process */
    CAM_STREAM_TYPE_PARM,         /* mct internal stream */
    CAM_STREAM_TYPE_ANALYSIS,     /* analysis stream */
    CAM_STREAM_TYPE_DEPTH,        /* Depth stream for depth sensor*/
    CAM_STREAM_TYPE_MAX,
} cam_stream_type_t;

• CAM_STREAM_TYPE_PREVIEW = 1
• CAM_STREAM_TYPE_METADATA = 7
• CAM_STREAM_TYPE_ANALYSIS = 11

//hal 层
D QCamera :  mm_stream_fsm_inited: 596: open dev name =/dev/video1 , fd = 78
D QCamera :  mm_stream_set_ext_mode: 1894: E, my_handle = 0x900, fd = 78, state = 1
D QCamera :  mm_stream_set_ext_mode: 1901: stream fd=78, rc=0, extended_mode=1

//元数据流
D QCamera :  mm_stream_calc_offset: 4876: Stream type 7 num_planes 1
D QCamera :  mm_stream_calc_offset: 4883: Plane 0, stride 764376, scanline 1, width 764376, height 1,length 764416

//预览流
D QCamera :  mm_stream_calc_offset: 4876: Stream type 1 num_planes 2
D QCamera :  mm_stream_calc_offset: 4883: Plane 0, stride 384, scanline 320, width 360, height 320, length 122880
D QCamera :  mm_stream_calc_offset: 4883: Plane 1, stride 384, scanline 160, width 360, height 160, length 61440

//人脸分析流
D QCamera :  mm_stream_calc_offset: 4876: Stream type 11 num_planes 1
D QCamera :  mm_stream_calc_offset: 4883: Plane 0, stride 544, scanline 480, width 540, height 480,length 261120

//kernel层
camera_v4l2_s_parm:session_id = 1,event_data->stream_id=1,sp->stream_id=1
camera_v4l2_s_parm:session_id = 1,event_data->stream_id=2,sp->stream_id=2
camera_v4l2_s_parm:session_id = 1,event_data->stream_id=3,sp->stream_id=3

Stay Hungry，Stay Foolish!