v4l2驱动框架分析-1

需要思考的问题:

(1) cimutils应用程序维护了哪些结构体,v4l2驱动框架维护了哪些结构体

(2)/dev/video0 这个节点怎么创建的

(3)应用层open 设备节点/dev/video0 的时候,内核中的调用关系和具体干的工作

(4)应用层ioctl 操作后,内核中的调用关系流程

(5) VIDIOC_QBUF / VIDIOC_STREAMON / VIDIOC_DQBUF 视频缓存队列是如何管理的?驱动中在哪里申请分配内存?怎么入列出列?

(6)I/O操作方式:V4L2_MEMORY_MMAP 和V4L2_MEMORY_USERPTR的区别

整个v4l2的框架分为三层:

    在应用层,我们可以在 /dev 目录发现 video0 类似的设备节点,上层的摄像头程序打开设备节点进行数据捕获,显示视频画面。设备节点的名字很统一,video0 video1 video2...这些设备节点在是核心层注册。struct video_device   video_register_device  /dev/vedio0

    核心层 v4l2-dev.c,承上启下,对于每一个硬件相关层注册进来的设备,设置一个统一的接口 v4l2_fops ,既然是统一的接口必然不是具体的视频设备的操作函数,应用层调用 v4l2_fops 中的函数最终将调用到硬件相关层的 video_device 的 fops 。

    硬件相关层,与具体的视频硬件打交道,分配、设置、注册 video_device 结构体。struct video_device   video_register_device  /dev/vedio0

 

v4l2_fops为video4linux2设备提供了统一的应用层接口

drivers/media/v4l2-core/v4l2-dev.c

static const struct file_operations v4l2_fops

.owner = THIS_MODULE,

    .read = v4l2_read,

    .write = v4l2_write,

    .open = v4l2_open,

    .get_unmapped_area = v4l2_get_unmapped_area,

    .mmap = v4l2_mmap,

    .unlocked_ioctl = v4l2_ioctl,

#ifdef CONFIG_COMPAT

    .compat_ioctl = v4l2_compat_ioctl32,

#endif

    .release = v4l2_release,

    .poll = v4l2_poll,

    .llseek = no_llseek,

};

drivers/media/platform/soc_camera/soc_camera.c

static struct v4l2_file_operations soc_camera_fops = {

    .owner      = THIS_MODULE,

    .open       = soc_camera_open,

    .release    = soc_camera_close,

    .unlocked_ioctl = video_ioctl2,

    .read       = soc_camera_read,

    .mmap       = soc_camera_mmap,

    .poll       = soc_camera_poll,

};

Video4Linux2是Linux内核中关于视频设备的内核驱动框架,为上层的访问底层的视频设备提供了统一的接口。

 

camera设备驱动开发:

涉及到的基础知识点:

(1)字符设备驱动 (2)设备模型 (3)平台设备驱动 (4)v4l2框架 (5)i2c驱动框架

 

涉及到的术语:

camera : 指的是整个camera,包括它本身的硬件连接方式及支持i2c控制的i2c设备

sensor : 指的是支持i2c控制的i2c设备,它属于camera的一部分,在内核实现里也能体现出来

camera host: 指的是与camera相连接的,一般内嵌在soc里面的控制器

 

涉及到的文件夹:

drivers/media/platform/soc_camera/ 主要存放camera host驱动,通用的camera驱动也存放在此

drivers/media/i2c/soc_camera/ 主要存放sensor驱动

(1)

Linux系统中视频输入设备主要包括以下四个部分:

(1)字符设备驱动程序核心:V4L2本身就是一个字符设备,具有字符设备所有的特性,暴露接口给用户空间;

(2)V4L2驱动核心:主要是构建一个内核中标准视频设备驱动的框架,为视频操作提供统一的接口函数;

(3)平台V4L2设备驱动:在V4L2框架下,根据平台自身的特性实现与平台相关的V4L2驱动部分,包括注册video_device和v4l2_dev。

(4)具体的sensor驱动:主要上电、提供工作时钟、视频图像裁剪、流IO开启等,实现各种设备控制方法供上层调用并注册v4l2_subdev。

V4L2的核心源码位于drivers/media/v4l2-core/,源码以实现的功能可以划分为四类:

(1)核心模块实现:由v4l2-dev.c实现,主要作用申请字符主设备号、注册class和提供video device注册注销等相关函数;

(2)V4L2框架:由v4l2-device.c、v4l2-subdev.c、v4l2-fh.c、v4l2-ctrls.c等文件实现,构建V4L2框架;

(3)Videobuf管理:由videobuf2-core.c、videobuf2-dma-contig.c、videobuf2-dma-sg.c、videobuf2-memops.c、videobuf2-vmalloc.c、v4l2-mem2mem.c等文件实现,完成videobuffer的分配、管理和注销。

(4)Ioctl框架:由v4l2-ioctl.c文件实现,构建V4L2ioctl的框架。

图1

(2)

I/O访问:

V4L2支持三种不同IO访问方式(内核中还支持了其它的访问方式,暂不讨论):

(1)read和write,是基本帧IO访问方式,通过read读取每一帧数据,数据需要在内核和用户之间拷贝,这种方式访问速度可能会非常慢;

(2)内存映射缓冲区(V4L2_MEMORY_MMAP),是在内核空间开辟缓冲区,应用通过mmap()系统调用映射到用户地址空间。这些缓冲区可以是大而连续DMA缓冲区、通过vmalloc()创建的虚拟缓冲区,或者直接在设备的IO内存中开辟的缓冲区(如果硬件支持);

(3)用户空间缓冲区(V4L2_MEMORY_USERPTR),是用户空间的应用中开辟缓冲区,用户与内核空间之间交换缓冲区指针。很明显,在这种情况下是不需要mmap()调用的,但驱动为有效的支持用户空间缓冲区,其工作将也会更困难。

Read和write方式属于帧IO访问方式,每一帧都要通过IO操作,需要用户和内核之间数据拷贝,而后两种是流IO访问方式,不需要内存拷贝,访问速度比较快。内存映射缓冲区访问方式是比较常用的方式。

内存映射缓存区方式

      硬件层的数据流传输

Camerasensor捕捉到图像数据通过并口或MIPI传输到CAMIF(camera interface),CAMIF可以对图像数据进行调整(翻转、裁剪和格式转换等)。然后DMA控制器设置DMA通道请求AHB将图像数据传到分配好的DMA缓冲区。

待图像数据传输到DMA缓冲区之后,mmap操作把缓冲区映射到用户空间,应用就可以直接访问缓冲区的数据。

图2

v4l2驱动代码在drivers\media\v4l2-core文件夹下:

videobuf-core和videobuf2-core;

v4l2-dev.c      ---->video_device   video_register_device  /dev/vedio0

v4l2-device.c   ---->v4l2_device      soc_camera_host_register--->  v4l2_device_register  / v4l2_device_register_subdev

v4l2-subdev     ----> v4l2_subdev    v4l2_i2c_new_subdev

v4l2-ioctl是实现ioctl

video驱动代码在driver/media/目录下,下面分好多子目录,platform目录存放的是不同SoC的驱动代码,对应video_device,其他大多子目录如i2c、mmc、usb、tuners、radio等对应subdev的实现

drivers/media/platform/soc_camera/jz_camera_v13.c

drivers/media/i2c/soc_camera/gc2155.c

设备实例(v4l2_device)

           |______子设备实例(v4l2_subdev)

           |______视频设备节点(video_device)

           |______文件访问控制(v4l2_fh)

           |______视频缓冲的处理(videobuf/videobuf2)

struct vb2_buffer

{

   struct v4l2_buffer

}

(3)

 soc_camera_device   和 soc_camera_host 

struct soc_camera_device {

    struct list_head list;      /* list of all registered devices */                                                                    

    struct soc_camera_desc *sdesc;                                                                                                      

    struct device *pdev;        /* Platform device */                                                                                   

    struct device *parent;      /* Camera host device */                                                                                

    struct device *control;     /* E.g., the i2c client */                                                                              

    s32 user_width;

    s32 user_height;                                                                                                                    

    u32 bytesperline;       /* for padding, zero if unused */                                                                           

    u32 sizeimage;

    enum v4l2_colorspace colorspace;                                                                                                    

    unsigned char iface;        /* Host number */                                                                                       

    unsigned char devnum;       /* Device number per host */                                                                            

    struct soc_camera_sense *sense; /* See comment in struct definition */                                                              

    struct video_device *vdev;

    struct v4l2_ctrl_handler ctrl_handler;                                                                                              

    const struct soc_camera_format_xlate *current_fmt;                                                                                  

    struct soc_camera_format_xlate *user_formats;

    int num_user_formats;

    enum v4l2_field field;      /* Preserve field over close() */                                                                       

    void *host_priv;        /* Per-device host private data */                                                                          

    /* soc_camera.c private count. Only accessed with .host_lock held */                                                                

    int use_count;

    struct file *streamer;      /* stream owner */

    union {

        struct videobuf_queue vb_vidq;

        struct vb2_queue vb2_vidq;

    };

};

struct soc_camera_host {

    struct v4l2_device v4l2_dev;

    struct list_head list;

    struct mutex host_lock;     /* Protect pipeline modifications */

    unsigned char nr;       /* Host number */

    u32 capabilities;

    void *priv;

    const char *drv_name;

    struct soc_camera_host_ops *ops;

};

应用层xioctl(fd, VIDIOC_QBUF, &buf)

-------------------------------------------------------------------------------

drivers/media/v4l2-core/v4l2-ioctl.c

static int v4l_qbuf(const struct v4l2_ioctl_ops *ops,

                struct file *file, void *fh, void *arg)

{

    struct v4l2_buffer *p = arg; 

    int ret = check_fmt(file, p->type);

    return ret ? ret : ops->vidioc_qbuf(file, fh, p);

}

drivers/media/platform/soc_camera/soc_camera.c

static const struct v4l2_ioctl_ops soc_camera_ioctl_ops = {

   .vidioc_qbuf         = soc_camera_qbuf,

}

static int soc_camera_qbuf(struct file *file, void *priv,                                                                               

               struct v4l2_buffer *p)                                                                                                   

{                                                                                                                                       

    struct soc_camera_device *icd = file->private_data;                                                                                 

    struct soc_camera_host *ici = to_soc_camera_host(icd->parent);                                                                                                                                                                                                             

    WARN_ON(priv != file->private_data);                                                                                                                                                                                                                                     

    if (icd->streamer != file)                                                                                                          

        return -EBUSY;                                                                                                                                                                                                                                                          

    if (ici->ops->init_videobuf)                                                                                                        

        return videobuf_qbuf(&icd->vb_vidq, p);  //vb1_buffer                                                                                       

    else                                                                                                                                

        return vb2_qbuf(&icd->vb2_vidq, p);       //vb2_buffer                                                                                      

}

应用层open 节点/dev/vedio0时,调用内核接口

v4l2_fops为video4linux2设备提供了统一的应用层接口

drivers/media/v4l2-core/v4l2-dev.c

static const struct file_operations v4l2_fops = {

    .owner = THIS_MODULE,

    .read = v4l2_read,

    .write = v4l2_write,

    .open = v4l2_open,

    .get_unmapped_area = v4l2_get_unmapped_area,

    .mmap = v4l2_mmap,

    .unlocked_ioctl = v4l2_ioctl,

#ifdef CONFIG_COMPAT

    .compat_ioctl = v4l2_compat_ioctl32,

#endif

    .release = v4l2_release,

    .poll = v4l2_poll,

    .llseek = no_llseek,

};

drivers/media/platform/soc_camera/soc_camera.c

static struct v4l2_file_operations soc_camera_fops = {

    .owner      = THIS_MODULE,

    .open       = soc_camera_open,

    .release    = soc_camera_close,

    .unlocked_ioctl = video_ioctl2,

    .read       = soc_camera_read,

    .mmap       = soc_camera_mmap,

    .poll       = soc_camera_poll,

};

static int soc_camera_open(struct file *file)

{

  ret = ici->ops->init_videobuf2(&icd->vb2_vidq, icd);

}

drivers/media/platform/soc_camera/jz_camera_v13.c 

static int jz_camera_init_videobuf2(struct vb2_queue *q, struct soc_camera_device *icd)--->

vb2_queue_init(q)

{

INIT_LIST_HEAD(&q->queued_list);

}

int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)

{

  list_add_tail(&vb->queued_entry, &q->queued_list);

 if (q->streaming)                                                                                                                   

        __enqueue_in_driver(vb);

    /* Fill buffer information for the userspace */                                                                                     

    __fill_v4l2_buffer(vb, b);                                                                                                          

                                 

}

list_add_tail(&vb->queued_entry, &q->queued_list);

struct vb2_queue *q

struct vb2_buffer *vb;

struct vb2_buffer {

struct list_head    queued_entry;

}

struct vb2_queue {

struct list_head        queued_list;

}

static void __enqueue_in_driver(struct vb2_buffer *vb)                                                                                  

{                                                                                                                                       

    struct vb2_queue *q = vb->vb2_queue;                                                                                                

    unsigned int plane;                                                                                                                 

                                                                                                                                        

    vb->state = VB2_BUF_STATE_ACTIVE;                                                                                                   

    atomic_inc(&q->queued_count);                                                                                                       

                                                                                                                                        

    /* sync buffers */                                                                                                                  

    for (plane = 0; plane < vb->num_planes; ++plane)                                                                                    

        call_memop(q, prepare, vb->planes[plane].mem_priv);                                                                             

                                                                                                                                        

    q->ops->buf_queue(vb);                                                                                                              

}  

drivers/media/platform/soc_camera/jz_camera_v13.c

static struct vb2_ops jz_videobuf2_ops = {                                                                                              

    .buf_init       = jz_buffer_init,                                                                                                   

    .queue_setup        = jz_queue_setup,                                                                                               

    .buf_prepare        = jz_buffer_prepare,                                                                                            

    .buf_queue      = jz_buffer_queue,                                                                                                  

    .start_streaming    = jz_start_streaming,                                                                                           

    .stop_streaming     = jz_stop_streaming,                                                                                            

    .wait_prepare       = soc_camera_unlock,                                                                                            

    .wait_finish        = soc_camera_lock,                                                                                              

}; 

应用层调用ioctl

ioctl(camera_v4l2->fd, VIDIOC_STREAMON, &type)

-----------------------------------------------------------------------------

(1)

drivers/media/v4l2-core/v4l2-ioctl.c

static int v4l_streamon(const struct v4l2_ioctl_ops *ops,                                                                               

                struct file *file, void *fh, void *arg)                                                                                 

{                                                                                                                                       

    return ops->vidioc_streamon(file, fh, *(unsigned int *)arg);                                                                        

(2)

drivers/media/platform/soc_camera/soc_camera.c 

static int soc_camera_streamon(struct file *file, void *priv,

                   enum v4l2_buf_type i)

{

    if (ici->ops->init_videobuf)

        ret = videobuf_streamon(&icd->vb_vidq);

    else 

        ret = vb2_streamon(&icd->vb2_vidq, i);

    if (!ret)

        v4l2_subdev_call(sd, video, s_stream, 1); //  

    return ret; 

}

struct v4l2_subdev {

const struct v4l2_subdev_ops *ops;

}

struct v4l2_subdev_ops {

    const struct v4l2_subdev_video_ops  *video;

}; 

(3)

drivers/media/i2c/soc_camera/gc2155.c

static struct v4l2_subdev_video_ops gc2155_subdev_video_ops = {                                                                         

    .s_stream   = gc2155_s_stream,    //开始视频采集                                                                                                  

    .g_mbus_fmt = gc2155_g_fmt,                                                                                                         

    .s_mbus_fmt = gc2155_s_fmt,                                                                                                         

    .try_mbus_fmt   = gc2155_try_fmt,                                                                                                   

    .cropcap    = gc2155_cropcap,                                                                                                       

    .g_crop     = gc2155_g_crop,                                                                                                        

    .enum_mbus_fmt  = gc2155_enum_fmt,                                                                                                  

    .g_mbus_config  = gc2155_g_mbus_config,                                                                                             

#if 0                                                                                                                                   

    .enum_framesizes = gc2155_enum_framesizes,                                                                                          

    .enum_frameintervals = gc2155_enum_frameintervals,                                                                                  

#endif                                                                                                                                  

};

将帧缓冲区在视频输入队列排队,并启动视频采集

在驱动程序处理视频的过程中,定义了两个队列:视频采集输入队列(incoming queues)和视频采集输出队列(outgoing queues),前者是等待驱动存放视频数据的队列,后者是驱动程序已经放入了视频数据的队列。

应用程序需要将上述帧缓冲区在视频采集输入队列排队(VIDIOC_QBUF),然后可启动视频采集

循环往复,采集连续的视频数据

启动视频采集后,驱动程序开始采集一帧数据,把采集的数据放入视频采集输入队列的第一个帧缓冲区,一帧数据采集完成,也就是第一个帧缓冲区存满一帧数据后,驱动程序将该帧缓冲区移至视频采集输出队列,等待应用程序从输出队列取出。驱动程序接下来采集下一帧数据,放入第二个帧缓冲区,同样帧缓冲区存满下一帧数据后,被放入视频采集输出队列。

应用程序从视频采集输出队列中取出含有视频数据的帧缓冲区,处理帧缓冲区中的视频数据,如存储或压缩。

最后,应用程序将处理完数据的帧缓冲区重新放入视频采集输入队列,这样可以循环采集

 

soc_camera_host,soc_camera_device,v4l2_device,v4l2_subdev关系如下:

(1) 理论上soc系统内可以有多个soc_camera_host(控制器),物理上soc_camera_host就是系统的camera处理模块驱动

(2) 一个soc_camera_host(控制器)可以对应多个soc_camera_device(设备),物理上soc_camera_device是一个camera接口,每个soc_camera_host对应一个v4l2_device

(3) 每个soc_camera_device,系统会为他们创建设备节点/dev/videoX。

(4) 每个soc_camera_device有多个v4l2_subdev,物理上v4l2_subdev可以是sensor,video AD芯片

v4l2_subdev可以通过i2c挂接到v4l2_device,也可以通过soc_camera_link提供的add_device来增加,这依赖于sensor和video AD芯片挂接到MCU camera接口的方式。

你可能感兴趣的:(linux,camera-V4L2驱动框架)