Video4Linux2 part 6a: Basic frame I/O

This series of articles onvideo drivers has been through several installments, but we have yet to transfer a single frame of video data. Atthis point, though, we have covered enough of the format negotiationdetails that we can begin to look at how video frames move between theapplication and device.

The Video4Linux2 API defines three different ways of transferring videoframes, two of which are actually available in the current implementation:

 

• The read() and write() system calls can be used in the normal way. Depending on the hardware and how the driver is implemented, this technique might be relatively slow - but it does not have to be that way.

   

• Frames can be streamed directly to and from buffers accessible to the application. Streaming is usually the most efficient way to move video data; this interface also allows for the transfer of some useful metadata with the image frames. There are two variants of the streaming technique, depending on whether the buffers are located in user or kernel space.

   

• The Video4Linux2 API specification provides for an asynchronous I/O mechanism for frame transfer. This mode has not been implemented, however, and cannot be used.

This article will look at the simple read() and write()interface; streaming transfers will be covered in the next installment.

 

read() and write()

Implementation of read() and write() is not required bythe Video4Linux2 specification. Many simpler applications expect thesesystem calls to be available, though, so, if possible, the driver writershould make them work. If the driver does support these calls, it shouldbe sure to set the V4L2_CAP_READWRITE bit in response to aVIDIOC_QUERYCAP call (described in part 3). In your editor'sexperience, however, most applications do not bother to check whether thesecalls are available before attempting to use them.

The driver's read() and/or write() methods must be storedin the fops field of the associated video_devicestructure. Note that the Video4Linux2 specification requires driversimplementing these methods to provide a poll() operation as well.

A naive implementation of read() on a frame grabber device isstraightforward: the driver tells the hardware to start capturing frames,delivers one to the user-space buffer, stops the hardware, and returns. Ifpossible, the driver should arrange for the DMA operation to transfer thedata directly to the destination buffer, but that is only possible if thecontroller can handle scatter/gather I/O. Otherwise, the driver will needto buffer the frame through the kernel. Similarly, write operations shouldgo directly to the device if possible, but be buffered through the kernelotherwise.

Less simplistic implementations are possible. Your editor's "Cafe" driver,for example, leaves the camera controller running in a speculative modeafter a read() operation. For the next fraction of a second,subsequent frames from the camera will be buffered in the kernel; if theapplication issues another read() call, it will be satisfied morequickly without the need to start up the hardware again. After a number ofunclaimed frames the controller is put back into an idlestate. Similarly, a write() operation could delay the first frameby a few tens of milliseconds with the idea of helping the applicationstream frames at the hardware's expected rate.

 

Streaming parameters

The VIDIOC_G_PARM and VIDIOC_S_PARM ioctl()calls adjust some parameters which are specific to read() andwrite() implementations - and some which are more general. Itappears to be a call where miscellaneous options with no obvious home wereput. We'll cover it here, even though some of the parameters affectstreaming I/O as well.

Video4Linux2 drivers supporting these calls provide the following twomethods:

 

    int (*vidioc_g_parm) (struct file *file, void *private_data,

    			  struct v4l2_streamparm *parms);

    int (*vidioc_s_parm) (struct file *file, void *private_data,

			  struct v4l2_streamparm *parms);

The v4l2_streamparm structure contains one of those unions whichshould be getting familiar to readers of this series by now:

 

    struct v4l2_streamparm

{

	enum v4l2_buf_type type;

	union

	{

		struct v4l2_captureparm	capture;

		struct v4l2_outputparm	output;

		__u8 raw_data[200];

	} parm;

    };

The type field describes the type of operation to be affected; itwill be V4L2_BUF_TYPE_VIDEO_CAPTURE for capture devices andV4L2_BUF_TYPE_VIDEO_OUTPUT for output devices. It can also beV4L2_BUF_TYPE_PRIVATE, in which case the raw_data fieldis used to pass some sort of private, non-portable, probably discourageddata through to the driver.

For capture devices, the parm.capture field will be of interest.That structure looks like this:

 

    struct v4l2_captureparm

    {

	__u32		   capability;

	__u32		   capturemode;

	struct v4l2_fract  timeperframe;

	__u32		   extendedmode;

	__u32              readbuffers;

	__u32		   reserved[4];

    };

capability is a set of capability flags; the only one currentlydefined is V4L2_CAP_TIMEPERFRAME which indicates that the devicecan vary its frame rate. capturemode is another flag field withexactly one flag defined: V4L2_MODE_HIGHQUALITY, intended to putthe hardware into a high-quality mode suitable for single-frame captures.This mode can make any number of sacrifices (in terms of the data formatssupported, exposure times, etc.) in order to get the best image qualitythat the device can handle.

The timeperframe field is used to specify the desired frame rate.It is yet another structure:

 

    struct v4l2_fract {

	__u32   numerator;

	__u32   denominator;

    };

The quotient described by numerator and denominator givesthe time between successive frames on the device. Another driver-specificfield is extendedmode, which has no defined meaning in the API.The readbuffers field is the number of buffers the kernel shoulduse for incoming frames when the read() method is being used.

For video output devices, the structure looks like:

 

    struct v4l2_outputparm

    {

	__u32		   capability;

	__u32		   outputmode;

	struct v4l2_fract  timeperframe;

	__u32		   extendedmode;

	__u32              writebuffers;

	__u32		   reserved[4];

    };

The capability, timeperframe, and extendedmodefields are exactly the same as for capture devices. outputmodeand writebuffers have the same effect as capturemode andreadbuffers, respectively.

When the application wishes to query the current parameters, it will issuea VIDIOC_G_PARM call, resulting in a call to the driver'svidioc_g_parm() method. The driver should provide the currentsettings, being sure to set the extendedmode field to zero if itis not being used, and the reserved field to zero always.

An attempt to set the parameters results in a call tovidioc_s_parm(). In this case, the driver should set theparameters as closely as possible to the application's request and adjustthe v4l2_streamparm structure to reflect the values which wereactually used. For example, the application might request a higher framerate than the hardware can provide; in this case, the fastest possible rateshould be programmed and the timeperframe field set to the actualframe rate.

If timeperframe is given as zero by the application, the drivershould program the nominal frame rate associated with the current videonorm. If readbuffers or writebuffers is zero, the drivershould return the current settings rather than getting rid of the currentbuffers.

At this point, we have covered enough to write a simple driver supportingframe transfer with read() or write(). Most seriousapplications will want to use streaming I/O, however: the streaming modemakes higher performance easier, and it allows frames to be packaged withrelevant metadata like sequence numbers. Tune in for the next installmentin this series which will discuss how to implement the streaming API invideo drivers.