python图像处理opencv笔记（二）：视频基本操作

视频基本操作

视频读取

opencv中通过VideoCaptrue类对视频进行读取操作以及调用摄像头，下面是该类的API：

import cv2

video = cv2.VideoCapture(0)		# 打开本地摄像头，如果是视频流，可将0替换为url

在C++中，CvCapture 是一个结构体，用来保存图像捕获所需要的信息。 opencv通过调用底层ffmpeg提供两种方式从外部捕获图像，打开摄像头或者解析视频流，直接再后面加0或者路径就可以了。而在python中也一样，VideoCapture()是用于从视频文件、图片序列、摄像头捕获视频的类，当我们创建好了上面的video对象后，就可以查看使用帮助和具体能调用的方法，使用help和dir：

$ help(video)
class VideoCapture(builtins.object)

 ......
 
 |  grab(...)
 |      grab() -> retval
 |      .   @brief Grabs the next frame from video file or capturing device.
 |      .   
 |      .       @return `true` (non-zero) in the case of success.
 |      .   
 |      .       The method/function grabs the next frame from video file or camera and returns true (non-zero) in
 |      .       the case of success.
 |      .   
 |      .       The primary use of the function is in multi-camera environments, especially when the cameras do not
 |      .       have hardware synchronization. That is, you call VideoCapture::grab() for each camera and after that
 |      .       call the slower method VideoCapture::retrieve() to decode and get frame from each camera. This way
 |      .       the overhead on demosaicing or motion jpeg decompression etc. is eliminated and the retrieved frames
 |      .       from different cameras will be closer in time.
 |      .   
 |      .       Also, when a connected camera is multi-head (for example, a stereo camera or a Kinect device), the
 |      .       correct way of retrieving data from it is to call VideoCapture::grab() first and then call
 |      .       VideoCapture::retrieve() one or more times with different values of the channel parameter.
 |      .   
 |      .       @ref tutorial_kinect_openni
 |  
 |  isOpened(...)
 |      isOpened() -> retval
 |      .   @brief Returns true if video capturing has been initialized already.
 |      .   
 |      .       If the previous call to VideoCapture constructor or VideoCapture::open() succeeded, the method returns
 |      .       true.
 |  
 |  open(...)
 |      open(filename[, apiPreference]) -> retval
 |      .   @brief  Opens a video file or a capturing device or an IP video stream for video capturing.
 |      .   
 |      .       @overload
 |      .   
 |      .       Parameters are same as the constructor VideoCapture(const String& filename, int apiPreference = CAP_ANY)
 |      .       @return `true` if the file has been successfully opened
 |      .   
 |      .       The method first calls VideoCapture::release to close the already opened file or camera.
 |      
 |      
 |      
 |      open(index[, apiPreference]) -> retval
 |      .   @brief  Opens a camera for video capturing
 |      .   
 |      .       @overload
 |      .   
 |      .       Parameters are same as the constructor VideoCapture(int index, int apiPreference = CAP_ANY)
 |      .       @return `true` if the camera has been successfully opened.
 |      .   
 |      .       The method first calls VideoCapture::release to close the already opened file or camera.
 |  
 |  read(...)
 |      read([, image]) -> retval, image
 |      .   @brief Grabs, decodes and returns the next video frame.
 |      .   
 |      .       @param [out] image the video frame is returned here. If no frames has been grabbed the image will be empty.
 |      .       @return `false` if no frames has been grabbed
 |      .   
 |      .       The method/function combines VideoCapture::grab() and VideoCapture::retrieve() in one call. This is the
 |      .       most convenient method for reading video files or capturing data from decode and returns the just
 |      .       grabbed frame. If no frames has been grabbed (camera has been disconnected, or there are no more
 |      .       frames in video file), the method returns false and the function returns empty image (with %cv::Mat, test it with Mat::empty()).
 |      .   
 |      .       @note In @ref videoio_c "C API", functions cvRetrieveFrame() and cv.RetrieveFrame() return image stored inside the video
 |      .       capturing structure. It is not allowed to modify or release the image! You can copy the frame using
 |      .       cvCloneImage and then do whatever you want with the copy.
 |  
 |  release(...)
 |      release() -> None
 |      .   @brief Closes video file or capturing device.
 |      .   
 |      .       The method is automatically called by subsequent VideoCapture::open and by VideoCapture
 |      .       destructor.
 |      .   
 |      .       The C function also deallocates memory and clears \*capture pointer.
 |  
 |  retrieve(...)
 |      retrieve([, image[, flag]]) -> retval, image
 |      .   @brief Decodes and returns the grabbed video frame.
 |      .   
 |      .       @param [out] image the video frame is returned here. If no frames has been grabbed the image will be empty.
 |      .       @param flag it could be a frame index or a driver specific flag
 |      .       @return `false` if no frames has been grabbed
 |      .   
 |      .       The method decodes and returns the just grabbed frame. If no frames has been grabbed
 |      .       (camera has been disconnected, or there are no more frames in video file), the method returns false
 |      .       and the function returns an empty image (with %cv::Mat, test it with Mat::empty()).
 |      .   
 |      .       @sa read()
 |      .   
 |      .       @note In @ref videoio_c "C API", functions cvRetrieveFrame() and cv.RetrieveFrame() return image stored inside the video
 |      .       capturing structure. It is not allowed to modify or release the image! You can copy the frame using
 |      .       cvCloneImage and then do whatever you want with the copy.
 |  
 |  set(...)
 |      set(propId, value) -> retval
 |      .   @brief Sets a property in the VideoCapture.
 |      .   
 |      .       @param propId Property identifier from cv::VideoCaptureProperties (eg. cv::CAP_PROP_POS_MSEC, cv::CAP_PROP_POS_FRAMES, ...)
 |      .       or one from @ref videoio_flags_others
 |      .       @param value Value of the property.
 |      .       @return `true` if the property is supported by backend used by the VideoCapture instance.
 |      .       @note Even if it returns `true` this doesn't ensure that the property
 |      .       value has been accepted by the capture device. See note in VideoCapture::get()
 |  
 |  setExceptionMode(...)
 |      setExceptionMode(enable) -> None
 |      .   Switches exceptions mode
 |      .        *
 |      .        * methods raise exceptions if not successful instead of returning an error code


$ print(dir(video))
['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'get', 'getBackendName', 'getExceptionMode', 'grab', 'isOpened', 'open', 'read', 'release', 'retrieve', 'set', 'setExceptionMode']

那么本篇就是介绍上面的函数使用方法和一些具体注意事项。

视频参数查看

如果是用python直接pip安装的opencv，一般都能直接创建成功videocapture对象，但若是编译的opencv用cv2.so文件，本地的ffmpeg相关依赖没有安装好，创建对象可能会报 ‘Segmentation fault’ with gpu/cpu video decoding，当时我在stack上搜索都是说的指针越界，后来又得从头开始编译安装，所以这是需要注意的一个点，然后拿到视频捕获对象，我们可以对其进行参数设置：

capture.set(CV_CAP_PROP_FRAME_WIDTH, 1080);//宽度 
capture.set(CV_CAP_PROP_FRAME_HEIGHT, 960);//高度
capture.set(CV_CAP_PROP_FPS, 30);//帧率 帧/秒
capture.set(CV_CAP_PROP_BRIGHTNESS, 1);//亮度 1
capture.set(CV_CAP_PROP_CONTRAST,40);//对比度 40
capture.set(CV_CAP_PROP_SATURATION, 50);//饱和度 50
capture.set(CV_CAP_PROP_HUE, 50);//色调 50
capture.set(CV_CAP_PROP_EXPOSURE, 50);//曝光 50

capture即video对象，上述很多都是默认的，如果不知道参数具体含义，就不要乱修改，因为都是通用参数，另外，有set那么就肯定会有get：

capture.get(CV_CAP_PROP_FRAME_WIDTH);
capture.get(CV_CAP_PROP_FRAME_HEIGHT);
capture.get(CV_CAP_PROP_FPS);
capture.get(CV_CAP_PROP_BRIGHTNESS);
capture.get(CV_CAP_PROP_CONTRAST);
capture.get(CV_CAP_PROP_SATURATION);
capture.get(CV_CAP_PROP_HUE);
capture.get(CV_CAP_PROP_EXPOSURE);

这里的get参数主要获取的是当前videocapture拿到的视频的宽高以及FPS，比如后面要介绍的视频录制，如果不知道当前流是什么比例，就需要用get去拿到相应的值 size = (int(vidcap.get(cv2.CAP_PROP_FRAME_WIDTH)),int(vidcap.get(cv2.CAP_PROP_FRAME_HEIGHT)))，另外还有一种简写的方式，参考 OpenCV VideoCapture.get()参数详解，如下：

param	define
cv2.VideoCapture.get(0)	视频文件的当前位置（播放）以毫秒为单位
cv2.VideoCapture.get(1)	基于以0开始的被捕获或解码的帧索引
cv2.VideoCapture.get(2)	视频文件的相对位置（播放）：0=电影开始，1=影片的结尾。
cv2.VideoCapture.get(3)	在视频流的帧的宽度
cv2.VideoCapture.get(4)	在视频流的帧的高度
cv2.VideoCapture.get(5)	帧速率
cv2.VideoCapture.get(6)	编解码的4字-字符代码
cv2.VideoCapture.get(7)	视频文件中的帧数
cv2.VideoCapture.get(8)	返回对象的格式
cv2.VideoCapture.get(9)	返回后端特定的值，该值指示当前捕获模式
cv2.VideoCapture.get(10)	图像的亮度(仅适用于照相机)
cv2.VideoCapture.get(11)	图像的对比度(仅适用于照相机)
cv2.VideoCapture.get(12)	图像的饱和度(仅适用于照相机)
cv2.VideoCapture.get(13)	色调图像(仅适用于照相机)
cv2.VideoCapture.get(14)	图像增益(仅适用于照相机)（Gain在摄影中表示白平衡提升）
cv2.VideoCapture.get(15)	曝光(仅适用于照相机)
cv2.VideoCapture.get(16)	指示是否应将图像转换为RGB布尔标志
cv2.VideoCapture.get(17)	× 暂时不支持
cv2.VideoCapture.get(18)	立体摄像机的矫正标注（目前只有DC1394 v.2.x后端支持这个功能）

读取图片

import numpy as np
import cv2

cap = cv2.VideoCapture(0)

while(True):
    # Capture frame-by-frame
    ret, frame = cap.read()

    # Our operations on the frame come here
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # Display the resulting frame
    cv2.imshow('frame',gray)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()

上述例子用到的是read方式进行图片读取，另外还有两种方式为retrieve和grab，具体的区别可以根据官方文档的说明详解：

reading_and_writing_images_and_video

一般来说，VideoCapure里的read是grab和retrieve的结合，grab是指向下一个帧，retrieve是解码并返回一个帧，而且retrieve比grab慢一些，所以当不需要当前的帧或画面时，可以使用grab跳过，这也叫跳帧。当不需要的时候可以多grab之后再read的话，就能比一直read更省时间，因为没有必要把不需要的帧解码。

ret,frame = cap.read()

success = vidcap.grab()

_,image = vidcap.retrieve()

cap.read()按帧读取视频，ret,frame是获cap.read()方法的两个返回值。其中ret是布尔值，如果读取帧是正确的则返回True，如果文件读取到结尾，它的返回值就为False。frame就是每一帧的图像。grab只会返回成功与否，而retrieve与read类似，会返回flag和img。

录制视频

录制视频这里的坑主要集中在视频编码器上，具体的看如下我之前写的例子：

    def new_MakeVideo(self,begin_pos,end_pos,vidcap):
        path = ""     # 不会重复的文件名
        size = (int(vidcap.get(cv2.CAP_PROP_FRAME_WIDTH)),int(vidcap.get(cv2.CAP_PROP_FRAME_HEIGHT)))   # 先试试自动适配大小
        # size = (int(vidcap.set(cv.CAP_PROP_FRAME_WIDTH,1080)),int(vidcap.set()))
        """
        在Fedora中：DIVX，XVID，MJPG，X264，WMV1，WMV2。（最好使用XVID。MJPG会生成大尺寸的视频。X264会生成非常小的尺寸的视频）
        """

        fps = 25
        out_video = cv2.VideoWriter(path, cv2.VideoWriter_fourcc('m','j','p','g'), fps, size)
        SaveTime = [[begin_pos,end_pos]]    # 可以是截取不同时刻
        print(SaveTime[0][0])
        now_frame = 0       # 帧数记录
        while (vidcap.isOpened()):
            ret, frame = vidcap.read()  # 捕获一帧图像，捕获到ret = True
            img_h, img_w, img_ch = frame.shape  # 分别拿出高度，长度和信道数
            # SaveTime = [[38 * 60 + 38, 39 * 60 + 59], [42 * 60 + 54, 44 * 60 + 11], [47 * 60 + 8, 48 * 60 + 24],[51 * 60 + 20, 52 * 60 + 39]]
            if ret:
                if img_ch == 1: # 如果当前信道数为1，不能直接将灰度或二值化的图片保存成视频,需要转换成彩色，RGB
                    frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
                for i in range(len(SaveTime)):
                    if now_frame > SaveTime[i][0] * fps and now_frame < SaveTime[i][1] * fps:   # 视频帧数在savetime之间
                        out_video.write(frame)
                        # print(now_frame,"now_frame..........")
                now_frame += 1
                if now_frame > SaveTime[0][1] * fps:
                    break
                # 0xFF 是一个十六进制常量，它是 11111111 < / code>二进制。通过对这个常数使用按位AND（& ），它只保留原始的最后8位（在这种情况下，不管 cv2.waitKey （0）是）
                k = cv2.waitKey(1) & 0xFF
                if k == 27:
                    break
                # cv2.waitKey(25)
            else:
                break
        out_video.release()

上面例子是本篇所有点的汇总，需要关注的是cv2.VideoWriter() 函数，它的第二个参数需要添加cv2.VideoWriter_fourcc()，这个是编码器的意思，需要调取本地视频编码器的支持类型，一般由四个字母组成，opencv-python包中有的编码器类型为：

cv2.VideoWriter_fourcc('M','J','P','G') = motion-jpeg codec

cv2.VideoWriter_fourcc('P','I','M','1') = MPEG-1 codec
cv2.VideoWriter_fourcc('M', 'P', '4', '2') = MPEG-4.2 codec
cv2.VideoWriter_fourcc('D', 'I', 'V', '3') = MPEG-4.3 codec
cv2.VideoWriter_fourcc('D', 'I', 'V', 'X') = MPEG-4 codec
cv2.VideoWriter_fourcc('U', '2', '6', '3') = H263 codec
cv2.VideoWriter_fourcc('I', '2', '6', '3') = H263I codec
cv2.VideoWriter_fourcc('F', 'L', 'V', '1') = FLV1 codec

上面有几个我也不确定有没有，比如说263的我没试过。一般保存mp4格式的视频就用mp42，或者XVID，但前者所用内存空间更小些，但上面的编码器之前都不能满足我的需求，如果你也遇到相同的情况，那么就要考虑重新编译opencv了。

opencv底层的编码器是集成了ffmpeg的部分，但还有非常多的编码器类型没有被集成，如果我们直接填入该编码器的名称，会出现如下情况：

h264
!!!analysis init sucess vith vision：1.0
detect_width: 416 330 (564, 712)
Could not find encoder for codec id 27: Encoder not found

mjpg
OpenCV: FFMPEG: tag 0x47504a4d/‘MJPG’ is not supported with codec id 7 and format ‘mp4 / MP4 (MPEG-4 Part 14)’
OpenCV: FFMPEG: fallback to use tag 0x7634706d/‘mp4v’

avc1
!!!analysis init sucess vith vision：1.0
detect_width: 416 330 (564, 712)
OpenCV: FFMPEG: tag 0x31435641/‘AVC1’ is not supported with codec id 27 and format ‘mp4 / MP4 (MPEG-4 Part 14)’
OpenCV: FFMPEG: fallback to use tag 0x31637661/‘avc1’
Could not find encoder for codec id 27: Encoder not found

所以我们需要先编译本地的ffmpeg，再进行opencv编译，ffmpeg具体可以看我之前的博客，然后opencv编译的还没写，最近没考虑在时间范围内，那么本篇博文到此结束。