Python+OpenCV：摄像机标定（Camera Calibration）

Python+OpenCV：摄像机标定（Camera Calibration）

理论

Some pinhole cameras introduce significant distortion to images. Two major kinds of distortion are radial distortion (径向畸变) and tangential distortion (切向畸变).

Radial distortion causes straight lines to appear curved. Radial distortion becomes larger the farther points are from the center of the image.

For example, one image is shown below in which two edges of a chess board are marked with red lines.

But, you can see that the border of the chess board is not a straight line and doesn't match with the red line.

All the expected straight lines are bulged out.

Radial distortion can be represented as follows:

Similarly, tangential distortion occurs because the image-taking lense is not aligned perfectly parallel to the imaging plane.

So, some areas in the image may look nearer than expected. The amount of tangential distortion can be represented as below:

In short, we need to find five parameters, known as distortion coefficients given by:

In addition to this, we need to some other information, like the intrinsic and extrinsic parameters of the camera.

Intrinsic parameters are specific to a camera. They include information like focal length ( fx,fy) and optical centers ( cx,cy).

The focal length and optical centers can be used to create a camera matrix, which can be used to remove distortion due to the lenses of a specific camera.

The camera matrix is unique to a specific camera, so once calculated, it can be reused on other images taken by the same camera.

It is expressed as a 3x3 matrix:

Extrinsic parameters corresponds to rotation and translation vectors which translates a coordinates of a 3D point to a coordinate system.

For stereo applications, these distortions need to be corrected first.

To find these parameters, we must provide some sample images of a well defined pattern (e.g. a chess board).

We find some specific points of which we already know the relative positions (e.g. square corners in the chess board).

We know the coordinates of these points in real world space and we know the coordinates in the image, so we can solve for the distortion coefficients.

For better results, we need at least 10 test patterns.

Camera Calibration in OpenCV

####################################################################################################
# 摄像机标定(Camera Calibration)
def lmc_cv_camera_calibration():
    """
        函数功能: 摄像机标定(Camera Calibration)。
    """

    # termination criteria
    criteria = (lmc_cv.TERM_CRITERIA_EPS + lmc_cv.TERM_CRITERIA_MAX_ITER, 30, 0.001)

    # prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
    objp = np.zeros((6 * 7, 3), np.float32)
    objp[:, :2] = np.mgrid[0:7, 0:6].T.reshape(-1, 2)

    # Arrays to store object points and image points from all the images.
    objpoints = []  # 3d point in real world space
    imgpoints = []  # 2d points in image plane.

    # 读取所有匹配的图像
    stacking_images = []
    images = glob.glob('D:/99-Research/TestData/cv/stereo/case1/left*.png')
    for image_name in images:
        image = lmc_cv.imread(image_name, lmc_cv.IMREAD_UNCHANGED)
        image = lmc_cv.cvtColor(image, lmc_cv.COLOR_BGR2RGB)
        corners_image = image.copy()
        gray_image = lmc_cv.cvtColor(image, lmc_cv.COLOR_BGR2GRAY)

        # Find the chess board corners
        ret, corners = lmc_cv.findChessboardCorners(gray_image, (7, 6), None)

        # If found, add object points, image points (after refining them)
        if ret:
            objpoints.append(objp)

            corners2 = lmc_cv.cornerSubPix(gray_image, corners, (11, 11), (-1, -1), criteria)
            imgpoints.append(corners)

            # Draw and display the corners
            lmc_cv.drawChessboardCorners(corners_image, (7, 6), corners2, ret)

            # stacking images side-by-side
            stacking_image = np.hstack((image, corners_image))
            stacking_images.append(stacking_image)

    # 显示图像
    for i in range(len(stacking_images)):
        pyplot.figure('Camera Calibration Corners Locations %d' % (i + 1), figsize=(16, 9))
        # pyplot.subplot(1, 1, 1)
        pyplot.imshow(stacking_images[i], 'gray')
        pyplot.title('Camera Calibration Corners Locations')
        pyplot.xticks([])
        pyplot.yticks([])
        pyplot.savefig('%02d.png' % (i + 1))
    pyplot.show()

    # Calibration
    ret, mtx, dist, rvecs, tvecs = lmc_cv.calibrateCamera(objpoints, imgpoints, gray_image.shape[::-1], None, None)
    # Undistortion
    image = lmc_cv.imread(images[11], lmc_cv.IMREAD_UNCHANGED)
    image = lmc_cv.cvtColor(image, lmc_cv.COLOR_BGR2RGB)
    h, w = image.shape[:2]
    newcameramtx, roi = lmc_cv.getOptimalNewCameraMatrix(mtx, dist, (w, h), 1, (w, h))

    # undistort using undistort()
    undistort_image = lmc_cv.undistort(image, mtx, dist, None, newcameramtx)
    # crop the image
    x, y, w, h = roi
    undistort_image = undistort_image[y:y + h, x:x + w]

    # undistort using remapping
    mapx, mapy = lmc_cv.initUndistortRectifyMap(mtx, dist, None, newcameramtx, (w, h), 5)
    remapping_image = lmc_cv.remap(image, mapx, mapy, lmc_cv.INTER_LINEAR)
    # crop the image
    x, y, w, h = roi
    remapping_image = remapping_image[y:y + h, x:x + w]

    # Re-projection Error
    mean_error = 0
    for i in range(len(objpoints)):
        imgpoints2, _ = lmc_cv.projectPoints(objpoints[i], rvecs[i], tvecs[i], mtx, dist)
        error = lmc_cv.norm(imgpoints[i], imgpoints2, lmc_cv.NORM_L2) / len(imgpoints2)
        mean_error += error
    print("总误差: {}".format(mean_error / len(objpoints)))

    # 显示图像
    titles = ['Original Image', 'Corners', 'Undistortion using undistort()', 'Undistortion using remapping']
    images = [image, corners_image, undistort_image, remapping_image]
    for i in range(len(images)):
        pyplot.figure('Camera Calibration')
        pyplot.subplot(2, 2, i + 1)
        pyplot.imshow(images[i], 'gray')
        pyplot.title(titles[i])
        pyplot.xticks([])
        pyplot.yticks([])
        pyplot.savefig('Camera Calibration.png')
    pyplot.show()

    # 根据用户输入保存图像
    if ord("q") == (lmc_cv.waitKey(0) & 0xFF):
        # 销毁窗口
        pyplot.close('all')
    return