解决问题
在图像测量过程以及机器视觉应用中，利用摄像机所拍摄到的图像来还原空间中的物体。
基本思路
假设摄像机所拍摄到的图像与三维空间中的物体之间存在以下一种简单的线性关系:[像]=M[物],这里，矩阵M可以看成是摄像机成像的几何模型。 M中的参数就是摄像机参数。通常，这些参数是要通过实验与计算来得到的。这个求解参数的过程就称为摄像机标定。
张正友平面标定方法
算法原理
算法描述
1、打印一张模板并贴在一个平面上;
2、从不同角度拍摄若干张模板图像;
3、检测出图像中的特征点;
4、求出摄像机的内参数和外参数;
5、求出畸变系数;
6、优化求精。
The Understanding & Solution from MATLAB
What is Camera Calibration: A wonderful explanation
OpenCV Implementation
Camera Calibration and 3D Reconstruction
个人理解
在现实应用中，经常需要利用摄像头拍摄的二维图像还原真实物理世界中的3D实物。为此，我们假定图像和实物之间存在线性对应关系: [像]=M[物]。求解这个对应关系M的过程，就是摄像头标定。M 就是摄像头参数。结合实际情况，摄像头参数分为两部分：
- 内参数：镜头固有参数。such as, its focal length, skew, distortion, and image center. 镜头中心位置 (Cx，Cy) 和 焦距大小 fx，fy。均用像素长度表达。
- 外参数：摄像机位置参数，物体所处的现实世界坐标系对摄像机坐标系的刚性变换，可表达为旋转R 和 平移的结合 [R t]。
在现实成像过程中，先对现实世界中的3D物体依外参数做刚性变换，转换到摄像机3D坐标系。再依内参数做投影变换，得到二维图像。
用矩阵表达就是：
这是理想条件下的成像，并未考虑透镜畸变(lens distortion)。考虑实际成像中必然会发生的透镜畸变，径向畸变和切向畸变，还要再对图像坐标进行修正。
算法实现
OpenCV 接口
- 用户接口
double calibrateCamera(InputArrayOfArrays objectPoints, InputArrayOfArrays imagePoints, Size imageSize, InputOutputArray cameraMatrix, InputOutputArray distCoeffs, OutputArrayOfArrays rvecs, OutputArrayOfArrays tvecs, int flags=0, TermCriteria criteria=TermCriteria( TermCriteria::COUNT+TermCriteria::EPS, 30, DBL_EPSILON) )Explanation of Parameters:
- objectPoints ：The points of Objects, they are in a pattern coordinate system, so they are 3D (e.g. std::vector
- imagePoints : The points of the projected images of calibration pattern points, (e.g. std::vector
- imageSize : Size of the image used only to initialize the intrinsic camera matrix (?)
- cameraMatrix : Output 3x3 floating-point camera matrix
- distCoeffs : Output vector of distortion coefficients (k1,k2,p1,p2[,k3[,k4,k5,k6]]) of 4, 5, or 8 elements.
- rvecs : Output vector of rotation vectors estimated for each pattern view (e.g. std::vector
- tvecs : Output vector of translation vectors estimated for each pattern view.
-
flags : Different flags that may be zero or a combination of the following values :
- CV_CALIAB_USE_INTRINSIC_GUESS cameraMatrix contains valid initial values of fx, fy, cx, cy that are optimized futher. Otherwise, (cx, cy) is initially set to the image center (the param imageSize if used), and focal distance are computed in a least-squares fashion. Note, that if intrinsic parameters are known, there is no need to use this function just to estimate extrinsic parameters. Use solvePnP() instead. 猜测内参数，需要在cameraMatrix给出初始值然后迭代优化。如果不开这个flag，表明内参数是已知的，那就没必要用这个函数来单纯的计算外参数了。所以，一般这个flag都是要开的。
- CV_CALIB_FIX_PRINCIPAL_POINT The principal point is not changed during the global optimization. It stays at the center or at a different location specified when CV_CALIB_USE_INTRINSIC_GUESS is set too. 开了这个flag，中心点就固定下来了，在之后的优化中不再改变。
- CV_CALIB_FIX_ASPECT_RATIO The functions considers only fy as a free parameter. The ratio fx/fy stays the same as in the input cameraMatrix. When CV_CALIB_USE_INTRINSIC_GUESS is not set, the actual input values of fx and fy are ignored, only their ratio is computed and used futher. 打开这个flag，fx/fy 比值固定，只有fy是一个自由参数去优化，fx随之而变。而如果 _GUESS flag 没开，当前的 fx， fy 就被忽略掉了，只计算一下比值留待后用。
- CV_CALIB_ZERO_TANGENT_DIST Tangential distortion coefficient(p1, p2) are set to zeros and stay zero. 开启这个flag， 切向畸变系数（p1， p2）设为0 而且保持为0.
- CV_CALIB_FIX_K1,...,CV_CALIB_FIX_K6 The corresponding radial distortion coefficient is not changed during the optimization. If _GUESS is set, the coefficient from the supplied distCoeffs matrix is used. Otherwise, it is set to 0. 开启flag，固定K1到K6指定的径向畸变系数。如果_GUESS 开启了，distCoeffs 里的数值就用上了。若_GUESS 没有开启，那么全部设为0.
- CV_CALIB_RATIONAL_MODEL Coefficients k4,k5,and k6 are enabled. This extra flag should be explicitly specified to make the calibration function use the rational model and return 8 coefficients. If the flag is not set, the function computes and returns only 5 distortion coefficients. 打开这个flag，计算并返回8个畸变系数。默认关闭，仅返回5个畸变系数。
- criteria : Termination criteria for the iterative optimization algorithm. 迭代优化算法的终止条件。
- term_crit : same as criteria.
The function estimates the intrinsic camera parameters and extrinsic parameters for each of the views. The algorithm is based on [Zhang2000] and [BouguetMCT]. The coordinates of 3D object points and their corresponding 2D projections in each view must be specified. That may be achieved by using an object with a known geometry and easily detectable feature points. Such an object is called a calibration rig or calibration pattern, and OpenCV has built-in support for a chessboard as a calibration rig. Currently, initialization of intrinsic parameters(when CV_CALIB_USE_INTRINSIC_GUESS is not set) is only implemented for planar calibration patterns(when Z-coordinates of the object points must be all zeros.) 3D calibration rigs can also be used as long as initial cameraMatrix is provided. 在 OpenCV 的内建算法中，内参数的自动初始化只针对二维平面标定板。只有给定了初始化的 cameraMatrix 才可以使用3D的标定板。
The algorithm performs the following steps: 算法总共分三步
- Compute the initial intrinsic parameters(the option only available for planar calibration patterns) or read them from the input parameters. The distortion coefficients are all set to zeros initially unless some of CV_CALIB_FIX_K? are specified. 先找内参数，直接计算初始化内参数（仅对平面标定板有效）或从输入参数里得到。畸变系数全部置为0，除非指定了FIX值。
- Estimate the initial camera pose as if the intrinsic parameters have been already known.This is done using solvePnP(). 假设得到内参数，开始估计初始相机姿态 外参数。
- Run the global Levenberg_Marquardt optimization algorithm to minimize the reprojection error, that is, the total sum of squared distances between the observed feature points imagePoints and the projected(using the current estimates for camera parameters and the poses) object points objectPoints. 首先定义 再投影误差 ：
实际投影得到的特征点与使用相机矩阵计算得到的理论特征点的距离的平方和。误差越小，说明相机矩阵越精确。随后就可以使用全局 LM 优化算法来减小再投影误差，一直迭代到退出条件为止。
The function returns the final re-projection error.
算法返回最终的再投影误差。
拓展 ： LM 算法
中文通俗解释
Wikipedia详细介绍
Caltech 开源 工具箱
Camera Calibration Toolbox for Matlab
相关论文
- 综述 ：摄像机标定的基本原理、实现及性能分析
- 课件 ： 中科院自动化所模式识别实验室
- 祝海江博士论文 ： 三维重建和摄像机标定技术研究