opencv直线检测直线提取算法与总结

有些情况,  我们会需要提取直线的详细参数, 下面介绍如何提取直线

霍夫变换(Hough Transformation)

其中很大一部分都在应用霍夫变换 及其 各种版本来提取直线

关于霍夫变换的理解,  这里有个检测圆的供参考 http://blog.csdn.net/traumland/article/details/51077293

     这里个人总结下:  霍夫变换的基本思想,   将原本的平面曲线参数方程转化成 , 以曲线上点作为已知,

把曲线方程的参数当作未知项,  在参数空间作出图形.(原本已知所有参数时可以求出曲线上的这些点)

     当交于某公共点上图形最多时(peak),  这个公共点(peak)对应的参数便可作为整个图形的参数.

霍夫变换最开始提出来时,  直线方程用的是斜截式

       因为考虑到直线两种极端情况,  水平和竖直

所以按斜率是否大于1,  使用了不同的方程

       k < 1 时       y = kx +b

       k>=1 时       y = 1/k x + 1/b  

就避免了因为 y = mx + c   中  m,c 过大而不好讨论(m,c代表直线方程的参数)

在Duda,Hart 71 中,  提出了用

        ρ = x cos θ + y sin θ
来表示直线,  也就是现在常见的表示形式

 

https://www.researchgate.net/profile/Frank_OGorman/publication/220331095_Finding_Picture_Edges_Through_Collinearity_of_Feature_Points/links/0912f5050615b5cf37000000.pdf

提供了减少计算量的方法

最好看看hough变换的源码,  因为opencv只提供了直线和圆的函数,  其他的需要自己去写

opencv标准霍夫变换源码及注释:http://blog.csdn.net/traumland/article/details/51319644

下面示例下opencv霍夫变换函数的使用

注: sobel算子使用的方法不太正确, 这里仅作示例用. 两图求得的edge不一样

详细请看http://blog.csdn.net/traumland/article/details/51074705

    cv::Mat G,Gx,Gy,G_otsu;

    cv::Sobel(srcEmpty,Gx,CV_8U,1,0,3);

    cv::Sobel(srcEmpty,Gy,CV_8U,0,1,3);

    G = Gx + Gy;

    cv::threshold(G,G_otsu,0,255,cv::THRESH_OTSU);

    if(debugflag)

        cv::imshow("Edge",G_otsu);

    cv::Mat dst(srcEmpty.size(),CV_8UC3);

    std::vector<cv::Vec2f> lines;

    cv::HoughLines(G_otsu, lines,1, CV_PI/180, 30,0,0);

    for( size_t i = 0; i < lines.size(); i++ ){

        float rho = lines[i][0];

        float theta = lines[i][1];

        double a = cos(theta), b = sin(theta);

        double x0 = a*rho, y0 = b*rho;

        cv::Point pt1(cvRound(x0 + 1000*(-b)), cvRound(y0 + 1000*(a)));

        cv::Point pt2(cvRound(x0 - 1000*(-b)), cvRound(y0 - 1000*(a)));

        cv::line( dst, pt1, pt2, cv::Scalar(0,0,255), 3, 8 );

    }

可以看到,  找出的直线并没有边界(直线的两端点)

至于为什么会出现右图多条直线的那种情况

因为用于投票的阈值 

太高只能提取长线, 短线被忽略

太低又会出现多条,  这种情况,  不好确定阈值

这个时候如果单单是调用houghLines的话,  可能不会有太好的结果

再来看houghLinesP的使用

    std::vector<cv::Vec4i> lines;

    cv::HoughLinesP( srcEmpty, lines, 1, CV_PI/180, 20, 20, 20);

    std::cout << lines.size() << std::endl;

    for( size_t i = 0; i < lines.size(); i++ ){

        cv::line( dst, cv::Point(lines[i][0], lines[i][1]),

        cv::Point(lines[i][2], lines[i][3]), cv::Scalar(0,0,255), 3, 8 );

    }

看起来好多了.但是我们看看本意提取的四条线 

cout << line.size() 的结果是8条

而且选定的参数也仅仅是针对这个边缘,  稍微变化一点参数又要变化

可以的话,  对两端点之内相交的线段,  定义夹角范围多少以内是同一条边,  

在这些边选取最长的线段作为整条边

对于本情况,  需要的并不是提取所有直线,  而是根据这个edge的特点,  找到四条满足大部分点的直线

因此,  opencv所提供的两个霍夫变换并不适合这种情况

(重点在对边缘直线长度未知的情况下,  没有很好的参数适用大部分情况)

需要对函数根据需要进行一些修改

例如

原图:   

结果:   

注 :  改动部分为   由返回符合threshold的多条直线,   变为    返回获得票数最高的直线

RANSAC

在opencv中,  涉及到ransac的函数只有跟立体标定有关

所以想直接调函数的还是不要想opencv了吧

本人所写的ransac 函数,  仅供参考 ,  传送门:  我的github

关于RANSAC    http://blog.csdn.net/traumland/article/details/51103095    其中有的连接给了c++代码

http://www.cnblogs.com/xrwang/archive/2011/03/09/ransac-1.html

http://grunt1223.iteye.com/blog/961063

关于RANSAC c++代码    很抱歉这里没有进行实验,  请自行甄别

https://github.com/srinath1905/GRANSAC/blob/master/include/GRANSAC.hpp

http://reference.mrpt.org/stable/classmrpt_1_1math_1_1_r_a_n_s_a_c___template.html 及

https://github.com/MRPT/mrpt/blob/9f72d70abe8db107f7331f3723cf2553d7c307d0/libs/base/include/mrpt/math/ransac.h

https://github.com/MRPT/mrpt/blob/9f72d70abe8db107f7331f3723cf2553d7c307d0/libs/base/src/math/ransac.cpp

  及

http://www.mrpt.org/tutorials/programming/maths-and-geometry/ransac-c-examples/

    要使用RANSAC拟合直线,  我们首先需要三个参数

1.The Number of Samples Required

    假设我们需要画n个点(对于拟合直线来说, 最少需要两个点; 对于圆则最少需要三个.),  

其中w是其中好点(good points, 有效点?)所占的比例,  

则需要画的次数(the number of draws k)的期望值是     

                                                          E[k] = w ^ (-n)

                        

             假设当只有bad samples 出现的概率为 z

                                         则

                                                 

这三种方法都可以求出k

其中每个sample里面包括the abstraction of interest所需要的最少点. 

2.Telling Whether a Point Is Close                  距离直线的最大距离的点

3.The Number of Points That Must Agree       (在前面所确定的距离内) 需要最少个数的点

    对于本情况,  对四条线段使用 RANSAC,  具体的想法是:

1.  对一组点集用投票法找出最高票数的直线,  找出peak后并删掉peak对应的数据

2.  找出下一条直线,   同样找出并删除对应的数据

3.  就这样循环,  直到找出4条直线,  终止

 

  

来源: computer vision : a modern approatch

拟合法

   相对于RANSAC对噪声不敏感,  因为拟合法是对当前所有的点进行的, 内在并没有一个检测和去除

坏点(gross point)的机制[Matin and Robert, 1980],  除了提供尽可能大的相关数据外

使用拟合法之前一定要把尽量无关的点去除

    拟合法在前面常提的课件(www.cse.psu.edu) , lecture14 有讲,  

以最小二乘法为例

比如直线点斜式方程

图片来源:  http://www.cse.psu.edu/~rtc12/CSE486/

对于直线一般方程,  一样的原理,  

ax+ by +c = 0 = d0

di = axi + byi +c

E(a,b,c) = ⅀ (di - d0 ) ^2

求导 ,  令导数为0,   就得到了 可以求解(a,b,c) 方程组

注意当所有点都在同一条直线上时,  方程组的参数矩阵的一个特征值为0,  此时齐次方程有无数多的解.

详细请自寻

opencv提供了直线拟合法函数fitLine(),  我用的3.1.0 调了很久都没有调通,  高度怀疑此处存在bug,具体点击这里

fitLine源码在下面,后面列出了参数参考

     用上面的存放edge图像的Mat时会出现

OpenCV Error: Assertion failed (npoints2 >= 0 || npoints3 >= 0) in fitLine,

 file /home/tau/opencv/opencv-3.1.0/modules/imgproc/src/linefit.cpp, line 603
terminate called after throwing an instance of 'cv::Exception'
  what():  /home/tau/opencv/opencv-3.1.0/modules/imgproc/src/linefit.cpp:603: 

error: (-215) npoints2 >= 0 || npoints3 >= 0 in function fitLine

    哪里出了问题呢?  我们来试试看看fitLine源码吧,  源码在下面

opencv 3.0 对这一部分用c++ 重写了一遍,  当然这里调用了c函数的FitLine2D, 这里不去深究

在这个函数发现错误提示来自这里

CV_Assert( npoints2 >= 0 || npoints3 >= 0

而int npoints2 = points.checkVector(2, -1, false);

函数的功能是: 当Mat的channels,depth,和连续性  满足checkVector的参数内容时,

返回(int)(total()*channels()/_elemChannels), 否则返回-1

看出问题没有? 

因为存放edge的Mat是连续的, 而checkVector(2,-1,false)要求是不连续的, 所以被返回了-1

且npoints2与npoints3 都被返回了-1

怎么解决呢?

首先肯定要想怎么把像素值不为0的点提取出来, 存成vector或Mat,

用findContours之类的. 或是干脆自己将edge分成四段线段,这样也避免了四条线相互影响

    虽然对于上面的edge四条线段确实可以用四条直线拟合,  

但是我还没有想好怎么方便把四条线段上的点分别存放, 以便直接使用fitLine

既然想到了findContours,  那就是把存放edge的Mat数据的非零点存成点集.

    在这里,  我先用了角点检测将完整的edge打断成四条线段,  但是不知道什么原因fitline函数总是无法正常运行

有关错误信息在这里http://blog.csdn.net/traumland/article/details/51292194

所以等到解决后再更新本文吧

关于points.checkVector()    

int cv::Mat::checkVector	(	int	elemChannels,
		int	depth = -1,
		bool	requireContinuous = true
	)		const

returns N if the matrix is 1-channel (N x ptdim) or ptdim-channel (1 x N) or (N x 1); negative number otherwise

虽然未对depth与requireContinuous进行说明,但从命名上看应该能看明白吧

int Mat::checkVector(int _elemChannels, int _depth, bool _requireContinuous) const
{
    return (depth() == _depth || _depth <= 0) &&  (isContinuous() || !_requireContinuous) &&
               ((dims == 2 && (((rows == 1 || cols == 1) && channels() == _elemChannels) ||(cols == _elemChannels && channels() == 1))) ||
        (dims == 3 && channels() == 1 && size.p[2] == _elemChannels && (size.p[0] == 1 || size.p[1] == 1) &&
         (isContinuous() || step.p[1] == step.p[2]*size.p[2])))
    ? (int)(total()*channels()/_elemChannels) : -1;
}

void cv::fitLine( InputArray _points, OutputArray _line, int distType, double param, double reps, double aeps )
{
    Mat points = _points.getMat();
    float linebuf[6]={0.f};
    int npoints2 = points.checkVector(2, -1, false);
    int npoints3 = points.checkVector(3, -1, false);
    CV_Assert( npoints2 >= 0 || npoints3 >= 0 );
    if( points.depth() != CV_32F || !points.isContinuous() )
    {
        Mat temp;
        points.convertTo(temp, CV_32F);
        points = temp;
    }
    if( npoints2 >= 0 )
        fitLine2D( points.ptr(), npoints2, distType, (float)param, (float)reps, (float)aeps, linebuf);
    else
        fitLine3D( points.ptr(), npoints3, distType, (float)param, (float)reps, (float)aeps, linebuf);
    Mat(npoints2 >= 0 ? 4 : 6, 1, CV_32F, linebuf).copyTo(_line);
}

其中distType可选参数

Parameters

points	Input vector of 2D or 3D points, stored in std::vector<> or Mat.
line	Output line parameters. In case of 2D fitting, it should be a vector of 4 elements (like Vec4f) - (vx, vy, x0, y0), where (vx, vy) is a normalized vector collinear to the line and (x0, y0) is a point on the line. In case of 3D fitting, it should be a vector of 6 elements (like Vec6f) - (vx, vy, vz, x0, y0, z0), where (vx, vy, vz) is a normalized vector collinear to the line and (x0, y0, z0) is a point on the line.
distType	Distance used by the M-estimator, see cv::DistanceTypes
param	Numerical parameter ( C ) for some types of distances. If it is 0, an optimal value is chosen.
reps	Sufficient accuracy for the radius (distance between the coordinate origin and the line).
aeps	Sufficient accuracy for the angle. 0.01 would be a good default value for reps and aeps.

void HoughLines( InputArray image, OutputArray lines, double rho, double theta, int threshold, double srn=0, 

                                                                                         double stn=0 )

image – 8-bit, single-channel binary source image. The image may be modified by the function.

lines –   Output vector of lines. Each line is represented by a two-element vector (ρ, θ) . 

              ρ is the distance from the coordinate origin (0, 0) (top-left corner of the image). θ is the line

              rotation angle in radians ( 0 ∼ vertical line, π/2 ∼ horizontal line ).

rho –      Distance resolution of the accumulator in pixels.

theta –    Angle resolution of the accumulator in radians.

threshold – Accumulator threshold parameter. Only those lines are returned that get enough votes ( > threshold ).

srn – For the multi-scale Hough transform, it is a divisor for the distance resolution rho .

         The coarse accumulator distance resolution is rho and the accurate accumulator resolution
         is rho/srn . If both srn=0 and stn=0 , the classical Hough transform is used. Otherwise,
         both these parameters should be positive.

stn – For the multi-scale Hough transform, it is a divisor for the distance resolution theta .

method – One of the following Hough transform variants:

– CV_HOUGH_STANDARD classical or standard Hough transform. Every line is repre-
sented by two floating-point numbers (ρ, θ) , where ρ is a distance between (0,0) point
and the line, and θ is the angle between x-axis and the normal to the line. Thus, the matrix
must be (the created sequence will be) of CV_32FC2 type

– CV_HOUGH_PROBABILISTIC probabilistic Hough transform (more efficient in case
if the picture contains a few long linear segments). It returns line segments rather than
the whole line. Each segment is represented by starting and ending points, and the matrix
must be (the created sequence will be) of the CV_32SC4 type.

– CV_HOUGH_MULTI_SCALE multi-scale variant of the classical Hough transform.
The lines are encoded the same way as CV_HOUGH_STANDARD .

void HoughLinesP( InputArray image, OutputArray lines, double rho, double theta, int threshold, 

                                                                 double minLineLength=0, double maxLineGap=0 )

image – 8-bit, single-channel binary source image. The image may be modified by the function.

lines – Output vector of lines. Each line is represented by a 4-element vector (x 1 , y 1 , x 2 , y 2 ),

                    where (x 1 , y 1 ) and (x 2 , y 2 ) are the ending points of each detected line segment.

rho – Distance resolution of the accumulator in pixels.

theta – Angle resolution of the accumulator in radians.

threshold – Accumulator threshold parameter. Only those lines are returned that get enough votes ( > threshold ).

minLineLength – Minimum line length. Line segments shorter than that are rejected.

maxLineGap – Maximum allowed gap between points on the same line to link them.

今天接触到LSD算法  http://blog.csdn.net/carson2005/article/details/9326847   改天增加这部分内容

http://docs.opencv.org/master/d1/dbd/classcv_1_1line__descriptor_1_1LSDDetector.html#gsc.tab=0