Halcon复习专题-镜头/匹配/标定/边缘/拟合/缺陷检测/
前言
该系列笔记主要用来记录Halcon学习过程中所需要的一些算子,和一些应用场景的算法流程方法
笔记的格式为:基本描述➕基本程序设计流程
Halcon的处理场景主要是下图所述
第一章 相机、镜头与光源
一般来说,项目的流程如下图所示,但是要注意Halcon完成的事视觉核心算法,通过Halcon程序可以导出成C++程序,然后利用VS、Qt等设计软件界面、通信等功能,从而完成项目.显然这里的意思是利用Halcon完成图形处理相关程序模块,然后导出到VS中.
为了更好的丰富理论知识,推荐学习相关的光学理论,如《工程光学》
一般来说,硬件的选择主要包括:镜头、光源打光、相机、采集卡、计算机、显示器和控制器.
相机中的CCD和CMOS主要是用来进行光电转换的器件.
光学系统大概分成:望远系统、显微系统、放大系统和摄影系统四大方向,而我们需要的就是摄影系统.
第一节 镜头选型
1.1镜头的基本概念
| 概念/记法 |
解释 |
备注 |
| 焦距/f |
镜头主点到焦点的距离 |
一般是3.5-8.0mm |
| 焦点/️ |
物方焦点、像方焦点 |
|
| 像方焦点/️ |
CMOS成像处的焦点 |
|
| 光圈/️ |
控制通光量的大小,即通光阀门 |
|
| 光圈直径/D |
光圈的直径距离 |
|
| 光圈数/F |
F=f/D,一般镜头上会写F1.4,有时也会写D/f=1:1.4 |
f相同情况下,D越大,光圈数F越低,镜头分辨率越高 |
| 景深/️ |
物体能够呈现清晰图像的前提下,物体可相对镜头前后移动的范围 |
⚠️相对于镜头而言是前后移动,即增加、减少物距 ⚠️光圈小,景深大/ 焦距小,景深大/物距大,景深大
|
| 镜头的MTF曲线 |
评价镜头成像好坏的参数 |
|
| 镜头接口/C,CS,F |
|
|
| 镜头分辨率 |
1mm内能分辨开的黑白相间线条的对数, |
单位:对数/毫米 |
|
|
|
|
1.2镜头选型
镜头选型要求对镜头的类型有比较全面的了解,不仅要知道成像原理,最重要的是掌握应用场景,这就要求掌握其优缺点.
着重介绍远心镜头,其包括物方远心镜头、像方远心镜头和双远心镜头.远心镜头的选择指标是放大倍率,一般来说0.5倍已经是很大的倍率.
物方远心镜头,目的是消除物体到镜头距离不同而产生的成像放大倍率不同.
物方远心镜头的原理是:把孔径光阑放在光学系统的像方焦平面上(下图的CCD不在焦平面位置,而在后方),两个B’点成为入瞳点,也可以看成物体的上下边缘.物体无论物距多少,上下边缘都会通过孔径光阑,通过光学系统得到一系列平行光,正是该平行光,使得在CCD/CMOS上得到固定的成像高度,即B1B2,因此消除了镜头距离导致的成像放大倍率变化问题.
像方远心镜头则是消除像方调焦不准带来的误差.
双远心镜头则是即避免物方距离改变导致的像方成像倍率变化,也避免了像方调焦不准带来的误差.
一般来说,镜头选型常常使用下图公示.
根据上述镜头选型的理论知识,接下来计算两道常规题目.
(1)已知客户观察范围是30mm*30mm,工作距离是100mm,CCD尺寸是1/3’,那么需要多少焦距的镜头才能满足需求?
解:
1⃣️假设客户需求是垂直成像, 那么查表得:CCD尺寸1/3 型对应的垂直方向的尺寸是3.6mm
2⃣️由焦距f=(工作距离*CCD尺寸)/视野范围,有焦距f=(100mm*3.6mm)/30mm=12mm
综上所述,焦距f为12mm
(2)已知客户要求的系统分辨率是0.06mm,像元大小是4.7um,工作距离大于100mm,光源采用白色LED灯,试问需要多少焦距的镜头?
解:
1⃣️若客户无需变焦,则选择定焦镜头,若客户需测量功能则选择畸变小的镜头或者远心镜头
2⃣️已知系统分辨率是0.06mm,也就是系统能分辨的物体最小距离是0.06mm,即物距最小为0.06mm=0.06*1000um,
3⃣️已知像元为4.7um即像距最小距离为4.7um
4⃣️由成像放大倍率M = 像距/物距or像高/物高,则有M = 4.7um/(0.06mm*1000)
5⃣️f = LM/(M+1), 其中L为工作距离
第二节 相机选型
| 概念/记法 |
解释 |
备注 |
| 曝光 |
光通过镜头达到CCD或者CMOS上,需要光电转换,进行电信号积累,这个时候需要曝光,曝光就是光电信号转换的程度,时间越久点信号积累的越多 |
|
| 相机分辨率 |
相机分辨率不是镜头分辨率,相机分辨率是由像素点排列组成的,一个像素点叫像元,一个像元一般是3.75um*3.75um的方格 |
行列有几个像素就是有多少分辨率,比如1920*1080个像素点 |
| 增益 |
用来协调曝光的参数,曝光时间越长,增益越小,曝光时间越短,增益就会适当增强,rgb三个通道都有增益 |
|
| 帧率/FPS |
帧率有奇场偶场之分,奇场+偶场=1帧 |
奇场就是奇数行,偶场就是偶数行 |
| 面阵相机 |
一场一场曝光,是实现像素帧的拍摄 |
全局曝光:拍出来一起曝光 卷帘曝光:拍一行曝光一行 |
| 线阵相机 |
一行一行曝光,结果是呈线状的 |
虽然也是二维图像,但是宽度只有几个像素,长度却十分长 |
| 尺寸 |
CCD和CMOS尺寸表 |
|
| 相机接口/USB/网口 |
|
|
| 白平衡 |
平衡rgb三通道亮度值 |
|
第三节 光源
光源的作用在于增加对比度,让成像的图片更加均匀.光源的选择直接决定图像分析的效果.就像数据集是否进行仔细清洗,直接决定了CNN网络预测的能力
光源在原则上应选择均匀、对比度高的,这一ROI区域会更加明显
过滤光线的硬件一般有激光片和偏振片,其中滤光片可以过滤不同波段的光,偏振片过滤不同方向的光,其中记法 | 表示上下震动, · 表示垂直屏幕震动.偏振片可以达到防止反光的作用,因而偏振片常用于直接组织反射光
偏振光简单的说就是存在多个反射方向的光,每个反射方向末端连接起来,如果是个圆⭕️则表示圆偏振光,椭圆则表示椭圆偏振光,若只是条线则表示线偏振光
3.1光源的分类与照射
光源往往分为条光、背光、球面光、点光、同轴光、组合光和环形光.
在测量的场景下,常选择远心镜头+背光源,尽可能选择短波光,因为衍射效应较差,长波光衍射效应较强(衍射效应会使得光在进入空洞后变宽,导致测量有误差).
互补与高反:若两种光是互补的,那么两光源照射重叠会导致目标变暗,相反,相同颜色光重叠会产生高反,高反则意味着更亮,图片中白色的块实际是红色的,利用高反效应突出了目标.
下图中左边的连线表示互补.
光的照射方法主要有:同轴光照射、角度照射、垂直照射、圆顶光照射、背光照射、条光照射、组合光照射、环形光照射、条形状照射和添加偏振片防反光照射等.具体如下图所示.
角度照射主要用于液晶矫正、塑胶容器检查、工件螺孔定位、标签检查、管脚检查、集成电路印字检查等场景.
垂直照射主要用:基地和线路板定位、晶片部件检查等
圆顶光照射主要用在:检测曲面的金属表面文字等
第二章 模版匹配
本章的要点主要在代码块中,代码块仅用来展示用法,不可复制,因为用了//来说明用法,而Halcon语法中的注释符号是*并不是//,直接复制运行会error.
模版匹配的学习方法就是在Halcon中按下ctrl+e,寻找例子,掌握各个应用场景的处理流程,然后实战即可.
模版匹配比较浅显,总的来说就是先获得一个模版图片,然后创建匹配模型,根据模型来对输入进行匹配对比,从而获得目标对象.
模版匹配一般有灰度匹配、轮廓匹配、相关性匹配、组件匹配、特征描述子匹配等方法,按照输入输出来说,则分为1对多和多对多匹配方式,即模版:对象=1:n和n:n两种.
在模版匹配中,常常使用一种显示结果的方法:仿射变换.一般来说仿射变换都是为了展示结果的,每次完成匹配后,都需要把模版位置转移到目标位置上去,一般操作包括:平移、旋转和缩放.
在Halcon中,所有的disp_开头的算子都是表示显示的.
第一节 相关性匹配
相关性匹配的输出结果是0-1的数值,该数值越接近1越表示相似程度越高.
使用场景常在:光照不均匀、目标有明暗变化和物体有纹理变化情况下
1.1相关性匹配标准流程
read_image (Image, 'cap_exposure/cap_exposure_03')
dev_close_window ()
dev_open_window_fit_image (Image, 0, 0, -1, -1, WindowHandle)
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
dev_update_off ()
gen_circle (Circle, 246, 336, 150)
area_center (Circle, Area, RowRef, ColumnRef)
reduce_domain (Image, Circle, ImageReduced)
//创建相关性匹配的模型,输入的单通道图片是你手里的模版图片,上边是生成了一个圆形图片
//create_ncc_model(输入单通道图片,金字塔层数,模板的最小旋转角度,旋转最大角度,匹配方法,输出模板模型句柄)
create_ncc_model (ImageReduced, 'auto', 0, 0, 'auto', 'use_polarity', ModelID)
// 设置绘制区域和颜色,每次设置以最后一次为准
dev_set_draw ('margin')
dev_display (Image)
dev_set_color ('yellow')
dev_display (Circle)
disp_message (WindowHandle, 'Trained NCC model', 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
Rows := []
Cols := []
for J := 1 to 10 by 1
read_image (Image, 'cap_exposure/cap_exposure_' + J$'02')
// 使用ncc模型进行匹配操作
// find_ncc_model(输入图片,模型句柄,模版最小旋转角度,模版旋转角度范围,匹配得分最小值,匹配得到的个数(0全要),最大的重叠度(0-1),是否采用亚像素精度,使用模型中金字塔第几层的模版,匹配结果的行列坐标和水平角度还有得分)
find_ncc_model (Image, ModelID, 0, 0, 0.5, 1, 0.5, 'true', 0, Row, Column, Angle, Score)
// 仿射变换,是把原来生成的圆圈的中心点RowRef和ColumnRef移动到Row和Column处,得到一个放射矩阵HomMat2D,把这个放射矩阵放到区域图片上,就能随着目标的变化而得到新的区域
vector_angle_to_rigid(RowRef,ColumnRef,0,Row,Column,0,HomMat2D)
// 应用到区域,输入圆圈,新的区域,放射矩阵,差值算法
affine_trans_region(Circle,RegionAffineTrans,HomMat2D,'nearest_neighbor')
// 往数组中追加数值的格式<数组名> := [数组名,value]
// Rows := [Row]是用Row来覆盖数组原有值
Rows := [Rows,Row]
Cols := [Cols,Column]
dev_display (Image)
dev_display_ncc_matching_results (ModelID, 'green', Row, Column, Angle, 0)
disp_message (WindowHandle, 'Found NCC model', 'window', 12, 12, 'black', 'true')
if (J < 10)
disp_continue_message (WindowHandle, 'black', 'true')
endif
stop ()
endfor
// 计算标准差,就是看看离散程度,标准差=sqrt(方差)
StdDevRows := deviation(Rows)
StdDevCols := deviation(Cols)
//清除模型
clear_ncc_model (ModelID)
1.2 带显示结果的相关性匹配
dev_close_window ()
read_image (ImageRef, 'pcb_focus/pcb_focus_telecentric_061')
get_image_size (ImageRef, Width, Height)
dev_open_window_fit_image (ImageRef, 0, 0, -1, -1, WindowHandle)
set_display_font (WindowHandle, 14, 'mono', 'true', 'false')
dev_update_off ()
dev_set_draw ('margin')
dev_set_color ('green')
dev_set_line_width (1)
*
* Create ncc model
gen_rectangle1 (ModelRegion, 81.5, 148.5, 419.5, 633.5)
reduce_domain (ImageRef, ModelRegion, TemplateImage)
dev_clear_window ()
dev_display (TemplateImage)
disp_message (WindowHandle, 'Creating the ncc model may take a few seconds ... ', 'window', 12, 12, 'black', 'true')
create_ncc_model (TemplateImage, 'auto', -rad(5), rad(10), 'auto', 'use_polarity', ModelID)
area_center (ModelRegion, Area, ModelRow, ModelColumn)
Rows := []
Columns := []
for Index := 1 to 121 by 1
read_image (Image, 'pcb_focus/pcb_focus_telecentric_' + Index$'03' + '.png')
find_ncc_model (Image, ModelID, -rad(5), rad(10), 0.5, 1, 0.5, 'true', 0, Row, Column, Angle, Score)
Rows := [Rows,Row]
Columns := [Columns,Column]
dev_display (Image)
dev_display_ncc_matching_results (ModelID, 'green', Row, Column, Angle, 0)
disp_message (WindowHandle, 'Finding ncc model in image: ' + Index$'03', 'window', 12, 12, 'white', 'false')
flush_buffer (WindowHandle)
endfor
*
set_window_param (WindowHandle, 'flush', 'true')
*
* Display results
dev_display (Image)
wait_seconds (1)
gen_contour_polygon_xld (Contour, Rows, Columns)
// 拟合直线,暂时不关注
fit_line_contour_xld (Contour, 'tukey', -1, 0, 5, 2, RowBegin, ColBegin, RowEnd, ColEnd, Nr, Nc, Dist)
gen_contour_polygon_xld (Regression, [RowBegin,RowEnd], [ColBegin,ColEnd])
* Resize window so that pixels appear square
Ratio := (394 - 389 + 1) / real(252 - 250 + 1)
dev_resize_window_fit_size (0, 0, Height * Ratio, Height, 640, 480)
dev_set_part (250, 389, 252, 394)
dev_display (ImageRef)
dev_set_color ('green')
dev_display (Contour)
dev_set_color ('yellow')
dev_display (Regression)
disp_message (WindowHandle, ['Deviation of ncc matches in (x,y)-direction','while defocusing lens'], 'image', 249.75, 389, 'white', 'false')
disp_message (WindowHandle, ' Deviation (green), Tukey\'s robust regression (yellow) in pixel', 'image', 251.75, 389.25, 'white', 'false')
*
* Close all handles
clear_ncc_model (ModelID)
第二节 轮廓形状匹配
轮廓匹配就是轮廓形状匹配也叫形状匹配,在日常中,轮廓形状匹配是使用最多的匹配方法,简而言之就是使用轮廓来寻找目标,该匹配方法对光照具有很高的要求.
轮廓形状匹配的流程一般如下:
2.1轮廓匹配标准流程
下述程序是基于blob分析的轮廓匹配方法
// 更新状态
dev_update_pc ('off')
dev_update_window ('off')
dev_update_var ('off')
//读取图片
read_image (Image, 'green-dot')
get_image_size (Image, Width, Height)
dev_close_window ()
dev_open_window (0, 0, Width, Height, 'black', WindowHandle)
// 设置绘制区域和显示图片
dev_set_color ('red')
dev_display (Image)
// 二值化
threshold (Image, Region, 0, 128)
// 区域连接连通,独立分割
connection (Region, ConnectedRegions)
// 根据面积选择目标区域
select_shape (ConnectedRegions, SelectedRegions, 'area', 'and', 10000, 20000)
// 填充区域
fill_up (SelectedRegions, RegionFillUp)
// 膨胀操作
dilation_circle (RegionFillUp, RegionDilation, 5.5)
// 抠图
reduce_domain (Image, RegionDilation, ImageReduced)
// 创建可比例缩放的形状模版,注意ncc模版创建时没有scaled即等比缩放性,显然这个形状匹配模版具有缩放性
/*
create_scaled_shape_model(Template : : NumLevels, AngleStart, AngleExtent, AngleStep, ScaleMin, ScaleMax, ScaleStep, Optimization, Metric, Contrast, MinContrast : ModelID)
*/
/*
参数(Template模版图片,NumbLevels模板金字塔层数,AngleStart模板最小旋转角度,AngleExtent模版旋转范围,AngleStep旋转步长,ScaleMin最小缩放比例,ScaleMax最大缩放比例,ScaleStep缩放步长,Optimization优化方法,Metric匹配标准,Contrast对比度,MinContrast最小对比度,ModelID生成的形状模版模型句柄)
*/
// 对比度小一点可能比较容易找到,但是容易带来干扰
// 这个算子会直接把模型坐标放到0,0角度调整到0
create_scaled_shape_model (ImageReduced, 5, rad(-45), rad(90), 'auto', 0.8, 1.0, 'auto', 'none', 'ignore_global_polarity', 40, 10, ModelID)
// 获得形状模版金字塔中的第1层的轮廓模板,上述的形状模版是有多个形状构成,这里的轮廓也是对应的轮廓,由于这个是在上述模型上得到的所以,这个轮廓也是在坐标0,0处,角度也是0
//参数列表中的1,表示Pyramid level for which the contour representation should be returned.
get_shape_model_contours (Model, ModelID, 1)
// 获得模版图片的中心点,注意不是形状模版模型也不是形状模版轮廓模型,这一步是为了把上述的形状轮廓模型仿射变换到原来的位置
area_center (RegionFillUp, Area, RowRef, ColumnRef)
// 获得放射矩阵,由于我们已知模版模型是在0,0角度0的位置,所以直接写0就行了
vector_angle_to_rigid (0, 0, 0, RowRef, ColumnRef, 0, HomMat2D)
// 仿射变换,利用上述的放射矩阵
affine_trans_contour_xld (Model, ModelTrans, HomMat2D)
// 到这里已经完成了把轮廓形状模型模版放射变换到指定位置了
dev_display (Image)
dev_display (ModelTrans)
read_image (ImageSearch, 'green-dots')
dev_display (ImageSearch)
//在输入图片中开始搜索匹配的区域
// 同样的道理,加上了scaled就表示支持模版的缩放,即使目标区域比模版大或者小也可以找到
// find_scaled_shape_model(Image输入图片, ModelID模型句柄, AngleStart旋转初始角度, AngleExtent角度范围, ScaleMin最小缩放比例, ScaleMax最大缩放比例, MinScore匹配最小得分, NumMatches匹配个数, MaxOverlap重叠率, SubPixel亚像素级精度, NumLevels模版金字塔第几层, Greediness贪心度,匹配结果的行列坐标,缩放比例和得分值:Row, Column, Angle, Scale, Score)
find_scaled_shape_model (ImageSearch, ModelID, rad(-45), rad(90), 0.8, 1.0, 0.5, 0, 0.5, 'least_squares', 5, 0.8, Row, Column, Angle, Scale, Score)
// 接下来的仿射变换,都是为了能够在各个匹配结果上,绘制一个bndbox,让我们更方便的查看具体结果
// 方法就是把我们得到的轮廓形状模版模型,根据每个匹配结果的中心坐标,旋转角度,缩放比例来仿射变换我们的模版模型,从而准确的框出来结果.
for I := 0 to |Score| - 1 by 1
// 生成恒等放射矩阵
hom_mat2d_identity (HomMat2DIdentity)
// 生成平移放射矩阵,坐标就是平移到目标的xy坐标值
hom_mat2d_translate (HomMat2DIdentity, Row[I], Column[I], HomMat2DTranslate)
// 生成旋转放射矩阵,旋转也是旋转到目标的角度值
hom_mat2d_rotate (HomMat2DTranslate, Angle[I], Row[I], Column[I], HomMat2DRotate)
// 生成缩放放射矩阵,缩放比例到目标的比例值
hom_mat2d_scale (HomMat2DRotate, Scale[I], Scale[I], Row[I], Column[I], HomMat2DScale)
// 把上述得到的形状轮廓模版模型,通过放射矩阵,准确的移动到第I个匹配结果上去
affine_trans_contour_xld (Model, ModelTrans, HomMat2DScale)
dev_display (ModelTrans)
endfor
clear_shape_model (ModelID)
2.2保存模版
介绍保存模版的方法
实际上只有一个算子是重要的,即write_shape_model()
dev_update_off ()
dev_close_window ()
read_image (Image, 'green-dot')
dev_open_window_fit_image (Image, 0, 0, -1, -1, WindowHandle)
dev_display (Image)
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
dev_set_draw ('margin')
dev_set_color ('green')
dev_set_line_width (3)
Message := 'This example shows how to create a shape model'
Message[1] := 'for scale invariant matching and how to save'
Message[2] := 'it in a file.'
disp_message (WindowHandle, Message, 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
// 这里使用的是blob分析
threshold (Image, Region, 0, 128)
connection (Region, ConnectedRegions)
select_shape (ConnectedRegions, SelectedRegions, 'area', 'and', 10000, 20000)
fill_up (SelectedRegions, RegionFillUp)
dilation_circle (RegionFillUp, RegionDilation, 5.5)
dev_display (Image)
dev_display (RegionDilation)
disp_message (WindowHandle, 'Template region', 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
reduce_domain (Image, RegionDilation, ImageReduced)
// 创建一个对形状模型的检查参数,相当于提前检查一下这个模版是否可以做模版
// 参数(输入图片,输出图片金字塔,输出金字塔区域,输入图片金字塔总层数,contrast对比度)
inspect_shape_model (ImageReduced, ModelImages, ModelRegions, 1, 40)
create_scaled_shape_model (ImageReduced, 5, rad(-45), rad(90), 0, 0.8, 1.0, 0, ['none','no_pregeneration'], 'ignore_global_polarity', 40, 10, ModelID)
dev_display (Image)
dev_display (ModelRegions)
disp_message (WindowHandle, 'Regions of the shape model', 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
// 保存创建好的模型,写成.shm文件
write_shape_model (ModelID, 'green-dot.shm')
Message := 'The shape model has been saved in the file'
Message[1] := '\'green-dot.shm\'.'
disp_message (WindowHandle, Message, 'window', 12, 12, 'black', 'true')
disp_end_of_program_message (WindowHandle, 'black', 'true')
clear_shape_model (ModelID)
2.3读取模版
介绍如何从保存的模版文件中读取模版文件
dev_update_window ('off')
read_image (ModelImage, 'rings_and_nuts')
get_image_pointer1 (ModelImage, Pointer, Type, Width, Height)
dev_close_window ()
dev_open_window (0, 0, Width, Height, 'white', WindowHandle)
dev_set_part (0, 0, Height - 1, Width - 1)
dev_display (ModelImage)
dev_set_color ('cyan')
dev_set_draw ('margin')
dev_set_line_width (2)
set_display_font (WindowHandle, 14, 'mono', 'true', 'false')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
Row := 324
Column := 279
Radius := 60
gen_circle (ROI1, Row, Column, Radius)
gen_circle (ROI2, Row, Column, 0.5 * Radius)
difference (ROI1, ROI2, ModelROI)
reduce_domain (ModelImage, ModelROI, ImageROI)
create_scaled_shape_model (ImageROI, 'auto', -rad(30), rad(60), 'auto', 0.6, 1.4, 'auto', 'none', 'use_polarity', 60, 10, ModelID)
inspect_shape_model (ImageROI, ShapeModelImage, ShapeModelRegion, 1, 30)
dev_clear_window ()
dev_display (ShapeModelRegion)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
// 保存模型
ModelFile := 'model_nut.sbm'
write_shape_model (ModelID, ModelFile)
// 保存模型对应图片
ModelRegionFile := 'model_region_nut.png'
// write_image(输入图片,图片格式,填充颜色默认0,文件名)
write_image (ImageROI, 'png', 0, ModelRegionFile)
clear_shape_model (ModelID)
// 读取模型
read_shape_model (ModelFile, ReusedModelID)
// 获取模型一系列参数
get_shape_model_contours (ReusedShapeModel, ReusedModelID, 1)
get_shape_model_origin (ReusedModelID, ReusedRefPointRow, ReusedRefPointCol)
get_shape_model_params (ReusedModelID, NumLevels, AngleStart, AngleExtent, AngleStep, ScaleMin, ScaleMax, ScaleStep, Metric, MinContrast)
read_image (ImageModelRegion, 'model_region_nut.png')
// 获得区域
get_domain (ImageModelRegion, DomainModelRegion)
dev_display (ImageModelRegion)
dev_display (DomainModelRegion)
stop ()
read_image (SearchImage, 'rings_and_nuts')
dev_display (SearchImage)
find_scaled_shape_model (SearchImage, ReusedModelID, AngleStart, AngleExtent, ScaleMin, ScaleMax, 0.65, 0, 0, 'least_squares', 0, 0.8, RowCheck, ColumnCheck, AngleCheck, ScaleCheck, Score)
for i := 0 to |Score| - 1 by 1
vector_angle_to_rigid (ReusedRefPointRow, ReusedRefPointCol, 0, RowCheck[i], ColumnCheck[i], AngleCheck[i], MovementOfObject)
hom_mat2d_scale (MovementOfObject, ScaleCheck[i], ScaleCheck[i], RowCheck[i], ColumnCheck[i], MoveAndScalingOfObject)
affine_trans_contour_xld (ReusedShapeModel, ModelAtNewPosition, MoveAndScalingOfObject)
dev_display (ModelAtNewPosition)
endfor
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
clear_shape_model (ModelID)
dev_update_window ('on')
2.4图像指针和缩放轮廓
//1⃣️预处理
dev_update_window ('off')
open_framegrabber ('File', 1, 1, 0, 0, 0, 0, 'default', -1, 'default', -1, 'default', 'pendulum/pendulum.seq', 'default', -1, 1, FGHandle)
grab_image (ModelImage, FGHandle)
// 获得图像指针
get_image_pointer1 (ModelImage, Pointer, Type, Width, Height)
dev_close_window ()
dev_open_window (0, 0, Width, Height, 'black', WindowHandle)
// 把图像进行放缩变换,参数就是(图像左上角坐标,右下角坐标)
dev_set_part (0, 0, Height - 1, Width - 1)
dev_display (ModelImage)
dev_set_color ('cyan')
dev_set_draw ('margin')
dev_set_line_width (2)
stop ()
//2⃣️ blob分析得到标签区域
threshold (ModelImage, BrightRegions, 200, 255)
connection (BrightRegions, ConnectedRegions)
fill_up (ConnectedRegions, FilledRegions)
dev_display (ModelImage)
dev_display (FilledRegions)
stop ()
select_shape (FilledRegions, Card, 'area', 'and', 1800, 1900)
// 目标图像进行放缩
dev_set_part (round(0.2 * Height), round(0.1 * Width) - 1, round(0.7 * Height) - 1, round(0.6 * Width) - 1)
dev_display (ModelImage)
dev_display (Card)
stop ()
reduce_domain (ModelImage, Card, ImageCard)
stop ()
//3⃣️ blob分析提取标签的logo区域
dev_set_color ('blue')
threshold (ImageCard, DarkRegions, 0, 230)
// 把独立的区域都分割开,生成新区域ConnectRegions
connection (DarkRegions, ConnectedRegions)
select_shape (ConnectedRegions, Characters, 'area', 'and', 150, 450)
// 把Characters中的区域联合起来,成一个新区域CharacterRegion
union1 (Characters, CharacterRegion)
dev_display (ModelImage)
dev_display (CharacterRegion)
stop ()
// 抠图log的时候把边缘进行膨胀操作,防止误差
dilation_circle (CharacterRegion, ROI, 1.5)
dev_display (ModelImage)
dev_display (ROI)
stop ()
// 4⃣️完成模版图片的获得,接下来进行创建模版模型
dev_set_part (0, 0, Height - 1, Width - 1)
dev_display (ModelImage)
reduce_domain (ModelImage, ROI, ImageROI)
/ //查看获得的形状模版金字塔
inspect_shape_model (ImageROI, ShapeModelImages, ShapeModelRegions, 5, 25)
//选择pyramid第几层为模版图片
select_obj (ShapeModelRegions, ShapeModelRegion, 1)
dev_display (ShapeModelRegion)
/
// 注意这个创建模版并不是可缩放的
create_shape_model (ImageROI, 3, 0, rad(360), 'auto', 'none', 'use_polarity', 30, 10, ModelID)
// 获得匹配模版轮廓
get_shape_model_contours (ShapeModel, ModelID, 1)
stop ()
//5⃣️开始匹配
for i := 1 to 30 by 1
grab_image (SearchImage, FGHandle)
// 进行模版匹配
find_shape_model (SearchImage, ModelID, 0, rad(360), 0.7, 1, 0.5, 'least_squares', 0, 0.5, RowCheck, ColumnCheck, AngleCheck, Score)
if (|Score| > 0)
// 快速获得放射矩阵,参数坐标角度,目标坐标角度,输出放射矩阵
vector_angle_to_rigid (0, 0, 0, RowCheck, ColumnCheck, AngleCheck, MovementOfObject)
affine_trans_contour_xld (ShapeModel, ModelAtNewPosition, MovementOfObject)
dev_display (SearchImage)
dev_display (ModelAtNewPosition)
endif
endfor
stop ()
dev_update_window ('on')
clear_shape_model (ModelID)
close_framegrabber (FGHandle)
2.5不规则轮廓匹配
一般来说,如果待匹配的对象是规则的图形,可以通过助手获得,即使需要放缩时,也可以通过带scale的算子进行等比例的放缩匹配,但是当遇到不规则的图形时,往往采用blob分析抠图或者直接使用不规则轮廓匹配方法
准备一个模型进行不等比例的匹配,可以使用不规则轮廓匹配方法
//find_aniso_shape_model.hdev
dev_update_off ()
dev_close_window ()
set_system ('int2_bits', 10)
read_image (Image, 'smd/smd_capacitors_01')
get_image_size (Image, Width, Height)
dev_open_window_fit_image (Image, 0, 0, -1, -1, WindowHandle)
dev_display (Image)
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
Message := 'This program shows how to use shape-based matching'
Message[1] := 'to find SMD capacitors that exhibit independent'
Message[2] := 'size changes in the row and column direction in'
Message[3] := 'images with a depth of 10 bits.'
Message[4] := 'First a synthetic model for the SMD capacitors'
Message[5] := 'is created. In the next step the created model'
Message[6] := 'is used to find the SMD capacitors.'
disp_message (WindowHandle, Message, 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
dev_close_window ()
dev_open_window (0, 0, Width, Height, 'black', WindowHandle)
dev_set_color ('green')
dev_set_line_width (3)
// 生成一个亚像素轮廓
gen_contour_polygon_rounded_xld (Contour, [50,100,100,50,50], [50,50,150,150,50], [6,6,6,6,6], 1)
// 形成一个空的图片,里面啥也米有
gen_image_const (Image, 'byte', 200, 150)
// 根据轮廓和图片生成grayValue=128的图片,图片大小是轮廓大小,注意生成的图片像素全是128
paint_xld (Contour, Image, ImageModel, 128)
/create_aniso_shape_model(Template输入模版图片,
NumLevels金字塔层数,
AngleStart,AngleExtent角度范围,
AngleStep,ScaleRMin行方向缩放比例,
ScaleRMax行方向缩放比例最大值,
ScaleRStep行方向缩放比例变化步长,
ScaleCMin,ScaleCMax,ScaleCStep列方向,
Optimization优化方法,
Metric度量标准,
Contrast对比度,
MinContrast,ModelID)
/
create_aniso_shape_model (ImageModel, 'auto', -rad(10), rad(20), 'auto', 0.9, 1.7, 'auto', 0.9, 1.1, 'auto', 'none', 'use_polarity', 'auto', 20, ModelID)
get_shape_model_contours (ModelContours, ModelID, 1)
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
for J := 1 to 4 by 1
read_image (Image, 'smd/smd_capacitors_' + J$'02d')
dev_display (Image)
count_seconds (S1)
find_aniso_shape_model (Image, ModelID, -rad(10), rad(20), 0.9, 1.7, 0.9, 1.1, 0.7, 0, 0.5, 'least_squares', 0, 0.8, Row, Column, Angle, ScaleR, ScaleC, Score)
count_seconds (S2)
Time := (S2 - S1) * 1000
Num := |Score|
disp_message (WindowHandle, Num$'d' + ' models found in ' + Time$'5.2f' + ' ms', 'window', 12, 12, 'black', 'true')
MeanColumn := mean(Column)
for I := 0 to Num - 1 by 1
hom_mat2d_identity (HomMat2D)
hom_mat2d_scale (HomMat2D, ScaleR[I], ScaleC[I], 0, 0, HomMat2D)
hom_mat2d_rotate (HomMat2D, Angle[I], 0, 0, HomMat2D)
hom_mat2d_translate (HomMat2D, Row[I], Column[I], HomMat2D)
affine_trans_contour_xld (ModelContours, ContoursTrans, HomMat2D)
dev_display (ContoursTrans)
ScaleRowStr := 'ScaleRow=' + ScaleR[I]$'5.3f'
ScaleColStr := 'ScaleCol=' + ScaleC[I]$'5.3f'
get_string_extents (WindowHandle, ScaleRowStr, AscentStr, DescentStr, WidthStr, HeightStr)
if (Column[I] <= MeanColumn)
disp_message (WindowHandle, [ScaleRowStr,ScaleColStr], 'image', Row[I] - 20, Column[I] - 60 - WidthStr, 'green', 'false')
else
disp_message (WindowHandle, [ScaleRowStr,ScaleColStr], 'image', Row[I] - 20, Column[I] + 60, 'green', 'false')
endif
endfor
if (J < 4)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
endif
endfor
clear_shape_model (ModelID)
set_system ('int2_bits', -1)
2.6轮廓的读取
从DXF文件中读取xld轮廓
dev_update_pc ('off')
dev_update_window ('off')
dev_update_var ('off')
dev_close_window ()
dev_open_window (0, 0, 646, 482, 'black', WindowHandle)
dev_set_part (0, 0, 481, 645)
dev_set_draw ('margin')
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
dev_set_line_width (3)
Colors := ['red','green','cyan']
gen_empty_obj (Models)
IndexS := []
IndexE := []
ModelIDs := []
for J := 1 to 3 by 1
dev_clear_window ()
// 参数(输出读取的contours,文件路径,输入dxf文件控制参数名,控制参数值,Dxf文件被读取的状态信息)
read_contour_xld_dxf (Contours, 'metal-part-' + J$'02', [], [], DxfStatus)
// 自定义函数,这个不在赘述,比较简单,就是把上边的contours变成一个图像
gen_model_image_of_bright_object_with_holes (Contours, Image, 3.38, 646, 482)
dev_display (Image)
dev_set_color ('green')
set_tposition (WindowHandle, 20, 20)
write_string (WindowHandle, 'Generating shape model ' + J$'d')
get_domain (Image, Domain)
area_center (Domain, Area, Row, Column)
inspect_shape_model (Image, ModelImages, ModelRegions, 1, 30)
connection (ModelRegions, ConnectedRegions)
select_shape (ConnectedRegions, SelectedRegions, 'area', 'and', 20, 100000)
union1 (SelectedRegions, ModelRegions)
gen_contours_skeleton_xld (ModelRegions, ModelContours, 1, 'filter')
dev_set_color ('red')
dev_display (ModelContours)
create_shape_model (Image, 5, rad(0), rad(360), 'auto', 'pregeneration', 'use_polarity', 30, 10, ModelID)
get_shape_model_contours (ModelCont, ModelID, 1)
select_shape_xld (ModelCont, ModelContours, 'contlength', 'and', 20, 1000)
count_obj (ModelContours, NumModel)
count_obj (Models, NumModels)
concat_obj (Models, ModelContours, Models)
IndexS := [IndexS,NumModels + 1]
IndexE := [IndexE,NumModels + NumModel]
ModelIDs := [ModelIDs,ModelID]
endfor
disp_message (WindowHandle, ['Press left button to start','and stop the demo'], 'image', 50, 20, 'yellow', 'false')
get_mbutton (WindowHandle, Row1, Column1, Button1)
wait_seconds (0.5)
dev_set_color ('red')
Button := 0
ImgNo := 1
while (Button != 1)
read_image (Image, 'metal-parts/metal-parts-' + ImgNo$'02d')
count_seconds (S1)
find_shape_models (Image, ModelIDs, rad(0), rad(360), 0.6, 0, 0.5, 'least_squares', 4, 0.3, Row, Column, Angle, Score, Model)
count_seconds (S2)
Time := (S2 - S1) * 1000.
dev_display (Image)
Num := |Score|
for J := 0 to Num - 1 by 1
* Select the correct XLD contours from the Models object.
copy_obj (Models, ModelSelected, IndexS[Model[J]], IndexE[Model[J]] - IndexS[Model[J]] + 1)
* Compute the transformation from the model object to
* the current instance.
vector_angle_to_rigid (0, 0, 0, Row[J], Column[J], Angle[J], HomMat2D)
affine_trans_contour_xld (ModelSelected, ModelTrans, HomMat2D)
dev_set_color (Colors[Model[J]])
dev_display (ModelTrans)
endfor
if (Num == 1)
disp_message (WindowHandle, Num$'1d' + ' object found in ' + Time$'4.2f' + 'ms', 'image', 20, 20, 'yellow', 'false')
else
disp_message (WindowHandle, Num$'1d' + ' objects found in ' + Time$'4.2f' + ' ms', 'image', 20, 20, 'yellow', 'false')
endif
ImgNo := ImgNo + 1
if (ImgNo > 15)
ImgNo := 1
endif
dev_error_var (Error, 1)
dev_set_check ('~give_error')
get_mposition (WindowHandle, R, C, Button)
dev_error_var (Error, 0)
dev_set_check ('give_error')
if (Error != H_MSG_TRUE)
Button := 0
endif
endwhile
for J := 0 to |ModelIDs| - 1 by 1
clear_shape_model (ModelIDs[J])
endfor
第三节 多对多的组件匹配
在模板:对象=n:n的场景下,往往需要区域数组和常规数组来进行辅助操作
Tuple数组:下面的Model用来存储图片区域的数组,下标从1开始
常规数组:常规控制变量的数组,下标从0开始,比如下面的IndexS和IndexE和ModelIDs的下标都是从0开始的
dev_update_pc ('off')
dev_update_window ('off')
dev_update_var ('off')
dev_close_window ()
dev_open_window (0, 0, 646, 482, 'black', WindowHandle)
dev_set_part (0, 0, 481, 645)
set_display_font (WindowHandle, 20, 'mono', 'true', 'false')
dev_set_draw ('margin')
dev_set_line_width (3)
Colors := ['red','green','cyan']
Row1 := [135,150,185]
Column1 := [250,170,220]
Row2 := [375,310,335]
Column2 := [355,395,375]
// 生成空的区域Models 这里的Models就像一个容器一样,用来存储轮廓
gen_empty_obj (Models)
IndexS := []
IndexE := []
ModelIDs := []
// 创建模版,存储模版
for J := 1 to 3 by 1
read_image (Image, 'metal-parts/metal-part-model-' + J$'02d')
dev_display (Image)
dev_set_color ('green')
// 设置显示窗口和位置
set_tposition (WindowHandle, 20, 20)
// 写字符串
write_string (WindowHandle, 'Generating shape model ' + J$'d')
gen_rectangle1 (Rectangle, Row1[J - 1], Column1[J - 1], Row2[J - 1], Column2[J - 1])
area_center (Rectangle, Area, Row, Column)
reduce_domain (Image, Rectangle, ImageReduced)
inspect_shape_model (Image, ModelImages, ModelRegions, 1, 30)
// 连接相同连通域,独立不同区域
connection (ModelRegions, ConnectedRegions)
// 面积过滤
select_shape (ConnectedRegions, SelectedRegions, 'area', 'and', 20, 100000)
union1 (SelectedRegions, ModelRegions)
// 把一个骨架转换成xld轮廓(输入轮廓骨架,输出轮廓,轮廓上点的最小数量,轮廓滤波模式)
gen_contours_skeleton_xld (ModelRegions, ModelContours, 1, 'filter')
dev_set_color ('red')
dev_display (ModelContours)
create_shape_model (ImageReduced, 5, rad(0), rad(360), 'auto', 'pregeneration', 'use_polarity', 30, 7, ModelID)
// 生成轮廓模型,注意模型位置是0,0,角度0
get_shape_model_contours (ModelCont, ModelID, 1)
// 筛选轮廓,根据周长
select_shape_xld (ModelCont, ModelContours, 'contlength', 'and', 20, 1000)
// 计算tuple数组中的元素个数(输入数组,输出个数)
// 计算ModelContours元素个数,给NumModel
count_obj (ModelContours, NumModel)
// 计算Models元素个数,给NumModels
count_obj (Models, NumModels)
// 追加轮廓(输入轮廓,输入轮廓,输出轮廓)
// 把ModelContours追加到Models
// concat_obj()无论是图像连接还是区域连接都是用这个追加. 相当于是个区域数组,它面向的对象时图形变量
// 注意这里的ModelContours包含了多个轮廓比如大小不同的轮廓,这些轮廓按1234排好顺序,第一个进入Models的轮廓模型放在Models的第一个工件空间,每次依次进入
// 这里是把每个轮廓模版真实的追加到了Models,由于Models长度是在这一步之前求的,所以长度变量NumModels是0
concat_obj (Models, ModelContours, Models)
// 追加元素到数组,这样的数组,面向的是控制变量
// 由于一开始Models即多轮廓组相组合的一个变量内没有放任何轮廓组,所以长度是0,所以要在索引标签IndexS上先+1,防止从0开始索引
IndexS := [IndexS,NumModels + 1]
// 由于要把MoldeContours即轮廓模型逐个放到模型组Models内,所以把每个ModelContours包含的轮廓个数也放到索引处,可以看我拍的图片
IndexE := [IndexE,NumModels + NumModel]
// 这里就是利用句柄把每个轮廓模块追加进了模块组Models
ModelIDs := [ModelIDs,ModelID]
endfor
// 设置匹配识别时的显示颜色窗口参数等
dev_set_color ('yellow')
set_tposition (WindowHandle, 50, 20)
write_string (WindowHandle, 'Press left button to start')
set_tposition (WindowHandle, 80, 20)
write_string (WindowHandle, 'and stop the demo.')
get_mbutton (WindowHandle, Row3, Column3, Button1)
wait_seconds (0.5)
dev_set_color ('red')
Button := 0
ImgNo := 1
// 开始寻找匹配
while (Button != 1)
read_image (Image, 'metal-parts/metal-parts-' + ImgNo$'02d')
// 计算时间
count_seconds (S1)
// 注意这个函数多了个s,其实都一样,只不过这里的模型句柄,是上边我们追加得到的一个句柄组,另外输出的是匹配结果的索引值,就是你匹配到是第几个模型的实例,这里是Model即索引值,注意这里Model是索引的第几个轮廓模版模型,而不是某个模型的第几个轮廓
find_shape_models (Image, ModelIDs, rad(0), rad(360), 0.5, 0, 0.5, 'least_squares', 0, 0.8, Row, Column, Angle, Score, Model)
count_seconds (S2)
Time := (S2 - S1) * 1000
dev_display (Image)
// 求匹配结果的个数,这个||是求长度的,不是绝对值
Num := |Score|
// 这个循环就是用来显示的,上边已经得到了匹配的RowColumAngleScore等信息
for J := 0 to Num - 1 by 1
// copy_obj拷贝一个目标到另一个变量里
// copy_obj(Objects : ObjectsSelected : Index, NumObj : )
// 参数(输入目标,输出结果,输入目标索引值从1开始,从索引值开始拷贝的个数)
// 这里就是把Models里每个模版中的每个小轮廓,从IndexS开始到IndexE结束这些个小轮廓拷贝出来,因为Models小轮廓的坐标是从1开始的,可以看下面的图片,
// 拷贝这些小轮廓就是为了接下来做仿射变换
// 这里indexS[Model[J]]是从第J个轮廓模版模型的第一个小轮廓在Models中的下标开始
// 这里的indexE就是为了获得模型组中的模型的轮廓个数
copy_obj (Models, ModelSelected, IndexS[Model[J]], IndexE[Model[J]] - IndexS[Model[J]] + 1)
// 仿射变换
vector_angle_to_rigid (0, 0, 0, Row[J], Column[J], Angle[J], HomMat2D)
affine_trans_contour_xld (ModelSelected, ModelTrans, HomMat2D)
dev_set_color (Colors[Model[J]])
dev_display (ModelTrans)
endfor
// 写一些字符串
dev_set_color ('yellow')
set_tposition (WindowHandle, 20, 20)
if (Num == 1)
write_string (WindowHandle, Num$'1d' + ' object found in ' + Time$'4.2f' + 'ms')
else
write_string (WindowHandle, Num$'1d' + ' objects found in ' + Time$'4.2f' + 'ms')
endif
ImgNo := ImgNo + 1
if (ImgNo > 15)
ImgNo := 1
endif
dev_error_var (Error, 1)
dev_set_check ('~give_error')
get_mposition (WindowHandle, R, C, Button)
dev_error_var (Error, 0)
dev_set_check ('give_error')
if (Error != H_MSG_TRUE)
Button := 0
endif
endwhile
for J := 0 to |ModelIDs| - 1 by 1
clear_shape_model (ModelIDs[J])
endfor
第三章 标定与测量(单相机)
测量是把像素距离转换成实际距离,拟合则是放在缺陷检测上,把目标形状进行几何化处理,从而直接从几何参数中进行数学测量
测量往往需要相机参数的标定,就是相机的内参和外参,根据这个参数来把像素距离转换成实际距离,⚠️测量往往会先进行图片畸变矫正
第一节 标定
1.1标定板
在相机实际项目中,我们会用到各种各样的标定板,如棋盘格标定板或圆形阵列标定板,如下图所示:
标定板的材质一般是玻璃、卡纸、塑料等
标定的尺寸也有不同,往往是标定板圆点的圆心之间的距离是4mm,一般是7*7的阵列
标定板具有如下特点:
- 一般标定板需要充满相机视野的1/3或1/4
- 标定板成像灰度值要大于128
- 拍摄标定板时,最好旋转标定板,拍9-16张不同旋转角度的标定板
- 标定板的拍摄数量和后续测量精度没有关系
1.2单相机标定
单相机标定的目的: 畸变矫正和一位二维图像测量(主要是1个像素代表的实际尺寸).
标定之所以很重要,是因为相机成像后往往会产生畸变和缩放,因此需要建立一个模型根据图像信息构造出物体的三维空间数据,那么就需要摄像机的几何参数和光学参数,以及摄像机相对于世界坐标系的方位.
在计算机进行测量时,首先要完成摄像机标定,即相机标定,所谓的相机标定就是确定相机的内参和外参.内参是指相机的固有属性,外参是指物体产生的畸变等
上述的世界坐标系就是几种常用坐标系的一个
| 名称/表示法 |
功能 |
| 图像像素坐标系/(u,v) |
以像素为单位,图像的左上点为原点的图像坐标系 |
| 图像物理坐标系/(x,y) |
以mm为单位,物理单位表示图像像素位置,坐标原点在相机和图像平面的交点 |
| 相机坐标系/(Xc,Yc,Zc) |
以mm为单位,以相机的光心作为原点 ,Zc轴和光轴重合,垂直于成像平面,摄像方向为正方向,Xc和Yc轴和图像物理坐标系平行 |
| 世界坐标系/(Xw,Yw,Zw) |
该坐标系描述环境中任何物体的位置,满足右手法则 |
图像像素坐标系和物理坐标系之间的关系,其实物理坐标系是实际物理尺寸坐标系,假设我们的图像像素坐标系坐标系原点在左上角,而物理坐标系在图像中心,假设:每个像素在u轴和v轴方向上的物理尺寸是dx和dy,那么:
u = x/dx + u0
v = y/dy + v0
其中,物理坐标系单位mm,像素坐标系单位是像素pixel,那么dx单位是mm/pixel毫米/像素,显然x/dx单位就是pixel,点(u0,v0)是光轴和图像平面的交点坐标,dx和dy分别是单位像素在x和y方向上的物理尺寸
上述一段话,转换成下图:
世界坐标系和像素坐标系同样可以通过上图来表示
上图中的M是3*4的投影矩阵,M1矩阵的元素数值完全由相机内参ax、ay、u0和v0决定,其中(u0,v0)是像素坐标系的原点坐标,ax和ay表示像素坐标系纵横轴上的尺度因子,即ax = f/dx, ay=f/dy, M2矩阵则是相机外参决定.
世界坐标系到像素坐标系,中间经过相机坐标系的过渡,Xw表示世界坐标系的x轴,单位是mm,uv表示像素坐标系的横纵轴,单位是pixel,从世界坐标系到像素坐标系的转换,就是从mm到pixel的转换.
在Halcon中,可以使用标定文件的算子来进行单相机标定.
//生成标定文件算子,输入标定板参数即可
gen_caltab( : : XNum每行黑色标志圆点数量,
YNum,
MarkDist两个就近圆点中心之间的距离单位是meter,
DiameterRatio黑色圆点直径和两个圆点中心距离比值,
CalPlateDescr存放标定板描述文件路径.descr文件,
CalPlatePSFile存放标定板图像文件路径.ps文件)
1.3单相机标定示例
首先,利用上述的算子gen_caltab()生成.descr 和 .ps文件,规定标定的参数
第二步, 在标定板选项中选择刚才生成的descr文件,如下图
第三步,选择标定,另外需要注意相机参数,选择相机时,需要考虑像元的实际宽高和焦距
第四步,选择采集助手,选择Gray灰度图片
第五步,点击采集获得图片,但需要查看图片的状态,如果状态显示:检测出品质问题不必在意,如果显示:失败则需要重新采集该图片.采集15张图片左右即可,要确保角度不同.
第六步,选择结果,其中摄像机参数就是内参,摄像机位姿就是外参.需要注意内参保存的文件是.cal 外参保存的文件是.dat
到这里就算完成了单相机的标定,接下来简单演示测量,具体的测量方法见第二节
第七步,测量助手,把上述的内参文件.cal和外参文件.dat加载进来
第八步,使用测量助手进行测量
此时,完成了相机的标定,完成了相机内参外参的设置,但还没有进行畸变矫正.
第二节 畸变矫正
单相机的标定就是为了获得相机的内参和外参,但是在测量前必须要完成的就是图像畸变的矫正,而在畸变矫正的过程中,往往不需要用到外参.
所谓的畸变是因为镜头内部结构导致的,存在畸变的对象往往是一维和二维的,因此外参对畸变的矫正并没有太大作用,但外参在三维对象的操作中却有很大作用.
畸变的矫正有如下六个环节:
显然,对于畸变的矫正,关键在于如何获得畸变矫正矩阵.接下来先看一张存在畸变的图片
显然上图的对象是一个存在明显畸变的标定板,如果使用它则会产生难以想象的误差.因此要通过畸变矫正来对它进行修复.
简单的说就是把相机的内参进行调整,从而获得具有无畸变效应的内参,如下面的程序所述:
/*通过矫正畸变的算子,把单相机标定好的内参外参,进行参数修改,获得正确的参数
*/
change_radial_distortion_cam_par(Mode,算法模式
CamParamIn, 输入相机内参
DistortionCoeffs,一般是0
CamParamOut输出相机内参)
/*产生映射,相当于放射矩阵*/
gen_radial_distortion_map(Map,输出映射数据
CamParamIn,原始相机内参即标定的内参
CamParamOut, 矫正过的相机内参
MapType算法模式)
/*应用变换到图片*/
map_image(Image,需要矫正的图片
Map,映射数据
ImageMapped,输出矫正后的图片)
通过畸变矫正矩阵的变换后,可以得到矫正后的图片
第三节 测量矩形
测量步骤:
- 把测量矩形放到待测量区域,Start和End以及夹角Phi表示待测量的方向,起始点角度等
- 在测量矩形内部寻找待测量图像的边缘
- 把边缘进行灰度投影,在垂直投影方向上找到边缘,然后计算哪个边和哪个边的距离
根据测量矩形的方向,把负边缘定义成是由亮到暗,正边缘是由暗到亮
实例一(基于边缘对)
结果图:
解释:
边缘对
检测流程:
- gen_measure_rectangle2()
- mesure_pairs()
- close_measure()
dev_update_window ('off')
dev_close_window ()
read_image (Fuse, 'fuse')
get_image_size (Fuse, Width, Height)
dev_open_window_fit_image (Fuse, 0, 0, Width, Height, WindowID)
set_display_font (WindowID, 12, 'mono', 'true', 'false')
dev_set_draw ('margin')
dev_set_line_width (3)
dev_display (Fuse)
set_display_font (WindowID, 12, 'mono', 'true', 'false')
disp_continue_message (WindowID, 'black', 'true')
stop ()
Row := 297
Column := 545
Length1 := 80
Length2 := 10
Angle := rad(90)
gen_rectangle2 (ROI, Row, Column, Angle, Length1, Length2)
// 生成测量矩形
// gen_measure_rectangle2( : : Row, Column, Phi, 矩形半宽Length1, 半高Length2, 图片Width, Height, Interpolation差值算法 : MeasureHandle输出测量矩形句柄)
gen_measure_rectangle2 (Row, Column, Angle, Length1, Length2, Width, Height, 'bilinear', MeasureHandle)
dev_display (ROI)
disp_continue_message (WindowID, 'black', 'true')
stop ()
/* 两个边缘作为一对,测量该对之间的距离,以及边缘对和边缘对之间的距离
注意这里的第一条边缘是说的每个边缘对中的第一个边缘,所以输出的是个数组
同理,第二条边缘也是每个边缘对的第二个,输出的也是个数组
边缘对内部的距离就是边缘对中第一个和第二个边缘的距离,同样输出的也是数组
边缘对之间的距离是每个边缘对的中心点到另一个边缘对的中心点的距离
*/
/*measure_pairs(Image : : MeasureHandle测量矩形句柄,
Sigma平滑参数,
Threshold最小边缘像素变化幅度,
Transition边缘类型是正还是负,
Select选择第几个边缘对,
RowEdgeFirst输出第一个边缘的中心行坐标,
ColumnEdgeFirst,
AmplitudeFirst输出第一个边缘的像素变化幅度即一阶差分导数值,
RowEdgeSecond输出第二个边的中心行坐标,
ColumnEdgeSecond,
AmplitudeSecond,
IntraDistance输出边缘对内部边缘之间的距离,
InterDistance第1边缘对的第2条边 到 第2边缘对第1个边缘的距离)
*/
measure_pairs (Fuse, MeasureHandle, 1, 1, 'negative', 'all', RowEdgeFirst, ColumnEdgeFirst, AmplitudeFirst, RowEdgeSecond, ColumnEdgeSecond, AmplitudeSecond, IntraDistance, InterDistance)
disp_continue_message (WindowID, 'black', 'true')
stop ()
// 解析来就是结果可视化了
for i := 0 to |RowEdgeFirst| - 1 by 1
// 获得多边形轮廓(输出轮廓,输入多边形轮廓的线段转折点的横坐标点,列坐标点)
gen_contour_polygon_xld (EdgeFirst, [-sin(Angle + rad(90)) * Length2 + RowEdgeFirst[i],-sin(Angle - rad(90)) * Length2 + RowEdgeFirst[i]], [cos(Angle + rad(90)) * Length2 + ColumnEdgeFirst[i],cos(Angle - rad(90)) * Length2 + ColumnEdgeFirst[i]])
// 另一个多边形轮廓
gen_contour_polygon_xld (EdgeSecond, [-sin(Angle + rad(90)) * Length2 + RowEdgeSecond[i],-sin(Angle - rad(90)) * Length2 + RowEdgeSecond[i]], [cos(Angle + rad(90)) * Length2 + ColumnEdgeSecond[i],cos(Angle - rad(90)) * Length2 + ColumnEdgeSecond[i]])
// 显示上述轮廓,上图中的青色就是第一个轮廓,洋红色就是第二个,
dev_set_color ('cyan')
dev_display (EdgeFirst)
dev_set_color ('magenta')
dev_display (EdgeSecond)
// 设置接下来的字体颜色是蓝色
dev_set_color ('blue')
if (i == 0)
set_tposition (WindowID, RowEdgeFirst[i] + 5, ColumnEdgeFirst[i] + 20)
else
set_tposition (WindowID, RowEdgeFirst[i] - 40, ColumnEdgeFirst[i] + 20)
endif
write_string (WindowID, 'width: ' + IntraDistance[i] + ' pix')
endfor
disp_continue_message (WindowID, 'black', 'true')
stop ()
close_measure (MeasureHandle)
dev_update_window ('on')
dev_clear_window ()
实例二(非边缘对)
实例一和实例二的输出测量的数组都是控制变量数组,即常规数组,所以索引从0开始
独立边缘测量步骤:
- gen_measrue_rectangle2()
- measure_pos()
- close_measure()
结果图:
dev_close_window ()
read_image (Image, 'ic_pin')
get_image_size (Image, Width, Height)
dev_open_window (0, 0, Width / 2, Height / 2, 'black', WindowHandle)
set_display_font (WindowHandle, 14, 'mono', 'true', 'false')
dev_display (Image)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
// draw_rectangle2 (WindowHandle, Row, Column, Phi, Length1, Length2)
Row := 47
Column := 485
Phi := 0
Length1 := 420
Length2 := 10
dev_set_color ('green')
dev_set_draw ('margin')
dev_set_line_width (3)
//生成一个矩形区域
gen_rectangle2 (Rectangle, Row, Column, Phi, Length1, Length2)
// 生成一个测量矩形在上述区域上
gen_measure_rectangle2 (Row, Column, Phi, Length1, Length2, Width, Height, 'nearest_neighbor', MeasureHandle)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
// 设置运行参数
dev_update_pc ('off')
dev_update_var ('off')
// n时测量次数
n := 100
count_seconds (Seconds1)
for i := 1 to n by 1
// 使用边缘对测量
measure_pairs (Image, MeasureHandle, 1.5, 30, 'negative', 'all', RowEdgeFirst, ColumnEdgeFirst, AmplitudeFirst, RowEdgeSecond, ColumnEdgeSecond, AmplitudeSecond, PinWidth, PinDistance)
endfor
count_seconds (Seconds2)
Time := Seconds2 - Seconds1
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
dev_set_color ('red')
disp_line (WindowHandle, RowEdgeFirst, ColumnEdgeFirst, RowEdgeSecond, ColumnEdgeSecond)
// sum是元素求和函数
avgPinWidth := sum(PinWidth) / |PinWidth|
avgPinDistance := sum(PinDistance) / |PinDistance|
numPins := |PinWidth|
dev_set_color ('yellow')
disp_message (WindowHandle, 'Number of pins: ' + numPins, 'image', 200, 100, 'yellow', 'false')
disp_message (WindowHandle, 'Average Pin Width: ' + avgPinWidth, 'image', 260, 100, 'yellow', 'false')
disp_message (WindowHandle, 'Average Pin Distance: ' + avgPinDistance, 'image', 320, 100, 'yellow', 'false')
* dump_window (WindowHandle, 'tiff_rgb', 'C:\\Temp\\pins_result')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
* draw_rectangle1 (WindowHandle, Row1, Column1, Row2, Column2)
Row1 := 0
Column1 := 600
Row2 := 100
Column2 := 700
dev_set_color ('blue')
disp_rectangle1 (WindowHandle, Row1, Column1, Row2, Column2)
stop ()
// 这个只是更改显示区域,并不是裁剪了图片
dev_set_part (Row1, Column1, Row2, Column2)
dev_display (Image)
dev_set_color ('green')
dev_display (Rectangle)
dev_set_color ('red')
disp_line (WindowHandle, RowEdgeFirst, ColumnEdgeFirst, RowEdgeSecond, ColumnEdgeSecond)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
close_measure (MeasureHandle)
dev_set_part (0, 0, Height - 1, Width - 1)
dev_display (Image)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
// 这里完成了第一次的测量,开始另一个区域测量
dev_set_color ('green')
* draw_rectangle2 (WindowHandle, Row, Column, Phi, Length1, Length2)
Row := 508
Column := 200
Phi := -1.5708
Length1 := 482
Length2 := 35
gen_rectangle2 (Rectangle, Row, Column, Phi, Length1, Length2)
gen_measure_rectangle2 (Row, Column, Phi, Length1, Length2, Width, Height, 'nearest_neighbor', MeasureHandle)
stop ()
//不是用边缘对的测量方法
// 不把边缘按对来分,每个边缘都是一个独立的边缘
/measure_pos(输入图片,
输出测量句柄,
平滑参数,
边缘差分最小阈值,
边缘正负类型,
选择哪个边缘,
输出边缘中心点的行坐标列坐标,
输出对应边缘差分值,
输出每个相邻边缘的中心点距离)
/
measure_pos (Image, MeasureHandle, 1.5, 30, 'all', 'all', RowEdge, ColumnEdge, Amplitude, Distance)
// 根据得到的信息来计算高度
PinHeight1 := RowEdge[1] - RowEdge[0]
PinHeight2 := RowEdge[3] - RowEdge[2]
dev_set_color ('red')
disp_line (WindowHandle, RowEdge, ColumnEdge - Length2, RowEdge, ColumnEdge + Length2)
disp_message (WindowHandle, 'Pin Height: ' + PinHeight1, 'image', RowEdge[1] + 40, ColumnEdge[1] + 100, 'yellow', 'false')
disp_message (WindowHandle, 'Pin Height: ' + PinHeight2, 'image', RowEdge[3] - 120, ColumnEdge[3] + 100, 'yellow', 'false')
* dump_window (WindowHandle, 'tiff_rgb', 'C:\\Temp\\pins_height_result')
close_measure (MeasureHandle)
dev_set_draw ('fill')
dev_set_line_width (1)
dev_close_window ()
read_image (Image, 'ic_pin')
get_image_size (Image, Width, Height)
dev_open_window (0, 0, Width / 2, Height / 2, 'black', WindowHandle)
set_display_font (WindowHandle, 14, 'mono', 'true', 'false')
dev_display (Image)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
* draw_rectangle2 (WindowHandle, Row, Column, Phi, Length1, Length2)
Row := 47
Column := 485
Phi := 0
Length1 := 420
Length2 := 10
dev_set_color ('green')
dev_set_draw ('margin')
dev_set_line_width (3)
gen_rectangle2 (Rectangle, Row, Column, Phi, Length1, Length2)
gen_measure_rectangle2 (Row, Column, Phi, Length1, Length2, Width, Height, 'nearest_neighbor', MeasureHandle)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
dev_update_pc ('off')
dev_update_var ('off')
n := 100
count_seconds (Seconds1)
for i := 1 to n by 1
measure_pairs (Image, MeasureHandle, 1.5, 30, 'negative', 'all', RowEdgeFirst, ColumnEdgeFirst, AmplitudeFirst, RowEdgeSecond, ColumnEdgeSecond, AmplitudeSecond, PinWidth, PinDistance)
endfor
count_seconds (Seconds2)
Time := Seconds2 - Seconds1
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
dev_set_color ('red')
disp_line (WindowHandle, RowEdgeFirst, ColumnEdgeFirst, RowEdgeSecond, ColumnEdgeSecond)
avgPinWidth := sum(PinWidth) / |PinWidth|
avgPinDistance := sum(PinDistance) / |PinDistance|
numPins := |PinWidth|
dev_set_color ('yellow')
disp_message (WindowHandle, 'Number of pins: ' + numPins, 'image', 200, 100, 'yellow', 'false')
disp_message (WindowHandle, 'Average Pin Width: ' + avgPinWidth, 'image', 260, 100, 'yellow', 'false')
disp_message (WindowHandle, 'Average Pin Distance: ' + avgPinDistance, 'image', 320, 100, 'yellow', 'false')
* dump_window (WindowHandle, 'tiff_rgb', 'C:\\Temp\\pins_result')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
* draw_rectangle1 (WindowHandle, Row1, Column1, Row2, Column2)
Row1 := 0
Column1 := 600
Row2 := 100
Column2 := 700
dev_set_color ('blue')
disp_rectangle1 (WindowHandle, Row1, Column1, Row2, Column2)
stop ()
dev_set_part (Row1, Column1, Row2, Column2)
dev_display (Image)
dev_set_color ('green')
dev_display (Rectangle)
dev_set_color ('red')
disp_line (WindowHandle, RowEdgeFirst, ColumnEdgeFirst, RowEdgeSecond, ColumnEdgeSecond)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
close_measure (MeasureHandle)
dev_set_part (0, 0, Height - 1, Width - 1)
dev_display (Image)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
dev_set_color ('green')
* draw_rectangle2 (WindowHandle, Row, Column, Phi, Length1, Length2)
Row := 508
Column := 200
Phi := -1.5708
Length1 := 482
Length2 := 35
gen_rectangle2 (Rectangle, Row, Column, Phi, Length1, Length2)
gen_measure_rectangle2 (Row, Column, Phi, Length1, Length2, Width, Height, 'nearest_neighbor', MeasureHandle)
stop ()
measure_pos (Image, MeasureHandle, 1.5, 30, 'all', 'all', RowEdge, ColumnEdge, Amplitude, Distance)
PinHeight1 := RowEdge[1] - RowEdge[0]
PinHeight2 := RowEdge[3] - RowEdge[2]
dev_set_color ('red')
disp_line (WindowHandle, RowEdge, ColumnEdge - Length2, RowEdge, ColumnEdge + Length2)
disp_message (WindowHandle, 'Pin Height: ' + PinHeight1, 'image', RowEdge[1] + 40, ColumnEdge[1] + 100, 'yellow', 'false')
disp_message (WindowHandle, 'Pin Height: ' + PinHeight2, 'image', RowEdge[3] - 120, ColumnEdge[3] + 100, 'yellow', 'false')
* dump_window (WindowHandle, 'tiff_rgb', 'C:\\Temp\\pins_height_result')
close_measure (MeasureHandle)
dev_set_draw ('fill')
dev_set_line_width (1)
实例三(形状匹配+定位测量)
- 匹配好目标形状,定位其位置
- 根据每一帧目标的变化,形成仿射矩阵
- 根据放射矩阵,把测量矩形的区域进行相同变换
- 对测量区域进行边缘测量
dev_update_pc ('off')
dev_update_window ('off')
dev_update_var ('off')
open_framegrabber ('File', 1, 1, 0, 0, 0, 0, 'default', -1, 'default', -1, 'default', 'board/board.seq', 'default', -1, 1, FGHandle)
grab_image (Image, FGHandle)
get_image_size (Image, Width, Height)
dev_close_window ()
dev_open_window (0, 0, Width, Height, 'black', WindowHandle)
dev_open_window (Height + 70, 0, Width, 120, 'black', WindowHandleText)
dev_set_window (WindowHandle)
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
set_display_font (WindowHandleText, 16, 'mono', 'true', 'false')
dev_set_color ('red')
dev_display (Image)
Row1 := 188
Column1 := 182
Row2 := 298
Column2 := 412
gen_rectangle1 (Rectangle, Row1, Column1, Row2, Column2)
area_center (Rectangle, Area, Row, Column)
Rect1Row := -102
Rect1Col := 5
Rect2Row := 107
Rect2Col := 5
RectPhi := 0
RectLength1 := 170
RectLength2 := 5
// 设置2个测量矩形的矩形区域
gen_rectangle2 (Rectangle1, Row + Rect1Row, Column + Rect1Col, RectPhi, RectLength1, RectLength2)
gen_rectangle2 (Rectangle2, Row + Rect2Row, Column + Rect2Col, RectPhi, RectLength1, RectLength2)
reduce_domain (Image, Rectangle, ImageReduced)
create_shape_model (ImageReduced, 4, 0, rad(360), rad(1), 'none', 'use_polarity', 30, 10, ModelID)
get_shape_model_contours (ShapeModel, ModelID, 1)
hom_mat2d_identity (HomMat2DIdentity)
hom_mat2d_translate (HomMat2DIdentity, Row, Column, HomMat2DTranslate)
affine_trans_contour_xld (ShapeModel, ShapeModelTrans, HomMat2DTranslate)
dev_display (Image)
dev_set_color ('green')
dev_display (ShapeModelTrans)
dev_set_color ('blue')
dev_set_draw ('margin')
dev_set_line_width (3)
dev_display (Rectangle1)
dev_display (Rectangle2)
dev_set_draw ('fill')
dev_set_line_width (1)
dev_set_color ('yellow')
disp_message (WindowHandle, ['Press left button to start','and stop the demo'], 'window', 12, 12, 'black', 'true')
get_mbutton (WindowHandle, Row3, Column3, Button1)
wait_seconds (0.5)
Button := 0
while (Button != 1)
dev_set_window (WindowHandle)
dev_set_part (0, 0, Height - 1, Width - 1)
grab_image (ImageCheck, FGHandle)
dev_display (ImageCheck)
count_seconds (S1)
find_shape_model (ImageCheck, ModelID, 0, rad(360), 0.7, 1, 0.5, 'least_squares', 4, 0.7, RowCheck, ColumnCheck, AngleCheck, Score)
count_seconds (S2)
dev_display (ImageCheck)
if (|Score| > 0)
dev_set_color ('green')
hom_mat2d_identity (HomMat2DIdentity)
hom_mat2d_translate (HomMat2DIdentity, RowCheck, ColumnCheck, HomMat2DTranslate)
hom_mat2d_rotate (HomMat2DTranslate, AngleCheck, RowCheck, ColumnCheck, HomMat2DRotate)
affine_trans_contour_xld (ShapeModel, ShapeModelTrans, HomMat2DRotate)
dev_display (ShapeModelTrans)
// 针对像素点进行仿射变换,参数(放射矩阵,像素行列坐标,输出变换后的行列坐标)
affine_trans_pixel (HomMat2DRotate, Rect1Row, Rect1Col, Rect1RowCheck, Rect1ColCheck)
affine_trans_pixel (HomMat2DRotate, Rect2Row, Rect2Col, Rect2RowCheck, Rect2ColCheck)
gen_rectangle2 (Rectangle1Check, Rect1RowCheck, Rect1ColCheck, AngleCheck, RectLength1, RectLength2)
gen_rectangle2 (Rectangle2Check, Rect2RowCheck, Rect2ColCheck, AngleCheck, RectLength1, RectLength2)
dev_set_color ('blue')
dev_set_draw ('margin')
dev_set_line_width (3)
dev_display (Rectangle1Check)
dev_display (Rectangle2Check)
dev_set_draw ('fill')
count_seconds (S3)
// 根据上述的测量矩形区域经过变换后得到仿射结果区域,然后测量该区域的边缘距离
gen_measure_rectangle2 (Rect1RowCheck, Rect1ColCheck, AngleCheck, RectLength1, RectLength2, Width, Height, 'bilinear', MeasureHandle1)
gen_measure_rectangle2 (Rect2RowCheck, Rect2ColCheck, AngleCheck, RectLength1, RectLength2, Width, Height, 'bilinear', MeasureHandle2)
measure_pairs (ImageCheck, MeasureHandle1, 2, 90, 'positive', 'all', RowEdgeFirst1, ColumnEdgeFirst1, AmplitudeFirst1, RowEdgeSecond1, ColumnEdgeSecond1, AmplitudeSecond1, IntraDistance1, InterDistance1)
measure_pairs (ImageCheck, MeasureHandle2, 2, 90, 'positive', 'all', RowEdgeFirst2, ColumnEdgeFirst2, AmplitudeFirst2, RowEdgeSecond2, ColumnEdgeSecond2, AmplitudeSecond2, IntraDistance2, InterDistance2)
close_measure (MeasureHandle1)
close_measure (MeasureHandle2)
count_seconds (S4)
// 显示结果
dev_set_color ('red')
disp_line (WindowHandle, RowEdgeFirst1 - RectLength2 * cos(AngleCheck), ColumnEdgeFirst1 - RectLength2 * sin(AngleCheck), RowEdgeFirst1 + RectLength2 * cos(AngleCheck), ColumnEdgeFirst1 + RectLength2 * sin(AngleCheck))
disp_line (WindowHandle, RowEdgeSecond1 - RectLength2 * cos(AngleCheck), ColumnEdgeSecond1 - RectLength2 * sin(AngleCheck), RowEdgeSecond1 + RectLength2 * cos(AngleCheck), ColumnEdgeSecond1 + RectLength2 * sin(AngleCheck))
disp_line (WindowHandle, RowEdgeFirst2 - RectLength2 * cos(AngleCheck), ColumnEdgeFirst2 - RectLength2 * sin(AngleCheck), RowEdgeFirst2 + RectLength2 * cos(AngleCheck), ColumnEdgeFirst2 + RectLength2 * sin(AngleCheck))
disp_line (WindowHandle, RowEdgeSecond2 - RectLength2 * cos(AngleCheck), ColumnEdgeSecond2 - RectLength2 * sin(AngleCheck), RowEdgeSecond2 + RectLength2 * cos(AngleCheck), ColumnEdgeSecond2 + RectLength2 * sin(AngleCheck))
dev_set_line_width (1)
NumLeads := |IntraDistance1| + |IntraDistance2|
MinDistance := min([InterDistance1,InterDistance2])
dev_set_window (WindowHandleText)
dev_set_part (0, 0, 119, Width - 1)
dev_clear_window ()
disp_message (WindowHandleText, 'Matching: Time: ' + ((S2 - S1) * 1000)$'5.2f' + 'ms , Score: ' + Score$'7.5f', 'image', 20, 20, 'green', 'false')
disp_message (WindowHandleText, 'Measure: Time: ' + ((S4 - S3) * 1000)$'5.2f' + ' ms, Num. leads: ' + NumLeads$'2d', 'image', 50, 20, 'red', 'false')
disp_message (WindowHandleText, ' Min. lead dist: ' + MinDistance$'6.3f', 'image', 80, 20, 'red', 'false')
endif
dev_error_var (Error, 1)
dev_set_check ('~give_error')
get_mposition (WindowHandle, R, C, Button)
dev_error_var (Error, 0)
dev_set_check ('give_error')
if (Error != H_MSG_TRUE)
Button := 0
endif
endwhile
dev_set_window (WindowHandleText)
dev_close_window ()
clear_shape_model (ModelID)
close_framegrabber (FGHandle)
第四节 测量弧形
一般步骤:
- 采集图像
- 标定图像
- 畸变矫正
- 测量
- 创建测量弧形
- 像素距离转换成实际距离(根据内参外参)
- 显示结果
需要测量的目标是个弧形,或者非直线,如下图
测量弧形结果
实例一(创建弧形)
// measure_arc.hdev
read_image (Zeiss1, 'zeiss1')
get_image_size (Zeiss1, Width, Height)
dev_close_window ()
dev_open_window (0, 0, Width / 2, Height / 2, 'black', WindowHandle)
set_display_font (WindowHandle, 14, 'mono', 'true', 'false')
dev_display (Zeiss1)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
// 画一个圆圈或者写入参数
* draw_circle (WindowHandle, Row, Column, Radius)
Row := 275
Column := 335
Radius := 107
AngleStart := -rad(55)
AngleExtent := rad(170)
dev_set_draw ('fill')
dev_set_color ('green')
dev_set_line_width (1)
// 获得椭圆上的一个点
// get_points_ellipse(get_points_ellipse( : : Angle=上图中的AgnleStart+AngleExtent, Row, Column椭圆中心点行列坐标, Phi椭圆长轴和水平的夹角, Radius1椭圆长半轴2 : RowPoint, ColPoint输出椭圆在Angle角度上的一个点)
get_points_ellipse (AngleStart + AngleExtent, Row, Column, 0, Radius, Radius, RowPoint, ColPoint)
// 显示一段圆弧,参数(窗口句柄,中心点行列坐标,角度范围,起点行列坐标)根据起点行列坐标再根据角度范围方向来绘制圆弧)
disp_arc (WindowHandle, Row, Column, AngleExtent, RowPoint, ColPoint)
dev_set_line_width (3)
//生成一个测量弧形
// gen_measure_arc( : : CenterRow, CenterCol, Radius, AngleStart, AngleExtent, AnnulusRadius见上图, Width, Height图片宽高, Interpolation 差值算法: MeasureHandle输出测量结果句柄)
gen_measure_arc (Row, Column, Radius, AngleStart, AngleExtent, 10, Width, Height, 'nearest_neighbor', MeasureHandle)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
count_seconds (Seconds1)
n := 10
for i := 1 to n by 1
// 开始测量弧形
// measure_pos(Image : : MeasureHandle测量弧形句柄, Sigma平滑参数, Threshold最小边缘幅度即梯度值, Transition是正还是负边缘, Select选择哪些边缘 : RowEdge, ColumnEdge输出边缘结果的行列坐标, Amplitude边缘幅度, Distance在一个测量弧形里面,各边缘之间的连续距离)
// ⚠️Distance注意两点,1⃣️在一个测量矩形内,2⃣️连续边缘之间的连续距离,即不是两个连续边缘的中心点直线距离,注意区别
measure_pos (Zeiss1, MeasureHandle, 1, 10, 'all', 'all', RowEdge, ColumnEdge, Amplitude, Distance)
endfor
count_seconds (Seconds2)
Time := (Seconds2 - Seconds1) / n
disp_continue_message (WindowHandle, 'black', 'true')
* stop ()
// pp表示两个点,即求两个点之间的距离
distance_pp (RowEdge[1], ColumnEdge[1], RowEdge[2], ColumnEdge[2], IntermedDist)
* dev_display (Zeiss1)
dev_set_color ('red')
* disp_circle (WindowHandle, RowEdge, ColumnEdge, RowEdge - RowEdge + 1)
disp_line (WindowHandle, RowEdge[1], ColumnEdge[1], RowEdge[2], ColumnEdge[2])
dev_set_color ('yellow')
disp_message (WindowHandle, 'Distance: ' + IntermedDist, 'image', 250, 80, 'yellow', 'false')
* dump_window (WindowHandle, 'tiff_rgb', 'C:\\Temp\\zeiss_result')
close_measure (MeasureHandle)
dev_set_line_width (1)
* disp_continue_message (WindowHandle, 'black', 'true')
stop ()
dev_clear_window ()
第四章-边缘
前言
在Halcon的例子中,有方法-边缘提取(像素精度)
边缘原理:参考DIP课本
区域和轮廓是没有像素值的,而图像是有像素值的,
在边缘检测中,有High阈值和Low阈值,像素值高于High时一定是边缘像素,低于Low一定不是边缘像素,若像素值在High和Low之间时,若该像素值的8邻域的像素有一个高于High,则该像素就判为边缘像素
Region区域和XLD轮廓只保存了位置信息,没有像素值信息
图像中提取边缘
edges_image(Image输入图像,
ImaAmp输出边缘幅度结果图像,
ImaDir输出边缘方向结果图像,
Filter边缘检测器一般是‘canny’,或者是lanser1一阶导,lanser2二阶导
Alpha平滑参数,越大越不平滑
NMS,
Low阈值,
High)
上述的ImaAmp和ImaDir都是图像,即有像素值,我们用threshold得到的区域或者xld轮廓是没有像素值的
一阶导数算子NMS
注意这里的输出都是图片,即有像素值
read_image (Image, 'fabrik')
// dir表示方向,amp表示幅度
//sobel_dir()可以产生边缘方向图和边缘幅度差分图
//sobel_amp()只能得到幅度图
//prewitt()同理
read_image (Image, 'fabrik')
sobel_dir(Image, EdgeAmplitude_sbbel_dir, EdgeDirection_sobel_dir, 'sum_abs', 3)
sobel_amp(Image,EdgeAmplitude_sobel_amp, 'sum_abs', 3)
prewitt_amp (Image, ImageEdgeAmp_pre_amp)
prewitt_dir (Image, ImageEdgeAmp_pre_dir, ImageEdgeDir_pre_dir)
//非极大值抑制算子
nonmax_suppression_dir(ImgAmp, ImgDir,ImageResult,Mode)
nonmax_suppression_amp(ImgAmp,ImageResult,Mode)示例
使用了skeleton骨架,作用就是提取区域骨架,效果就是区域框架更细了,可以通过skeleton转换成contours轮廓
骨架往往是一个连通域,而轮廓是多个连通域
skeleton是一个连通域
contours是多个不同的连通域
read_image (Image, 'fabrik')
edges_image (Image, ImaAmp, ImaDir, 'lanser2', 0.5, 'nms', 12, 22)
threshold (ImaAmp, Edges, 1, 255)
//提取骨架
skeleton (Edges, Skeleton)
// 骨架转换成xld亚像素轮廓
gen_contours_skeleton_xld (Skeleton, Contours, 1, 'filter')
dev_display (Image)
dev_set_colored (6)
dev_display (Contours)
区域中提取边缘
区域无像素值
该方法是基于形态学操作产生的,其中边界类型包括了外边界和内边界,
内边界=原区域-腐蚀 外边界=膨胀-原区域
boundary(Region输入区域,
RegionBorder输出区域边界,
BoundaryType边界的类型‘inner’或‘outer’)
第五章-拟合
拟合
所有fit开头的算子都是和拟合相关的算子
一般测量时往往是通过blob分析获得区域然后设置测量矩形,或者通过助手直接设置测量矩形,这里提供了一种特殊的方法就是直接把目标进行拟合成几何图形,然后根据几何图形的参数来计算测量距离
拟合流程:
- 采集图像,
- 预处理,
- 边缘提取,
- 轮廓联合分割
- 拟合,获得几何参数
- 根据拟合结果求去目标距离
Halcon的拟合示例在ctrl+E中的方法:边缘提取(亚像素)&几何测量中
通过拟合,你可以获得相应图形的几何表示,更方便求距离,面积,等参数,进而根据标定板或者像素尺寸转换来获得实际测量距离
轮廓联合与分割
示例1:拟合圆
拟合3个圆圈⭕️,提取圆的一段轮廓
拟合结果如图中的弧线
* circles.hdev
* The edges in the image are segmented into lines and circles.
* For the edges that are part of a circle, the circle parameters
* are estimated and the resulting circle is displayed.
read_image (Image, 'double_circle')
*
* Init window
dev_close_window ()
get_image_size (Image, Width, Height)
dev_open_window (0, 0, Width, Height, 'black', WindowHandle)
*
* Segment a region containing the edges
fast_threshold (Image, Region, 0, 120, 7)
// 对区域求边缘
boundary (Region, RegionBorder, 'inner')
//裁剪区域
clip_region_rel (RegionBorder, RegionClipped, 5, 5, 5, 5)
dilation_circle (RegionClipped, RegionDilation, 2.5)
reduce_domain (Image, RegionDilation, ImageReduced)
*
* In the subdomain of the image containing the edges,
* extract subpixel precise edges.
edges_sub_pix (ImageReduced, Edges, 'canny', 2, 20, 60)
segment_contours_xld (Edges, ContoursSplit, 'lines_circles', 5, 4, 3)
count_obj (ContoursSplit, Number)
dev_display (Image)
dev_set_draw ('margin')
dev_set_color ('white')
dev_update_window ('off')
for I := 1 to Number by 1
select_obj (ContoursSplit, ObjectSelected, I)
get_contour_global_attrib_xld (ObjectSelected, 'cont_approx', Attrib)
* Fit a circle to the line segment that are arcs of a circle
if (Attrib > 0)
fit_circle_contour_xld (ObjectSelected, 'ahuber', -1, 2, 0, 3, 2, Row, Column, Radius, StartPhi, EndPhi, PointOrder)
gen_circle_contour_xld (ContCircle, Row, Column, Radius, 0, rad(360), 'positive', 1.0)
dev_display (ContCircle)
endif
endfor
dev_set_colored (12)
dev_set_line_width (3)
dev_display (ContoursSplit)
步骤分析
1. blob分析,获得目标区域
read_image (Image, 'double_circle')
*
* Init window
dev_close_window ()
get_image_size (Image, Width, Height)
dev_open_window (0, 0, Width, Height, 'black', WindowHandle)
*
* Segment a region containing the edges
fast_threshold (Image, Region, 0, 120, 7)
// 对区域求边缘
boundary (Region, RegionBorder, 'inner')
//裁剪区域
clip_region_rel (RegionBorder, RegionClipped, 5, 5, 5, 5)
dilation_circle (RegionClipped, RegionDilation, 2.5)
reduce_domain (Image, RegionDilation, ImageReduced)算子简介
clip_region_rel(Region输入区域,
RegionClipped输入裁剪结果,
Top,
Bottom,
Left,
Right)
2. 亚像素边缘提取
edges_sub_pix (ImageReduced目标区域图片,
Edges输出亚像素边缘轮廓,
'canny'边缘检测算法,
2, 平滑系数(canny算子平滑系数越大越平滑,其他相反)
20, 边缘差分阈值
60)
3. 轮廓分割
上述轮廓需要拟合成三个圆,
/segment_contours_xld(Contours输入轮廓,
ContoursSplit输出分割的轮廓,
Mode分割模式, 即你想要分割成直线lines还是几个圆lines_circles或者椭圆lines_ellipses
SmoothCont, 用来平滑轮廓的点数
MaxLineDist1, 第一次迭代轮廓到多边形的距离阈值
MaxLineDist2第二次迭代轮廓到多边形的距离阈值)
/
segment_contours_xld (Edges, ContoursSplit, 'lines_circles', 5, 4, 3)
4. 判断拟合成什么
别忘了图形数组是从下标1开始的
拟合的目的就是获得拟合得到的几何对象的参数,比如圆心,半径,长度等
// 计算圆弧的数量
count_obj (ContoursSplit, Number)
dev_display (Image)
dev_set_draw ('margin')
dev_set_color ('white')
dev_update_window ('off')
for I := 1 to Number by 1
//选择图形算子
select_obj (ContoursSplit, ObjectSelected, I)
//自动判断拟合对象算子,你不知道到底拟合成什么比较好,所以这个算子就是告诉你答案的
//Attrib一般是0,-1或者1, 1是圆,
/get_contour_global_attrib_xld (ObjectSelected输入轮廓,
'cont_approx'判断算法,
Attrib输出结果)
/
get_contour_global_attrib_xld (ObjectSelected, 'cont_approx', Attrib)
5. 开始拟合圆,圆弧
* Fit a circle to the line segment that are arcs of a circle
if (Attrib > 0)
/拟合圆的算子
/*fit_circle_contour_xld(Contours,输入轮廓
Algorithm,拟合算法,如最小二乘法
MaxNumPoints,拟合时采用轮廓最多的点数,-1表示全部参与
MaxClosureDist,是指Maximum distance between the end points of a contour to be considered as 'closed'用来区分拟合成圆还是圆弧 ,就是要拟合的最后一个点和被认为把轮廓封闭的点,这两个点距离
ClippingEndPoints,拟合时忽略轮廓开始和结束点的数量
Iterations,拟合迭代的最大次数
ClippingFactor,离群值的裁剪因子,越小离群值越多
Row, 输出圆的坐标
Column,
Radius,输出圆半径
StartPhi,输出圆开始角度
EndPhi,输出圆终止角度
PointOrder边界点正反向)
*/
fit_circle_contour_xld (ObjectSelected, 'ahuber', -1, 2, 0, 3, 2, Row, Column, Radius, StartPhi, EndPhi, PointOrder)
//根据拟合结果生成拟合的几何图形
gen_circle_contour_xld (ContCircle, Row, Column, Radius, 0, rad(360), 'positive', 1.0)
dev_display (ContCircle)
endif
endfor
dev_set_colored (12)
dev_set_line_width (3)
dev_display (ContoursSplit)
示例2:拟合直线
* measure_metal_part.hdev: inspects metal part by fitting lines and circles
*
dev_close_window ()
dev_update_window ('off')
* ****
* step: acquire image
* ****
read_image (Image, 'metal-parts/metal-parts-01')
get_image_size (Image, Width, Height)
dev_open_window_fit_image (Image, 0, 0, Width, Width, WindowID)
set_display_font (WindowID, 14, 'mono', 'true', 'false')
dev_set_draw ('margin')
dev_set_line_width (3)
dev_display (Image)
disp_continue_message (WindowID, 'black', 'true')
stop ()
* ****
* step: create contours
* ****
edges_sub_pix (Image, Edges, 'lanser2', 0.5, 40, 90)
dev_display (Edges)
disp_continue_message (WindowID, 'black', 'true')
stop ()
* ****
* step: process contours
* ****
segment_contours_xld (Edges, ContoursSplit, 'lines_circles', 6, 4, 4)
sort_contours_xld (ContoursSplit, SortedContours, 'upper_left', 'true', 'column')
dev_clear_window ()
dev_set_colored (12)
dev_display (SortedContours)
disp_continue_message (WindowID, 'black', 'true')
stop ()
* ****
* step: perform fitting
* ****
ROI := [115,225,395,535]
dev_open_window (0, round(Width / 2), (ROI[3] - ROI[1]) * 2, (ROI[2] - ROI[0]) * 2, 'black', WindowHandleZoom)
dev_set_part (round(ROI[0]), round(ROI[1]), round(ROI[2]), round(ROI[3]))
set_display_font (WindowHandleZoom, 14, 'mono', 'true', 'false')
count_obj (SortedContours, NumSegments)
dev_display (Image)
NumCircles := 0
NumLines := 0
for i := 1 to NumSegments by 1
select_obj (SortedContours, SingleSegment, i)
get_contour_global_attrib_xld (SingleSegment, 'cont_approx', Attrib)
if (Attrib == 1)
NumCircles := NumCircles + 1
fit_circle_contour_xld (SingleSegment, 'atukey', -1, 2, 0, 5, 2, Row, Column, Radius, StartPhi, EndPhi, PointOrder)
gen_ellipse_contour_xld (ContEllipse, Row, Column, 0, Radius, Radius, 0, rad(360), 'positive', 1.0)
dev_set_color ('white')
dev_display (ContEllipse)
set_tposition (WindowHandleZoom, Row - Radius - 10, Column)
write_string (WindowHandleZoom, 'C' + NumCircles)
ResultText := 'C' + NumCircles + ': radius = ' + Radius
else
NumLines := NumLines + 1
fit_line_contour_xld (SingleSegment, 'tukey', -1, 0, 5, 2, RowBegin, ColBegin, RowEnd, ColEnd, Nr, Nc, Dist)
gen_contour_polygon_xld (Line, [RowBegin,RowEnd], [ColBegin,ColEnd])
dev_set_color ('yellow')
dev_display (Line)
distance_pp (RowBegin, ColBegin, RowEnd, ColEnd, Length)
set_tposition (WindowHandleZoom, (RowBegin + RowEnd) / 2 - Nr * 10, (ColBegin + ColEnd) / 2)
write_string (WindowHandleZoom, 'L' + NumLines)
ResultText := 'L' + NumLines + ': length = ' + Length
endif
set_tposition (WindowHandleZoom, 275 + i * 10, 230)
write_string (WindowHandleZoom, ResultText)
endfor
disp_continue_message (WindowID, 'black', 'true')
stop ()
dev_set_window (WindowHandleZoom)
dev_close_window ()
dev_clear_window ()
轮廓排序
sort_contours_xld (ContoursSplit, SortedContours, 'upper_left', 'true', 'column')
拟合直线
/*fit_line_contour_xld(Contours,输入轮廓
Algorithm,拟合算法
MaxNumPoints,最大参与拟合点数量
ClippingEndPoints, 拟合时忽略轮廓开始和结束点的数量
Iterations, 迭代次数
ClippingFactor,离群值因子
RowBegin, 直线起点坐标
ColBegin,
RowEnd, 直线重点坐标
ColEnd,
Nr, 直线的normal vector法向量的row坐标
Nc, 直线的normal vector法向量的column坐标
Dist直线延长线和原点0,0的垂直距离)
*/
fit_line_contour_xld (SingleSegment, 'tukey', -1, 0, 5, 2, RowBegin, ColBegin, RowEnd, ColEnd, Nr, Nc, Dist)轮廓联合
联合主要是联合三种,一种是共线,一种是邻近,一种是共圆
union_adjacent_contours_xld(Contours : UnionContours : MaxDistAbs, MaxDistRel, Mode : )
union_cocircular_contours_xld(Contours : UnionContours : MaxArcAngleDiff, MaxArcOverlap, MaxTangentAngle, MaxDist, MaxRadiusDiff, MaxCenterDist, MergeSmallContours, Iterations : )
union_collinear_contours_xld(Contours : UnionContours : MaxDistAbs, MaxDistRel, MaxShift, MaxAngle, Mode : )
union_collinear_contours_ext_xld(Contours : UnionContours : MaxDistAbs, MaxDistRel, MaxShift, MaxAngle, MaxOverlap, MaxRegrError, MaxCosts, WeightDist, WeightShift, WeightAngle, WeightLink, WeightRegr, Mode : )
上述都是在把轮廓分割后然后对分割的小轮廓的基础上进行拟合的
这里介绍轮廓如何联合
* close_contour_gaps.hdev: closes gaps in extracted straight contours
*
dev_close_window ()
dev_update_window ('off')
* ****
* step: create synthetic image
* ****
gen_rectangle1 (Rectangle, 30, 20, 100, 100)
// 转换区域成二值图像,注意这一并非只有0和1,可以是任意两个灰度值,只要只存在两种就行
/*region_to_bin(Region输入要转换的区域,
BinImage输出二值图像,
ForegroundGray前景灰度值,
BackgroundGray,背景灰度值
Width, Height输出宽高 )
*/
region_to_bin (Rectangle, BinImage, 130, 100, 120, 130)
/裁剪一个矩形出来,以前的reduce_domain()是裁剪一个固定的区域,这里的是裁剪一个设计好的矩形
/
rectangle1_domain (BinImage, ImageReduced, 20, 48, 40, 52)
// 均值滤波算子(输入图片,kernel,kernel尺寸)
mean_image (ImageReduced, SmoothedImage, 15, 15)
paint_gray (SmoothedImage, BinImage, Image)
get_image_size (Image, Width, Height)
dev_open_window_fit_image (Image, 0, 0, Width, Height, WindowID)
dev_set_draw ('margin')
dev_set_line_width (3)
dev_display (Image)
stop ()
* ****
* step: create contours
* ****
rectangle1_domain (Image, ImageReduced, 5, 5, 125, 115)
// 输出xld亚像素轮廓,注意是轮廓没有像素值哦
edges_sub_pix (ImageReduced, Edges, 'lanser2', 1.1, 22, 30)
dev_display (Image)
dev_display (Edges)
stop ()
* ****
* step: process contours
* ****
// 分割边缘
segment_contours_xld (Edges, LineSegments, 'lines', 5, 4, 2)
//计算回归线到XLD轮廓线的参数
/regress_contours_xld(Contours带拟合轮廓,
RegressContours输出xld轮廓,
Mode离群值处理方法,
Iterations迭代拟合次数)
/
regress_contours_xld (LineSegments, RegressContours, 'no', 1)
// 联合共线轮廓
/union_collinear_contours_xld(Contours输入轮廓,
UnionContours输出合并轮廓,
MaxDistAbs两个轮廓之间的绝对距离,这里参考帮助窗口
MaxDistRel,
MaxShift,
MaxAngle,
Mode)
/
union_collinear_contours_xld (RegressContours, UnionContours, 10, 1, 2, 0.1, 'attr_keep')
//轮廓排序
sort_contours_xld (UnionContours, SortedContours, 'upper_left', 'true', 'column')
dev_display (Image)
colored_display (SortedContours, ['yellow','white','white','yellow'])
dev_update_window ('on')
测量拟合实战一
利用测量助手
* Measure 05: Code generated by Measure 05
*利用测量助手工具实现
*1采集图像
dev_close_window ()
read_image (Image, 'D://gongjian.bmp')
get_image_size (Image, Width, Height)
dev_open_window (0, 0, Width / 2, Height / 2, 'black', WindowHandle)
set_display_font (WindowHandle, 14, 'mono', 'true', 'false')
dev_display (Image)
disp_continue_message (WindowHandle, 'black', 'true')
AmplitudeThreshold := 40
RoiWidthLen2 := 28.5
set_system ('int_zooming', 'true')
* Measure 05: Coordinates for line Measure 05 [0]
LineRowStart_Measure_05_0 := 263.191
LineColumnStart_Measure_05_0 := 63.9394
LineRowEnd_Measure_05_0 := 283.792
LineColumnEnd_Measure_05_0 := 1036.39
* Measure 05: Convert coordinates to rectangle2 type
TmpCtrl_Row := 0.5*(LineRowStart_Measure_05_0+LineRowEnd_Measure_05_0)
TmpCtrl_Column := 0.5*(LineColumnStart_Measure_05_0+LineColumnEnd_Measure_05_0)
TmpCtrl_Dr := LineRowStart_Measure_05_0-LineRowEnd_Measure_05_0
TmpCtrl_Dc := LineColumnEnd_Measure_05_0-LineColumnStart_Measure_05_0
TmpCtrl_Phi := atan2(TmpCtrl_Dr, TmpCtrl_Dc)
TmpCtrl_Len1 := 0.5*sqrt(TmpCtrl_Dr*TmpCtrl_Dr + TmpCtrl_Dc*TmpCtrl_Dc)
TmpCtrl_Len2 := RoiWidthLen2
* Measure 05: Create measure for line Measure 05 [0]
* Measure 05: Attention: This assumes all images have the same size!
gen_measure_rectangle2 (TmpCtrl_Row, TmpCtrl_Column, TmpCtrl_Phi, TmpCtrl_Len1, TmpCtrl_Len2, 1280, 960, 'nearest_neighbor', MsrHandle_Measure_05_0)
* Measure 05: Coordinates for line Measure 05 [1]
LineRowStart_Measure_05_1 := 164.307
LineColumnStart_Measure_05_1 := 529.66
LineRowEnd_Measure_05_1 := 559.843
LineColumnEnd_Measure_05_1 := 506.228
* Measure 05: Convert coordinates to rectangle2 type
TmpCtrl_Row := 0.5*(LineRowStart_Measure_05_1+LineRowEnd_Measure_05_1)
TmpCtrl_Column := 0.5*(LineColumnStart_Measure_05_1+LineColumnEnd_Measure_05_1)
TmpCtrl_Dr := LineRowStart_Measure_05_1-LineRowEnd_Measure_05_1
TmpCtrl_Dc := LineColumnEnd_Measure_05_1-LineColumnStart_Measure_05_1
TmpCtrl_Phi := atan2(TmpCtrl_Dr, TmpCtrl_Dc)
TmpCtrl_Len1 := 0.5*sqrt(TmpCtrl_Dr*TmpCtrl_Dr + TmpCtrl_Dc*TmpCtrl_Dc)
TmpCtrl_Len2 := RoiWidthLen2
* Measure 05: Create measure for line Measure 05 [1]
* Measure 05: Attention: This assumes all images have the same size!
gen_measure_rectangle2 (TmpCtrl_Row, TmpCtrl_Column, TmpCtrl_Phi, TmpCtrl_Len1, TmpCtrl_Len2, 1280, 960, 'nearest_neighbor', MsrHandle_Measure_05_1)
* Measure 05: Coordinates for line Measure 05 [2]
LineRowStart_Measure_05_2 := 364.135
LineColumnStart_Measure_05_2 := 664.397
LineRowEnd_Measure_05_2 := 380.616
LineColumnEnd_Measure_05_2 := 892.864
* Measure 05: Convert coordinates to rectangle2 type
TmpCtrl_Row := 0.5*(LineRowStart_Measure_05_2+LineRowEnd_Measure_05_2)
TmpCtrl_Column := 0.5*(LineColumnStart_Measure_05_2+LineColumnEnd_Measure_05_2)
TmpCtrl_Dr := LineRowStart_Measure_05_2-LineRowEnd_Measure_05_2
TmpCtrl_Dc := LineColumnEnd_Measure_05_2-LineColumnStart_Measure_05_2
TmpCtrl_Phi := atan2(TmpCtrl_Dr, TmpCtrl_Dc)
TmpCtrl_Len1 := 0.5*sqrt(TmpCtrl_Dr*TmpCtrl_Dr + TmpCtrl_Dc*TmpCtrl_Dc)
TmpCtrl_Len2 := RoiWidthLen2
* Measure 05: Create measure for line Measure 05 [2]
* Measure 05: Attention: This assumes all images have the same size!
gen_measure_rectangle2 (TmpCtrl_Row, TmpCtrl_Column, TmpCtrl_Phi, TmpCtrl_Len1, TmpCtrl_Len2, 1280, 960, 'nearest_neighbor', MsrHandle_Measure_05_2)
* Measure 05: ***************************************************************
* Measure 05: * The code which follows is to be executed once / measurement *
* Measure 05: ***************************************************************
* Measure 05: The image is assumed to be made available in the
* Measure 05: variable last displayed in the graphics window
copy_obj (Image, Image, 1, 1)
* Measure 05: Execute measurements
measure_pairs (Image, MsrHandle_Measure_05_0, 1, 40, 'all', 'all', Row1_Measure_05_0, Column1_Measure_05_0, Amplitude1_Measure_05_0, Row2_Measure_05_0, Column2_Measure_05_0, Amplitude2_Measure_05_0, Width_Measure_05_0, Distance_Measure_05_0)
measure_pairs (Image, MsrHandle_Measure_05_1, 1, 40, 'all', 'all', Row1_Measure_05_1, Column1_Measure_05_1, Amplitude1_Measure_05_1, Row2_Measure_05_1, Column2_Measure_05_1, Amplitude2_Measure_05_1, Width_Measure_05_1, Distance_Measure_05_1)
measure_pairs (Image, MsrHandle_Measure_05_2, 1, 40, 'all', 'all', Row1_Measure_05_2, Column1_Measure_05_2, Amplitude1_Measure_05_2, Row2_Measure_05_2, Column2_Measure_05_2, Amplitude2_Measure_05_2, Width_Measure_05_2, Distance_Measure_05_2)
* Measure 05: Do something with the results
* Measure 05: Clear measure when done
disp_line (WindowHandle, Row1_Measure_05_0, Column1_Measure_05_0, Row2_Measure_05_0,Column2_Measure_05_0)
* 当前图形窗口右键菜单设置:
* 图形窗口右键菜单设置结束
disp_line (WindowHandle, Row1_Measure_05_1, Column1_Measure_05_1, Row2_Measure_05_1,Column2_Measure_05_1)
disp_line (WindowHandle, Row1_Measure_05_2, Column1_Measure_05_2, Row2_Measure_05_2,Column2_Measure_05_2)
m_width1:=Column2_Measure_05_0-Column1_Measure_05_0
m_width2:=Row2_Measure_05_1-Row1_Measure_05_1
m_width3:=Column2_Measure_05_2-Column1_Measure_05_2
disp_message (WindowHandle, '长 ' + m_width1, 'image', 0, 0, 'red', 'false')
disp_message (WindowHandle, ' 宽 ' + m_width2, 'image', 100, 0, 'red', 'false')
disp_message (WindowHandle, '直径 ' + m_width3, 'image', 200, 0, 'red', 'false')
close_measure (MsrHandle_Measure_05_0)
close_measure (MsrHandle_Measure_05_1)
close_measure (MsrHandle_Measure_05_2)
利用拟合方法
//1. 图片预处理
*识别拟合求距离
rgb1_to_gray(Image,Imagegray)
*预处理（定位）
dev_set_draw ('fill')
*threshold (Imagegray, Region, 100, 255)
threshold (Imagegray, Regions, 0, 114)
connection(Regions,Regionsconnect)
select_shape (Regionsconnect, SelectedRegions, 'area', 'and', 207798, 275688)
area_center (SelectedRegions, AreaRegion, RowCenterRegion, ColumnCenterRegion)
orientation_region (SelectedRegions, OrientationRegion)
//进行显示
dev_display (SelectedRegions)
dev_display (Image)
disp_message (WindowHandle, 'Center Row: ' + RowCenterRegion$'.5', 'window', 20, 10, 'blue', 'false')
disp_message (WindowHandle, 'Area: ' + AreaRegion + ' pixel', 'window', 20, 300, 'blue', 'false')
disp_message (WindowHandle, 'Center Column: ' + ColumnCenterRegion$'.5', 'window', 60, 10, 'blue', 'false')
disp_message (WindowHandle, 'Orientation: ' + OrientationRegion$'.3' + ' rad', 'window', 60, 300, 'blue', 'false')
disp_cross (WindowHandle, RowCenterRegion, ColumnCenterRegion, 15, 0)
disp_arrow (WindowHandle, RowCenterRegion, ColumnCenterRegion, RowCenterRegion - 60 * sin(OrientationRegion), ColumnCenterRegion + 60 * cos(OrientationRegion), 2)
*reduce_domain (Imagegray, SelectedRegions, ImageReduced)
*select_shape_std (ConnectedRegions, RectangleRegions, 'rectangle2', 80)
*intersection_lines
//2. 提取边缘并拟合
//2.1 提取目标垂直边缘
edges_sub_pix (Imagegray, Edges, 'canny', 0.6, 20, 40)
// 选择需要的xld轮廓
// 这里往往是利用特征直方图工具,选择特征选项来插入代码,需要选择xld_width,xld_height
select_shape_xld (Edges, SelectedXLD, ['height','width'], 'and', [100,600], [500,1000])
segment_contours_xld(SelectedXLD, ContoursSplit, 'lines_circles', 5, 4, 2)
dev_set_colored (12)
select_shape_xld (ContoursSplit, SelectedXLD1, ['height','width'], 'and', [200,0], [400,30])
sort_contours_xld(SelectedXLD1, SortedContour1, 'character', 'true', 'row')
fit_line_contour_xld(SortedContour1, 'tukey', -1, 0, 5, 2, RowBegin0, ColBegin0, RowEnd0, ColEnd0, Nr, Nc, Dist)
gen_contour_polygon_xld(line1, [RowBegin0[0],RowEnd0[0]], [ColBegin0[0],ColEnd0[0]])
gen_contour_polygon_xld(line2, [RowBegin0[1],RowEnd0[1]], [ColBegin0[1],ColEnd0[1]])
//2.2提取目标水平边缘
//也是通过特征检测工具得到的
select_shape_xld (ContoursSplit, SelectedXLD1_2, ['height','width'], 'and', [0,600], [50,1000])
sort_contours_xld(SelectedXLD1_2, SortedContour2, 'character', 'true', 'row')
fit_line_contour_xld(SortedContour2, 'tukey', -1, 0, 5, 2, RowBegin1, ColBegin1, RowEnd1, ColEnd1, Nr1, Nc1, Dist1)
gen_contour_polygon_xld(line3, [RowBegin1[0],RowEnd1[0]], [ColBegin1[0],ColEnd1[0]])
gen_contour_polygon_xld(line4, [RowBegin1[1],RowEnd1[1]], [ColBegin1[1],ColEnd1[1]])
//2.3计算距离
distance_pl(RowBegin0[0], ColBegin0[0], RowBegin0[1], ColBegin0[1], RowEnd0[1], ColEnd0[1], Distance1)
disp_message (WindowHandle, '竖直边的距离 ' +Distance1, 'window',300, 10, 'blue', 'false')
distance_pl(RowBegin1[0], ColBegin1[0], RowBegin1[1], ColBegin1[1], RowEnd1[1], ColEnd1[1], Distance1)
disp_message (WindowHandle, '水平边的距离 ' +Distance1, 'window',360, 10, 'blue', 'false')
//2.4提取目标内部圆形边缘并计算半径等距离
edges_sub_pix(Imagegray, Edges3, 'canny', 1, 20, 30)
*select_shape_xld (Edges3, SelectedXLD2, 'width', 'and', 34.86, 640.37)
select_shape_xld (Edges3, SelectedXLD2, ['width','circularity'], 'and', [34.86,0.12752], [640.37,1])
* select_shape_xld (Edges3, SelectedXLD3, ['width','circularity'], 'and', [0,0.35], [120,0.5])
union_cocircular_contours_xld (SelectedXLD2, UnionContours, 1.6, 0.9, 0.9, 30, 40, 40, 'true', 1)
fit_circle_contour_xld(UnionContours, 'algebraic', -1, 0, 0, 3, 2, Row1, Column1, Radius1, StartPhi1, EndPhi1, PointOrder1)
gen_circle_contour_xld(ContCircle1, Row1, Column1, Radius1, 0, 6.28318, 'positive', 1)
distance_pl(Row1, Column1, RowBegin0[0], ColBegin0[0], RowEnd0[0], ColEnd0[0], DistanceR_left)
disp_message (WindowHandle, '圆心到左边直线的距离 ' +DistanceR_left+'半径'+Radius1, 'window',400, 10, 'blue', 'false')
distance_pp(Row1, Column1, RowEnd1[0], ColEnd1[0],DistanceR_upendpoint)
angle_ll(RowBegin0[1],ColBegin0[1],RowEnd0[1],ColEnd0[1],RowBegin1[0],ColBegin1[0],RowEnd1[0],ColEnd1[0],m_angle)
disp_message (WindowHandle, '圆心到上边直线末尾点（左边点）的距离 ' +DistanceR_upendpoint+'半径'+Radius1, 'window',460, 10, 'blue', 'false')
第六章-缺陷检测
关于缺陷检测
回顾工业视觉的任务
常见的缺陷与对应算法
常见的缺陷有:
- 对象轮廓凸凹不平
- 非曲面轮廓的: 凸点斜光打亮,凹点垂直光打暗
- 曲面轮廓的:
- 对象内部污点、内部凸凹点、瑕疵点、空洞和破损等
- 对象表面划痕
- 打光,采用低角度环形光 & 同轴光
总结:检测缺陷的几个方法如下
- 打光,光的角度
- 光源选型,如波长
- 图像处理算法:
- blob分析+特征检测
- blob分析+特征检测+差分
- 频域分析+空间域分析
- 光度立体法
- 特征训练(深度学习领域)
- 测量+拟合方法(通过测量结果知道有缺陷)
Halcon中的缺陷检测案例
案例一(blob+特征)
检测饼干完整性
这里的blob+特征中的特征意思是,根据对象的形状特征来判断好坏,比如这里通过对分割得到的区域的面积阈值来判断是否有碎块
//check_hazelnut_wafers.hdev
* This example demonstrates a quality inspection on hazelnut wavers.
* Using the morphology tools the waver is extracted and examined
* according to a few shape features like Rectangularity and AreaHoles.
* This program also shows the use of the operator area_holes.
*
read_image (Image, 'food/hazelnut_wafer_01')
dev_close_window ()
dev_open_window_fit_image (Image, 0, 0, -1, -1, WindowHandle)
dev_update_window ('off')
dev_set_line_width (3)
dev_set_draw ('margin')
set_display_font (WindowHandle, 20, 'mono', 'true', 'false')
*
for Index := 1 to 24 by 1
read_image (Image, 'food/hazelnut_wafer_' + Index$'.02')
binary_threshold (Image, Foreground, 'smooth_histo', 'light', UsedThreshold)
opening_circle (Foreground, FinalRegion, 8.5)
area_holes (FinalRegion, AreaHoles)
rectangularity (FinalRegion, Rectangularity)
dev_display (Image)
if (AreaHoles > 300 or Rectangularity < 0.92)
dev_set_color ('red')
Text := 'Not OK'
else
dev_set_color ('forest green')
Text := 'OK'
endif
dev_display (FinalRegion)
disp_message (WindowHandle, Text, 'window', 12, 12, '', 'false')
if (Index < 24)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
endif
endfor
案例二(blob+差分)
差分就是减法操作,其实是相对的像素值求导
实现工件突刺毛刺凸起的检测
背光源打光,选择了亮的一边作为背景
* fin.hdev: Detection of a fin
*
dev_update_window ('off')
read_image (Fins, 'fin' + [1:3])
get_image_size (Fins, Width, Height)
dev_close_window ()
dev_open_window (0, 0, Width[0], Height[0], 'black', WindowID)
set_display_font (WindowID, 14, 'mono', 'true', 'false')
for I := 1 to 3 by 1
select_obj (Fins, Fin, I)
dev_display (Fin)
binary_threshold (Fin, Background, 'max_separability', 'light', UsedThreshold)
dev_set_color ('blue')
dev_set_draw ('margin')
dev_set_line_width (4)
dev_display (Background)
disp_continue_message (WindowID, 'black', 'true')
stop ()
// 闭操作:先膨胀后腐蚀,这里因为背景是白色,所以采用了250即对背景进行了一次膨胀腐蚀操作,从而抹去了毛刺
closing_circle (Background, ClosedBackground, 250)
dev_set_color ('green')
dev_display (ClosedBackground)
disp_continue_message (WindowID, 'black', 'true')
stop ()
// 区域做减法:用带有毛刺的图片-抹去毛刺的图片=毛刺
difference (ClosedBackground, Background, RegionDifference)
opening_rectangle1 (RegionDifference, FinRegion, 5, 5)
dev_display (Fin)
dev_set_color ('red')
dev_display (FinRegion)
area_center (FinRegion, FinArea, Row, Column)
if (I < 3)
disp_continue_message (WindowID, 'black', 'true')
stop ()
endif
endfor
案例三(blob+特征+极坐标变换)
检测瓶嘴
因为是圆形的,所以使用极坐标的效果会更好一些
可以看到右侧是利用坐标变换把圆形拉直的结果(看OCR那里有这个知识点)
* This example checks bottle necks for defects.
* First, the bottle is detected with basic morphology,
* edge detection and circle fitting.
* Then, the neck area is transformed with a polar transformation.
* After that, in the transformed image a dynamic threshold is used
* to detect defects. Finally, the results are displayed.
*
*
* tuning parameters
SmoothX := 501
ThresholdOffset := 25
MinDefectSize := 50
*
* initialization
PolarResolution := 640
RingSize := 70
get_system ('store_empty_region', StoreEmptyRegion)
set_system ('store_empty_region', 'false')
read_image (Image, 'bottles/bottle_mouth_01')
dev_update_off ()
dev_close_window ()
dev_close_window ()
dev_open_window_fit_image (Image, 0, 0, 640, 512, WindowHandle1)
set_display_font (WindowHandle1, 16, 'mono', 'true', 'false')
dev_display (Image)
dev_set_draw ('margin')
dev_set_line_width (3)
dev_open_window_fit_size (0, 648, RingSize, PolarResolution, 150, 512, WindowHandle)
dev_set_draw ('margin')
dev_set_line_width (3)
dev_set_color ('red')
*
* Main loop
*
* Detect defects in bottle necks
for Index := 1 to 16 by 1
read_image (Image, 'bottles/bottle_mouth_' + Index$'.02')
*
* Part 1: Use basic morphology to detect bottle
auto_threshold (Image, Regions, 2)
select_obj (Regions, DarkRegion, 1)
opening_circle (DarkRegion, RegionOpening, 3.5)
closing_circle (RegionOpening, RegionClosing, 25.5)
fill_up (RegionClosing, RegionFillUp)
boundary (RegionFillUp, RegionBorder, 'outer')
dilation_circle (RegionBorder, RegionDilation, 3.5)
reduce_domain (Image, RegionDilation, ImageReduced)
*
* Find the bottle center by fitting a circle to extracted edges
edges_sub_pix (ImageReduced, Edges, 'canny', 0.5, 20, 40)
segment_contours_xld (Edges, ContoursSplit, 'lines_circles', 5, 4, 2)
union_cocircular_contours_xld (ContoursSplit, UnionContours, 0.9, 0.5, 0.5, 200, 50, 50, 'true', 1)
length_xld (UnionContours, Length)
select_obj (UnionContours, LongestContour, sort_index(Length)[|Length| - 1] + 1)
fit_circle_contour_xld (LongestContour, 'ahuber', -1, 0, 0, 3, 2, Row, Column, Radius, StartPhi, EndPhi, PointOrder)
*
* Part 2: Transform the ring-shaped bottle neck region to a rectangle
gen_circle (Circle, Row, Column, Radius)
dilation_circle (Circle, RegionDilation, 5)
erosion_circle (Circle, RegionErosion, RingSize - 5)
difference (RegionDilation, RegionErosion, RegionDifference)
reduce_domain (Image, RegionDifference, ImageReduced)
polar_trans_image_ext (ImageReduced, ImagePolar, Row, Column, 0, rad(360), Radius - RingSize, Radius, PolarResolution, RingSize, 'nearest_neighbor')
*
* Part 3: Find defects with a dynamic threshold
* Note the strong smoothing in x-direction in the transformed image.
scale_image_max (ImagePolar, ImageScaleMax)
mean_image (ImageScaleMax, ImageMean, SmoothX, 3)
dyn_threshold (ImageScaleMax, ImageMean, Regions1, 55, 'not_equal')
connection (Regions1, Connection)
select_shape (Connection, SelectedRegions, 'height', 'and', 9, 99999)
* ignore noise regions
closing_rectangle1 (SelectedRegions, RegionClosing1, 10, 20)
union1 (RegionClosing1, RegionUnion)
* re-transform defect regions for visualization
polar_trans_region_inv (RegionUnion, XYTransRegion, Row, Column, 0, rad(360), Radius - RingSize, Radius, PolarResolution, RingSize, 1280, 1024, 'nearest_neighbor')
*
* Part 4: Display results
* display original image with results
dev_set_window (WindowHandle1)
dev_display (Image)
dev_set_color ('blue')
dev_display (RegionDifference)
dev_set_color ('red')
dev_display (XYTransRegion)
* display polar transformed inspected region with results
* The image and resulting region are rotated by 90 degrees
* only for visualization purposes! (I.e. to fit better on the screen)
* The rotation is NOT necessary for the detection algorithm.
dev_set_window (WindowHandle)
rotate_image (ImagePolar, ImageRotate, 90, 'constant')
dev_display (ImageRotate)
count_obj (RegionUnion, Number)
if (Number > 0)
mirror_region (RegionUnion, RegionMirror, 'diagonal', PolarResolution)
mirror_region (RegionMirror, RegionMirror, 'row', PolarResolution)
dev_display (RegionMirror)
disp_message (WindowHandle1, 'Not OK', 'window', 12, 12, 'red', 'false')
else
disp_message (WindowHandle1, 'OK', 'window', 12, 12, 'forest green', 'false')
endif
if (Index < 16)
disp_continue_message (WindowHandle1, 'black', 'true')
stop ()
endif
endfor
* Reset system parameters
set_system ('store_empty_region', StoreEmptyRegion)
案例四(blob+特征)
这里的blob利用了局部二值化,提取局部的信息进行二值化操作,局部动态阈值操作
光照稳定,环境简单的应用场合
* The task of this example is to detect defects on a
* web using the operator dyn_threshold. In this way,
* the operator can be used to find textures that
* differ from the rest of the image.
dev_update_window ('off')
read_image (Image, 'plastic_mesh/plastic_mesh_01')
dev_close_window ()
get_image_size (Image, Width, Height)
dev_open_window_fit_image (Image, 0, 0, Width, Height, WindowHandle)
set_display_font (WindowHandle, 18, 'mono', 'true', 'false')
dev_set_draw ('margin')
dev_set_line_width (3)
* Each of the images is read and smoothed. Subsequently
* dyn_threshold is performed and connected regions are
* looked for. The parameter 'area' of the operator select_shape
* makes it possible to find regions that differ in the area
* size. Found errors are finally counted and displayed.
for J := 1 to 14 by 1
read_image (Image, 'plastic_mesh/plastic_mesh_' + J$'02')
mean_image (Image, ImageMean, 49, 49)
dyn_threshold (Image, ImageMean, RegionDynThresh, 5, 'dark')
connection (RegionDynThresh, ConnectedRegions)
// 根据面积来提取图像的特征
select_shape (ConnectedRegions, ErrorRegions, 'area', 'and', 500, 99999)
count_obj (ErrorRegions, NumErrors)
dev_display (Image)
dev_set_color ('red')
dev_display (ErrorRegions)
* If the number of errors exceeds zero, the message 'Mesh not
* OK' is displayed. Otherwise the web is undamaged
* and 'Mesh OK' is displayed.
if (NumErrors > 0)
disp_message (WindowHandle, 'Mesh not OK', 'window', 24, 12, 'black', 'true')
else
disp_message (WindowHandle, 'Mesh OK', 'window', 24, 12, 'black', 'true')
endif
* If the sequence number of the image to be inspected is
* lower than 14, the request to press 'Run' to continue appears.
* If the last image is read, pressing 'Run' will clear the SVM.
if (J < 14)
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
endif
endfor
对于图片:动态二值化,首先需要把图片进行均值滤波,这个动态二值化相当于对两张图片做了一次像素级别的差分运算
对于区域:差分运算使用difference算子
dyn_threshold(OrigImage, ThresholdImage : RegionDynThresh : Offset, LightDark : )
//ThresholdImage是输入的等待二值化的图片,Offset是二值化偏移量,OrigImage是原图片,RegionDynThresh是输出结果
//二值化是把OrigImage和ThresholdImage两个图片中的像素,一个像素一个像素的比较
案例五(blob+特征+差分)
检测PCB板的毛刺和断开点
打光均匀,对比度高,稳定的场合
灰度图像进行腐蚀或者开运算,图像暗的像素会增多
灰度图像进行膨胀或者闭运算,图像亮的像素会增多
开运算=先腐蚀后膨胀
read_image (Image, 'pcb')
dev_close_window ()
get_image_size (Image, Width, Height)
dev_open_window (0, 0, Width, Height, 'black', WindowHandle)
dev_display (Image)
* detect defects ...
gray_opening_shape (Image, ImageOpening, 7, 7, 'octagon')
gray_closing_shape (Image, ImageClosing, 7, 7, 'octagon')
//动态二值化实现差分操作
dyn_threshold (ImageOpening, ImageClosing, RegionDynThresh, 75, 'not_equal')
dev_display (Image)
dev_set_color ('red')
dev_set_draw ('margin')
dev_display (RegionDynThresh)
案例六(检测定位+blob+特征+差分)
- 定位胶囊
- 把胶囊位置调正,调水平
- 把每个胶囊放到区域数组中,每个胶囊对应到一个标准的胶囊格子中(用来测量面积)
- 将胶囊面积和胶囊格子面积进行对比,若小于低阈值则正常,大于高阈值则异常
* This example demonstrates an application from the pharmaceutical
* industry. The task is to check the content of automatically filled
* blisters. The first image (reference) is used to locate the chambers
* within a blister shape as a reference model, which is then used to
* realign the subsequent images along to this reference shape. Using
* blob analysis the content of each chamber is segmented and finally
* classified by a few shape features.
*
dev_close_window ()
dev_update_off ()
read_image (ImageOrig, 'blister/blister_reference')
dev_open_window_fit_image (ImageOrig, 0, 0, -1, -1, WindowHandle)
set_display_font (WindowHandle, 14, 'mono', 'true', 'false')
dev_set_draw ('margin')
dev_set_line_width (3)
*
* In the first step, we create a pattern to cut out the chambers in the
* subsequent blister images easily.
access_channel (ImageOrig, Image1, 1)
threshold (Image1, Region, 90, 255)
// 拟合成椭圆
shape_trans (Region, Blister, 'convex')
orientation_region (Blister, Phi)
area_center (Blister, Area1, Row, Column)
vector_angle_to_rigid (Row, Column, Phi, Row, Column, 0, HomMat2D)
affine_trans_image (ImageOrig, Image2, HomMat2D, 'constant', 'false')
gen_empty_obj (Chambers)
for I := 0 to 4 by 1
// 人工手动划分胶囊格子
Row := 88 + I * 70
for J := 0 to 2 by 1
Column := 163 + J * 150
gen_rectangle2 (Rectangle, Row, Column, 0, 64, 30)
concat_obj (Chambers, Rectangle, Chambers)
endfor
endfor
affine_trans_region (Blister, Blister, HomMat2D, 'nearest_neighbor')
difference (Blister, Chambers, Pattern)
union1 (Chambers, ChambersUnion)
orientation_region (Blister, PhiRef)
PhiRef := rad(180) + PhiRef
area_center (Blister, Area2, RowRef, ColumnRef)
*
*
* Each image read will be aligned to this pattern and reduced to the area of interest,
* which is the chambers of the blister
Count := 6
for Index := 1 to Count by 1
read_image (Image, 'blister/blister_' + Index$'02')
threshold (Image, Region, 90, 255)
connection (Region, ConnectedRegions)
select_shape (ConnectedRegions, SelectedRegions, 'area', 'and', 5000, 9999999)
shape_trans (SelectedRegions, RegionTrans, 'convex')
*
* Align pattern along blister of image
orientation_region (RegionTrans, Phi)
area_center (RegionTrans, Area3, Row, Column)
vector_angle_to_rigid (Row, Column, Phi, RowRef, ColumnRef, PhiRef, HomMat2D)
affine_trans_image (Image, ImageAffineTrans, HomMat2D, 'constant', 'false')
*
* Segment pills
reduce_domain (ImageAffineTrans, ChambersUnion, ImageReduced)
decompose3 (ImageReduced, ImageR, ImageG, ImageB)
var_threshold (ImageB, Region, 7, 7, 0.2, 2, 'dark')
connection (Region, ConnectedRegions0)
closing_rectangle1 (ConnectedRegions0, ConnectedRegions, 3, 3)
fill_up (ConnectedRegions, RegionFillUp)
select_shape (RegionFillUp, SelectedRegions, 'area', 'and', 1000, 99999)
opening_circle (SelectedRegions, RegionOpening, 4.5)
connection (RegionOpening, ConnectedRegions)
select_shape (ConnectedRegions, SelectedRegions, 'area', 'and', 1000, 99999)
shape_trans (SelectedRegions, Pills, 'convex')
*
* Classify segmentation results and display statistics
count_obj (Chambers, Number)
gen_empty_obj (WrongPill)
gen_empty_obj (MissingPill)
for I := 1 to Number by 1
select_obj (Chambers, Chamber, I)
intersection (Chamber, Pills, Pill)
area_center (Pill, Area, Row1, Column1)
if (Area > 0)
min_max_gray (Pill, ImageB, 0, Min, Max, Range)
if (Area < 3800 or Min < 60)
concat_obj (WrongPill, Pill, WrongPill)
endif
else
concat_obj (MissingPill, Chamber, MissingPill)
endif
endfor
*
dev_clear_window ()
dev_display (ImageAffineTrans)
dev_set_color ('forest green')
count_obj (Pills, NumberP)
count_obj (WrongPill, NumberWP)
count_obj (MissingPill, NumberMP)
dev_display (Pills)
if (NumberMP > 0 or NumberWP > 0)
disp_message (WindowHandle, 'Not OK', 'window', 12, 12 + 600, 'red', 'true')
else
disp_message (WindowHandle, 'OK', 'window', 12, 12 + 600, 'forest green', 'true')
endif
*
Message := '# Correct pills: ' + (NumberP - NumberWP)
Message[1] := '# Wrong pills : ' + NumberWP
Message[2] := '# Missing pills: ' + NumberMP
*
Colors := gen_tuple_const(3,'black')
if (NumberWP > 0)
Colors[1] := 'red'
endif
if (NumberMP > 0)
Colors[2] := 'red'
endif
disp_message (WindowHandle, Message, 'window', 12, 12, Colors, 'true')
dev_set_color ('red')
dev_display (WrongPill)
dev_display (MissingPill)
if (Index < Count)
disp_continue_message (WindowHandle, 'black', 'true')
endif
stop ()
endfor