【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法

目录

  • 目录
  • 说明
  • 图片格式介绍
  • 识别与截取的方法
    • 1 识别与截取初设想
    • 2 实际情况下的识别与截取方法
  • 实现代码
    • 1 为图片加边框的代码
    • 2 识别与截取图片的代码
  • 要说的

1 说明

又是一篇总结的文章,上个月弄了一下Low-Power Image Recognition Challenge (LPIRC)这个比赛中的track3部分。

LPIRC比赛内容就是低功耗图像识别比赛,图像识别的方法大多队伍都用的是神经网络识别方法。
比赛中的Track1项目,就是从比赛方服务器上down下来图片,在自己的机器上用神经网络识别,再把识别结果传回服务器上计算识别的精度,同时还会有功耗仪器来记录这段过程的功耗,最终根据你的所用时间、识别精度、消耗的瓦特计算你的分数。

与之前不同的是,今年比赛放在比赛前半个月突然弄出来一个新的比赛项目Track3,与Track1不同之处在于获取图片的方式不同,Track3需要你自己带一个摄像头,不断的拍摄放在一个黑盒子中的显示器,当然这个显示器会显示带二维码边框的图片(用脚本控制)。
这个文章写的就是,如何从的二维码边框中识别出图片的ID、宽度、高度,再根据识别出的宽度、高度,把图片(用于输入到神经网络中)从带二维码边框的图片截取出来。

比赛内容详见:http://lpirc.net/index.html 。
Track3比赛实拍视频(YouTube):https://youtu.be/EEpsL2twsFw
Track3比赛实拍视频(YouKu):http://v.youku.com/v_show/id_XMTYyNDgwMDIyOA==.html
【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第1张图片

2 图片格式介绍

【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第2张图片
这是一张示例图片
每个图片的四个角都有四个白块也就是Marker。
【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第3张图片
其中白块表示1,黑块表示0
图片中上面的二进制码用来表示图片的宽度,下边表示图片的深度,从左到右为11->0位。
左右两边的二进制码合起来表示图片的ID共20位,左边从上到下表示19->10位,右边同理表示9->0位。
【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第4张图片
上下边边长为1800,左右边长为1000,单位是pixel
Maker块:宽60,高40
左右边块:宽60,高100
上下边块:款150,高40
【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第5张图片
在生成边框的时候会根据图片的分辨率,自动适应边框,所以会有两种情况
一种是wide_image:原始图片分辨率的宽高比例大于1800/1000 (上下边边长为1800,左右边长为1000)
一种是tall_image: 原始图片分辨率的宽高比例小于1800/1000

【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第6张图片
最终从这样一张带二维码边框的图片中我们可以得到这样的信息
上边的二维码是:0100 1011 0000 得出宽度信息:1200
下边的二维码是:0111 0000 0110 得出高度信息:1798
左右两边二维码:10000 00001 10000 00111 得到ID信息:525831

3 识别与截取的方法

3.1 识别与截取初设想

【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第7张图片
一开始设想了一种理想条件下的识别和截取的方法
1,首先摄像头照这样一帧图片
2,然后将这一帧图片二值化,图片变为白色255,黑色0,方便识别,同时去除部分干扰
3,插探测针识别黑白块:因为在边框中,相邻黑块或白块位置和大小相对来说都是固定的,所以根据比例运算,在每个黑块或白块的中心位置插这样一个探测针,探测针其实用于统计在某一位置一定长度或宽度区间上有多少的白色像素点,把这些统计到的像素点的值相加,如果得出的和大于一定阈值,我们就认定这一位置是一个白块,识别出1。同理小于一定阈值就可以识别出0来。
4,根据得出的宽度和高度信息,按照比例运算截取出图片来

【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第8张图片
不过以上方法只限定在十分理想的条件下,也就是要摄像头和显示器完全平行,并且要求照到的照片中四个边框的形状是一个完美矩形,因为插针是按照比例运算插下去的,只有在完美矩形的条件下才能准确的识别出来。
(需要说明的是,图片中白色的探针是我后画上去的,探针只是我用来方便你描述这种方法的名词,例如对于上边框每个块的高度是一定的为h,那我们就统计每个块宽度中间位置上,高度从0到h-1这些像素点值的和,把h个像素点的和与一个预设的阈值进行对比,从而识别出1或者0。)

3.2 实际情况下的识别与截取方法

【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第9张图片
在实际条件下,由于摄像头和显示器相对位置的不同,拍摄下来的照片中的边框不一定是矩形的,大多情况是梯形。
如上图和原始照片相比较,实际照片中四个边框已经变成了梯形。
这就会出为题,识别十分不准的问题,因为插针过程是按照矩形比例计算的进行的,插出来的也是一个矩形,如果在梯形的边框插入一个矩形形状的探测针,那肯定就是不对的,有的针会在边框外或边框内,有的针会插错位置,总之就是大错特错。
这种情况下怎么解决呢?
一个方法就是用到warpPerspective这个opencv函数,把梯形边框校准回理想的矩形条件下。
【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第10张图片
不过这个函数需要待矫正图片的四个端点的坐标信息,也就是上左图四个红点的坐标。
为了得到这四个点坐标,我们先把图片转换成二进制图。
【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第11张图片
然后使用一种滑动探针的方法:也就是一定长度的探针,已照片的四个端点为起点,从上到下,从左到右的滑动,一共滑八次,得到四个红点的坐标信息。
把带矫正的坐标信息和想要的坐标信息输入到getPerspectiveTransform(),计算变换矩阵。
再把变换矩阵和拍摄的照片输入到warpPerspective(),就得出了矫正后的矩形图片。
然后我们再跑一遍理想条件下的识别方法,插一遍探测针就识别出了图片的相关信息。
【opencv学习】 Low-Power Image Recognition Challenge (LPIRC):Track 3 用摄像头从带二维码边框的图片中提取照片及信息的方法_第12张图片
不过矫正函数warpPerspective和计算矩阵函数getPerspectiveTransform,还是相当耗时的。

4 实现代码

4.1 为图片加边框的代码

#!/usr/bin/env python
import cv2
import numpy as np
from cv2 import *

def creat (n):
    img=imread("C://Users//Joe_Quan//Desktop//display//"+str(n)+".jpg")
    #imshow("show",img)
    ori_height,ori_width,s=img.shape
    print ori_height
    print ori_width
    h_step = 100;
    w_step = 150;
    height = 1080;
    width = 1920;
    inner_frame_width  = 1800;
    inner_frame_height = 1000;
    ori_rate = 0;#compresion two ratio to decide how to cut this image  decide its a tall image or wide image 
    inner_rate = 0;
    corner_h = 40;
    corner_w = 60;

    blankspace=np.zeros((1080,1920,3),dtype=np.uint8)
    #blankspace[0:height-1,0:width-1]=255
    blankspace[0:corner_h-1,0:corner_w-1]=255
    blankspace[height-1-corner_h:height-1,0:corner_w-1]=255
    blankspace[0:corner_h-1,width-1-corner_w:width-1]=255
    blankspace[height-1-corner_h:height-1,width-1-corner_w:width-1]=255
    j=0

    for i in range (width-1-corner_w,corner_w-1,-w_step):
        #print j
        if (ori_width&(2**j)) != 0:
#           print 1
            blankspace[0:corner_h,i-w_step:i]=255
        if (ori_height&(2**j))!= 0:
            blankspace[height-1-corner_h:height-1,i-w_step:i]=255
        j=j+1

    value_right=n
#   print n
    k=0
    for i in range (height-1-corner_h,corner_h-1,-h_step):

        if k<10:
            if(value_right&(2**k))!=0:
                blankspace[i-h_step:i,width-1-corner_w:width-1]=255
        k=k+1

    l=10
    for i in range (height-1-corner_h,corner_h-1,-h_step):
        if l<20:

            if(value_right&(2**l))!=0:

                blankspace[i-h_step:i,0:corner_w-1]=255
        l=l+1

    ori_rate = float(1920) / float(1080);
    inner_rate = float(ori_width )/ float(ori_height);
    print inner_rate
    print ori_rate
    if ori_rate>inner_rate:#its a tall image 
        cut_x = corner_w-1 + ((inner_frame_width - (inner_frame_height*ori_width / ori_height)) / 2);
        cut_width = inner_frame_height *ori_width / ori_height;
        print cut_width
        frame_change=cv2.resize(img,(cut_width,inner_frame_height))
        #imwrite(str(n)+"show.jpg",frame_change)
        blankspace[corner_h-1:corner_h+inner_frame_height-1,cut_x-1:cut_x+cut_width-1]=frame_change
        imwrite("C://Users//Joe_Quan//Desktop//display_out//"+str(n)+".jpg",blankspace)
    else:#its a wide image 
        cut_y = corner_h -1+ ((inner_frame_height - (inner_frame_width*ori_height / ori_width)) / 2);
        cut_height = inner_frame_width *ori_height / ori_width;
        print cut_height
        frame_change=cv2.resize(img,(inner_frame_width,cut_height))
        #imwrite(str(n)+"show.jpg",frame_change)
        blankspace[cut_y:cut_y+cut_height,corner_w-1:corner_w+inner_frame_width-1]=frame_change
        imwrite("C://Users//Joe_Quan//Desktop//display_out//"+str(n)+".jpg",blankspace)
    #imwrite(str(n)+"show_ blan.jpg",blankspace)
    #waitKey(1)

if __name__ == '__main__':
    n=1
    while n<5001:
        creat(n)
        n=n+1

4.2 识别与截取图片的代码

python:

import cv2  
import sys
import numpy as np  
import multiprocessing
from cv2 import *
import time
def detec(img,key):
# initialize the camera
    threshold_binary = 175#binary threshold very important
    if key==1:#if key equal to 1 open execution door
        a=time.time()#record title
        frame=img;
        frame_per=frame
        gray_image=cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)#trun one frame to gray_image
        imshow("perchange_s",gray_image)
        retval,binary_image=cv2.threshold(gray_image,threshold_binary,255,cv2.THRESH_BINARY)#turn one frmae to binary
        TOP_left_x = 0; #for corner detection
        TOP_left_y = 0;
        TOP_right_x = 0;
        TOP_right_y = 0;
        Bottom_left_x = 0; 
        Bottom_left_y = 0;  
        Bottom_right_x = 0;
        Bottom_right_y = 0;
        binary_height,binary_width = binary_image.shape
        #========================================================corner detection
        for i in range(0,binary_width/2):
            sum_tl_x=sum(binary_image[1:binary_height/3,i])
            if sum_tl_x>binary_height*0.01*255 :
                #print 'topleftx:',i
                TOP_left_x=i
                break
        for i in range(0,binary_height/2):
            sum_tl_y=sum(binary_image[i,1:binary_width/3])
            if sum_tl_y>binary_width*0.01*225:
                #print 'toplefty:',i
                TOP_left_y=i
                break

        for i in range(binary_width-1,binary_width/2,-1):
            sum_tr_x=sum(binary_image[1:binary_height/3,i])
            if sum_tr_x>binary_height*0.01*225:
                #print 'toprightx:',i
                TOP_right_x=i
                break
        for i in range(0,binary_height/2):
            sum_tr_y=sum(binary_image[i,binary_width*2/3:binary_width])
            if sum_tr_y>binary_width*0.01*225:
                #print 'toprighty:',i
                TOP_right_y=i
                break
        for i in range(0,binary_width/2):
            sum_bl_x=sum(binary_image[binary_height*2/3:binary_height,i])
            if sum_bl_x>binary_height*0.01*225:
                #print 'bottomleftx:',i
                Bottom_left_x=i
                break
        for i in range(binary_height-1,binary_height/2,-1):
            sum_bl_y=sum(binary_image[i,1:binary_width/3])
            if sum_bl_y>binary_width*0.01*225:
                #print 'bottomlefty:',i
                Bottom_left_y=i
                break

        for i in range(binary_width-1,binary_width/2,-1):
            sum_br_x=sum(binary_image[binary_height*2/3:binary_height,i])
            if sum_br_x>binary_height*0.01*225:
                #print 'bottomrightx',i
                Bottom_right_x=i
                break
        for i in range(binary_height-1,binary_height/2,-1):
            sum_br_y=sum(binary_image[i,binary_width*2/3:binary_width])
            if sum_br_y>binary_width*0.01*225:
                #print 'bottomrighty',i
                Bottom_right_y=i
                break
        #=====================================================================persperctive change to turn original image inti retangle image
        srcQuad = np.float32([[TOP_left_x,TOP_left_y],[TOP_right_x - 1, TOP_right_y],[Bottom_left_x,Bottom_left_y - 1],[Bottom_right_x - 1,Bottom_right_y - 1]])
        #dstQuad = np.float32([[0,0],[binary_width-1,0],[0,binary_height-1],[binary_width-1,binary_height-1]])
        dstQuad = np.float32([[0,0],[1920-1,0],[0,1080-1],[1920-1,1080-1]])
        transform = cv2.getPerspectiveTransform(srcQuad,dstQuad)#caculate persperctive change matrix
        #pers_height,pers_width = 0
        frame_per_fix = cv2.warpPerspective(frame_per,transform,(1920,1080))#execute persperctive change save to frame_per_fix
        #===========================================================================end of persperctive change
        gray_image_fix=cv2.cvtColor(frame_per_fix,cv2.COLOR_BGR2GRAY)#turn gary
        retval_fix,binary_image_fix=cv2.threshold(gray_image_fix,threshold_binary,255,cv2.THRESH_BINARY)#turn binary
        #imshow("perchange_s",frame_per_fix)
        binary_height,binary_width,ch = frame_per_fix.shape#got the height 1080 and width  1920 of frame_per_fix 
        #============================================================================side detection
        h_start = 0;
        h_end = 0;
        w_start = 0;
        w_end = 0;
        for i in range(0,binary_height):
            sum_t=sum(binary_image_fix[i,:])
            if sum_t>binary_width*0.01*255:
                #print 'h_start',i
                h_start=i
                break
        for i in range(binary_height-1,0,-1):
            sum_b=sum(binary_image_fix[i,:])
            if sum_b>binary_width*0.01*255:
                #print 'h_end',i
                h_end=i 
                break
        for i in range(0,binary_width):
            sum_l=sum(binary_image_fix[:,i])
            if sum_l>binary_height*0.01*255:
                #print 'w_start',i
                w_start=i
                break
        for i in range(binary_width-1,0,-1):
            sum_r=sum(binary_image_fix[:,i])
            if sum_r>binary_height*0.01*255:
                #print 'w_end',i
                w_end=i
                break
        #=================================================================pin detection to caculate the height width and id from binary block
        h_step = 0;
        w_step = 0;
        height = 0;
        width = 0;
        value_h_s = 0;
        value_h_e = 0;
        value_w_s = 0;
        value_w_e = 0;
        height = h_end - h_start;
        width = w_end - w_start;
        h_step = int(height * 10 / 108)
        w_step = int(width * 5 / 64)
        w_p_s = 0;
        w_p_e = 0;
        h_p_s = 0;
        h_p_e = 0;
        corner_h = 0;
        corner_w = 0;
        id_value = 0;
        w_p_s = int(w_start + w_step / 2);#width_pin_start location 
        w_p_e = int(w_end - w_step / 2);#width_pin_end location
        h_p_s = int(h_start + h_step / 2);
        h_p_e =int( h_end - h_step / 2);
        corner_h = int(h_step / 2.5);#the height width of for corner block
        corner_w = int(w_step / 2.5);
        for i in range(w_p_s+corner_w,w_end-corner_w,w_step):
            sum_top=0
            sum_bottom=0
            sum_top=sum(binary_image_fix[h_start:h_start+corner_h,i])
            sum_bottom=sum(binary_image_fix[h_end-corner_h:h_end,i])
            if sum_top>corner_h*0.5*255:
                value_h_s=value_h_s<<1
                value_h_s=value_h_s+1
            else:
                value_h_s=value_h_s<<1
            if sum_bottom>corner_h*0.5*255:
                value_h_e=value_h_e<<1
                value_h_e=value_h_e+1
            else:
                value_h_e=value_h_e<<1
        #print 'top value',value_h_s
        #print 'bottom value',value_h_e
        for i in range(h_p_s+corner_h,h_end-corner_h,h_step):
            sum_left=0
            sum_left=sum(binary_image_fix[i,w_start:w_start+corner_w])
            if sum_left>corner_w*0.5*255:
                id_value=id_value<<1
                id_value=id_value+1
                #print '1'
            else:
                id_value=id_value<<1
                #print '0'
        for i in range(h_p_s+corner_h,h_end-corner_h,h_step):
            sum_right=0
            sum_right=sum(binary_image_fix[i,w_end-corner_w:w_end])
            if sum_right>corner_w*0.5*255:
                id_value=id_value<<1
                id_value=id_value+1
                #print '1'
            else:
                id_value=id_value<<1
                #print '0'
        #print 'ID:value',id_value
        #==============================================================cut final image from frame_per_fix 
        ori_height = float(value_h_e);#the height width value caculate from binary block
        ori_width = float(value_h_s);
        if ori_height==0 or ori_width==0:
            return frame
        cut_x = 0;#rect need the star point for imgae cut
        cut_y = 0;#rect need the star point for imgae cut
        cut_height = 0;#rect need the height information  for imgae cut
        cut_width = 0;#rect need the width information  for imgae cut
        inner_frame_x = 0;#the start point of the image inside the makers for the cutting of central image 
        inner_frame_y = 0;
        inner_frame_width = 0;
        inner_frame_height = 0;
        ori_rate = 0;#compresion two ratio to decide how to cut this image  decide its a tall image or wide image 
        inner_rate = 0;
        inner_frame_x = w_start + corner_w;#start point x,y of inner frame (the image in the makers)
        inner_frame_y = h_start + corner_h;
        inner_frame_width = float(width - corner_w - corner_w);
        inner_frame_height = float(height - corner_h - corner_h);
        ori_rate = float(ori_width / ori_height);
        inner_rate = float(inner_frame_width / inner_frame_height);
        #print 'ori_rate',ori_rate
        #print 'inner_rate',inner_rate
        #imshow("final_pre",frame_per_fix)
        if ori_rate#its a tall image 
            cut_x = inner_frame_x + ((inner_frame_width - (inner_frame_height*ori_width / ori_height)) / 2);
            cut_width = inner_frame_height *ori_width / ori_height;
            cutImage=frame_per_fix[inner_frame_y+10:inner_frame_y-3+inner_frame_height,cut_x:cut_x+cut_width]
            return cutImage
            #imshow("final_tall",cutImage)
        else:#its a wide image 
            cut_y = inner_frame_y + ((inner_frame_height - (inner_frame_width*ori_height / ori_width)) / 2);
            cut_height = inner_frame_width *ori_height / ori_width;
            cutImage=frame_per_fix[ cut_y:cut_y+cut_height,inner_frame_x:inner_frame_x+inner_frame_width]
            return cutImage
            #imshow("final_wide",cutImage)
        b=time.time()
        execute_time=b-a
        print 'time',execute_time
        key=0


if __name__=='__main__':
    #w = multiprocessing.Process(target = main)
    #nw = multiprocessing.Process(target = main)
    #w.start()
    #nw.start()
    key=0#the key to execute detection 
    cam = VideoCapture(0)   # 0 -> index of camera
    cam.set(cv.CV_CAP_PROP_FRAME_WIDTH, 1280);
    cam.set(cv.CV_CAP_PROP_FRAME_HEIGHT, 720);
    print cam.get(cv.CV_CAP_PROP_FRAME_WIDTH);
    #img = imread("C://Users//Joe_Quan//Desktop//test.jpg");#load image to excute instead of cam
    while cam.isOpened():#while cam is opened
        key=0
        s, img = cam.read() #read one frame to img
        if waitKey(1)&0xFF ==ord('q'):
            break
        if waitKey(1)&0xFF ==ord('1'):
            key=1
        imshow("cam-test",img) 
        im=detec(img,key)
        if key==1:
            imshow("cam-final",im) 
    cam.release()
    cv2.destroyAllWindows()

c++

 #include "opencv2/video/tracking.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "cv.h"   
#include "cxcore.h"   
#include "highgui.h"   
#include 
#include 
#include 
#include 
#include 
#include   
using namespace cv;
using namespace std;
int key = 0;
static void onMouse(int event, int x, int y, int /*flags*/, void* /*param*/){
    //Mat mouse_img;
    //namedWindow("on mouse", 1);
    //  Mat imageresize;
    //  imageresize=cvCreateImage(cvSize(1280, 720), IPL_DEPTH_8U, 3);;
    if (event == CV_EVENT_LBUTTONDOWN){

    }
    else if (event == CV_EVENT_LBUTTONUP){
        key = 1;
    }
    //else if ((event == CV_EVENT_MOUSEMOVE) && (left_mouse == true)){

    //  }
}

Mat image_detec(Mat cam_frame)
{
    int TOP_left_x, TOP_right_x, Bottom_left_x, Bottom_right_x;
    int TOP_left_y, TOP_right_y, Bottom_left_y, Bottom_right_y;
    int h_start, h_end, w_start, w_end;
    bool left_mouse = false;
    int rect_width = 0, rect_height = 0;
    Point tmpPoint;
    int num = 0;
    int value_h_s;
    int value_h_e;
    int value_w_s;
    int value_w_e;

    //#pragma comment(linker, "/subsystem:\"windows\" /entry:\"mainCRTStartup\"")  
    Mat srcimage;
    Mat gray_image;
    Mat gray_image_fix;
    Mat binary_image;
    Mat binary_image_fix;
    Mat persrc;
    Mat perdst;

    //Mat frame_temp;
    int threshold_binary = 180;
    int counter = 0;
    Mat imageresize;

    namedWindow("cam_frame",1);
    //setMouseCallback("cam_frame", onMouse, 0);

        Mat frame;
        //Mat cam_frame = imread("C://Users//Joe_Quan//Desktop//test.jpg");
    //  cap.read(frame);
        cam_frame.copyTo(srcimage);
        //srcimage = frame;
        imshow("cam_frame", srcimage);
        waitKey(1);

            cvtColor(srcimage, gray_image, CV_BGR2GRAY);
            threshold(gray_image, binary_image, threshold_binary, 255, CV_THRESH_BINARY);
            //imshow("mat version", binary_image);
            cv::Size  BiSize = binary_image.size();
            TOP_left_x = 0;
            TOP_left_y = 0;
            TOP_right_x = 0;
            TOP_right_y = 0;
            Bottom_left_x = 0;
            Bottom_left_y = 0;
            Bottom_right_x = 0;
            Bottom_right_y = 0;
            //  Mat A_e = img.col(i);
            //  Mat SubMat_e = A_e.rowRange(height_end - corner_h, height_end);
            for (int i = 0; i < BiSize.width / 2; i++) {
                Mat temp_l = binary_image.col(i);
                int sum_tl = cv::sum(temp_l.rowRange(0, BiSize.height / 3))[0];
                if (sum_tl > BiSize.height*0.01 * 255) {
                    //      cout << "TOP_left_x:" << i << " " << endl;
                    TOP_left_x = i;
                    break;
                }
            }
            for (int i = 0; i < BiSize.height / 2; i++) {
                Mat temp_l = binary_image.row(i);
                int sum_tl = cv::sum(temp_l.colRange(0, BiSize.width / 3))[0];
                if (sum_tl > BiSize.width*0.01 * 255) {
                    //      cout << "TOP_left_y:" << i << " " << endl;
                    TOP_left_y = i;
                    break;
                }
            }

            for (int i = BiSize.width - 1; i > BiSize.width / 2; i--) {
                Mat temp_tr = binary_image.col(i);
                int sum = cv::sum(temp_tr.rowRange(0, BiSize.height / 4))[0];
                if (sum > BiSize.height*0.01 * 255) {
                    //      cout << "TOP_right_x:" << i << " " << endl;
                    TOP_right_x = i;
                    break;
                }
            }
            for (int i = 0; i < BiSize.height / 2; i++) {
                Mat temp_tr = binary_image.row(i);
                int sum = cv::sum(temp_tr.colRange(BiSize.width * 2 / 3, BiSize.width))[0];
                if (sum > BiSize.width*0.01 * 255) {
                    //      cout << "TOP_right_y:" << i << " " << endl;
                    TOP_right_y = i;
                    break;
                }
            }
            for (int i = 0; i < BiSize.width / 2; i++) {
                Mat temp_bl = binary_image.col(i);
                int sum_bl = cv::sum(temp_bl.rowRange(BiSize.height * 2 / 3, BiSize.height))[0];
                if (sum_bl > BiSize.height*0.01 * 255) {
                    //  cout << "Bottom_left_x:" << i << " " << endl;
                    Bottom_left_x = i;
                    break;
                }
            }
            for (int i = BiSize.height - 1; i > BiSize.height / 2; i--) {
                Mat temp_bl = binary_image.row(i);
                int sum_bl = cv::sum(temp_bl.colRange(0, BiSize.width / 3))[0];
                if (sum_bl > BiSize.width*0.01 * 255) {
                    //  cout << "Bottom_left_y:" << i << " " << endl;
                    Bottom_left_y = i;
                    break;
                }
            }
            for (int i = BiSize.width - 1; i > BiSize.width / 2; i--) {
                Mat temp_br = binary_image.col(i);
                int sum_br = cv::sum(temp_br.rowRange(BiSize.height * 2 / 3, BiSize.height))[0];
                if (sum_br > BiSize.height*0.01 * 255) {
                    //      cout << "Bottom_right_x:" << i << " " << endl;
                    Bottom_right_x = i;
                    break;
                }
            }
            for (int i = BiSize.height - 1; i > BiSize.height / 2; i--) {
                Mat temp_br = binary_image.row(i);
                int sum_br = cv::sum(temp_br.colRange(BiSize.width * 2 / 3, BiSize.width))[0];
                if (sum_br > BiSize.width*0.01 * 255) {
                    //  cout << "Bottom_right_y:" << i << " " << endl;
                    Bottom_right_y = i;
                    break;
                }
            }
            Point2f  srcQuad[4], dstQuad[4];
            persrc = srcimage;
            perdst = persrc;
            //  perdst = Mat::zeros(perdst.size(),perdst.type());

            srcQuad[0] = Point2f(TOP_left_x, TOP_left_y);                                //src top left
            srcQuad[1] = Point2f(TOP_right_x - 1, TOP_right_y);                    //src top right
            srcQuad[2] = Point2f(Bottom_left_x, Bottom_left_y - 1);                            //src bottom left 
            srcQuad[3] = Point2f(Bottom_right_x - 1, Bottom_right_y - 1);                    //src bottom right 

            dstQuad[0] = Point2f(0, 0);
            dstQuad[1] = Point2f(1920 - 1, 0);
            dstQuad[2] = Point2f(0, 1080 - 1);
            dstQuad[3] = Point2f(1920 - 1, 1080 - 1);

            Mat transform = getPerspectiveTransform(srcQuad, dstQuad);
            warpPerspective(persrc, perdst, transform, Size(1920, 1080));
            //imshow("mat version perschange", perdst);


        cvtColor(perdst, gray_image_fix, CV_BGR2GRAY);
        threshold(gray_image_fix, binary_image_fix, threshold_binary, 255, CV_THRESH_BINARY);
        //imshow("mat version binary", binary_image_fix);

        //Side_detc(binary_image_fix);//detect four side of the image :height_start height_end width_start width_end
        h_start = 0;
        h_end = 0;
        w_start = 0;
        w_end = 0;
        BiSize = binary_image_fix.size();
        /// cout <for (int i = 0; i < BiSize.height; i++) {
            int sum = cv::sum(binary_image_fix.row(i))[0];
            if (sum > BiSize.width*0.01 * 255) {
                //  cout <<"h_start:"<< i << " " <<  endl;
                h_start = i;
                break;
            }
        }

        for (int i = BiSize.height - 1; i >= 0; i--) {
            int sum = cv::sum(binary_image_fix.row(i))[0];
            if (sum > BiSize.width*0.01 * 255) {
                //      cout << "h_end:" << i << " " << endl;
                h_end = i;
                break;
            }
        }

        for (int i = 0; i < BiSize.width; i++) {
            int sum = cv::sum(binary_image_fix.col(i))[0];
            if (sum > BiSize.height*0.01 * 255) {
                //      cout << "w_start:" << i << " " << endl;
                w_start = i;
                break;
            }
        }

        for (int i = BiSize.width - 1; i >= 0; i--) {
            int sum = cv::sum(binary_image_fix.col(i))[0];
            if (sum > BiSize.height*0.01 * 255) {
                //      cout << "w_end:" << i << " " << endl;
                w_end = i;
                break;
            }
        }
        //int value = 0;
        double h_step = 0;//the height of bloack in left&right side
        double w_step = 0;//the width og top&bottom block
        double height = 0;//for caculate h_step
        double width = 0;//caculate w_step with image ratio information 
        value_h_s = 0;
        value_h_e = 0;
        value_w_s = 0;
        value_w_e = 0;

        int w_p_s = 0;
        int w_p_e = 0;
        int h_p_s = 0;
        int h_p_e = 0;
        double corner_h = 0;//four corner block has the same size ratio
        double corner_w = 0;//four corner block has the same size ratio
        int id_value = 0;
        height = h_end - h_start;
        width = w_end - w_start;
        h_step = double(height * 10 / 108);
        w_step = double(width * 5 / 64);
        corner_h = h_step / 2.5;
        corner_w = w_step / 2.5;
        w_p_s = w_start + w_step / 2;
        w_p_e = w_end - w_step / 2;
        h_p_s = h_start + h_step / 2;
        h_p_e = h_end - h_step / 2;


        //for (int i = w_p_e - corner_w; i>width_start + corner_w; i = i - w_step)
        for (int i = w_p_s + corner_w; i < w_end - corner_w; i = i + w_step)
        {
            int sum_s = 0;
            int sum_e = 0;
            Mat A_s = binary_image_fix.col(i);
            Mat A_e = binary_image_fix.col(i);
            Mat SubMat = A_s.rowRange(h_start, h_start + corner_h);
            Mat SubMat_e = A_e.rowRange(h_end - corner_h, h_end);
            sum_e = cv::sum(SubMat_e)[0];
            sum_s = cv::sum(SubMat)[0];

            //cout << "sum:" << sum << endl;
            if (sum_s > corner_h*0.5 * 255) {
                value_h_s = value_h_s << 1;
                value_h_s = value_h_s + 1;
                //  cout << "1 ";
            }
            else {
                value_h_s = value_h_s << 1;
                //  cout << "0 ";
            }

            if (sum_e > corner_h*0.5 * 255) {
                value_h_e = value_h_e << 1;
                value_h_e = value_h_e + 1;
                //  cout << "1 ";
            }
            else {
                value_h_e = value_h_e << 1;
                //  cout << "0 ";
            }
        }
        //for (int i = h_p_e - corner_h; i>height_start + corner_h; i = i - h_step)


        for (int i = h_p_s + corner_h; iint sum_s = 0;
            Mat A_col = binary_image_fix.row(i);
            Mat SubMat_w_s = A_col.colRange(w_start, w_start + corner_w);
            sum_s = cv::sum(SubMat_w_s)[0];
            if (sum_s > corner_w*0.5 * 255) {
                id_value = id_value << 1;
                id_value = id_value + 1;
                //  cout << "1 ";
            }
            else {
                id_value = id_value << 1;
                //  cout << "0 ";
            }
        }
        for (int i = h_p_s + corner_h; iint sum_s = 0;
            Mat A_col_r = binary_image_fix.row(i);
            Mat SubMat_w_e = A_col_r.colRange(w_end - corner_w, w_end);
            sum_s = cv::sum(SubMat_w_e)[0];
            if (sum_s > corner_w*0.5 * 255) {
                id_value = id_value << 1;
                id_value = id_value + 1;
                //  cout << "1 ";
            }
            else {
                id_value = id_value << 1;
                //  cout << "0 ";
            }
        }


        //  cout << endl;
        //  cout << endl << "ID-value:" << id_value << endl;
        //  cout << endl << "width-value:" << value_h_s << endl;
        //  cout << endl << "height-value:" << value_h_e << endl;


        Mat cutImage = perdst;

        double ori_height = value_h_e;
        double ori_width = value_h_s;
        double cut_x = 0;//rect need the star point for imgae cut
        double cut_y = 0;//rect need the star point for imgae cut
        double cut_height = 0;//rect need the height information  for imgae cut
        double cut_width = 0;//rect need the width information  for imgae cut



        double inner_frame_x = 0;//the start point of the image inside the makers for the cutting of central image 
        double inner_frame_y = 0;
        double inner_frame_width = 0;
        double inner_frame_height = 0;
        double ori_rate = 0;//for deciding how two cut this image 
        double inner_rate = 0;

        inner_frame_x = w_start + corner_w;
        inner_frame_y = h_start + corner_h;
        inner_frame_width = width - corner_w - corner_w;
        inner_frame_height = height - corner_h - corner_h;
        ori_rate = double(ori_width / ori_height);
        inner_rate = double(inner_frame_width / inner_frame_height);

        if (ori_rate < inner_rate)//for tall image 
        {
            cut_x = inner_frame_x + ((inner_frame_width - (inner_frame_height*ori_width / ori_height)) / 2);
            cut_width = inner_frame_height *ori_width / ori_height;
            Rect rect2(cut_x + 3, inner_frame_y + 1, cut_width - 2, inner_frame_height - 5);
            Mat image_roi = cutImage(rect2);
            imshow("final_img", image_roi);
            return image_roi;
        }
        else//for cutting wide image 
        {
            cut_y = inner_frame_y + ((inner_frame_height - (inner_frame_width*ori_height / ori_width)) / 2);
            cut_height = inner_frame_width *ori_height / ori_width;
            Rect rect2(inner_frame_x + 2, cut_y + 2, inner_frame_width - 2, cut_height - 2);
            Mat image_roi = cutImage(rect2);
            imshow("final_img", image_roi);
            return image_roi;
        }
        counter = counter + 1;
        cout << counter << endl;
    cvWaitKey(0);
    //system("pause");

}

int main(int argc, char** argv)
{
    Mat result ;
    VideoCapture cap;//mat version
    cap.open(0);
    if (!cap.isOpened())
    {
        cout << "error";
    }
    Mat cam_frame = imread("C://Users//Joe_Quan//Desktop//test.jpg");
    result = image_detec(cam_frame);

    imshow("final_img", result);
    cvWaitKey(0);
    system("pause");
    return 0;
}

5 要说的

代码写得很烂,请各位见谅!
以上方法的只是Track3获取图像的一个小接口,用于给神经网络输入图片,这一接口的识别速度还有待提升。
如果您看了这篇博客,并且您有更好的解决方法,欢迎您评论留言!

你可能感兴趣的:(opencv)