图像的视差匹配（Stereo Matching）

这里要求用我们自己计算得到的视差图和给的视差图作比较来比较我们得到的视差图的好坏程度，我视差图返回的值是计算得到的视差乘以3之后的图，所以在计算时我不是两个值相差大于1，而是大于3，因为两个图像都乘3了，所以要大于3，我传入的参数是两个图像的矩阵，因为我是写了一个脚本咯跑所有测例的，在脚本里边已经把图像读出来了

function [percentnumberbadpixels] = PercentBadPixels(mydisparitymap,groundtruthdisparitymap)
mydisparitymap = double(mydisparitymap);
groundtruthdisparitymap = double(groundtruthdisparitymap);
[Rmydisparitymap, Cmydisparitymap] = size(mydisparitymap);
numbadpixels = 0;
for i = 1:Rmydisparitymap
    for j = 1:Cmydisparitymap
        if (abs(mydisparitymap(i,j) - groundtruthdisparitymap(i,j)) > 3)
              numbadpixels = numbadpixels+1;
        end
    end
end
percentnumbadpixels = numbadpixels/(Rmydisparitymap*Cmydisparitymap*1.0);
percentnumbadpixels = floor(percentnumbadpixels*100);
str = strcat(num2str(percentnumbadpixels),'%');
percentnumberbadpixels = str;
end

SSD:

  左视差图

  右视差图

   

实现细节：

   输入的图像是24位的图像，但是这些计算都是基于强度（intensities）来计算的，在matlab中把彩色图转为灰度图的函数rgb2gray实质上就是转为其强度灰度图像。

原理：

如图：

对于同一图像，左眼看到的图像相对于右眼看到的图像的位置是不一样的，如上图，对于相同位置在右眼看到的图像的位置在左眼看到的图像的位置的前边，所以左视差图和右视差图的搜索方法不同，对于左视差图：

 由于其位置比右眼看到的位置要远，则对于在右眼图像相同的物理位置应该往该位置的前边搜索才可能找到最相近的部分，即要减去d。

同理，对于右视差图则刚好相反，要加d。

实现细节：

1）首先把图像通过intensities转为灰度图像：

2）设定搜索深度d的值和patch的值，然后对图像进行补边界，使得可以从原图像的第一个像素开始进行匹配，我用的补边界的方法和以前做滤波时候原理一样，我都是补周围的值

3）对左图或右图中的每一个像素（i，j）作为patch的中心在另外的左图或右图上来寻找与之最相近的patch部分的搜索深度d，具体为：

 对于模板图像中每个不移动的patch，把该patch里边的每一个像素和另外一张目标图对应得patch（加上搜索深度d的）上的像素点做差的平方和然后再累加起来，每一个d对应一个patch，每一个patch对应一个值（sum），然后把求出在搜索深度d的范围内最小的sum，从而得到该最小的sum的搜索深度d的值，当遇到边界时用判断来处理，具体实现如下： 

如果是求右视差图的话，上边原理以及介绍了，要加上d来进行搜索

NCC

左视差图：

右视差图:

实现细节：

图像转为灰度图及边界处理和SSD相同，公式实现：

对左图或右图中的每一个像素（i，j）作为patch的中心，在另外的左图或右图上来寻找与之最相近的patch部分的搜索深度d。

 1）对于模板图像中每个不移动的patch，把该patch里边的每一个像素和另外一张目标图对应的patch（加上搜索深度d的）上的对应像素点分别相乘相加，把模板图中的patch和目标图的patch中的像素值分别平方相加，然后再两个值相乘开方。

2) 每一个d对应一个patch，每一个patch对应一个值（ncctemp），然后把求出在搜索深度d的范围内最大的ncctemp，从而得到该最大的ncctemp的搜索深度d的值，当遇到边界时用判断来处理

NCC与SSD的区别：

NCC抗光照性强，SSD抗光照性弱

如：Add a small constant amount of intensity (e.g. 10) to all right eye images, and re-run the above two methods. Analyze how the intensity change affects the results (i.e. the quality) of the two methods. Explain in which ways that NCC is a better matching cost than SSD.

这里我在view5加了常数10

NCC结果如下：Aloe disp1坏点率为24%

View5强度没有加10是坏点率为：24%

SSD 结果如下：Aloe disp1坏点率为38%

View5强度没有加10是坏点率为：26%

分析：

由上边公式可SSD是根据两幅图上patch里两个像素点差值的平方求和来找到最小的值，根据这个最小值我们认为距离为d的patch是匹配的部分，但是在拍照片是，会受到一些因素的影响，比如光照强度等，这样会照成左眼和右眼看到的图像的强度不一样，假如右眼看到的图像强度增加10，这样对于patch里边像素值相减得到的结果就会有很大的变化，从而造成求得的最小值的d也会不一样，因此的图像和右眼图像没有强度变化的结果有很大的不同。

而对于NCC来说，分子上是两个patch里边的像素分别相乘相加，而分母是每个patch里边的像素值平方然后求和在相乘开方，即使右眼的亮度加了，但是分子和分母的值都会同时增大，而且增大的值差不多相同，增大的值相除接近为1，所以得到的结果和右眼强度没有加10的结果基本上一样，没有多大的变化。

所以如果左眼图和右眼图受光照强度干扰比较大的情况下，NCC比SSD好。

nccdispl.m

function [output_img] = nccdispl(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
disparitymap = zeros(nrright,ncright);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end
if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
d = 70;
winsize = 11;
disparitymap = zeros(nrright,ncright);
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrright+win
    for j = 1+win:ncright+win
        min = -9999999;
        for k = 0:d
            numerator = 0.0;
            powerrightwin = 0.0;
            powerleftwin = 0.0;
            for a=-win:win
                for b = -win:win
                    if j+b-k > 0
                        numerator = numerator+(leftimg(i+a,j+b)*rightimg(i+a,j+b-k));
                        powerleftwin = powerleftwin+(leftimg(i+a,j+b)*leftimg(i+a,j+b));
                        powerrightwin = powerrightwin+(rightimg(i+a,j+b-k)*rightimg(i+a,j+b-k));
                    end
                end
            end
            ncctemp = numerator/(sqrt(powerrightwin*powerleftwin));
            if (min < ncctemp && j+b-k > 0)
                min = ncctemp;
                recordk = k;
            end
        end
        disparitymap(i-win,j-win) = recordk;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\nccdisp1.png'));
end

ssddispl.m
 
  
 
  
 
  
function [output_img] = ssddispl(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end

if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
disparitymap = zeros(nrleft,ncleft);
d = 70;
winsize = 11;
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrright+win
    for j = 1+win:ncright+win
        max = 99999999;
        for k = 0:d
            sum = 0.0;
            for a=-win:win
                for b = -win:win
                    if j+b-k > 0
                         temp = leftimg(i+a,j+b)-rightimg(i+a,j+b-k);
                         temp = temp*temp;
                         sum = sum+temp;
                    end                   
                end
            end
            if (max > sum && j+b-k > 0)
                max = sum;
                mind = k;
            end
        end
        disparitymap(i-win,j-win) = mind;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\ssddisp1.png'));
end
 
  
 
  
 
  
 
  
nccdispr.m
 
  
 
  
function [output_img] = nccdispr(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end
if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
d = 70;
winsize = 11;
disparitymap = zeros(nrleft,ncleft);
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrleft+win
    for j = 1+win:ncleft+win
        min = -999999;
        for k = 0:d
             numerator = 0.0;
             powerrightwin = 0.0;
             powerleftwin = 0.0;
            for a = -win:win
                for b = -win:win
                    if j+b+k <= ncleft+2*win
                        numerator = numerator+(rightimg(i+a,j+b)*leftimg(i+a,j+b+k));
                        powerrightwin = powerrightwin+(rightimg(i+a,j+b)*rightimg(i+a,j+b));
                        powerleftwin = powerleftwin+(leftimg(i+a,j+b+k)*leftimg(i+a,j+b+k));
                    end
                     
                end
            end
            ncctemp = numerator/(sqrt(powerrightwin*powerleftwin));
            if (min < ncctemp && j+b+k <= ncleft+2*win)
                min = ncctemp;
                recordk = k;
            end
        end
        disparitymap(i-win,j-win) = recordk;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\nccdisp5.png'));
end
 
  
ssddispr.m
 
  
 
  
function [output_img] = ssddispr(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end
if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
disparitymap = zeros(nrleft,ncleft);
d = 70;
winsize = 11;
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrleft+win
    for j = 1+win:ncleft+win
        max = 999999999;
        for k = 0:d
            sum = 0.0;
            for a=-win:win
                for b = -win:win
                    if j+b+k <= ncleft+2*win
                         temp = rightimg(i+a,j+b)-leftimg(i+a,j+b+k);
                         temp = temp*temp;
                         sum = sum+temp;
                    end
                end
            end
            if (max > sum && j+b+k <= ncleft+win)
                max = sum;
                mind = k;
            end
        end
        disparitymap(i-win,j-win) = mind;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\ssddisp5.png'));
end