CV toolbox (2) - bilateral filter

bilateral_filter

$1.$ algrithm_introduction

$1.1$ gaussian_linear_filter

$1.1.1$ guassian_function

The Gaussian Linear Filter functin is the basic filter function at spacial denoising, though not function well at edge preserving. To apply, we need to do a set of steps.
First of all, I want to introduce the gaussian function, as follow.
$$G_{\sigma }(x)=ex{p}^{-\frac{x^2}{2{\sigma }^2}}\text{\hspace{1em}(1.1.1.1)}$$
maybe the unvisible expression is too difficult to understand, so we can plot gaussian function on MATLAB, code is given.
In the gragh, we can learn that guassian function empower a much larger value to the center part of the gragh, while empower a much smaller value to the surrouding areas.

$fig.1.1.1.1$ The right figure is guassian function plot on 2-dimension, the left fighre is guassian function plot on 3-dimension. The parameter $\sigma$ of gaussian function in left pic is $\sigma =0.2,0.4,0.6,0.8$. The parameter $\sigma$ of guassian function in right pic is $\sigma =2$.

%% plot 2-dimension gaussian-function
x = [-5: 0.1: 5];
gaussian = @(x, sigma) exp((-x.^2) ./ (2*sigma^2));
y = zeros(size(x));
figure(1);
for i = 1: 4
    y = gaussian(x, 0.2 * i);
    subplot(2, 2, i);
    plot(x, y);
end

%% plot 3-dimension gaussian-function
x = [-5: 0.1: 5];
map = zeros(size(x, 2), size(x, 2));
figure(2);
for i = 1: size(x, 2)
    for j = 1: size(x, 2)
        map(i, j) = x(i)^2 + x(j)^2;
    end
end
mesh(x, x, gaussian(map, 2));

$1.1.2$ apply guassian function to filter

It is avaliable to get the empower map through guassian function, then we can apply it to a filter to carry some computational photogragh task like denoising. Here we gave the filter epxpression as follow.
$$G_{\sigma }(f)(x)=\frac{1}{Z}\sum _y{G}_{\sigma }(x-y)f(y)\text{\hspace{1em}(1.1.2.1)}$$
In this function $Z={\sum }_y{G}_{\sigma }(y)$, always we standlize all value in $f(y)$ to make $Z=1$.
To understand expression $1.1.2.1$ , I want to have the aid of some concept in coputational photogragh. If we see $x$ as target pixel, which have some noise in it. To denoise it, we can reference the surrounding pixels to repower the value to pixel $x$. Moreover, we often apply surrounding pixels in a squre.
Suspend $va{l}_{m,n}$ is value matrix of pixel $x$'s surrounding pixels, $G_{m,n}$ is gaussian function matrix. Sum of $G_{m,n}$ equals $1$.Apply $G_{m,n}$ to $va{l}_{m,n}$, output the denoising temp martrix. After a series of steps, we can get the value of pixel $x$. Details as follow.
$$Guassian(x{)}_{m,n}=va{l}_{m,n}.\ast G_{m,n}$$$$x=sum(Guassian(x{)}_{m,n})$$
Then we can get the $x$ output function, as follow:
$$x=sum(va{l}_{m,n}.\ast G_{m,n})\text{\hspace{1em}(1.1.2.2)}$$
Applying this function to all pixels, then we can get gaussian function denoising gragh.

$1.1.3$ accelerate gaussian function

Using FFT method, through two-dimensional fast fourier transform, we could reduce the calculating work.
Here is the method page, paper.

$1.1.4$ code is here

code is based on MATLAB.

% x - input data
% sigma - guassian funciton parameter
% n*n - size of gragh
gaussian = @(x, sigma) exp((-x.^2) ./ (2*sigma^2)); 
x = [0: n/2-1, -n/2: -1];
[Y, X] = meshgrid(x, x);
GaussianFilt = @(F, sigma) exp((-X.^2 -Y.^2) ./ (2*sigma^2));
Filter = @(F,s)real( ifft2( fft2(F).*repmat( fft2(GaussianFilt(s)), [1 1 size(F,3)])));

suggest we have a gragh whose size is n*n, we can process several steps to denoise by guassian filter(here we adopt FFT method).

% name - path of a RGB image
% n*n - size of gragh
% sigma - parameter of guassina function
f0 = imread(name);
f0 = rescale(crop(f0, 3), n);
imshow(Filter(f0, sigma));

$1.2$ bilateral_filter

$1.2.1$ basic funciton introduction

Basic function introduction is posted in this blog.

$1.2.2$ bilateral_filter_stacking

Based on basic bilateral filter function, it is of vital importance to accelerate the processing.
$$B_{{\sigma }_x,{\sigma }_v}(f)(x)=F_v(x)=\frac{[G_{{\sigma }_x}(f\cdot W_v)](x)}{[G_{{\sigma }_x]}(W_v)](x)}\text{\hspace{1em}(1.2.2.1)}$$$$W_v(x)=G_{{\sigma }_v}(v-f(x))\text{\hspace{1em}(1.2.2.2)}$$

$2.$ using

$2.1$ denoising

Ideas from Bilateral Filtering for Gray and Color Images.

$2.1.1$ introduction

If we treat bilateral fiter as “black box”, noise gragh can be denoise through the “black box”.

f

black box

f0

$fig2.1.1.1$ denoise processing, $f$ is image with noise, $f_0$ is denoised image, blak box is bilateral filter.

$2.1.2$ code is here

Initially, we generate a noise gragh. Suspend f0 is a clear image without noise, we put some random noise on it.

% mu - put noise in a controled limit
% f - a noise gragh with noise, based on f0
mu = 0.05;
f = f0 + rand(n, n)*mu;

Then, apply bilateral filter to denoise the gragh.

% sx, sv - parameter of bilateral spacial, valued-dimensional
imshow(clamp(bilateral_lin(f, sx, sv)));

Now, processing is over.

$2.2$ detail_enhancement

$2.2.1$ introduction

Ideas from $part2.1$, we konw that we can seperate a gragh to a 2-layer data structer. The first layer we define as base layer, which cotains the bisic contents of the image；the second layer we dined as detail layer, which contains the details of a image, sometimes noise. To enhance the detail, we need to work on the detail layer.
To seperate detail layer and base layer To understand the base layer and detail layer, there is a gragh that could help.

$fig2.2.1.1$ Right gragh is denoised gragh, which is processed by bilateral filter; left contains noise.
Contract to right gragh, left gragh contains lots of details. When it comes to a face picture, maybe there are some freckles in it, which is the detail we want to boost.
To get denoised gragh, jump to $part2.1$.
To enhance detail To enhance gragh, initially is to find what to enhance. Idea from $part2.1$. Define $detail$ as detail layer, $f$ is denoised layer, $f_0$ is original layer with noise. Give the funciton.
$$detail=f-f_0$$Define $\gamma$ as enhance rate. Output enhanced gragh as $f_{enhs}$, give the funciton.
$$f_{enhs}=f_0+\gamma \ast detai{l}^{\alpha }$$Alwaysly, set $\alpha =1$.
Now, gragh enhancement is completed.

$2.2.2$ code is here

% f - input image matrix
% f0 - denoise image
% mu - enhance rate
f0 = clamp(bilateral_lin(f, sx, sv));
detail = f - f0;
im_output = f0 + mu*detail;
imshow(im_output);

$2.3$ HDR_range_tone

Idea from Fast Bilateral Filtering for the Display of High-Dynamic-Range Images.

$2.3.1$ introduction

Retell algrithm

$Begin$
$1st$, $inputIntensity=\frac{20R+40G+B}{61}$
$2nd$, $r=\frac{R}{inputIntensity}$; $g=\frac{G}{inputIntensity}$: $b=\frac{B}{inputIntensity}$
$3rd$, $bas{e}_{log}=Bilateral(log(InputIntensity))$
$4th$, $detai{l}_{log}=log(inputIntensity)-bas{e}_{log}$
$5th$, $outputIntensit{y}_{log}=bas{e}_{log}\ast compressionFactor+detai{l}_{log}-absoluteScal{e}_{log}$
$5th$, $outpu{t}_r=r\ast 10^{outputIntensit{y}_{log}}$
$End$

where :
$compressioonFactor=\frac{tragetContrast}{max(bas{e}_{log})}-min(bas{e}_{log})$ Alwaysly, $targetContract=log(5)$
$absoluteScal{e}_{log}=max(bas{e}_{log}\ast compressionFactor)$

To learn how to process this, use gragh as follow.

From numerical-tours, It can be learned that

A simple way to process a color image is to perform a change of color space and to process only the intensity channel.
For instance one can use a HSV color space and process the intensity channel

$fig2.3.1.1$ Graghs on top is HSV color space intensity for 3 channelsplot graghs. Graghs at bottom is standard RGB channels intensity plot gragh.

$fig2.3.1.2$ Graghs on top is HSV color space intensity for 3 channels value.

Therefore, in the following steps, our operations to details are based on the 3rd channel of HSV image, define this layer as $f_v$.

A popular contract modification is the $\gamma$ correction, give the function.
$$\psi (t)=(t+ϵ{)}^{\alpha }\text{\hspace{1em}(2.3.1.1)}$$where $ϵ>0$ is small, $\alpha \in [0,1]$.
Several other operators:
$\psi (t)=\frac{t}{t+ϵ}$
$\psi (t)=log(t+ϵ)$

However, we process contract modification through local algrithm, which have better effect contract to above global algrithm.
To realize that, there are several steps as follow, which is referred in the above reference of algrithm introduction.

$Begin$
$1st$, recompose the logarithm of $f_v$, and standarlize it in $[0,1]$.
$$F_v^{\prime }=log(f_v+ϵ)$$Then standarlize $f_v$, $a$、$b$ is the minimum and maximum of $F_v^{\prime }$.
$$F_v=\frac{F_v-a}{b-a}$$In all,
$$F_v=\frac{log(f_v+ϵ)-\alpha }{b-a}\text{\hspace{1em}(2.3.1.2)}$$
$2nd$, get the base layer
Give the function,
$$B_v=B_{{\sigma }_x,{\sigma }_v}(F_v)\text{\hspace{1em}(2.2.1.3)}$$
$3rd$, get the tone map
$${\tilde{f}}_V=e^{(\gamma B_V+F_V-B_v)(b-a)+a}-ϵ$$
$End$

Transform into RGB gragh, then output.

$2.3.2$ code is here

% f- input hdr gragh
p = min(size(f,1),size(f,2));
f = rescale( f(1:p,1:p,:) );
f = rescale( clamp( image_resize(f,n,n) ) );
fhsv = rgb2hsv(f);
fV = fhsv(:,:,3);
color_recompose = @(fV)hsv2rgb( cat(3, fhsv(:,:,1:2), rescale(fV) ) );
epsilon = 1e-5;
FV = log(fV+epsilon);
a = min(FV(:)); b = max(FV(:));
FV = (FV-a)/(b-a);
sx = 5; sv = .02;
FV1 = bilateral_lin(FV, sx,sv);
remap = @(FV,FV1,gamma)exp( ( gamma*FV1 + FV-FV1 ) * (b-a) + a ) - epsilon;
gamma = .1;
FVmapped = remap(FV,FV1,gamma);
imageplot(color_recompose(FVmapped), ['\gamma=' num2str(gamma)]);

official matlab hsv2rgb.m

function [rout,g,b] = hsv2rgb(hin,s,v)
%HSV2RGB Convert hue-saturation-value colors to red-green-blue.
%   M = HSV2RGB(H) converts an HSV color map to an RGB color map.
%   Each map is a matrix with any number of rows, exactly three columns,
%   and elements in the interval 0 to 1.  The columns of the input matrix,
%   H, represent hue, saturation and value, respectively.  The columns of
%   the resulting output matrix, M, represent intensity of red, blue and
%   green, respectively.
%
%   RGB = HSV2RGB(HSV) converts the HSV image HSV (3-D array) to the
%   equivalent RGB image RGB (3-D array).
%
%   As the hue varies from 0 to 1, the resulting color varies from
%   red, through yellow, green, cyan, blue and magenta, back to red.
%   When the saturation is 0, the colors are unsaturated; they are
%   simply shades of gray.  When the saturation is 1, the colors are
%   fully saturated; they contain no white component.  As the value
%   varies from 0 to 1, the brightness increases.
%
%   The colormap HSV is hsv2rgb([h s v]) where h is a linear ramp
%   from 0 to 1 and both s and v are all 1's.
%
%   CLASS SUPPORT
%   -------------
%   If the input is an HSV image, it can be of class logical, single, or 
%   double. The output image is single when the input is single. For all 
%   other input data types, the output image is double. If the input is a 
%   colormap, it can be of class logical, single, or double. The output 
%   colormap is single when the input is single. For all other input data 
%   types, the output colormap is double.
%
%   See also RGB2HSV, COLORMAP, RGBPLOT.

%   Undocumented syntaxes:
%   [R,G,B] = HSV2RGB(H,S,V) converts the HSV image H,S,V to the
%   equivalent RGB image R,G,B.
%
%   RGB = HSV2RGB(H,S,V) converts the HSV image H,S,V to the
%   equivalent RGB image stored in the 3-D array (RGB).
%
%   [R,G,B] = HSV2RGB(HSV) converts the HSV image HSV (3-D array) to
%   the equivalent RGB image R,G,B.

%   See Alvy Ray Smith, Color Gamut Transform Pairs, SIGGRAPH '78.
%   Copyright 1984-2011 The MathWorks, Inc.

threeD = ndims(hin)==3;
if nargin == 1 % HSV colormap
    if threeD
        if(size(hin,3) ~= 3)
            error(message('MATLAB:hsv2rgb:invalidInputSizeHSV'));
        end
        
        validateattributes(hin, {'double', 'single', 'logical'}, {'real'}, mfilename, 'HSV', 1);
        
        h = hin(:,:,1); s = hin(:,:,2); v = hin(:,:,3);
    elseif ismatrix(hin)
        if(size(hin,2) ~=3)
            error(message('MATLAB:hsv2rgb:invalidSizeForColormap'));
        end
        
        validateattributes(hin, {'double', 'single', 'logical'}, {'real','nonempty','nonsparse'}, mfilename, 'H');
        
        if((any(hin(:) < 0) || any(hin(:) > 1)))
            error(message('MATLAB:hsv2rgb:badMapValues'));
        end
        
        h = hin(:,1); s = hin(:,2); v = hin(:,3);
    else
        error(message('MATLAB:hsv2rgb:invalidInputSize'));
    end
elseif nargin == 3
    
    validateattributes(hin, {'double', 'single', 'logical'}, {'real', '2d'}, mfilename, 'HIN', 1);
    validateattributes(s, {'double', 'single', 'logical'}, {'real', '2d'}, mfilename, 'S', 2);
    validateattributes(v, {'double', 'single', 'logical'}, {'real', '2d'}, mfilename, 'V', 3);
    
    if ~isequal(size(hin),size(s),size(v)),
        error(message('MATLAB:hsv2rgb:InputSizeMismatch'));
    end
    h = hin;
else
    error(message('MATLAB:hsv2rgb:WrongInputNum'));
end

h = 6.*h;
k = floor(h);
p = h-k;
t = 1-s;
n = 1-s.*p;
p = 1-(s.*(1-p));

% Processing each value of k separately to avoid simultaneously storing
% many temporary matrices the same size as k in memory
kc = (k==0 | k==6);
r = kc;
g = kc.*p;
b = kc.*t;

kc = (k==1);
r = r + kc.*n;
g = g + kc;
b = b + kc.*t;

kc = (k==2);
r = r + kc.*t;
g = g + kc;
b = b + kc.*p;

kc = (k==3);
r = r + kc.*t;
g = g + kc.*n;
b = b + kc;

kc = (k==4);
r = r + kc.*p;
g = g + kc.*t;
b = b + kc;

kc = (k==5);
r = r + kc;
g = g + kc.*t;
b = b + kc.*n;

if nargout <= 1
    if nargin == 3 || threeD
        rout = cat(3,r,g,b);
    else
        rout = [r g b];
    end
    if isempty(rout)
        return
    else
        rout = bsxfun(@times, v./max(rout(:)), rout);
    end
else
    if isempty(r) || isempty(g) || isempty(b)
        rout = r;
        return
    else
        f = v./max([max(r(:)); max(g(:)); max(b(:))]);
    end
    rout = f.*r;
    g = f.*g;
    b = f.*b;
end