OpenCV教程(11)-锐化滤波器

图像滤波器

Mask operations on matrices

---

Mask operations on matrices are quite simple. The idea is that we recalculate each pixels value in an image according to a mask matrix (also known as kernel). This mask holds values that will adjust how much influence neighboring pixels (and the current pixel) have on the new pixel value. From a mathematical point of view we make a weighted average, with our specified values.

---

Our test case 

Let us consider the issue of an image contrast enhancement method. Basically we want to apply for every pixel of the image the following formula:

The first notation is by using a formula, while the second is a compacted version of the first by using a mask. You use the mask by putting the center of the mask matrix (in the upper case noted by the zero-zero index) on the pixel you want to calculate and sum up the pixel values multiplied with the overlapped matrix values. It's the same thing, however in case of large matrices the latter notation is a lot easier to look over.

Now let us see how we can make this happen by using the basic pixel access method or by using the cv::filter2D function.

---

The Basic Method

Here's a function that will do this:

void Sharpen(const Mat& myImage,Mat& Result)  
 /*Image enhancement - Sharpen operator*/
{
    CV_Assert(myImage.depth() == CV_8U);  // accept only uchar images
    const int nChannels = myImage.channels();    //to get the number of the image channels.  
    Result.create(myImage.size(),myImage.type());
    for(int j = 1 ; j < myImage.rows-1; ++j)
    {
        const uchar* previous = myImage.ptr(j - 1);
        const uchar* current  = myImage.ptr(j    );
        const uchar* next     = myImage.ptr(j + 1);
        uchar* output = Result.ptr(j);
        for(int i= nChannels;i < nChannels*(myImage.cols-1); ++i)
        {
            *output++ = saturate_cast(5*current[i]
                         -current[i-nChannels] - current[i+nChannels] - previous[i] - next[i]);
        }
    }
    Result.row(0).setTo(Scalar(0));
    Result.row(Result.rows-1).setTo(Scalar(0));
    Result.col(0).setTo(Scalar(0));
    Result.col(Result.cols-1).setTo(Scalar(0));
}

At first we make sure that the input images data is in unsigned char format. For this we use the cv::CV_Assert function that throws an error when the expression inside it is false.

 CV_Assert(myImage.depth() == CV_8U);  /* accept only uchar images, 
uchar in between 0~255, gray level image only. */

We create an output image with the same size and the same type as our input. As you can see in the storing section, depending on the number of channels we may have one or more subcolumns.

We will iterate through them via pointers so the total number of elements depends on this number.

    const int nChannels = myImage.channels();
    Result.create(myImage.size(),myImage.type());

We'll use the plain C [] operator to access pixels. Because we need to access multiple rows at the same time we'll acquire the pointers for each of them (a previous, a current and a next line). We need another pointer to where we're going to save the calculation. Then simply access the right items with the [] operator. For moving the output pointer ahead we simply increase this (with one byte) after each operation:

for(int j = 1 ; j < myImage.rows-1; ++j)
    {
        const uchar* previous = myImage.ptr(j - 1);
        const uchar* current  = myImage.ptr(j    );
        const uchar* next     = myImage.ptr(j + 1);
        uchar* output = Result.ptr(j);
        for(int i= nChannels;i < nChannels*(myImage.cols-1); ++i)
        {
            *output++ = saturate_cast(5*current[i]
                         -current[i-nChannels] - current[i+nChannels] - previous[i] - next[i]);
        }
    }

On the borders of the image the upper notation results inexistent pixel locations (like minus one - minus one). In these points our formula is undefined. A simple solution is to not apply the kernel in these points and, for example, set the pixels on the borders to zeros:

    Result.row(0).setTo(Scalar(0));
    Result.row(Result.rows-1).setTo(Scalar(0));
    Result.col(0).setTo(Scalar(0));
    Result.col(Result.cols-1).setTo(Scalar(0));

---

The filter2D function 

Applying such filters are so common in image processing that in OpenCV there exist a function that will take care of applying the mask (also called a kernel in some places). For this you first need to define an object that holds the mask:

Mat kernel = (Mat_(3,3) <<  0, -1,  0,
                                   -1,  5, -1,
                                    0, -1,  0);

Then call the cv::filter2D function specifying the input, the output image and the kernel to use:

 filter2D( src, dst1, src.depth(), kernel );

The function even has a fifth optional argument to specify the center of the kernel, a sixth for adding an optional value to the filtered pixels before storing them in K and a seventh one for determining what to do in the regions where the operation is undefined (borders).

 

For example:

/**
 * Created with Clion IDEA.
 * Description: 
 * 
 * User: haowang
 * Date: 2019-01-23
 * Time: 19:10
 */

#include 
#include 
#include 
#include 

using namespace std;
using namespace cv;

static void help(char *progName) {
	cout << endl
	     << "This program shows how to filter images with mask: the write it yourself and the"
	     << "filter2d way. " << endl
	     << "Usage:" << endl
	     << progName << " [image_path -- default ../data/lena.jpg] [G -- grayscale] " << endl << endl;
}

double getPSNR(const Mat &I1, const Mat &I2);  
/* Peak Signal to Noise Ratio */

void Sharpen(const Mat &myImage, Mat &Result);
/* Hand written sharpen function  */

int main(int argc, char *argv[]) {
	help(argv[0]);
	const char *filename = argc >= 2 ? argv[1] : "/Users/haowang/Desktop/Workspace_C++/400px-Chess.JPG";

	Mat src, dst0, dst1;

	if (argc >= 3 && !strcmp("G", argv[2]))
		src = imread(filename, IMREAD_GRAYSCALE);
	else
		src = imread(filename, IMREAD_COLOR);

	if (src.empty()) {
		cerr << "Can't open image [" << filename << "]" << endl;
		return -1;
	}

	namedWindow("Input", WINDOW_AUTOSIZE);
	namedWindow("Output", WINDOW_AUTOSIZE);

	imshow("Input", src);

	double t = (double) getTickCount();

	Sharpen(src, dst0);

	t = ((double) getTickCount() - t) / getTickFrequency();

	double psnr_hand = getPSNR(src, dst0);
	cout << "Hand written function time passed in seconds: " << t << endl;
	cout << "Hand written funciton sharpen psnr is: " << psnr_hand << "dB" << endl;

	imshow("Output", dst0);
	waitKey();

	//![kern]
	Mat kernel = (Mat_(3, 3) << 0, -1, 0,
			-1, 5, -1,
			0, -1, 0);
	//![kern]

//	double t = (double) getTickCount();

	//![filter2D]
	filter2D(src, dst1, src.depth(), kernel);
	//![filter2D]
	t = ((double) getTickCount() - t) / getTickFrequency();

	double psnr_filter2D = getPSNR(src, dst1);
	cout << "Built-in filter2D time passed in seconds:     " << t << endl;
	cout << "Built-in filter2D  sharpen psnr is: " << psnr_filter2D << "dB" << endl;
	imshow("Output_Filter2D", dst1);

	waitKey();
	return 0;
}

//! [basic_method]
void Sharpen(const Mat &myImage, Mat &Result) {
	//! [8_bit]
	CV_Assert(myImage.depth() == CV_8U);  // accept only uchar images
	//! [8_bit]

	//! [create_channels]
	const int nChannels = myImage.channels();
	Result.create(myImage.size(), myImage.type());
	//! [create_channels]

	//! [basic_method_loop]
	for (int j = 1; j < myImage.rows - 1; ++j) {
		const uchar *previous = myImage.ptr(j - 1);
		const uchar *current = myImage.ptr(j);
		const uchar *next = myImage.ptr(j + 1);

		uchar *output = Result.ptr(j);

		for (int i = nChannels; i < nChannels * (myImage.cols - 1); ++i) {
			*output++ = saturate_cast(5 * current[i]
			                                 - current[i - nChannels] - current[i + nChannels] - previous[i] - next[i]);
		}
	}
	//! [basic_method_loop]

	//! [borders]
	Result.row(0).setTo(Scalar(0));
	Result.row(Result.rows - 1).setTo(Scalar(0));
	Result.col(0).setTo(Scalar(0));
	Result.col(Result.cols - 1).setTo(Scalar(0));
	//! [borders]
}
//! [basic_method]

double getPSNR(const Mat &I1, const Mat &I2) {
	Mat s1;
	absdiff(I1, I2, s1);       // |I1 - I2|
	s1.convertTo(s1, CV_32F);  // cannot make a square on 8 bits
	s1 = s1.mul(s1);           // |I1 - I2|^2

	Scalar s = sum(s1);        // sum elements per channel

	double sse = s.val[0] + s.val[1] + s.val[2]; // sum channels

	if (sse <= 1e-10) // for small values return zero
		return 0;
	else {
		double mse = sse / (double) (I1.channels() * I1.total());
		double psnr = 10.0 * log10((255 * 255) / mse);
		return psnr;
	}
}

Result:

This function is shorter, less verbose and, because there are some optimizations, it is usually faster than the hand-coded method. For example in my test while the second one took only 5.34 milliseconds the first took around 1.23 milliseconds. Quite some difference.

                                         (Left is the origional image; another one is the output of basic_sharpen_method )

                                            (Left is the origional image; another one is the output of sharpen_filter2D )

Peak Signal To Noise Ratio shown below：

Hand written function time passed in seconds: 0.0053403
Hand written funciton sharpen psnr is: 15.2533dB
Built-in filter2D time passed in seconds:     0.00123578
Built-in filter2D  sharpen psnr is: 29.7174dB

---