效果图
点击ShowImg后
点击match,然后点击showmatch,可以不断点击showmatch。
主要功能描述:显示在SD卡上已经存在的图片test.jpg,根据图片在cameraframe对于每一帧计算和test.ipg的匹配并显示。
一个JavaCameraView用来显示帧相当于是相机的预览,两个ImgView一个用来显示要匹配的图像,一个用来显示最后得到的匹配图。三个Button对应三个View,ShowImg用来显示SD卡上的test.jpg,match开始匹配,showmatch,显示匹配的结果。(忽略名字的不统一。。。。)
采用线性布局的嵌套来实现布局,首先最外面是一个水平方向的线性布局,然后每行又是一个垂直方向的线性布局,一个垂直方向的线性布局里放的是JavaCameraView和match button,另一个则是两个ImgView和另两个button。
<?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="fill_parent" android:layout_height="fill_parent" android:orientation="horizontal" > <LinearLayout android:layout_width="fill_parent" android:layout_height="fill_parent" android:layout_weight="1" android:orientation="vertical" > <org.opencv.android.JavaCameraView android:id="@+id/objectMatch" android:layout_width="fill_parent" android:layout_height="fill_parent" android:layout_weight="1" /> <Button android:id="@+id/button_match" android:layout_width="wrap_content" android:layout_height="wrap_content" android:text="@string/Match" /> </LinearLayout> <LinearLayout android:layout_width="fill_parent" android:layout_height="fill_parent" android:layout_weight="1" android:orientation="vertical" > <ImageView android:id="@+id/ImgPhoto" android:layout_width="match_parent" android:layout_height="match_parent" android:layout_weight="1" android:src="@drawable/ic_launcher" /> <Button android:id="@+id/button_show" android:layout_width="wrap_content" android:layout_height="wrap_content" android:text="@string/showImg" /> <ImageView android:id="@+id/Imgmatch" android:layout_width="match_parent" android:layout_height="match_parent" android:layout_weight="1" android:src="@drawable/ic_launcher" /> <Button android:id="@+id/button_showmatch" android:layout_width="wrap_content" android:layout_height="wrap_content" android:text="@string/showmatch" /> </LinearLayout> </LinearLayout>
按照一般图像匹配的流程,即提训练和测试图像关键点,计算关键点的特征表示,计算训练测试图片的匹配点数,并画图。这个流程需要用的OpenCV4android API有
FeatureDetector .create,detect,match见前面章节的介绍
http://blog.csdn.net/h2008066215019910120/article/details/42650231
还有就是bitmap到mat,mat到bitmap之间的转换Utils.bitmapToMat,Utils.matToBitmap。
package com.example.objectmatch; import android.app.Activity; import android.os.Bundle; import android.view.Menu; import android.view.MenuItem; import java.io.File; import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.IOException; import java.sql.Date; import java.text.SimpleDateFormat; import java.util.ArrayList; import java.util.List; import org.opencv.android.BaseLoaderCallback; import org.opencv.android.CameraBridgeViewBase; import org.opencv.android.LoaderCallbackInterface; import org.opencv.android.OpenCVLoader; import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame; import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2; import org.opencv.android.Utils; import org.opencv.core.Core; import org.opencv.core.CvType; import org.opencv.core.Mat; import org.opencv.core.MatOfDMatch; import org.opencv.core.MatOfKeyPoint; import org.opencv.core.MatOfPoint; import org.opencv.core.Rect; import org.opencv.core.Scalar; import org.opencv.features2d.DescriptorExtractor; import org.opencv.features2d.DescriptorMatcher; import org.opencv.features2d.FeatureDetector; import org.opencv.features2d.Features2d; import org.opencv.features2d.KeyPoint; import org.opencv.imgproc.Imgproc; import android.app.Activity; import android.content.Context; import android.content.pm.PackageManager; import android.graphics.Bitmap; import android.graphics.BitmapFactory; import android.hardware.Camera; import android.hardware.Camera.PictureCallback; import android.os.Bundle; import android.os.Environment; import android.util.Log; import android.view.View; import android.view.WindowManager; import android.widget.Button; import android.widget.FrameLayout; import android.widget.ImageView; public class MainActivity extends Activity implements CvCameraViewListener2 { private Bitmap testimg; private Bitmap matchbitmap; private CameraBridgeViewBase mOpenCvCameraView; private Mat mRgba; private Mat mGray; private Mat mByte; private Scalar CONTOUR_COLOR; private boolean isProcess = false; private String filepath = "/sdcard/test.jpg"; private static final String TAG = "Dawn"; private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) { @Override public void onManagerConnected(int status) { switch (status) { case LoaderCallbackInterface.SUCCESS: { Log.i(TAG, "OpenCV loaded successfully"); mOpenCvCameraView.enableView(); } break; default: { super.onManagerConnected(status); } break; } } }; @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON); setContentView(R.layout.activity_main); mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.objectMatch); mOpenCvCameraView.setCvCameraViewListener(this); final ImageView showimg = (ImageView) findViewById(R.id.ImgPhoto); final ImageView matchimg = (ImageView) findViewById(R.id.Imgmatch); Button showButton = (Button) findViewById(R.id.button_show); Button matchButton = (Button) findViewById(R.id.button_match); Button showmatchButton = (Button) findViewById(R.id.button_showmatch); showButton.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { // show img in the Imageview File file = new File(filepath); if (file.exists()) { testimg = BitmapFactory.decodeFile(filepath); // 将图片显示到ImageView中 showimg.setImageBitmap(testimg); } } }); matchButton.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { // show img in the Imageview isProcess = !isProcess; } }); showmatchButton.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { // show img in the Imageview matchimg.setImageBitmap(matchbitmap); } }); } @Override protected void onPause() { super.onPause(); if (mOpenCvCameraView != null) mOpenCvCameraView.disableView(); } public void onResume() { super.onResume(); OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_10, this, mLoaderCallback); } @Override protected void onDestroy() { Log.e("onDestroy", "INITIATED"); super.onDestroy(); if (mOpenCvCameraView != null) mOpenCvCameraView.disableView(); } public void onCameraViewStarted(int width, int height) { mRgba = new Mat(height, width, CvType.CV_8UC3); mByte = new Mat(height, width, CvType.CV_8UC1); } public void onCameraViewStopped() { // Explicitly deallocate Mats mRgba.release(); } public Mat onCameraFrame(CvCameraViewFrame inputFrame) { Bitmap s_testimg; Mat testimage = new Mat(); Mat grayimage=new Mat(); mRgba = inputFrame.rgba(); mGray = inputFrame.gray(); CONTOUR_COLOR = new Scalar(255); MatOfDMatch matches = new MatOfDMatch(); MatOfKeyPoint keypoint_train = new MatOfKeyPoint(); MatOfKeyPoint keypoint_test = new MatOfKeyPoint(); KeyPoint kpoint = new KeyPoint(); Mat mask = Mat.zeros(mGray.size(), CvType.CV_8UC1); Mat output = new Mat(); // Mat train=new Mat(); Mat Mat test = new Mat(); Mat train = new Mat(); if (isProcess) { FeatureDetector detector_train = FeatureDetector .create(FeatureDetector.ORB); detector_train.detect(mGray, keypoint_train); // Features2d.drawKeypoints(mGray, keypoint_train, output, new Scalar( // 2, 254, 255), Features2d.DRAW_RICH_KEYPOINTS); DescriptorExtractor descriptor_train = DescriptorExtractor .create(DescriptorExtractor.ORB); descriptor_train.compute(mGray, keypoint_train, train); s_testimg = Bitmap.createScaledBitmap(testimg, mGray.width(), mGray.height(), false); Utils.bitmapToMat(s_testimg, testimage); Imgproc.cvtColor(testimage, grayimage, Imgproc.COLOR_RGB2GRAY); FeatureDetector detector_test = FeatureDetector .create(FeatureDetector.ORB); detector_test.detect(grayimage, keypoint_test); // Features2d.drawKeypoints(testimage, keypoint_test, output, // new Scalar(2, 254, 255), Features2d.DRAW_RICH_KEYPOINTS); DescriptorExtractor descriptor_test = DescriptorExtractor .create(DescriptorExtractor.ORB); descriptor_test.compute(grayimage, keypoint_test, test); DescriptorMatcher descriptormatcher = DescriptorMatcher .create(DescriptorMatcher.BRUTEFORCE_HAMMING); descriptormatcher.match(test, train, matches); Features2d.drawMatches(grayimage,keypoint_test,mGray, keypoint_train, matches, output); matchbitmap=Bitmap.createScaledBitmap(testimg, output.width(), output.height(), false); Utils.matToBitmap(output, matchbitmap); return mRgba; } return mRgba; } }
原因:可能是SIFT这个属于nonfree模块,在手机上无法运行,也可能是SIFT计算时间长,存储空间太大。每个关键点128维,每维需要4个字节(float型)来存储,而一张图片可能有上百或者上千个关键点。
解决:改用其他特征来进行识别
可选项
static int
BRIEF
static int
BRISK
static int
FREAK
static int
OPPONENT_BRIEF
static int
OPPONENT_BRISK
static int
OPPONENT_FREAK
static int
OPPONENT_ORB
static int
OPPONENT_SIFT
static int
OPPONENT_SURF
static int
ORB
BRIEF:主要思路就是在特征点附近随机选取若干点对,将这些点对的灰度值的大小,组合成一个二进制串,并将这个二进制串作为该特征点的特征描述子
分析见:http://www.cnblogs.com/ronny/p/4081362.html
ORB:
RIEF的优点在于速度,缺点也相当明显:1:不具备旋转不变性。2:对噪声敏感3:不具备尺度不变性。ORB就解决上述1和2两个问题。
分析见:http://www.cnblogs.com/ronny/p/4083537.html
这些特征区别于SIFT在于:
SIFT特征采用了128维的特征描述子,由于描述子用的浮点数,所以它将会占用512 bytes的空间。类似地,对于SURF特征,常见的是64维的描述子,它也将占用256bytes的空间。如果一幅图像中有1000个特征点(不要惊讶,这是很正常的事),那么SIFT或SURF特征描述子将占用大量的内存空间,对于那些资源紧张的应用,尤其是嵌入式的应用,这样的特征描述子显然是不可行的。而且,越占有越大的空间,意味着越长的匹配时间。但是实际上SFIT或SURF的特征描述子中,并不是所有维都在匹配中有着实质性的作用。我们可以用PCA、LDA等特征降维的方法来压缩特征描述子的维度。还有一些算法,例如LSH,将SIFT的特征描述子转换为一个二值的码串,然后这个码串用汉明距离进行特征点之间的匹配。这种方法将大大提高特征之间的匹配,因为汉明距离的计算可以用异或操作然后计算二进制位数来实现,在现代计算机结构中很方便。
01-24 10:23:58.251: E/AndroidRuntime(3535): CvException [org.opencv.core.CvException: cv::Exception: /hdd2/buildbot/slaves/slave_ardbeg1/50-SDK/opencv/modules/features2d/src/draw.cpp:208: error: (-215) i2 >= 0 && i2 < static_cast<int>(keypoints2.size()) in function void cv::drawMatches(const cv::Mat&, const std::vector<cv::KeyPoint>&, const cv::Mat&, const std::vector<cv::KeyPoint>&, const std::vector<cv::DMatch>&, cv::Mat&, const Scalar&, const Scalar&, const std::vector<char>&, int)
原因:descriptormatcher.match(test, train, matches);
Features2d.drawMatches(grayimage,keypoint_test,mGray, keypoint_train, matches, output);的参数写反了
第一个参数应该是test的图片,第二个参数是train的图片。要跟前面的match函数匹配起来。
两个参数写反了
OpenCV4android的资料确实太少,API和文档都不是很完善,需要先找到C++的实现然后转换到java上来。
这个程序还存在不少问题,很多功能都没有完善待添加功能
1.能够从图库中选取或者拍照得到测试的图片。
2.但测试图片从帧图像中找到匹配后,停止并在界面上显示匹配的图像。
代码地址:https://github.com/dawnminghuang/objectMatch