Transplanting OpenCV2.2.0 to ARM cortex-A9 Platform

移植 OpenCV 2.2.0 到 ARM Cortex-A9 平台

Keyword: OpenCV，2.2.0，ARM，Cortex-A9，OMAP4430，PandaBoard,cross-compile

这篇文章我也上传到了google doc ，方便大家一起参与编辑与完善。

link: https://docs.google.com/document/d/1Yk2Fv5kC1V8kZuDDpuMUqUqNaYj9MULrfGVubhyt22g/edit?pli=1

Develop environment

Host: ubuntu 10.10

OpenCV version: 2.2.0

Cross-Compiler version: 4.5.1

Target-Platform:PandaBoard (Omap 4430)

Kernel version: 2.6.35

Recently I got anproject want to use OpenCV in ARM platform . Though we got lot ofguides and manuals from Internet. But because of the board I have isomap4430 which contains dual-core Crotex-A9. I have to usehigh-version cross-compiler and high-version OpenCV source code .

I spend nearly twoweeks to finish this work. Not only the processing I record but alsothe problems I got and the solution I found in this guide . I reallyhope this guide will help you transplant smoothly. You are also verywelcomed to commit problem or advice you got to help me completethis guide.

As we know , OpenCVusing cmake to configure and generate Makefile from version2.1.0(including 2.1.0). That has a lot of different with“./configure+arguments”mode. cmake-curses-gui is a tool of cmake,which can help us configure and modify arguments with a simple GUIinstead of complex command arguments. Before transplantation we haveto prepare tools, cross-compiler and sources.

1.Install cmake

sudo apt-get install cmake

2.Install cmake-curses-gui

sudo apt-get installcmake-curses-gui

NOTE:

Problem you may encounter:cmake-curses-gui depends （=2.8.0-5ubuntu1）But cmake was installed depends 2.8.2-2ubuntu0.1library.

Solution: UseSynaptic package Manager remove high-version cmake youhad. Then force install cmake in version 2.8.0-5ubuntu1 . Afterthat cmake-curses-gui will install successful. Don't forget to“Apply” them.

3.DownloadCross-Compiler and Source Code we need

1. Download and installCodesourcery 2010.12 .

DownloadURL:http://www.codesourcery.com/sgpp/lite/arm/portal/release1600 This arm-linux-gcc version is 4.5.1.

2. Unzip and export it toyour path.

tar -xvjf./arm-2010.09-50-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2

-C /opt

exportPATH=$PATH:/opt/arm-2010.12/bin

To check you haveinstalled and configure it correct.

arm-none-linux-gnueabi-g++-v

If you installed andconfigure it successful, you will see following information :

Configured with:/scratch/nathan/arm-lite/src/gcc-4.5-2010.09/configure--build=i686-pc-linux-gnu --host=i686-pc-linux-gnu--target=arm-none-linux-gnueabi --enable-threads --disable-libmudflap--disable-libssp --disable-libstdcxx-pch--enable-extra-sgxxlite-multilibs --with-arch=armv5te --with-gnu-as--with-gnu-ld --with-specs='%{save-temps: -fverbose-asm}%{funwind-tables|fno-unwind-tables|mabi=*|ffreestanding|nostdlib:;:-funwind-tables}-D__CS_SOURCERYGXX_MAJ__=2010 -D__CS_SOURCERYGXX_MIN__=9-D__CS_SOURCERYGXX_REV__=50 %{O2:%{!fno-remove-local-statics:-fremove-local-statics}}%{O*:%{O|O0|O1|O2|Os:;:%{!fno-remove-local-statics:-fremove-local-statics}}}' --enable-languages=c,c++ --enable-shared--enable-lto --enable-symvers=gnu --enable-__cxa_atexit--with-pkgversion='Sourcery G++ Lite 2010.09-50'--with-bugurl=https://support.codesourcery.com/GNUToolchain/--disable-nls --prefix=/opt/codesourcery--with-sysroot=/opt/codesourcery/arm-none-linux-gnueabi/libc--with-build-sysroot=/scratch/nathan/arm-lite/install/arm-none-linux-gnueabi/libc--with-gmp=/scratch/nathan/arm-lite/obj/host-libs-2010.09-50-arm-none-linux-gnueabi-i686-pc-linux-gnu/usr--with-mpfr=/scratch/nathan/arm-lite/obj/host-libs-2010.09-50-arm-none-linux-gnueabi-i686-pc-linux-gnu/usr--with-mpc=/scratch/nathan/arm-lite/obj/host-libs-2010.09-50-arm-none-linux-gnueabi-i686-pc-linux-gnu/usr--with-ppl=/scratch/nathan/arm-lite/obj/host-libs-2010.09-50-arm-none-linux-gnueabi-i686-pc-linux-gnu/usr--with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic-lm'--with-cloog=/scratch/nathan/arm-lite/obj/host-libs-2010.09-50-arm-none-linux-gnueabi-i686-pc-linux-gnu/usr--with-libelf=/scratch/nathan/arm-lite/obj/host-libs-2010.09-50-arm-none-linux-gnueabi-i686-pc-linux-gnu/usr--disable-libgomp --enable-poison-system-directories--with-build-time-tools=/scratch/nathan/arm-lite/install/arm-none-linux-gnueabi/bin--with-build-time-tools=/scratch/nathan/arm-lite/install/arm-none-linux-gnueabi/bin

Thread model: posix

gcc version 4.5.1(Sourcery G++ Lite 2010.09-50)

3. Then download and unzipOpenCV sources.

DownloadURL:http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.2/

tar -xvjfOpenCV-2.2.0.tar.bz2 -C /YOURDIR

NOTE: you'd better choosea directory which have permission to compile.

4.Create a builddirectory to avoid mixing install files with source code.

mkdir/YOURDIR/build && cd /YOURDIR build

5.Create atoolchain.cmake file .This file is used to tell cmake how tocross-compile,and some other infromation

gedit./toolchain.cmake

6.Paste following totoolchain.cmake and save it . Last /DIR means the path you want toinstall.

set( CMAKE_SYSTEM_NAMELinux )

set(CMAKE_SYSTEM_PROCESSOR arm )

set( CMAKE_C_COMPILER arm-none-linux-gnueabi-gcc ) set(CMAKE_CXX_COMPILER arm-none-linux-gnueabi-g++)

set( CMAKE_FIND_ROOT_PATH /DIR )

NOTE: If you not sure to your PATH. I suggest using an absolute pathlike this: /opt/toolschain/4.5.1/bin

7.Edit CMakeLists.txt

Important:The CMake build setup in OpenCV 2.2 and earlier does not have NEONENABLE flag to to take advantage of neon accelearation. Hence pleasemake the following modifications to the file CMakeLists.txt in OpenCV2.2 to enable neon acceleration.

#Other optimizations

if(USE_O2)

set(EXTRA_C_FLAGS_RELEASE "${EXTRA_C_FLAGS_RELEASE} -O2-mfpu=neon")

endif()

This may not be requiredin later versions of OpenCV since OpenCV 2.3 has the option to enableneon in the CMake build process.

8.Run Cmake to processand generate Makefile

cmake-DCMAKE_TOOLCHAIN_FILE=toolchain.cmake /YOURDIR

9.Use ccmke toconfigure

ModifyCMAKE_INSTALL_PREFIX to which PATH you want to install if necessary.

Make sure that thefollowing items are OFF. These items based 3^rd partylibrary.

• BUILD_NEW_PYTHON_SUPPORT

• BUILD_TESTS

• WITH_1394

• WITH_CUDA

• WITH_EIGEN2

• WITH_FFMPEG

• WITH_GSTREAMER

• WITH_GTK

• WITH_JASPER

• WITH_JPEG

• WITH_OPENEXR

• WITH_PNG

• WITH_PVAPI

• WITH_QT

• WITH_QT_OPENGL

• WITH_TBB

• WITH_TIFF

• WITH_UNICAP

• WITH_V4L

• WITH_XINE

If you want to use any ofthem,please cross-compile it before transplant OpenCV.

NOTE: Do not forget tokeep the USE_O2 option ON in the configuration to take advantage ofneon acceleration that you added to this option.

10.Now press “c”to configure and “g” to generate Makefile again.

11.Everything seemsready now. Run “make” from command line to generate OpenCVlibraries. “make install” will help you install libraries to“CMAKE_INSTALL_PREFIX” you modified before.

You will see thesedynamic libraries in ./.libs and your install prefix directory.

Problem you may encounter:

> [ 86%] Built targetpch_Generate_opencv_test
>Linking CXX executable ../../bin/opencv_test
>CMakeFiles/opencv_test.dir/src/anearestneighbors.o: Infunction
`CV_FlannSavedIndexTest::createModel(cv::Matconst&)':
>
anearestneighbors.cpp:(.text._ZN22CV_FlannSavedIndexTest11createModelERKN2cv3Mat\
E+0x78):warning: the use of `tmpnam' is dangerous, better use `mkstemp'
>../../lib/libopencv_features2d.so.2.2.0: undefined referenceto
`cv::SIFT::SIFT(double,bool, bool, int, int, int, int)'
>../../lib/libopencv_features2d.so.2.2.0: undefined referenceto
`cv::SIFT::CommonParams::CommonParams()'
>../../lib/libopencv_features2d.so.2.2.0: undefined referenceto
`cv::SIFT::DescriptorParams::DescriptorParams()'
>../../lib/libopencv_features2d.so.2.2.0: undefined referenceto
`cv::SIFT::DetectorParams::DetectorParams()'
>../../lib/libopencv_features2d.so.2.2.0: undefined referenceto
`cv::SIFT::operator()(cv::Matconst&, cv::Mat const&,std::vector<cv::KeyPoint,
std::allocator<cv::KeyPoint>>&, cv::Mat&, bool) const'
>../../lib/libopencv_features2d.so.2.2.0: undefined referenceto
`cv::SIFT::operator()(cv::Matconst&, cv::Mat const&,std::vector<cv::KeyPoint,
std::allocator<cv::KeyPoint>>&) const'
>../../lib/libopencv_features2d.so.2.2.0: undefined referenceto
`cv::SIFT::SIFT(double,double, int, int, int, int)'
>collect2: ld returned 1 exit status
>make[2]: *** [bin/opencv_test] Error 1
>make[1]: *** [tests/cv/CMakeFiles/opencv_test.dir/all] Error 2
>make: *** [all] Error 2

solution:Findbranches/2.2/opencv/modules/features2d/src/sift.cpp ¶

Modify from 49-58 like following.

#include"precomp.hpp"

#ifdef__arm__

#defineARM_NO_SIFT

#endif

#ifdefANDROID

#undefARM_NO_SIFT

#endif//ANDROID

TO

//#ifdef__arm__

//#defineARM_NO_SIFT

//#endif

//#ifdefANDROID

//#undefARM_NO_SIFT

//#endif//ANDROID

#ifndefARM_NO_SIFT

This may help you fix thatproblem .

12.So far, we have compiled OpenCVlibraries to ARM-Arch . Test your dynamic libraries.

arm-none-linux-gnueabi-g++ -I/home/boat/arm/include/opencv -I/home/boat/arm/include-L/home/boat/arm/lib -lopencv_core -lopencv_imgproc -lopencv_highgui-lopencv_ml -lopencv_video -lopencv_features2d -lopencv_calib3d-lopencv_objdetect -lopencv_contrib -lopencv_legacy -lopencv_flann ./opencv_test.c -o ./opencv_test_arm.o

Following is the opencv_test.c Imodify from an opencv sample. This sample is used for edge detectionwithout GUI.

Code:

#include "cv.h"

#include "highgui.h"

IplImage* doCanny(

IplImage* in,

double lowThresh,

double highThresh,

double aperture)

{

if (in->nChannels != 1)

return(0); // Canny onlyhandles gray scale images

IplImage* out =cvCreateImage(

cvGetSize( in ),

in->depth,//IPL_DEPTH_8U,

1);

cvCanny( in, out, lowThresh,highThresh, aperture );

return( out );

};

int main( int argc, char** argv)

{

IplImage* img_rgb =cvLoadImage( argv[1] );

IplImage* img_gry =cvCreateImage( cvSize( img_rgb->width,img_rgb->height ),img_rgb->depth, 1);

cvCvtColor(img_rgb, img_gry,CV_BGR2GRAY);

// cvNamedWindow("ExampleGray", CV_WINDOW_AUTOSIZE );

// cvNamedWindow("ExampleCanny", CV_WINDOW_AUTOSIZE );

// cvShowImage("ExampleGray", img_gry );

IplImage* img_cny = doCanny(img_gry, 10, 100, 3 );

if(cvSaveImage(argv[2],img_cny,0)!=0)

printf("Save ImageSuccessful\n");

// cvShowImage("ExampleCanny", img_cny );

// cvWaitKey(0);

cvReleaseImage( &img_rgb);

cvReleaseImage( &img_gry);

cvReleaseImage( &img_cny);

// cvDestroyWindow("ExampleGray");

// cvDestroyWindow("ExampleCanny");

return 0;

}

If you have any question with “-l-L” arguments ,please “man gcc”.

13.Copy libraries and executablefile we have compiled and some test image(bmp format) to ARM board.

cp /home/boat/arm/lib/* ~/targetfs/usr/lib

cp ./opencv_tar-arm.o~/targetfs/home/code

cp ./lena.bmp~/targetfs/home/code

NOTE: Don't forget to “chmod+x” to executable file.

Then switch to ARM board terminaland run it.

./opencv_test-arm.o ./lena.bmp

NOTE：Wecan only handle bmp format image . Because when configure “ccmake”,we OFF all the 3^rd party libraries including JPEG PNG.

Problem you may encounter:

~/Code # ./opencv_test-arm.o ./lena.bmp

./opencv_test-arm.o: error whileloading shared libraries: libstdc++.so.6: cannot open shared object

Solution:

Copy libstdc++.* from/arm-none-linux-gnueabi/libc/usr/lib/ in your host PC to ARM-platform/usr/lib.

NOTE:I copied it from 4.5.1libraries but it seems has some version problem. So I use the otherversion lib (4.4.3). It finally works.

Reference documentation &&link:

http://opencv.willowgarage.com/wiki/

http://processors.wiki.ti.com/index.php/Building_OpenCV_for_ARM_Cortex-A8

https://code.ros.org/trac/opencv/changeset/4812

https://groups.google.com/forum/?fromgroups#!topic/android-opencv/rp9DMrrI1DU

http://karytech.blogspot.com/2011/02/opencv-22-on-ubuntu-1010.html

http://blog.csdn.net/noodies/article/details/5798434

http://blog.csdn.net/gfocean/article/details/6341155

http://ubuntuforums.org/showthread.php?t=1809300

You are very welcomed to commitproblem or advice you got (e.g. Transplant 3^rd partylibrary)to help me complete this guide.

Boat Jiang

24/4/2012

Following are some problemsencountered during my transplantation (lower version )which I haven'tsolved.

As you know,OpenCV below2.0(including 2.0) version use “./configure + arguments” mode toconfigure Makefile. I had tried a lot versions of cross-compiler andopencv source. But always has ridiculous problem. So I record some ofthem above for anyone which can solve them.

I had tried OpenCV2.0 source witharm-linux-gcc-4.3.2 to cross compile.

Configure arguments:

sudo ./configure--host=arm-linux --without-gtk --without-carbon --without-quicktime--without-1394libs --without-ffmpeg --with-v4l --without-python--without-swig --enable-static --enable-shared --disable-appsCXX=/opt/toolschain/4.3.2/bin/arm-none-linux-gnueabi-g++CPPFLAGS=-I/opt/toolschain/4.3.2/arm-none-linux-gnueabi/include LDFLAGS=-L/opt/toolschain/4.3.2/arm-none-linux-gnueabi/libc/lib --prefix=/home/boat/arm --libdir=/home/boat/arm/lib–includedir=/home/boat/arm/include

If I use “--enable-shared ”argument to make .I got these Error information:

/opt/toolschain/4.3.2/bin/../lib/gcc/arm-none-linux-gnueabi/4.3.2/../../../../arm-none-linux-gnueabi/bin/ld:../3rdparty/.libs/lib_clapack.a(dlasyf.o): Relocations in generic ELF(EM: 3)

../3rdparty/.libs/lib_clapack.a(dlasyf.o):could not read symbols: File in wrong format

collect2: ld returned 1 exitstatus

make[2]: *** [libcxcore.la]Error 1

make[2]: Leaving directory`/usr/src/OpenCV-Source/OpenCV-2.0.0-new/build/src'

make[1]: *** [all-recursive]Error 1

make[1]: Leaving directory`/usr/src/OpenCV-Source/OpenCV-2.0.0-new/build'

make: *** [all] Error 2

So I change it to“--disable-shared” ,that I hope can link manual by staticlibraries.

It really works to generate staticlibraries. But when I try to use these static libraries tocross-compile samples code . I got this problem:

opencv-arm/lib/libcxcore.a(dgetri.o):Relocations in generic ELF (EM: 3) libcxcore.a: could not readsymbols: File in wrong format

“readelf -h ./libcxcore.a”in command line to check format of this file.

File: libcxcore.a(dgeqr2.o)

ELF Header:

Magic: 7f 45 4c 46 01 01 0100 00 00 00 00 00 00 00 00

Class: ELF32

Data: 2's complement, little endian

Version: 1 (current)

OS/ABI: UNIX - System V

ABI Version: 0

Type: REL (Relocatable file)

Machine: Intel 80386

Version: 0x1

Entry point address: 0x0

Start of program headers: 0 (bytes into file)

Start of section headers: 712 (bytes into file)

Flags: 0x0

Size of this header: 52 (bytes)

Size of program headers: 0 (bytes)

Number of program headers: 0

Size of section headers: 40 (bytes)

Number of section headers: 11

Section header string tableindex: 8

It actully 80386 format! But others*.a files are all ARM format. They are generated by the same Makefile. It really confused me. The other interesting thing is lib_cxcore.ais ARM-Ach format. But I link it manual when a cross-compile asample, It still can't work.

Because of these ridiculousproblems, I raise both opencv and cross-compiler version. And finallytransplant successfully.

If anyone who got the reason ofthese problems even solved them. Please commit or connect me to helpme complete this guide. Thank you.

Email:[email protected]

Boat Jiang

24/4/2012