[3D-Caffe]Anaconda2+CUDA8+cuDNN5+OpenCV3编译3D-Caffe中遇到各种错误的解决方案

---

环境

系统：Ubuntu 14.04.1
显卡：GeForce GTX 1080 Ti (Driver Version: 384.69)

Anaconda2
CUDA版本：v8.0.44
cuDNN版本：v5
OpenCV版本：v3.0.0

---

配置

3D-Caffe源码下载地址：https://github.com/faustomilletari/3D-Caffe

  $  cd ~   
  $ git clone https://github.com/faustomilletari/3D-Caffe (先把源码拖到本地)
  $ cd 3D-Caffe
  $ cp Makefile.config.example Makefile.config (拷贝一份配置)
  $ vi Makefile.config  (对配置进行修改)

配置修改如下：

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
# USE_CUDNN := 1 【!!!去掉注释!!!】

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
        -gencode arch=compute_20,code=sm_21 \
        -gencode arch=compute_30,code=sm_30 \
        -gencode arch=compute_35,code=sm_35 \
        -gencode arch=compute_50,code=sm_50 \
        -gencode arch=compute_50,code=compute_50

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \ 【!!!加上注释!!!】
        /usr/lib/python2.7/dist-packages/numpy/core/include 【!!!加上注释!!!】
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda 【!!!去掉注释!!!并修改对应anaconda的安装路径】
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \ 【!!!去掉注释!!!】
        # $(ANACONDA_HOME)/include/python2.7 \ 【!!!去掉注释!!!】
        # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \ 【!!!去掉注释!!!】

# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib  【!!!加上注释!!!】
# PYTHON_LIB := $(ANACONDA_HOME)/lib 【!!!去掉注释!!!】

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include 【!!!添加hdf5路径,例如：/usr/include/hdf5/serial】
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

---

编译中遇到的错误

错误1

$ make all -j8

In file included from /usr/include/boost/python/detail/prefix.hpp:13:0,
                 from /usr/include/boost/python/args.hpp:8,
                 from /usr/include/boost/python.hpp:11,
                 from src/caffe/layer_factory.cpp:4:
/usr/include/boost/python/detail/wrap_python.hpp:50:23: fatal error: pyconfig.h: No such file or directory
# include 
                       ^
compilation terminated.
make: *** [.build_release/src/caffe/layer_factory.o] Error 1
make: *** Waiting for unfinished jobs....

解决方案1

$ make clean 
$ export CPLUS_INCLUDE_PATH=/usr/include/python2.7 
$ make all -j8

错误2

$ make all -j8

.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/caffe.bin] Error 1
make: *** Waiting for unfinished jobs....
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/upgrade_net_proto_binary.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/convert_imageset.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/extract_features.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/compute_image_mean.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/upgrade_net_proto_text.bin] Error 1

解决方案2

$ pkg-config --modversion opencv
$ 3.0.0

检查OpenCV安装正常，版本3.0.0，应该是库链接出了问题。解决办法：把找不到的lib添加到Makefile文件中。
修改Makefile文件，在文件172-174行，LIBRARIES 添加opencv_imgcodecs。

...省略...
##############################
# Derive include and lib directories
##############################
CUDA_INCLUDE_DIR := $(CUDA_DIR)/include

CUDA_LIB_DIR :=
# add /lib64 only if it exists
ifneq ("$(wildcard $(CUDA_DIR)/lib64)","")
        CUDA_LIB_DIR += $(CUDA_DIR)/lib64
endif
CUDA_LIB_DIR += $(CUDA_DIR)/lib

INCLUDE_DIRS += $(BUILD_INCLUDE_DIR) ./src ./include
ifneq ($(CPU_ONLY), 1)
        INCLUDE_DIRS += $(CUDA_INCLUDE_DIR)
        LIBRARY_DIRS += $(CUDA_LIB_DIR)
        LIBRARIES := cudart cublas curand
endif
LIBRARIES += glog gflags protobuf leveldb snappy \
        lmdb boost_system hdf5_hl hdf5 m \
        opencv_core opencv_highgui opencv_imgproc 【!!!此处加上opencv_imgcodecs】
PYTHON_LIBRARIES := boost_python python2.7
WARNINGS := -Wall -Wno-sign-compare

##############################
# Set build directories
##############################
...省略...

错误3

$ make all -j8

make: *** [.build_release/examples/siamese/convert_mnist_siamese_data.bin] Error 1
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFReadDirectory@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFWriteEncodedStrip@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFIsTiled@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFOpen@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFReadEncodedStrip@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFSetField@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFWriteScanline@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFGetField@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFScanlineSize@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFNumberOfStrips@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFSetWarningHandler@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFSetErrorHandler@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFReadEncodedTile@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFReadRGBATile@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFClose@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFRGBAImageOK@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so: undefined reference to `TIFFReadRGBAStrip@LIBTIFF_4.0'
collect2: error: ld returned 1 exit status

解决方案3

权限问题，切换到root执行

$ sudo su
$ make all -j8

错误4

$ make pycaffe -j8

CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
python/caffe/_caffe.cpp:1:52: fatal error: Python.h: No such file or directory
 #include  // NOLINT(build/include_alpha)
                                                    ^
compilation terminated.
make: *** [python/caffe/_caffe.so] Error 1

解决方案4

Anaconda路径不对，修改Makefile.config中的ANACONDA_HOME

 ANACONDA_HOME := $(HOME)/anaconda 【!!!修改为anaconda安装路径，例如$(HOME)/anaconda2】
 PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
                 $(ANACONDA_HOME)/include/python2.7 \
                 $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include

错误5

$ make pycaffe -j8

ERROR: test_solver (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_solver
Traceback (most recent call last):
  File "/home/lgw/anaconda2/lib/python2.7/unittest/loader.py", line 254, in _find_tests
    module = self._get_module_from_name(name)
  File "/home/lgw/anaconda2/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
    __import__(name)
  File "/home/lgw/3D-Caffe/python/caffe/test/test_solver.py", line 7, in <module>
    import caffe
  File "caffe/__init__.py", line 1, in <module>
    from .pycaffe import Net, SGDSolver
  File "caffe/pycaffe.py", line 14, in <module>
    import caffe.io
  File "caffe/io.py", line 8, in <module>
    from caffe.proto import caffe_pb2
  File "caffe/proto/caffe_pb2.py", line 4, in <module>
    from google.protobuf.internal import enum_type_wrapper
ImportError: No module named google.protobuf.internal

解决方案5

用Anaconda2中的conda命令安装protobuf

$ sudo chmod 777 -R anaconda2
$ conda install protobuf

错误6

$ make pytest -j8

======================================================================
ERROR: test_net_spec (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_net_spec
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/loader.py", line 254, in _find_tests
    module = self._get_module_from_name(name)
  File "/usr/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
    __import__(name)
  File "/home/lgw/3D-Caffe/python/caffe/test/test_net_spec.py", line 3, in <module>
    import caffe
  File "caffe/__init__.py", line 1, in <module>
    from .pycaffe import Net, SGDSolver
  File "caffe/pycaffe.py", line 13, in <module>
    from ._caffe import Net, SGDSolver
ImportError: libhdf5_hl.so.100: cannot open shared object file: No such file or directory

解决方案6

把anaconda2库目录下的文件拷贝一份到/usr/lib目录下

$ sudo cp ~/anaconda2/lib/libhdf5_hl.so.100 /usr/lib/libhdf5_hl.so.100
$ sudo ldconfig

---

编译通过

在python中导入caffe包正常

0123@user:~/3D-Caffe$ python
Python 2.7.14 |Anaconda, Inc.| (default, Dec  7 2017, 17:05:42) 
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
>>> 

---

参考

1. https://github.com/BVLC/caffe/issues/410
2. http://blog.sina.com.cn/s/blog_721a75e50102wfig.html
3. http://blog.csdn.net/yhl_leo/article/details/51371936
4. http://blog.csdn.net/wuzuyu365/article/details/52431062