在上一节,我们安装好了所有caffe可能用到的软件,我这里再列举一下:
- NVIDIA驱动367.57
- CUDA8.0
- Cudnn v5.1
- MKL(这里我提一句,最后BLAS可以不用MKL,用openblas或者atlas都可以)
- MATLAB2014a
- Anaconda(Python 2.7版)
- OpenCV 2.4.13
接下来,我们进入caffe的安装与编译环节。
在安装caffe前,我们先来安装一些依赖
sudo apt-get install -y libprotobuf-dev libleveldb-dev libsnappy-dev libhdf5-serial-dev protobuf-compiler;
sudo apt-get install -y libatlas-base-dev;
sudo apt-get install -y --no-install-recommends libboost-all-dev;
sudo apt-get install -y libgflags-dev libgoogle-glog-dev liblmdb-dev;
sudo apt-get install -y python-pip python-dev python-numpy python-scipy;
sudo apt-get install -y libopencv-dev;
安装依赖后,我们再来修改Make.config文件,如下:
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
USE_OPENCV := 0
USE_LEVELDB := 0
USE_LMDB := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
-gencode arch=compute_20,code=sm_21 \
-gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_50,code=compute_50
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
#BLAS_INCLUDE :=/opt/intel/mkl/include
#BLAS_LIB :=/opt/intel/mkl/lib/intel64
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
MATLAB_DIR := /usr/local/MATLAB/R2014a
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
#PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := $(HOME)/anaconda2
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
$(ANACONDA_HOME)/include/python2.7 \
$(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \
# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
# /usr/lib/python3.5/dist-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
#PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1
# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
Q ?= @
最后我们来编译caffe和python,matlab与caffe的接口,并测试。
编译caffe:
sudo make all -j16;
sudo make test -j16;
sudo make runtest -j16;
编译pycaffe和matcaffe:
sudo make pycaffe -j16;
sudo make matcaffe -j16;
sudo make pytest -j16;
sudo make mattest -j16;
如果以上命令全部成功执行,我们就完全安装好了caffe。
如果安装过程中出现了错误,那该怎么处理呢?
我在这里就列举一下,当初我安装caffe与编译pycaffe和matcaffe时,出现的错误。
编译caffe时出现的错误
boost错误
CXX src/caffe/layers/hinge_loss_layer.cpp
CXX src/caffe/layers/threshold_layer.cpp
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/relu_layer.hpp:6,
from src/caffe/layers/relu_layer.cpp:4:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/power_layer.hpp:6,
from src/caffe/layers/power_layer.cpp:3:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/relu_layer.o] Error 1
make: *** Waiting for unfinished jobs....
make: *** [.build_release/src/caffe/layers/power_layer.o] Error 1
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/filler.hpp:10,
from src/caffe/layers/bias_layer.cpp:3:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/cudnn_relu_layer.hpp:6,
from src/caffe/layers/cudnn_relu_layer.cpp:4:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/cudnn_relu_layer.o] Error 1
make: *** [.build_release/src/caffe/layers/bias_layer.o] Error 1
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/silence_layer.hpp:6,
from src/caffe/layers/silence_layer.cpp:3:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/reshape_layer.hpp:6,
from src/caffe/layers/reshape_layer.cpp:3:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/silence_layer.o] Error 1
make: *** [.build_release/src/caffe/layers/reshape_layer.o] Error 1
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/threshold_layer.hpp:6,
from src/caffe/layers/threshold_layer.cpp:3:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/threshold_layer.o] Error 1
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/sigmoid_layer.hpp:6,
from src/caffe/layers/sigmoid_layer.cpp:4:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/sigmoid_layer.o] Error 1
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/concat_layer.hpp:6,
from src/caffe/layers/concat_layer.cpp:3:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/concat_layer.o] Error 1
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/multinomial_logistic_loss_layer.hpp:6,
from src/caffe/layers/multinomial_logistic_loss_layer.cpp:5:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/hinge_loss_layer.hpp:6,
from src/caffe/layers/hinge_loss_layer.cpp:4:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/multinomial_logistic_loss_layer.o] Error 1
make: *** [.build_release/src/caffe/layers/hinge_loss_layer.o] Error 1
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/cudnn_tanh_layer.hpp:6,
from src/caffe/layers/cudnn_tanh_layer.cpp:4:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/cudnn_tanh_layer.o] Error 1
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/flatten_layer.hpp:6,
from src/caffe/layers/flatten_layer.cpp:3:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/flatten_layer.o] Error 1
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/cudnn_lcn_layer.hpp:6,
from src/caffe/layers/cudnn_lcn_layer.cpp:4:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/cudnn_lcn_layer.o] Error 1
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/cudnn_conv_layer.hpp:6,
from src/caffe/layers/cudnn_conv_layer.cpp:5:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/cudnn_conv_layer.o] Error 1
In file included from ./include/caffe/blob.hpp:8:0,
from ./include/caffe/layers/softmax_loss_layer.hpp:6,
from src/caffe/layers/softmax_loss_layer.cpp:5:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/softmax_loss_layer.o] Error 1
解决方法:
这是由于没有安装boost导致的错误。要想使用caffe,安装的boost版本必须高于1.55。所以在命令行执行以下即可。
sudo apt-get install libboost1.55-*
cblas错误
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/filler.hpp:13,
from src/caffe/layers/bias_layer.cpp:3:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/bias_layer.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/cudnn_tanh_layer.hpp:7,
from src/caffe/layers/cudnn_tanh_layer.cpp:4:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/cudnn_tanh_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/concat_layer.hpp:7,
from src/caffe/layers/concat_layer.cpp:3:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/concat_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/multinomial_logistic_loss_layer.hpp:7,
from src/caffe/layers/multinomial_logistic_loss_layer.cpp:5:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/multinomial_logistic_loss_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/softmax_loss_layer.hpp:7,
from src/caffe/layers/softmax_loss_layer.cpp:5:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/softmax_loss_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/cudnn_conv_layer.hpp:7,
from src/caffe/layers/cudnn_conv_layer.cpp:5:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/cudnn_conv_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/cudnn_relu_layer.hpp:7,
from src/caffe/layers/cudnn_relu_layer.cpp:4:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/relu_layer.hpp:7,
from src/caffe/layers/relu_layer.cpp:4:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/cudnn_relu_layer.o] Error 1
make: *** [.build_release/src/caffe/layers/relu_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/reshape_layer.hpp:7,
from src/caffe/layers/reshape_layer.cpp:3:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/reshape_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/power_layer.hpp:7,
from src/caffe/layers/power_layer.cpp:3:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/power_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/silence_layer.hpp:7,
from src/caffe/layers/silence_layer.cpp:3:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/silence_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/cudnn_lcn_layer.hpp:7,
from src/caffe/layers/cudnn_lcn_layer.cpp:4:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/cudnn_lcn_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/threshold_layer.hpp:7,
from src/caffe/layers/threshold_layer.cpp:3:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/threshold_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/sigmoid_layer.hpp:7,
from src/caffe/layers/sigmoid_layer.cpp:4:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/sigmoid_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/flatten_layer.hpp:7,
from src/caffe/layers/flatten_layer.cpp:3:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/flatten_layer.o] Error 1
In file included from ./include/caffe/util/math_functions.hpp:11:0,
from ./include/caffe/layer.hpp:12,
from ./include/caffe/layers/hinge_loss_layer.hpp:7,
from src/caffe/layers/hinge_loss_layer.cpp:4:
./include/caffe/util/mkl_alternate.hpp:14:19: fatal error: cblas.h: No such file or directory
#include
^
compilation terminated.
make: *** [.build_release/src/caffe/layers/hinge_loss_layer.o] Error 1
libcudart 错误
当编译caffe时,出现有关libcudart错误时,往往是cuda的路径没有设置对。这个时候我们要修改Makefile.config文件和ld.conf.d文件夹下的cuda.conf文件(如果该文件夹下没有该文件则要新建一个)
# 修改makefile文件,添加/usr/local/cuda/include和/usr/local/cuda/lib64
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/local/cuda/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/local/cuda/lib64
#修改目录/etc/ld.so.conf.d下的cuda.conf文件,将/usr/local/cuda/include和/usr/local/cuda/lib64添加进去
cd /etc/ld.so.conf.d
sudo vim cuda.conf
#insert two paths above
至此caffe应该可以顺利的编译完毕。
接下来我们看看编译pycaffe时会出现的问题
安装一些python依赖
进入caffe目录下的python文件夹然后执行下列操作:
for req in $(cat requirements.txt); do pip install $req; done
这样就安装了一些python的依赖
skimage.io错误
如果出现该错误,说明有一些依赖没有正确的安装,执行以下程序:
sudo apt-get install python-numpy python-scipy python-matplotlib python-sklearn python-skimage python-h5py python-protobuf python-leveldb python-networkx python-nose python-pandas python-gflags Cython ipython;#主要是看看python-skimage有没有安装
ImportError: No module named google.protobuf.internal
这是由于没有安装protobuf导致的。
pip install protobuf
就ok了
import caffe时出现了问题:Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so
这个跟anaconda有关,一般情况下,我们使用Linux自带的python就可以了,没必要整anaconda。如果非要使用anaconda的话。使用一下办法解决这个问题。
conda install nomkl numpy scipy scikit-learn numexprconda remove mkl mkl-service
安装一些包,移除一些包就可以了。
最后就是编译matcaffe出现的问题了。
我最近重装了一次标准版的caffe,编译matcaffe时只出现了一个问题。就是要用-std=c++11的gcc去编译matcaffe。这个很好解决,只需要在Makefile文件中查找CXXFLAG,然后在一个什么mp的表达式下面加上一句话就行了。
编译完之后的路径设置问题。
我们编译好了caffe和pycaffe以及matcaffe后,需要将caffe和pycaffe的库文件路径添加到系统路径中,这样我们才能正常的使用caffe。
涉及到的文件有:
编译完之后的测试问题
以上是编译普通caffe 的方法,这一套流程我已经走过很多遍了,应该没有问题。
这里需要注意的是,当你需要编译ssd 版的caffe或者别的研究团队改过的caffe时,最好使用cmake 的方法编译,具体内容以后有时间我会补充上来的。