ubuntu 16.04(no gui)+conda python3.5+cuda10.1+cudnn7.4.1+opencv3.4.1+caffe

由于本人穷（只有一个机器）又事多（想把caffe安到anaconda的某个环境下，避免包冲突），探索了这么一个麻烦的安装caffe的方法

另外Turing显卡据说只能支持cuda10，就安cuda10。

我的想法是尽量少的使用apt-get来安装。

同时我是新安装的系统，比较纯净，只有很少用apt-get安装的开发包。另外我源码编译了protobuf3.5.1。所以我这方法如果不适用于你，我也没办法，但是能提供一些排坑的思路。

 

0.1 关于一些包

过程中没有绕开的包有cmake, libpng：

atlas用conda的会出现caffe找不到的问题，conda的libpng涉及到zlib会出问题（zlib引用）

sudo apt-get install cmake libpng16-16 #libpng不建议提前装好

选安装的有unzip（用于解压包），atlas编译caffe所用的blas包

sudo apt-get install unzip libatlas-base-dev

cmake使用3.5.1

安装完成后conda 环境中的包和版本如下所示，不建议提前都装好，可能会出一些问题。建议越少越好，咱么需要啥装啥

 

# packages in environment at /home/zhangxun/work/anaconda3/envs/caffe35:
#
# Name                    Version                   Build  Channel
blas                      1.0                    openblas
boost                     1.67.0                   py35_4
bzip2                     1.0.6                h14c3975_5
ca-certificates           2019.1.23                     0
certifi                   2018.8.24                py35_1
cloudpickle               0.8.0                    pypi_0    pypi
cycler                    0.10.0                   pypi_0    pypi
dask                      1.1.4                    pypi_0    pypi
decorator                 4.4.0                    pypi_0    pypi
gflags                    2.2.2                he6710b0_0
glog                      0.3.5                hf484d3e_1
hdf5                      1.10.4               hb1b8bf9_0
icu                       58.2                 h9c2bf20_1
intel-openmp              2019.1                      144
jpeg                      9b                   h024ee3a_2
kiwisolver                1.0.1                    pypi_0    pypi
leveldb                   1.20                 hf484d3e_1
libboost                  1.67.0               h46d08c1_4
libedit                   3.1.20181209         hc058e9b_0
libffi                    3.2.1                hd88cf55_4
libgcc-ng                 8.2.0                hdf63c60_1
libgfortran-ng            7.3.0                hdf63c60_0
libopenblas               0.3.3                h5a2b251_3
libprotobuf               3.5.2                h6f1eeef_0
libstdcxx-ng              8.2.0                hdf63c60_1
lmdb                      0.9.23               he6710b0_0
matplotlib                3.0.3                    pypi_0    pypi
mkl                       2018.0.3                      1
mkl_fft                   1.0.1            py35h3010b51_0
mkl_random                1.0.1            py35h629b387_0
ncurses                   6.1                  he6710b0_1
networkx                  2.2                      pypi_0    pypi
nomkl                     3.0                           0
numpy                     1.15.2           py35h99e49ec_0
numpy-base                1.15.2           py35h2f8d375_0
openblas                  0.3.3                         3
openblas-devel            0.3.3                         3
openssl                   1.0.2r               h7b6447c_0
pillow                    5.4.1                    pypi_0    pypi
pip                       10.0.1                   py35_0
protobuf                  3.5.2            py35hf484d3e_0
py-boost                  1.67.0           py35h04863e7_4
pyparsing                 2.3.1                    pypi_0    pypi
python                    3.5.6                hc3d631a_0
python-dateutil           2.8.0                    pypi_0    pypi
pywavelets                1.0.2                    pypi_0    pypi
readline                  7.0                  h7b6447c_5
scikit-image              0.14.2                   pypi_0    pypi
scipy                     1.2.1                    pypi_0    pypi
setuptools                40.2.0                   py35_0
six                       1.12.0                   pypi_0    pypi
snappy                    1.1.7                hbae5bb6_3
sqlite                    3.27.2               h7b6447c_0
tk                        8.6.8                hbc83047_0
toolz                     0.9.0                    pypi_0    pypi
wheel                     0.31.1                   py35_0
xz                        5.2.4                h14c3975_4
zlib                      1.2.11               h7b6447c_3

0.2 关于zlib

     这个东西造成了比较多的困扰，原因在于它是一个非常底层的包，对于系统对于python都是必须的。ubuntu16.04（headers是4.4.0.62）自带的是zlib是1.2.8的，而很多我们使用conda的过程中安装的包都是用的zlib1.2.11（包含了zlib1.2.9），包引用的混乱造成了 “inflateValidate@ZLIB_1.2.9’ 未定义的引用”的问题。

解决的方法是可能有系统zlib升级，conda的zlib降级，和通过设置路径不解决三个办法

• 系统zlib升级，我没敢试，怕系统应用出现问题
• conda zlib降级，带来的麻烦也挺多，会导致conda环境中的包版本下降，包括protobuf，如下图所示。开始时因为opencv3.4.1和caffe的编译都用protobuf，而本人以前源码编译了protobuf3.5.1，实在是怕出问题，于是没使用这个方法

The following packages will be SUPERSEDED by a higher-priority channel:

  boost                      pkgs/main::boost-1.67.0-py35_4 --> pkgs/free::boost-1.60.0-py35_0
  hdf5                    pkgs/main::hdf5-1.10.4-hb1b8bf9_0 --> pkgs/free::hdf5-1.8.17-2
  libprotobuf        pkgs/main::libprotobuf-3.5.2-h6f1eeef~ --> pkgs/free::libprotobuf-3.4.0-0
  python                 pkgs/main::python-3.5.6-hc3d631a_0 --> pkgs/free::python-3.5.4-0
  readline               pkgs/main::readline-7.0-h7b6447c_5 --> pkgs/free::readline-6.2-2
  sqlite                pkgs/main::sqlite-3.27.2-h7b6447c_0 --> pkgs/free::sqlite-3.13.0-0
  tk                         pkgs/main::tk-8.6.8-hbc83047_0 --> pkgs/free::tk-8.5.18-0
  zlib                    pkgs/main::zlib-1.2.11-h7b6447c_3 --> pkgs/free::zlib-1.2.8-3

• 路径设置，最后caffe.bin编译过程中是可以设置library rpath的，他会有一个优先寻找库的路径，绕过运行库的问题。而本人又狗屎运，编译过程中也绕过了编译库的问题，导致编译侥幸通过。。反正最后算是成功了、、

 

0.3 anaconda3

安装的时候conda的版本是4.6.8

新建了一个环境 caffe35

conda create -n caffe35 python=3.5 
conda activate caffe35

我们进入到caffe35的环境中来，这样在这里安装的各种库不会影响到其他的环境

我们在编译完opencv和caffe后，会在~/.bashrc中加入

export PYTHONPATH=~/work/caffe/caffe/python:$PYTHONPATH
export PYTHONPATH=~/work/caffe/protobuf-3.5.1/python/:$PYTHONPATH

这样会使得其他环境中的python也能import caffe和import cv2，但是会因为缺少库而不能成功。总之影响不大。

1 cuda10.1

没有图形界面基本没什么问题，（有图形界面的话）具体请上网搜索安装教程

安装完记得加路径（类似以下的，具体可以自己搜，路径自己设置，）

export PATH=$PATH:/usr/local/cuda-10.1/bin
export LD_LIBRARY_PATH=/usr/local/cuda-10.1/lib64$LD_LIBRARY_PATH

 

2 opencv 3.4.1

对cuda10.1的支持不太好，主要是dynlink_nvcuvid.h 在cuda10中弃用了，但opencv3.4.1带的文件依然要include这个头文件（其他版本opencv未实验），具体参考

https://www.cnblogs.com/penguins/p/10214817.html

https://github.com/opencv/opencv/issues/9544

要改的文件包括opencv-3.4.1/modules/cudacodec/src/下的大部分hpp文件

要使用的nvcuvid.h需要在下载

https://developer.nvidia.com/designworks/video_codec_sdk/downloads/v8.2-ga2

 

在opencv路径下

mkdir build
cd build
cmake .. #根据需要添加指令
make -j8
make install

其中在cmake ..这个步骤，会出现设置的展示

………………………………
…………………………
Media I/O:
--     ZLib:                        /home/XXX/work/anaconda3/envs/caffe35/lib/libz.so (ver 1.2.11)
--     JPEG:                        /home/XXX/work/anaconda3/envs/caffe35/lib/libjpeg.so (ver 90)
--     WEBP:                        build (ver encoder: 0x020e)
--     PNG:                         /home/XXX/work/anaconda3/envs/caffe35/lib/libpng.so (ver ..)
--     TIFF:                        build (ver 42 - 4.0.9)
--     JPEG 2000:                   build (ver 1.900.1)
--     OpenEXR:                     build (ver 1.7.1)
…………………………………………………………………………………………………………………………
 Python 3:
--     Interpreter:                 /home/XXX/work/anaconda3/envs/caffe35/bin/python3 (ver 3.5.6)
--     Libraries:                   /home/XXX/work/anaconda3/envs/caffe35/lib/libpython3.5m.so (ver 3.5.6)
--     numpy:                       /home/XXX/work/anaconda3/envs/caffe35/lib/python3.5/site-packages/numpy/core/include (ver 1.15.2)
--     packages path:               lib/python3.5/site-packages
--
--   Python (for build):            /usr/bin/python2.7
……………………………………………………………………………………………………………………………………………………
--   Install to:                    /home/XXX/work/anaconda3/envs/caffe35

我们比较关心上面的几条，特别是ZLib PNG两个，提前conda install png使得cmake能找到环境caffe35中的库

另外需要conda install numpy，不然编译的caffe不能import

如果python3 的参数找不到的话 需要指令设置，类似如下，加入-DPARA=PATH等

cmake -DPYTHON3_LIBRARIES=/home/XXX/work/anaconda3/envs/caffe35/lib/libpython3.5m.so -DCMAKE_INSTALL_PREFIX=/home/XXX/work/anaconda3/envs/caffe35/ -DBUILD_opencv_python3=true ..

其中-D 是define的意思，变量名需要到cmaklist.txt里去确定

如果安装到/usr/local里面可能要 sudo make install，我们安装到caffe35环境的文件夹下

 

其他没有的包，opencv自带的第三方库会自己编译，实在不行conda install安装试一下（一般不会缺包）

make install后，python中import cv2验证

如不行试试~/.bashrc中加入

export PYTHONPATH=~/work/caffe/protobuf-3.5.1/python/:$PYTHONPATH

然后source ~/.bashrc 一下

其他还有问题就自己搜一搜吧

 

3 caffe

caffe具体参考官方安装http://caffe.berkeleyvision.org/install_apt.html

官方教程http://caffe.berkeleyvision.org/install_apt.html

              http://caffe.berkeleyvision.org/installation.html#compilation

cp Makefile.config.example Makefile.config
# Adjust Makefile.config (for example, if using Anaconda Python, or if cuDNN is desired)
make all
make pycaffe
make test
make runtest

 

过程中需要的可能有boost，hdf5，gflags，glog，lmdb，leveldb，snappy，openblas（或其他blas）等等

关于Makefile.config中的设置，我们要设置用caffe35中的python和lib和include

blas库用openblas比较好一些。 cuda-arch记得屏蔽前两行

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
USE_OPENCV := 1
# USE_LEVELDB := 0
# USE_LMDB := 0
# This code is taken from https://github.com/sh1r0/caffe-android-lib
# USE_HDF5 := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#	You should not set this flag if you will be reading LMDBs with any
#	possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda-10.1
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
CUDA_ARCH := #-gencode arch=compute_20,code=sm_20 \
		#-gencode arch=compute_20,code=sm_21 \
		-gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
		-gencode arch=compute_50,code=sm_50 \
		-gencode arch=compute_52,code=sm_52 \
		-gencode arch=compute_60,code=sm_60 \
		-gencode arch=compute_61,code=sm_61 \
		-gencode arch=compute_61,code=compute_61

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := open
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
# PYTHON_INCLUDE := /usr/include/python2.7 \
		# /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := $(HOME)/work/anaconda3/envs/caffe35
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		# $(ANACONDA_HOME)/include/python2.7 \
		# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include

# Uncomment to use Python 3 (default is Python 2)
PYTHON_LIBRARIES := boost_python35 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include
PYTHON_INCLUDE := $(ANACONDA_HOME)/include/ \
		$(ANACONDA_HOME)/include/python3.5m/ \
		$(ANACONDA_HOME)/lib/python3.5/site-packages/numpy/core/include/
		
# We need to be able to find libpythonX.X.so or .dylib.
# PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

另外CUDA_ARCH在rtx2070以上的显卡是sm_75，需要修改文件，具体可以在第46行后加入

-gencode arch=compute_75,code=sm_75 \ #我未验证

 

make test之后，基本就算编译完成，编译过程中，缺什么库，就conda安装什么，其中缺少skimage的过程中，我是使用了pip install scikit-image。大家可以试试conda install scikit-image

 

唯一要注意的是 png的包会爆zlib1.2.9的错误，这个时候，我的处理方式是conda remove png，然后sudo apt-get install libpng16-16 之后就编译过了，这个里面必然存在问题，但我不清楚怎么弄是对的，既然通过了我也没再关心。

 

 

make runtest

开始不成功，原因就是 caffe_root/build/tools/caffe.bin在运行的时候找不到各种库，这些库全在caffe35的库里面，这说明在编译过程中 library rpath没有设置去envs/caffe35/lib去找。可以通过以下命令验证

(caffe35) XXX@DS-GPU10:~/caffe/build/tools$ readelf -a caffe.bin | grep rpath
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../lib]

可见caffe.bin只会去caffe/build/lib下面去找库。

使用ldconfig的方式我不太会，于是我使用软链接的方式一劳永逸了（应该会造成其他坑）。。

在caffe/build/lib下 （具体的路径自己设置，需要软链接的包也取决于编译的时候用了多少conda的库，所以可能会有所不一样）

ln -s ~/anaconda3/envs/caffe35/lib/libglog.so.0 .
ln -s ~/anaconda3/envs/caffe35/lib/libgflags.so.2.2 .
ln -s ~/anaconda3/envs/caffe35/lib/libboost_system.so.1.67.0 .
ln -s ~/anaconda3/envs/caffe35/lib/libboost_filesystem.so.1.67.0 .
ln -s ~/anaconda3/envs/caffe35/lib/libleveldb.so.1 .
ln -s ~/anaconda3/envs/caffe35/lib/liblmdb.so .
ln -s ~/anaconda3/envs/caffe35/lib/libhdf5_hl.so.100 .
ln -s~/anaconda3/envs/caffe35/lib/libhdf5.so.103 .
ln -s ~/anaconda3/envs/caffe35/lib/libboost_thread.so.1.67.0 .
ln -s ~/anaconda3/envs/caffe35/lib/libopenblas.so.0 .
ln -s ~/anaconda3/envs/caffe35/lib/libsnappy.so.1 .
ln -s ~/anaconda3/envs/caffe35/lib/libz.so.1.2.11 libz.so.1

这样make runtest应该就可以用了，python中也能import caffe了。不行的话试试下面

#我们在编译完caffe后，会在~/.bashrc中加入
export PYTHONPATH=~/work/caffe/caffe/python:$PYTHONPATH

删除这些软链接

rm libglog.so.0 libgflags.so.2.2 libboost_system.so.1.67.0 libboost_filesystem.so.1.67.0 libleveldb.so.1 liblmdb.so libhdf5_hl.so.100 libhdf5.so.103 libboost_thread.so.1.67.0 libopenblas.so.0 libsnappy.so.1 libz.so.1