[caffe]安装中遇到的问题
1.identifier “cudnnTensor4dDescriptor_t” is undefined
Solution_1:
CMakelist.txt :
Change
caffe_option(USE_CUDNN “Build Caffe with cuDNN libary support” ON IF NOT CPU_ONLY)
to
caffe_option(USE_CUDNN “Build Caffe with cuDNN libary support” OFF IF NOT CPU_ONLY)
Solution_2:
[
原来tsubame里已经安装了cudnn v3….我找到了路径…
http://tsubame.gsic.titech.ac.jp/docs/guides/tsubame2/html_en/programming.html#cuda-compiler
]
其实这个到底是怎么解决的我记不清了,可能只是进行了(1), 或者(1)(2)都进行了
(1)
cd /usr/apps.sp3/cuda
ls
5.0 5.5 6.0 6.5 7.0
cd 7.0
. ./cuda.sh
(2)
After cmake -DCMAKE_INSTALL_PREFIX=$LOCAL -DBOOST_ROOT:PATHNAME=$LOCAL -DCMAKE_CXX_FLAGS=-L$LOCAL/lib -DBUILD_matlab=ON ..
Change the CMakeCache.txt
//Path to cuDNN include directory.
CUDNN_INCLUDE:PATH=/usr/apps.sp3/cuda/7.0/include
//Path to cuDNN library.
CUDNN_LIBRARY:FILEPATH=/usr/apps.sp3/cuda/7.0/lib64/libcudnn.so
//CUDNN root folderCUDNN_ROOT:
PATH=2. When I successfully “cmake”, I failed in “make”
[ 98%] Building Matlab interface: /work1/lisa-caffe-public/matlab/caffe/caffe.mexa64Building with ‘g++’./usr/bin/ld: cannot find -lpython2
However, I read those make lines and I found it successfully locate the python path.
-- Python:-- Interpreter : /home/usr9/local/bin/python2.7 (ver. 2.7.7)-- Libraries : /home0/usr9/local/lib/libpython2.7.so (ver 2.7.7)-- NumPy : /home/usr9/local/lib/python2.7/site-packages/numpy/core/include (ver 1.10.1)There are several ways to solve this problem, however, because I don’t have permision to make symbolic link in university server, I just “cmake” again while this time I removed the matlab part..
cmake -DCMAKE_INSTALL_PREFIX=$LOCAL -DBOOST_ROOT:PATHNAME=$LOCAL -DCMAKE_CXX_FLAGS=-L$LOCAL/lib -DBUILD_matlab=ON ..cmake -DCMAKE_INSTALL_PREFIX=$LOCAL -DBOOST_ROOT:PATHNAME=$LOCAL -DCMAKE_CXX_FLAGS=-L$LOCAL/lib ..3. something about the groupID
when I do cmake, it said “cannot found numpy”(and some other things) though I have installed.
Some software are pre-installed in the server, so maybe the groupID?
I changed my groudID from users to t2g-xxxx2011, then this time “cmake.file” error (I installed cmake while the groupID was “users”), I echo $LD_LIBRARY_PATH and find it changed!
If I set groupID=t2g-xxxx2011 -> error A
If I set groupID=users -> no error A but error B
My tutor found that the directory “caffe/” is drwxr-xr-x, then he -> chmod +s caffe/ (all subdirectories except files) [I forgot the command… maybe “g+s” ? I only remember that he also used | xargs and d to find only directories to add s]
Then, “ls -l” , it looks like:
-rw-r--r-- 1 15Mxxxxx t2g-xxxx2011 2102 Jan 8 15:02 README.md
drwxr-sr-x 13 15Mxxxxx t2g-xxxx2011 4096 Jan 8 21:01 buildIt works! And I don’t have to change groupID from “users” to “t2g-xxxx2011”
[I download caffe in my local computer and then upload to server, so next time I have to pay attention to that]
found a command:
find . -type d -print0 | xargs -0 chmod g+s
4. mnist example
1)
when I “./examples/mnist/create_mnist.sh”
the error:
Check failed: mdb_env_open(env, filename_->c_str(), 0, 0664) == 0 (12 vs. 0) mdb_env_open failed
I find the solution here:https://groups.google.com/forum/#!msg/caffe-users/m4iCrEK2Qy8/2DRRy2VMZUEJ
” errno 12 means no enough memory, so I change the 1TB value to 1GB in the above line, like that:mdb_env_set_mapsize(mdb_env_, 1099511627776/1024), MDB_SUCCESS). My platform is suse SLES 11, I’m not sure why it doesnot work with 1TB size.”
So I changed the caffe/examples/mnist/convert_mnist_data.cpp,
then removed “build” directory and “cmake”, “make” again.
2)
when I “./examples/mnist/train_lenet.sh”
Check failed: mdb_status == 0 (12 vs. 0) Cannot allocate memory
Solution from: https://github.com/BVLC/caffe/issues/2709
“I solved it by changing the following:
in convert_mnist_data.cpp
CHECK_EQ(mdb_env_set_mapsize(mdb_env, 1099511627776), MDB_SUCCESS) // 1TB
to:
CHECK_EQ(mdb_env_set_mapsize(mdb_env, 1073741824), MDB_SUCCESS) // 1GB
and compile again
and also in :
/src/caffe/util/db_lmdb.cpp
change
LMDB_MAP_SIZE
to
4294967296”
So I changed the LMDB_MAP_SIZE to 1073741824(not 4294967296), and “cmake”,”make” again.
5. use python in caffe
.bashrc.txt
umask 022
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_ALL=en_US.UTF-8
export WORK=/work1/t2g-shinoda2011/15M54105
export LOCAL=$WORK/local
export INTALL_RN=$WORK/install_rn
export PATH=~/local/bin:~/.gem/ruby/2.0.0/bin:$LOCAL/bin:$INTALL_RN/yasm/bin:$WORK/lisa-caffe-public/examples/LRCN_activity_recognition:$PATH
export MANPATH=~/local/share/man:$LOCAL/share/man:$MANPATH
export PYTHONPATH=$WORK/parallel0/caffe/python:$WORK/newTest1/lisa-caffe-public/python:~/local/lib/python2.7/site-packages:$PYTHONPATH
export GEM_HOME=~/.gem/ruby/gems/2.0.0:$GEM_HOME
export RUBYLIB=~/.gem/ruby/gems/2.0.0:$RUBYLIB
export INCLUDE_PATH=~/local/include:$LOCAL/include:$INCLUDE_PATH
export C_INCLUDE_PATH=~/local/include:$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH=~/local/include:$CPLUS_INCLUDE_PATH
export LIBRARY_PATH=$LOCAL/lib:~/local/lib:$LIBRARY_PATH
export LD_LIBRARY_PATH=$LOCAL/lib:~/local/lib:$LD_LIBRARY_PATH
export INCLUDE_PATH=~/local/include/ncursesw:~/local/include:$INCLUDE_PATH
export C_INCLUDE_PATH=~/local/include/ncursesw:~/local/include:$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH=~/local/include/ncursesw:~/local/include:$CPLUS_INCLUDE_PATH
export http_proxy="http://localhost:3128"
export https_proxy="http://localhost:3128"
export ftp_proxy="http://localhost:3128"1.To import the caffe Python module after completing the installation, add the module directory to your $PYTHONPATH by export PYTHONPATH=/path/to/caffe/python:$PYTHONPATH or the like. You should not import the module in the caffe/python/caffe directory!
2.To compile the Python and MATLAB wrappers do make pycaffe and make matcaffe respectively. Be sure to set your MATLAB and Python paths in Makefile.config first!
3.Distribution: run make distribute to create a distribute directory with all the Caffe headers, compiled libraries, binaries, etc. needed for distribution to other machines.
4.Speed: for a faster build, compile in parallel by doing make all -j8 where 8 is the number of parallel threads for compilation (a good choice for the number of threads is the number of cores in your machine)
6 HDF5报错: HDF5 header version与HDF5 library不匹配
http://www.cnblogs.com/platero/p/4077934.html
I0323 14:01:28.488822 10457 layer_factory.hpp:77] Creating layer data
Warning! ***HDF5 library version mismatched error***
The HDF5 header files used to compile this application do not match
the version used by the HDF5 library to which this application is linked.
Data corruption or segmentation faults may occur if the application continues.
This can happen when an application was compiled by one version of HDF5 but
linked with a different version of static or shared HDF5 library.
You should recompile the application or check your shared library related
settings such as 'LD_LIBRARY_PATH'.
'HDF5_DISABLE_VERSION_CHECK' environment variable is set to 1, application will
continue at your own risk.
Headers are 1.8.16, library is 1.8.13
SUMMARY OF THE HDF5 CONFIGURATION
=================================
General Information:
-------------------
HDF5 Version: 1.8.13
Configured on: Fri May 16 14:09:09 EDT 2014
Configured by: djmarsha@annlnxdjmarsha11
Configure mode: production
Host system: x86_64-unknown-linux-gnu
Uname information: Linux annlnxdjmarsha11 2.6.18-371.1.2.el5 #1 SMP Tue Oct 22 12:51:53 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux
Byte sex: little-endian
Libraries: static, shared
Installation point: /home/compilations/hdf5/1.8.13/Linux64/install
Compiling Options:
------------------
Compilation Mode: production
C Compiler: /opt/anss/bin/gcc ( gcc (GCC) 4.6.1)
CFLAGS:
H5_CFLAGS: -std=c99 -pedantic -Wall -Wextra -Wundef -Wshadow -Wpointer-arith -Wbad-function-cast -Wcast-qual -Wcast-align -Wwrite-strings -Wconversion -Waggregate-return -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -Wnested-externs -Winline -Wno-long-long -Wfloat-equal -Wmissing-format-attribute -Wmissing-noreturn -Wpacked -Wdisabled-optimization -Wformat=2 -Wendif-labels -Wdeclaration-after-statement -Wold-style-definition -Winvalid-pch -Wvariadic-macros -Wnonnull -Winit-self -Wmissing-include-dirs -Wswitch-default -Wswitch-enum -Wunused-macros -Wunsafe-loop-optimizations -Wc++-compat -Wstrict-overflow -Wlogical-op -Wlarger-than=2048 -Wvla -Wsync-nand -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wstrict-aliasing -Wstrict-overflow=5 -Wjump-misses-init -Wunsuffixed-float-constants -Wdouble-promotion -Wsuggest-attribute=const -Wtrampolines -O3 -fomit-frame-pointer -finline-functions
AM_CFLAGS:
CPPFLAGS:
H5_CPPFLAGS: -D_POSIX_C_SOURCE=199506L -DNDEBUG -UH5_DEBUG_API
AM_CPPFLAGS: -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_BSD_SOURCE
Shared C Library: yes
Static C Library: yes
Statically Linked Executables: no
LDFLAGS:
H5_LDFLAGS:
AM_LDFLAGS:
Extra libraries: -lz -lrt -ldl -lm
Archiver: ar
Ranlib: ranlib
Debugged Packages:
API Tracing: no
Languages:
----------
Fortran: no
C++: no
Features:
---------
Parallel HDF5: no
High Level library: yes
Threadsafety: no
Default API Mapping: v18
With Deprecated Public Symbols: yes
I/O filters (external): deflate(zlib)
I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset
MPE: no
Direct VFD: no
dmalloc: no
Clear file buffers before write: yes
Using memory checker: no
Function Stack Tracing: no
Strict File Format Checks: no
Optimization Instrumentation: no
Large File Support (LFS): yes
Traceback (most recent call last):
File "/work1/t2g-shinoda2011/15M54105/parallel0/caffe_final/examples/LRCN_activity_recognition/sequence_input_layer.py", line 18, in <module>
import h5py
File "/home/usr9/15M54105/local/lib/python2.7/site-packages/h5py/__init__.py", line 31, in <module>
from .highlevel import *
File "/home/usr9/15M54105/local/lib/python2.7/site-packages/h5py/highlevel.py", line 13, in <module>
from ._hl.base import is_hdf5, HLObject
File "/home/usr9/15M54105/local/lib/python2.7/site-packages/h5py/_hl/base.py", line 78, in <module>
dlapl = default_lapl()
File "/home/usr9/15M54105/local/lib/python2.7/site-packages/h5py/_hl/base.py", line 65, in default_lapl
lapl = h5p.create(h5p.LINK_ACCESS)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/Scratch/scr_host/tmp/pip-build-3fEnyY/h5py/h5py/_objects.c:2574)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/Scratch/scr_host/tmp/pip-build-3fEnyY/h5py/h5py/_objects.c:2533)
File "h5py/h5p.pyx", line 130, in h5py.h5p.create (/Scratch/scr_host/tmp/pip-build-3fEnyY/h5py/h5py/h5p.c:2596)
ValueError: Not a property list class (Not a property list class)tsubame的系统里装的应该只有1.8.13阿,1.8.16是哪里来的?
后来我采取了简单粗暴的方式,直接在tsubame里安装1.8.16
第一次安装失败了,因为我的安装路径是在一个已经装了很多软件的文件夹下,于是我新建了一个空的文件夹,重新安装,安装成功.
修改了.bashrc文件,
export HDF5=$WORK/software/hdf5
export PATH=$WORK/software/hdf5/bin:~/local/bin:~/.gem/ruby/2.0.0/bin:$LOCAL/bin:$INTALL_RN/yasm/bin:$WORK/lisa-caffe-public/examples/LRCN_activity_recognition:$PATH
export MANPATH=~/local/share/man:$LOCAL/share/man:$MANPATH
export PYTHONPATH=$WORK/parallel0/caffe_final/python:$WORK/newTest1/lisa-caffe-public/python:~/local/lib/python2.7/site-packages:$PYTHONPATH
export GEM_HOME=~/.gem/ruby/gems/2.0.0:$GEM_HOME
export RUBYLIB=~/.gem/ruby/gems/2.0.0:$RUBYLIB
export INCLUDE_PATH=$WORK/software/hdf5/include:~/local/include:$LOCAL/include:$INCLUDE_PATH
export C_INCLUDE_PATH=$WORK/software/hdf5/include:~/local/include:$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH=$WORK/software/hdf5/include:~/local/include:$CPLUS_INCLUDE_PATH
export LIBRARY_PATH=$WORK/software/hdf5/lib:$LOCAL/lib:~/local/lib:$LIBRARY_PATH
export LD_LIBRARY_PATH=$WORK/software/hdf5/lib:$LOCAL/lib:~/local/lib:$LD_LIBRARY_PATH
export INCLUDE_PATH=~/local/include/ncursesw:~/local/include:$INCLUDE_PATH
export C_INCLUDE_PATH=~/local/include/ncursesw:~/local/include:$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH=~/local/include/ncursesw:~/local/include:$CPLUS_INCLUDE_PATH可是仍旧报一样的错.
于是查看CAFFE的CMakeCache.txt, 发现里面的lib还是链接到了tsubame系统里的1.8.13的路径上,于是手动修改了CMakeCache.txt中所有的hdf5的lib路径,重新进行cmake, 再make.
/work1/t2g-shinoda2011/15M54105/software/hdf5/lib
7. caffe某cpp里新增加一个函数,opencv报错.说未定义.
image_data_layer.cpp
#include 在make的时候,大概到百分之九十几报错,
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: skipping incompatible /work1/t2g-shinoda2011/15M54105/local/lib/libstdc++.so when searching for -lstdc++
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: skipping incompatible /work1/t2g-shinoda2011/15M54105/local/lib/libstdc++.a when searching for -lstdc++
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: skipping incompatible /work1/t2g-shinoda2011/15M54105/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: skipping incompatible /work1/t2g-shinoda2011/15M54105/local/lib/libgcc_s.so when searching for -lgcc_s
../lib/libcaffe.so: undefined reference to `cv::Directory::GetListFiles(std::basic_string, std::char_traits, std::char_traits 因为我加了#include
就需要把这个lib也加进去.
查看CmakeCache.txt,里面只有opencv_core;opencv_highgui;opencv_imgproc这三个,于是在里面加入opencv_contrib;general;(我也不造general是什么意思)
//Dependencies for the target
caffe_LIB_DEPENDS:STATIC=general;proto;general;proto;general;/work1/t2g-shinoda2011/15M54105/local/lib/libboost_system.so;general;/work1/t2g-shinoda2011/15M54105/local/lib/libboost_thread.so;general;/work1/t2g-shinoda2011/15M54105/local/lib/libboost_filesystem.so;general;-lpthread;general;/work1/t2g-shinoda2011/15M54105/local/lib/libglog.so;general;/work1/t2g-shinoda2011/15M54105/local/lib/libgflags.so;optimized;/work1/t2g-shinoda2011/15M54105/local/lib/libprotobuf.so;debug;/work1/t2g-shinoda2011/15M54105/local/lib/libprotobuf.so;general;-lpthread;general;/usr/apps.sp3/isv/ansys_inc/16.2/v162/Framework/bin/Linux64/libhdf5_hl.so;general;/usr/apps.sp3/isv/ansys_inc/16.2/v162/Framework/bin/Linux64/libhdf5.so;general;/work1/t2g-shinoda2011/15M54105/local/lib/liblmdb.so;general;/work1/t2g-shinoda2011/15M54105/local/lib/libleveldb.so;general;/work1/t2g-shinoda2011/15M54105/local/lib/libsnappy.so;general;/usr/apps.sp3/cuda/7.0/lib64/libcudart.so;general;/usr/apps.sp3/cuda/7.0/lib64/libcurand.so;general;/usr/apps.sp3/cuda/7.0/lib64/libcublas.so;general;/usr/apps.sp3/cuda/7.0/lib64/libcudnn.so;general;opencv_core;general;opencv_highgui;general;opencv_imgproc;general;opencv_contrib;general;/work1/t2g-shinoda2011/15M54105/local/lib/liblapack.a;general;/work1/t2g-shinoda2011/15M54105/local/lib/libptcblas.a;general;/work1/t2g-shinoda2011/15M54105/local/lib/libatlas.a;general;/home0/usr9/15M54105/local/lib/libpython2.7.so;general;/work1/t2g-shinoda2011/15M54105/local/lib/libboost_python.so;
但是改过之后在cmake,一看cmakecache.txt里修改的东西都不见了.
先放弃.
在caffe根目录下
grep -r opencv
grep -r OPENCV
grep -r OpenCV
终于找到了cmake/Dependencies.cmake
# ---[ OpenCV
if(USE_OPENCV)
find_package(OpenCV QUIET COMPONENTS core highgui imgproc contrib imgcodecs)
if(NOT OpenCV_FOUND) # if not OpenCV 3.x, then imgcodecs are not found
find_package(OpenCV REQUIRED COMPONENTS core highgui imgproc contrib)
endif()
include_directories(SYSTEM ${OpenCV_INCLUDE_DIRS})
list(APPEND Caffe_LINKER_LIBS ${OpenCV_LIBS})
message(STATUS "OpenCV found (${OpenCV_CONFIG_PATH})")
add_definitions(-DUSE_OPENCV)
endif()