Ubuntu14.04配置Detectron,及问题.
安装依赖(txt文件有的就不用了)opencv-python>=3.2 future
源码编译:
make && make ops
编译了一天:PROJECT_SOURCE_DIR为包含PROJECT()的最近一个CMakeLists.txt文件所在的文件夹。
cmake问题 要学着去分析CMakeLists.txt.内容
出现问题1:
CMake Error at cmake/Dependencies.cmake:8 (find_package):
By not providing "FindCaffe2.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "Caffe2", but
CMake did not find one.
Could not find a package configuration file provided by "Caffe2" with any
of the following names:
Caffe2Config.cmake
caffe2-config.cmake
Add the installation prefix of "Caffe2" to CMAKE_PREFIX_PATH or set
"Caffe2_DIR" to a directory containing one of the above files. If "Caffe2"
provides a separate development package or SDK, be sure it has been
installed.
Call Stack (most recent call first):
CMakeLists.txt:14 (include)
-- Configuring incomplete, errors occurred!
See also "/home/vivian/HelloWorld/tracker/DetectAndTrack/lib/build/CMakeFiles/CMakeOutput.log".
make: *** [ops] Error 1
o check if Caffe2 GPU build was successful
或者
By not providing "FindEigen3.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "Eigen3", but
CMake did not find one.
Could not find a package configuration file provided by "Eigen3" with any
of the following names:
Eigen3Config.cmake
eigen3-config.cmake
[也就是找不到Eigen3Config.cmake ]
[解决办法]确保“Caffe2”已安装好.全局搜索 Eigen3Config.cmake,
放到lib/cmake/Modules下,你自己改CMakeLists.txt文件下的路径也行
把
# Add CMake modules.
list(APPEND CMAKE_MODULE_PATH ${PROJECT_SOURCE_DIR}/cmake/Modules)
改成
# Add CMake modules.
set(CMAKE_MODULE_PATH ${PROJECT_SOURCE_DIR}/cmake/Modules)
find_package(Eigen3 REQUIRED)
出现问题2
~/lib$ make && make ops
[ 12%] Building CXX object CMakeFiles/caffe2_detectron_custom_ops.dir/ops/affine_channel_nd_op.cc.o
In file included from /home/vivian/anaconda3/envs/detect_and_track/lib/python2.7/site-packages/torch/lib/include/caffe2/proto/caffe2_pb.h:4:0,
from /home/vivian/anaconda3/envs/detect_and_track/lib/python2.7/site-packages/torch/lib/include/caffe2/core/context_base.h:7,
from /home/vivian/anaconda3/envs/detect_and_track/lib/python2.7/site-packages/torch/lib/include/caffe2/core/context.h:10,
from /home/vivian/HelloWorld/tracker/DetectAndTrack/lib/ops/affine_channel_nd_op.h:16,
from /home/vivian/HelloWorld/tracker/DetectAndTrack/lib/ops/affine_channel_nd_op.cc:13:
/home/vivian/anaconda3/envs/detect_and_track/lib/python2.7/site-packages/torch/lib/include/caffe2/proto/caffe2.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
#error This file was generated by a newer version of protoc which is
^
/home/vivian/anaconda3/envs/detect_and_track/lib/python2.7/site-packages/torch/lib/include/caffe2/proto/caffe2.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
#error incompatible with your Protocol Buffer headers. Please update
^
/home/vivian/anaconda3/envs/detect_and_track/lib/python2.7/site-packages/torch/lib/include/caffe2/proto/caffe2.pb.h:14:2: error: #error your headers.
#error your headers.
^
In file included from /home/vivian/anaconda3/envs/detect_and_track/lib/python2.7/site-packages/torch/lib/include/caffe2/proto/caffe2_pb.h:4:0,
from /home/vivian/anaconda3/envs/detect_and_track/lib/python2.7/site-packages/torch/lib/include/caffe2/core/context_base.h:7,
from /home/vivian/anaconda3/envs/detect_and_track/lib/python2.7/site-packages/torch/lib/include/caffe2/core/context.h:10,
from /home/vivian/HelloWorld/tracker/DetectAndTrack/lib/ops/affine_channel_nd_op.h:16,
from /home/vivian/HelloWorld/tracker/DetectAndTrack/lib/ops/affine_channel_nd_op.cc:13:
/home/vivian/anaconda3/envs/detect_and_track/lib/python2.7/site-packages/torch/lib/include/caffe2/proto/caffe2.pb.h:23:35: fatal error: google/protobuf/arena.h: No such file or directory
#include
^
compilation terminated.
make[2]: *** [CMakeFiles/caffe2_detectron_custom_ops.dir/ops/affine_channel_nd_op.cc.o] Error 1
make[1]: *** [CMakeFiles/caffe2_detectron_custom_ops.dir/all] Error 2
make: *** [all] Error 2
(detect_and_track) vivian@vivian-Super-Server:~/HelloWorld/tracker/DetectAndTrack/lib$ protoc --version
libprotoc 3.5.1
(我系统的肯定不是这个)
:~$ whereis protoc
protoc: /usr/bin/protoc /usr/bin/X11/protoc /usr/share/man/man1/protoc.1.gz
这三个都是旧的
:~$ which protoc
/home/vivian/anaconda3/envs/detect_and_track/bin/protoc 用的是新的 版本是3.51
:~$ protoc --version
libprotoc 3.5.1
[protoc版本不一致问题]
[解决方法]:
因为上面说的是太新了.所以用旧的生成一个,到提示出错的文件夹下
正确指令: cd /home/vivian/HelloWorld/tracker/tools/pytorch/caffe2/proto/
~/HelloWorld/tracker/tools/pytorch/caffe2/proto$ /usr/bin/protoc --cpp_out=/home/vivian/HelloWorld/tracker/DetectAndTrack/errors/ caffe2.proto
问题3:
OSError: /c/Development/densepose/build/libcaffe2_detectron_custom_ops_gpu.so: undefined symbol: _ZN6google8protobuf8internal9ArenaImpl28AllocateAlignedAndAddCleanupEmPFvPvE
还是啥的,记不清了,改了就是了
[参考解决办法,根据自己情况来]
One of simple solutions is to use protobuf from the conda installation
# you need to set CONDA_ROOT env variable.
set(PROTOBUF_LIB $ENV{CONDA_ROOT}/lib/libprotobuf.a)
If you built caffe2 from source, then
# I'm using virtualenv.
set(PROTOBUF_LIB "$ENV{VIRTUAL_ENV}/lib/python2.7/site-packages/torch/lib/libprotobuf.a")
我虚拟环境这个路径没有这个文件,因为如果从源码编译 libprotobuf.a是在地址的在虚拟环境下输入conda list protobuf 得到
(detect_and_track)~DetectAndTrack/lib$ conda list protobuf
# packages in environment at /home/vivian/anaconda3/envs/detect_and_track:
#
libprotobuf 3.5.2 0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
protobuf 3.5.2 py27hf484d3e_1 defaults
那么我的就是在 libprotobuf.a这里(源码编译的直接是set(PROTOBUF_LIB "~/pytorch/build/lib/libprotobuf.a")),综合起来就是
set(PROTOBUF_LIB "$ENV{VIRTUAL_ENV}/lib/libprotobuf.a")
#上一句等同于 set(PROTOBUF_LIB "/home/vivian/anaconda3/envs/detect_and_track/lib/libprotobuf.a")
这句话供参考set(PROTOBUF_LIB "/home/vivian/HelloWorld/tracker/tools/pytorch/build/lib/libprotobuf.a")
[解决办法]
在lib/CMakeLists.txt
target_link_libraries(caffe2_detectron_custom_ops caffe2)
install(TARGETS caffe2_detectron_custom_ops DESTINATION lib)两句改为
# static protobuf library 改成这五句
add_library(libprotobuf STATIC IMPORTED)
set(PROTOBUF_LIB "/home/vivian/anaconda3/envs/detect_and_track/lib/libprotobuf.a")
set_property(TARGET libprotobuf PROPERTY IMPORTED_LOCATION "${PROTOBUF_LIB}")
target_link_libraries(caffe2_detectron_custom_ops caffe2_library libprotobuf)
install(TARGETS caffe2_detectron_custom_ops DESTINATION lib)
相应的把
target_link_libraries(caffe2_detectron_custom_ops caffe2)
install(TARGETS caffe2_detectron_custom_ops DESTINATION lib)
改为
target_link_libraries(caffe2_detectron_custom_ops_gpu caffe2_gpu_library libprotobuf)
install(TARGETS caffe2_detectron_custom_ops_gpu DESTINATION lib)
问题四:各种头文件找不到,添加环境变量也没用
In my case, adding these two lines at the beginning of Detectron/CMakeLists.txt solved the problem. Please modify the /path/to/pytorch/ part.
set(Caffe2_DIR "/path/to/pytorch/torch/share/cmake/Caffe2/")
include_directories("/path/to/pytorch/torch/lib/include")
也就是
set(Caffe2_DIR "/home/vivian/HelloWorld/tracker/tools/pytorch/torch/share/cmake/Caffe2/")
include_directories("/home/vivian/HelloWorld/tracker/tools/pytorch/torch/lib/include/")
一定不要忘了/ 一忘了 会各种奇奇怪怪的错误.被这玩意儿耽误了半天时间,
make 成功了 但是ops不行
$ cd $DETECTRON
(detect_and_track) ~/detectron$ make ops
home/vivian/HelloWorld/tracker/tools/pytorch/torch/lib/include/caffe2/utils/cblas.h:8:23: fatal error: mkl_cblas.h: No such file or directory
#include
^
compilation terminated.
In file included from /home/vivian/HelloWorld/tracker/tools/pytorch/torch/lib/include/caffe2/utils/math.h:9:0,
from /home/vivian/HelloWorld/tracker/tools/pytorch/torch/lib/include/caffe2/utils/filler.h:8,
from /home/vivian/HelloWorld/tracker/tools/pytorch/torch/lib/include/caffe2/core/operator_schema.h:16,
from /home/vivian/HelloWorld/tracker/tools/pytorch/torch/lib/include/caffe2/core/net.h:17,
from /home/vivian/HelloWorld/tracker/tools/pytorch/torch/lib/include/caffe2/core/operator.h:16,
from /home/vivian/detectron/detectron/ops/zero_even_op.h:21,
from /home/vivian/detectron/detectron/ops/zero_even_op.cu:19:
/home/vivian/HelloWorld/tracker/tools/pytorch/torch/lib/include/caffe2/utils/cblas.h:8:23: fatal error: mkl_cblas.h: No such file or directory
#include
^
compilation terminated.
CMake Error at caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o.cmake:219 (message):
Error generating
/home/vivian/detectron/build/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/detectron/ops/./caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o
make[3]: *** [CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/detectron/ops/caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o] Error 1
make[3]: Leaving directory `/home/vivian/detectron/build'
make[2]: *** [CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
make[3]: *** [CMakeFiles/caffe2_detectron_custom_ops.dir/detectron/ops/zero_even_op.cc.o] Error 1
make[3]: Leaving directory `/home/vivian/detectron/build'
make[2]: *** [CMakeFiles/caffe2_detectron_custom_ops.dir/all] Error 2
make[2]: Leaving directory `/home/vivian/detectron/build'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/vivian/detectron/build'
make: *** [ops] Error 2
from /home/vivian/HelloWorld/tracker/DetectAndTrack/lib/ops/affine_channel_nd_op.cc:13:
/usr/include/mkl_cblas.h:24:23: fatal error: mkl_types.h: No such file or directory
#include "mkl_types.h"
^
compilation terminated.
make[2]: *** [CMakeFiles/caffe2_detectron_custom_ops.dir/ops/affine_channel_nd_op.cc.o] Error 1
make[1]: *** [CMakeFiles/caffe2_detectron_custom_ops.dir/all] Error 2
make: *** [all] Error 2
关于gpupython -m caffe2.python.operator_test.thresholded_relu_op_test.py
WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
CRITICAL:root:Cannot load caffe2.python. Error: No module named caffe2_pybind11_state
python -m caffe2.python.operator_test.relu_op_test
/home/vivian/anaconda3/envs/detect_and_track/bin/python: No module named caffe2.python.operator_test.relu_op_test
问题5:
nvcc fatal : redefinition of argument 'std'
CMake Error at caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o.cmake:215 (message):
Error generating
/home/vivian/HelloWorld/tracker/DetectAndTrack/lib/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/./caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o
make[2]: *** [CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o] Error 1
make[1]: *** [CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/all] Error 2
make: *** [all] Error 2
把 ~/DetectAndTrack/lib/cmake/Cuda.cmake的第208行
list(APPEND CUDA_NVCC_FLAGS "-std=c++11")注释掉 #
成功记录:
(detect_and_track) ~lib$ make && make ops
-- Caffe2: CUDA detected: 8.0
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 8.0
-- Found cuDNN: v7.2.1 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so)
-- Autodetected CUDA architecture(s): 3.5;3.0
-- Added CUDA NVCC flags for: -gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_30,code=sm_30
-- CUDA detected: 8.0
-- Added CUDA NVCC flags for: sm_20 sm_21 sm_30 sm_35 sm_50 sm_52 sm_60 sm_61
-- Found libcuda: /usr/lib/x86_64-linux-gnu/libcuda.so
-- Found libnvrtc: /usr/local/cuda/lib64/libnvrtc.so
-- Found cuDNN: v7.2.1 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so)
-- Summary:
-- CMake version : 3.10.3
-- CMake command : /home/vivian/anaconda3/envs/detect_and_track/bin/cmake
-- System name : Linux
-- C++ compiler : /usr/bin/c++
-- C++ compiler version : 4.9.4
-- CXX flags : -std=c++11 -O2 -fPIC -Wno-narrowing
-- Caffe2 version : 1.0.0
-- Caffe2 include path : /home/vivian/HelloWorld/tracker/tools/pytorch/torch/lib/include
-- Have CUDA : TRUE
-- CUDA version : 8.0
-- CuDNN version : 7.2.1
-- Configuring done
-- Generating done
-- Build files have been written to: /home/vivian/HelloWorld/tracker/DetectAndTrack/lib
[ 37%] Built target caffe2_detectron_custom_ops
[ 50%] Building NVCC (Device) object CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o
[ 62%] Building NVCC (Device) object CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/caffe2_detectron_custom_ops_gpu_generated_affine_channel_nd_op.cu.o
[ 75%] Linking CXX shared library libcaffe2_detectron_custom_ops_gpu.so
[100%] Built target caffe2_detectron_custom_ops_gpu
1.
Caffe2 comes with performance profiling support which you may find useful for benchmarking or debugging your operators (see BatchPermutationOp test for example usage). Profiling support is not built by default and you can enable it by setting the -DUSE_PROF=ON flag when running Caffe2 CMake.
Caffe2提供了性能分析支持，您可能会发现它对基准测试或调试操作员很有用（有关示例用法，请参阅BatchPermutationOp测试）。默认情况下不会构建性能分析支持，您可以通过在运行Caffe2 CMake时设置-DUSE_PROF = ON标志来启用它。
2.CMake 找不到 CUDA and cuDNN
Sometimes CMake has trouble with finding CUDA and cuDNN dirs on your machine.
When building Caffe2, you can point CMake to CUDA and cuDNN dirs by running:
cmake .. \
# insert your Caffe2 CMake flags here
-DCUDA_TOOLKIT_ROOT_DIR=/path/to/cuda/toolkit/dir \
-DCUDNN_ROOT_DIR=/path/to/cudnn/root/dir
Similarly, when building custom Detectron operators you can use:
cd $DETECTRON
mkdir -p build && cd build
cmake .. \
-DCUDA_TOOLKIT_ROOT_DIR=/path/to/cuda/toolkit/dir \
-DCUDNN_ROOT_DIR=/path/to/cudnn/root/dir
make
Note that you can use the same commands to get CMake to use specific versions of CUDA and cuDNN out of possibly multiple versions installed on your machine.
3.Protobuf Errors
Caffe2 uses protobuf as its serialization format and requires version 3.2.0 or newer. If your protobuf version is older, you can build protobuf from Caffe2 protobuf submodule and use that version instead.
记录一下 gcc4.8+Caffe2(0.8)+onnx(1.2)+onnx-caffe2.我没用过