caffe+ubuntu 32位+atlas最少配置方法

caffe+Ubuntu14.0.4 64bit 环境配置说明(无CUDA,caffe在CPU下运行) --for --Amd

caffe是一个简洁高效的深度学习框架，具体介绍可以看这里，caffe环境配置过程可以参考这里，我在搭建环境时搜集了许多资料，这里整理了一下，介绍一下caffe在无CUDA的环境下如何配置。

1. 安装build-essentials

安装开发所需要的一些基本包

 

sudo apt-get install build-essential

如果出现essential包不可用的情况，可以执行下列命令解决：

sudo apt-get update

2. 安装ATLAS for Ubuntu

执行命令:

 

sudo apt-get install libatlas-base-dev

注：ATLAS, MKL,或OpenBLAS都可以，我这里选择安装ATLAS

3. 安装OpenCV
这个尽量不要手动安装， Github上有人已经写好了完整的安装脚本：https://github.com/jayrambhia/Install-OpenCV
下载该脚本，解压".zip"文件,解压命令:

 

unzip /home/joe/Install-OpenCV-master.zip

进入Ubuntu/2.4 目录, 给所有shell脚本加上可执行权限：

 

chmod +x *.sh

然后安装最新版本 （当前为2.4.9）：

 

sudo ./opencv2_4_9.sh

4. 安装其他依赖项

 

Ubuntu14.04用户执行

 

sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler

使用其它系统的可以参考官网介绍。

5. 编译Caffe
完成了上述环境的配置，就可以编译Caffe了！
下载caffe安装包,下载地址:https://github.com/BVLC/caffe
解压该压缩包，解压缩命令:

 

unzip /home/joe/caffe-master.zip

注:如果解压位置出错了,可以使用以下命令删除该目录及所有的子目录:

sudo rm -rf caffe-master

进入caffe根目录，首先复制一份Makefile.config，命令：

 

cp Makefile.config.example Makefile.config

然后修改里面的内容(Makefile.config)，主要需要修改的参数包括：

    CPU_ONLY 是否只使用CPU模式，由于我没有NVIDIA的显卡，就没有安装CUDA，因此需要打开这个选项。
其余的一些配置可以根据需要修改：
    BLAS (使用intel mkl还是OpenBLAS)
    MATLAB_DIR 如果需要使用MATLAB wrapper的同学需要指定matlab的安装路径, 如我的路径为 /usr/local/MATLAB/R2013b (注意该目录下需要包含bin文件夹，bin文件夹里应该包含mex二进制程序)

DEBUG 是否使用debug模式，打开此选项则可以在eclipse或者NSight中debug程序

注意：最懒的办法就是只去掉CPU_ONLY的注释即可，默认使用atlas和python（自带）。

完成上述设置后，开始编译：

 

1. make all -j4  

2. make test  

3. make runtest

注意：-j4 是指使用几个线程来同时编译，可以加快速度，j后面的数字可以根据CPU core的个数来决定，如果CPU是4核的，则参数为-j4，也可以不添加这个参数，直接使用“make all”，这样速度可能会慢一点儿。

6.使用MNIST数据集进行测试

    Caffe默认情况会安装在$CAFFE_ROOT，就是解压到的那个目录，例如：$ home/username/caffe-master，所以下面的工作，
默认已经切换到了该工作目录。下面的工作主要是测试Caffe是否工作正常，不做详细评估。具体设置请参考官网：
http://caffe.berkeleyvision.org/gathered/examples/mnist.html
    (1)数据预处理
    可以用下载好的数据集，也可以重新下载，直接下载的具体操作如下：

 

1. $ cd caff-master   (go caff home dir )

2. $ sudo sh data/mnist/get_mnist.sh

       (2)重建LDB文件，就是处理二进制数据集为Caffe识别的数据集，以后所有的数据，包括jpe文件都要处理成这个格式，执行命令如下：

 

$ sudo sh ./examples/mnist/create_mnist.sh

生成mnist-train-leveldb/ 和 mnist-test-leveldb/文件夹，这里包含了LDB格式的数据集

注:新版caffe都需要从根目录上执行，如果使用下列命令执行：

 

1. # cd examples/ mnist 2. # sudo sh ./create_mnist.sh

可能会遇到这个错误：./create_mnist.sh: 16: ./create_mnist.sh: build/examples/mnist/convert_mnist_data.bin: not found

 

      (3)训练mnist
      如果没有GPU,只有CPU的话,需要先修改examples/mnist目录下lenet_solver.prototxt文件,将solver_mode:GPU改为 solver_mode:CPU，修改后结果如下所示：

 

1. # solver mode: CPU or GPU  

2. solver_mode: CPU

修改时可以使用vi编辑命令，如果是只读文件，不能编辑，可以用sudo命令，比如:

 

sudo vi lenet_solver.prototxt

先进入命令模式，使用a进入编辑模式，修改完之后，使用esc退出编辑模式，进入末行模式，再使用“:wq”保存修改并退出(“:q!”为退出但不保存修改)
修改完成后，再执行下面的命令进行训练:

1. $ cd examples/mnist  

2. $ sudo sh ./train_lenet.sh

注意：此处尝试后发现会出现与上面一样的错误，解决办法与上面相同。

最终训练完的模型存储为一个二进制的protobuf文件，至此，Caffe安装的所有步骤完结。

 

 

----------------------------------------------------------------------------------

参考资料:

Caffe 实例测试一： MNIST
http://blog.sciencenet.cn/blog-1583812-843207.html

Caffe + Ubuntu 14.04 64bit + CUDA 6.5 配置说明
http://www.cnblogs.com/platero/p/3993877.html

CNN之Caffe配置
http://www.cnblogs.com/alfredtofu/p/3577241.html
 
Training LeNet on MNIST with Caffe
http://caffe.berkeleyvision.org/gathered/examples/mnist.html

Caffe+Ubuntu14.04+CUDA6.5新手安装配置指南
http://www.haodaima.net/art/2823705

caffe安装指南
http://www.haodaima.net/art/2823705

 

 

 参考链接：

http://94it.net/a/jingxuanboke/2015/0322/493702.html

http://www.cnblogs.com/outline/p/4381464.html

关于32位ubuntu中会产生错误，解决方案如下：

I get an error after running the last command:

sudo apt-get install -y git
git clone  https://github.com/BVLC/caffe.git

cd caffe && git checkout dev
cp Makefile.config.example Makefile.config

make -j 4 all

make -j 4 runtest -------> After 15 minutes of testing I get the following error:



F0618 11:23:23.786288 30901 db.hpp:109] Check failed: mdb_status == 0 (-30792 vs. 0) MDB_MAP_FULL: Environment mapsize limit reached
*** Check failure stack trace: ***
@ 0x432d9060 (unknown)
@ 0x432d8f5c (unknown)
@ 0x432d8b78 (unknown)
@ 0x432daf98 (unknown)
@ 0x43bd63fc caffe::db::LMDBTransaction::Put()
@ 0x1421a2 caffe::DBTest_TestWrite_Test<>::TestBody()
@ 0x24665c testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x240a82 testing::Test::Run()
@ 0x240b12 testing::TestInfo::Run()
@ 0x240bea testing::TestCase::Run()
@ 0x242442 testing::internal::UnitTestImpl::RunAllTests()
@ 0x242634 testing::UnitTest::Run()
@ 0x7d430 main
@ 0x43eec632 (unknown)


I have tried this twice now and get the same error. I'm not sure what is going on

#1

Posted 06/18/2015 03:47 PM   

Unfortunately the master branch of Caffe has a really large value for LMDB_MAP_SIZE in src/caffe/util/db.cpp, which confuses the 32-bit ARM processor on the Jetson, eventually leading to Caffe tests failing with errors like MDB_MAP_FULL: Environment mapsize limit reached. Caffe GitHub issue #1861 has some discussion about this and maybe it will be fixed eventually, but for the moment if you manually adjust the value from 1099511627776 to 536870912, you’ll be able to run all the Caffe tests successfully.

This information was gathered from three places:

http://planspace.org/20150614-the_nvidia_jetson_tk1_with_caffe_on_mnist/
https://github.com/BVLC/caffe/issues
http://jetsonhacks.com/2015/01/17/nvidia-jetson-tk1-caffe-deep-learning-framework/

jetsonhacks.com

#2

Posted 06/18/2015 04:04 PM   

said:

Unfortunately the master branch of Caffe has a really large value for LMDB_MAP_SIZE in src/caffe/util/db.cpp, which confuses the 32-bit ARM processor on the Jetson, eventually leading to Caffe tests failing with errors like MDB_MAP_FULL: Environment mapsize limit reached. Caffe GitHub issue #1861 has some discussion about this and maybe it will be fixed eventually, but for the moment if you manually adjust the value from 1099511627776 to 536870912, you’ll be able to run all the Caffe tests successfully.

This information was gathered from three places:


http://planspace.org/20150614-the_nvidia_jetson_tk1_with_caffe_on_mnist/

https://github.com/BVLC/caffe/issues

http://jetsonhacks.com/2015/01/17/nvidia-jetson-tk1-caffe-deep-learning-framework/

Now the tests run successfully, but when I add the line:

build/tools/caffe time --model=models/bvlc_alexnet/deploy.prototxt --gpu=0 

I get the following error message:
ubuntu@tegra-ubuntu:~/caffe$ build/tools/caffe time --model=models/bvlc_alexnet/deploy.prototxt --gpu=0 
build/tools/caffe: error while loading shared libraries: libcaffe.so: cannot open shared object file: No such file or directory

#3

Posted 06/18/2015 05:36 PM   

You should compare where your executable thinks that the 'libcaffe.so' library is versus where it is actually located.

jetsonhacks.com

#4

Posted 06/18/2015 05:58 PM   

said:

You should compare where your executable thinks that the 'libcaffe.so' library is versus where it is actually located.

I can't find the library anywhere

#5

Posted 06/18/2015 06:04 PM   

Actaully, its in the lib folder

caffe/build/lib

#6

Posted 06/18/2015 06:05 PM    The path it tells is caffe/libcaffe.so

#7

Posted 06/18/2015 06:09 PM     

I cut the library and pasted it where the device thinks it is located (right in the caffe folder). Now it gets to this stage:

I0618 14:14:03.554016 27880 caffe.cpp:225] *** Benchmark begins ***
I0618 14:14:03.554700 27880 caffe.cpp:226] Testing for 50 iterations.


before saying "Killed"

#8

Posted 06/18/2015 06:15 PM    Do you have any other applications running? It may be running out of memory. Try rebooting the machine, and then running the test.

jetsonhacks.com

#9

Posted 06/18/2015 06:45 PM   

said:

Do you have any other applications running? It may be running out of memory. Try rebooting the machine, and then running the test.

Hmm it started working randomly....thanks!

#10

Posted 06/18/2015 06:57 PM   

I get an error right after I invoke the "make -j 4 all" command, When I get to the “make -j 4 all” step, I get:
CXX .build_release/src/caffe/proto/caffe.pb.cc
CXX src/caffe/common.cpp
CXX src/caffe/layers/deconv_layer.cpp
CXX src/caffe/layers/dropout_layer.cpp

Then I get a bunch of compile errors, about 300 or so lines of them, that have a lot to do with the ./include/caffe/ directory. Any advice? Thanks!

#11

Posted 07/30/2015 09:58 PM   

转载自：https://devtalk.nvidia.com/default/topic/846361