My Machine is Lenovo Y480, and Ubuntu 16.04, NVIDIA GTX650. What a poor Boy! - -

Step 1 Install Dependencies

sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler 
sudo apt-get install --no-install-recommends libboost-all-dev
sudo apt-get install libopenblas-dev liblapack-dev libatlas-base-dev
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev

Step 2 Install NVIDIA Driver

Find your driver version in http://www.nvidia.com/Download/index.aspx?lang=en-us, and mine is 375.39 .
Run sudo apt-get remove --purge nvidia* to uninstall the older nvidia drivers.

sudo add-apt-repository ppa:xorg-edgers/ppa
sudo apt-get update
sudo apt-get install nvidia-375 #This is mine, may you have another one.

Run sudo nvidia-smi and if get some GPU infos, the NVDIA Driver is installed well. Before this step, you may restart your system first.

Step 3 Install CUDA 8.0

Download a cuda 8.0 runfile in https://developer.nvidia.com/cuda-downloads.
You have to read a very much long text, continuously press Enter to reach the bottom.
sudo sh cuda_8.0.27_linux.run to install cuda 8.0. During the installation, you need choose y or n for several times, when you see Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 361.62?, please choose no, while the rest could be yes.
sudo nano ~/.bashrc and add

export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda8.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

into the bash file, run souece ~/.bashrc to make it work.

cd /usr/local/cuda-8.0/samples/1_Utilities/deviceQuery
make
sudo ./deviceQuery

CUDA 8.0 installation is done if you see some GPU infos.

Step 4 Set up CuDNN 5.1

Download a cuDNN 5.1 from https://developer.nvidia.com/rdp/cudnn-download.
tar -zxvf cudnn_file_name.tgz and the directory may be 'cuda'.
cd cuda/include and sudo cp cudnn.h /usr/local/cuda/include/
cd cuda/lib64 and

sudo cp lib* /usr/local/cuda/lib64/ 
cd /usr/local/cuda/lib64/
sudo chmod 777 libcudnn.so.5.1.10   # mine is 5.1.10, may you have another one.
sudo rm -rf libcudnn.so libcudnn.so.5  
sudo ln -s libcudnn.so.5.1.10 libcudnn.so.5
sudo ln -s libcudnn.so.5 libcudnn.so

Step 5 Install OpenCV 3.1

Download openCV 3.1 from opencv.org/opencv-3-1.html and tar -zxvf opencv-3.1.0.tar.gz.
Download ippicv_linux_20151201.tgz, and do

cp ippicv_linux_20151201.tgz opencv-3.1.0/3rdparty/ippicv/downloads/linux-808b791a6eac9ed78d32a7666804320e

and nano opencv-3.1.0/modules/cudalegacy/src/graphcuts.cpp, modify the file like

repalce
#if !defined (HAVE_CUDA) || defined (CUDA_DISABLER)
by
#if !defined (HAVE_CUDA) || defined (CUDA_DISABLER) || (CUDART_VERSION >= 8000)

otherwise you may fail to make openCV.

mkdir opencv-3.1.0/build
cd opencv-3.1.0/build
cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local ..
make -j8
make install

Step 6 Set up Caffe

git clone https://github.com/BVLC/caffe.git
sudo cp Makefile.config.example Makefile.config
Make some modifes
a. sudo gedit Makefile.config

uncomment
#USE_CUDNN := 1
to
USE_CUDNN := 1
and
uncomment
#OPENCV_VERSION := 3 
to
OPENCV_VERSION := 3
and
replace
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib 
by 
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/
and
replace
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
by
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/

b. sudo gedit Makefile

replace
NVCCFLAGS +=-ccbin=$(CXX) -Xcompiler-fPIC $(COMMON_FLAGS)
by
NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)
and
replace
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5
by
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs

c. sudo gedit /usr/local/cuda/include/host_config.h

comment
#error-- unsupported GNU version! gcc versions later than 4.9 are not supported!
to
//#error-- unsupported GNU version! gcc versions later than 4.9 are not supported!

make all -j8
If there is still some errors while making caffe such as libcudnn.so.5 cannot open shared object file: No such file or directory, you should run

sudo cp /usr/local/cuda-8.0/lib64/libcudnn.so.5 /usr/local/lib/libcudnn.so.5 && sudo ldconfig

, same as some other similar errors. And then rerun make all -j8.

Run sudo make runtest to have a test for caffe, and you may see many 'RUN OK's while a 'PASSED' is in the end of output.

Step 7 Have a Try

# enter the root directory of caffe
cd caffe
# get the mnist data set
./data/mnist/get_mnist.sh
# create
./examples/mnist/create_mnist.sh
# train
./examples/mnist/train_lenet.sh

Done.

Caffe + CUDA 8.0 + CuDNN 5.1 Configuration on Ubuntu 16.04