caffe 对分类结果画混淆矩阵（confusion matrix）

引用链接：

1.Caffe python layer to print confusion matrix
链接：http://gcucurull.github.io/caffe/python/deep-learning/2016/06/29/caffe-confusion-matrix/
2.github主页：https://github.com/gcucurull/caffe-conf-matrix
3.Caffe Python Layer编译：https://chrischoy.github.io/research/caffe-python-layer/

---

1.编译python layer

WITH_PYTHON_LAYER=1 make && make pycaffe

打开makefile.config中 WITH_PYTHON_LAYER=1的开关，然后make编译，再然后make pycaffe 编译。
2.在hands_test.protxt文件中定义python layer如下：

layer {
  type: 'Python'
  name: 'py_accuracy'
  top: 'py_accuracy'
  bottom: 'ip2'
  bottom: 'label'
  python_param {
    # the module name -- usually the filename -- that needs to be in $PYTHONPATH
    module: 'python_confmat'
    # the layer name -- the class name in the module
    layer: 'PythonConfMat'
    # this is the number of test iterations, it must be the same as defined in the solver.
    param_str: '{"test_iter":100}'
  }
  include {
    phase: TEST
  }
}

注意：test_iter要和batch size对应，以取遍所有测试集样本。

python_confmat.py如下：

import caffe
import json
import numpy as np
import sys

import sklearn.metrics

class PythonConfMat(caffe.Layer):
    """
    Compute the Accuracy with a Python Layer
    """

    def setup(self, bottom, top):
        # check input pair
        if len(bottom) != 2:
            raise Exception("Need two inputs.")

        self.num_labels = bottom[0].channels
        params = json.loads(self.param_str)
        self.test_iter = params['test_iter']
        self.conf_matrix = np.zeros((self.num_labels, self.num_labels))
        self.current_iter = 0

    def reshape(self, bottom, top):
        # bottom[0] are the net's outputs
        # bottom[1] are the ground truth labels

        # Net outputs and labels must have the same number of elements
        if bottom[0].num != bottom[1].num:
            raise Exception("Inputs must have the same number of elements.")

        # accuracy output is scalar
        top[0].reshape(1)

    def forward(self, bottom, top):
        self.current_iter += 1

        # predicted outputs
        pred = np.argmax(bottom[0].data, axis=1)
        accuracy = np.sum(pred == bottom[1].data).astype(np.float32) / bottom[0].num
        top[0].data[...] = accuracy

        # compute confusion matrix
        self.conf_matrix += sklearn.metrics.confusion_matrix(bottom[1].data, pred, labels=range(self.num_labels))

        if self.current_iter == self.test_iter:
            self.current_iter = 0
            sys.stdout.write('\nCAUTION!! test_iter = %i. Make sure this is the correct value' % self.test_iter)
            sys.stdout.write('\n"param_str: \'{"test_iter":%i}\'" has been set in the definition of the PythonLayer' % self.test_iter)
            sys.stdout.write('\n\nConfusion Matrix')
            sys.stdout.write('\t'*(self.num_labels-2)+'| Accuracy')
            sys.stdout.write('\n'+'-'*8*(self.num_labels+1))
            sys.stdout.write('\n')
            for i in range(len(self.conf_matrix)):
                for j in range(len(self.conf_matrix[i])):
                    sys.stdout.write(str(self.conf_matrix[i][j].astype(np.int))+'\t')
                sys.stdout.write('| %3.2f %%' % (self.conf_matrix[i][i]*100 / self.conf_matrix[i].sum()))
                sys.stdout.write('\n')
            sys.stdout.write('Number of test samples: %i \n\n' % self.conf_matrix.sum())
            # reset conf_matrix for next test phase
            self.conf_matrix = np.zeros((self.num_labels, self.num_labels))

    def backward(self, top, propagate_down, bottom):
        pass

注意： 调用test命令可能出现import error：no module named python_confmat

解决方案：终端输入export PYTHONPATH=/home/xxx/caffe/python即可。

然后调用test命令即可生成混淆矩阵。
生成效果如下：

延伸阅读：

1.https://github.com/borisgin/caffe/blob/confusion_matrix/src/caffe/layers/confusion_matrix_layer.cpp
直接在caffe层里添加了confusion_matrix_layer
2.https://gist.github.com/stardust2602/79c818add4f7100397dd
Caffe script to compute accuracy and confusion matrix based on training input ( .txt file )
3.https://github.com/giladsharir/caffe
4.https://groups.google.com/forum/#!topic/caffe-users/u-pyqqZzlOM
5.http://blog.csdn.net/guoyilin/article/details/42047615
如何用python画好confusion matrix