混淆矩阵和语义分割评价指标

语义分割评价指标

混淆矩阵和评价指标

https://cocodataset.org/#stuff-eval

---

混淆矩阵（Confusion Matrix)

指标	公式
准确率、正确率	$$\frac{TP+TN}{ТР+TN+FP+FN}$$
精确率、精度	$$P=\frac{TP}{ТР+FP}$$
真正类率、召回率	$$R=\frac{TP}{ТР+FN}$$
F1-Score：精确度和召回率的调和平均值	$$F1=2\frac{P\times R}{P+R}$$

---

举例

注：大多数博客的类别像素准确率都是$\frac{TP}{TP+FP}$，而代码中都是$\frac{TP}{ТР+FN}$，其实类别像素准确率应该为混淆矩阵的召回率。

指标	公式	举例
像素准确率	$$PA=\frac{TP+TN}{ТР+TN+FP+FN}$$	$$PA=\frac{a+e+i}{a+b+c+d+e+f+g+h+i}$$
类别像素准确率	$$CP{A}_i=\frac{T{P}_i}{T{P}_i+F{N}_i}$$	$$CP{A}_{类别1}=\frac{a}{a+b+c};CP{A}_{类别2}=\frac{e}{d+e+f};CP{A}_{类别3}=\frac{i}{g+h+i}$$
类别平均像素准确率	$$MPA=\frac{\Sigma (CP{A}_i)}{N}$$	$$MPA=\frac{CP{A}_{类别1}+CP{A}_{类别2}+CP{A}_{类别3}}{3}$$
交并比	$$Io{U}_i=\frac{T{P}_i}{T{P}_i+F{P}_i+F{N}_i}$$	$$Io{U}_{类别1}=\frac{a}{a+b+c+d+g}$$$$Io{U}_{类别2}=\frac{e}{d+e+f+b+h}$$$$Io{U}_{类别3}=\frac{i}{g+h+i+c+f}$$
平均交并比	$$MIoU=\frac{\Sigma (Io{U}_i)}{N}$$	$$MIoU=\frac{Io{U}_{类别1}+Io{U}_{类别2}+Io{U}_{类别3}}{3}$$
频权平均交并比	$$FWMIoU=\frac{{\sum }_{k=1}^K{n}_{k}T{P}_k}{{\sum }_{k=1}^K{n}_k\ast (T{P}_k+F{P}_k+F{N}_k)}$$	$$freq1=\frac{a+b+c}{a+b+c+d+e+f+g+h+i}$$$$freq2=\frac{d+e+f}{a+b+c+d+e+f+g+h+i}$$$$freq3=\frac{g+h+i}{a+b+c+d+e+f+g+h+i}$$ $$FWMIoU=\frac{Io{U}_{类别1}\ast freq1+Io{U}_{类别2}\ast freq2+Io{U}_{类别3}\ast freq3}{freq1+freq2+freq3}$$
$F1-score$	$$2\ast TP/(2\ast TP+FP+FN)$$	$$F{1}_{类别1}=\frac{2a}{2a+b+c+d+g+h}$$$$F{1}_{类别2}=\frac{2e}{2e+d+f+b+e+h}$$$$F{1}_{类别3}=\frac{2i}{2i+g+h+c+f}$$$$F1=\frac{F{1}_{类别1}+F{1}_{类别2}+F{1}_{类别3}}{3}$$
Dice系数
ROC和AUC
RR/MRR

"""
refer to https://github.com/jfzhang95/pytorch-deeplab-xception/blob/master/utils/metrics.py
"""
import numpy as np

__all__ = ['SegmentationMetric']

"""
                  预测
                P    N
        P      TP    FN
真实
        N      FP    TN
"""


class SegmentationMetric(object):
    def __init__(self, numClass):
        self.numClass = numClass
        self.confusionMatrix = np.zeros((self.numClass,) * 2)

    def genConfusionMatrix(self, imgPredict, imgLabel):  # 同FCN中score.py的fast_hist()函数
        # remove classes from unlabeled pixels in gt image and predict
        mask = (imgLabel >= 0) & (imgLabel < self.numClass)
        label = self.numClass * imgLabel[mask] + imgPredict[mask]
        count = np.bincount(label, minlength=self.numClass ** 2)
        confusionMatrix = count.reshape(self.numClass, self.numClass)
        return confusionMatrix

    def addBatch(self, imgPredict, imgLabel):
        assert imgPredict.shape == imgLabel.shape
        self.confusionMatrix += self.genConfusionMatrix(imgPredict, imgLabel)

    def pixelAccuracy(self):
        # 返回所有类别的整体像素准确度
        #  PA = acc = (TP + TN) / (TP + TN + FP + TN)
        acc = np.diag(self.confusionMatrix).sum() / self.confusionMatrix.sum()
        return acc

    def classPixelAccuracy(self):
        # 返回每个类别的像素精度（称为精度的更准确方法）
        # acc = (TP) / TP + FN
        classAcc = np.diag(self.confusionMatrix) / self.confusionMatrix.sum(axis=1)
        return classAcc  # 返回的是一个列表值，如：[0.90, 0.80, 0.96]，表示类别1 2 3各类别的预测准确率

    def meanPixelAccuracy(self):
        classAcc = self.classPixelAccuracy()
        meanAcc = np.nanmean(classAcc)  # np.nanmean 求平均值，nan表示遇到Nan类型，其值取为0
        return meanAcc  # 返回单个值，如：np.nanmean([0.90, 0.80, 0.96, nan, nan]) = (0.90 + 0.80 + 0.96） / 3 =  0.89

    def IntersectionOverUnion(self):
        # Intersection = TP Union = TP + FP + FN
        # IoU = TP / (TP + FP + FN)
        intersection = np.diag(self.confusionMatrix)  # 取对角元素的值，返回列表
        union = np.sum(self.confusionMatrix, axis=1) + np.sum(self.confusionMatrix, axis=0) - np.diag(
            self.confusionMatrix)  # axis = 1表示混淆矩阵行的值，返回列表； axis = 0表示取混淆矩阵列的值，返回列表
        IoU = intersection / union  # 返回列表，其值为各个类别的IoU
        return IoU

    def meanIntersectionOverUnion(self):
        IoU = self.IntersectionOverUnion()
        mIoU = np.nanmean(IoU)  # 求各类别IoU的平均
        return mIoU

    def FrequencyWeightedIntersectionOverUnion(self):
        # FWIOU =     [(TP+FN)/(TP+FP+TN+FN)] *[TP / (TP + FP + FN)]
        freq = np.sum(self.confusionMatrix, axis=1) / np.sum(self.confusionMatrix)
        iu = np.diag(self.confusionMatrix) / (
                np.sum(self.confusionMatrix, axis=1) + np.sum(self.confusionMatrix, axis=0) -
                np.diag(self.confusionMatrix))
        FWIoU = (freq[freq > 0] * iu[freq > 0]).sum()
        return FWIoU

    def reset(self):
        self.confusionMatrix = np.zeros((self.numClass, self.numClass))


if __name__ == '__main__':
    imgPredict = np.array([0, 1, 2, 2, 0, 1])
    imgLabel = np.array([0, 1, 2, 2, 1, 2])
    metric = SegmentationMetric(3)  # 3表示有3个分类
    metric.addBatch(imgPredict, imgLabel)
    pa = metric.pixelAccuracy()
    cpa = metric.classPixelAccuracy()
    mpa = metric.meanPixelAccuracy()
    IoU = metric.IntersectionOverUnion()
    mIoU = metric.meanIntersectionOverUnion()
    fwmIoU = metric.FrequencyWeightedIntersectionOverUnion()
    print('混淆矩阵:\n', metric.confusionMatrix)
    print('pa:\t', pa)
    print('cpa:\t', cpa)
    print('mpa:\t', mpa)
    print('IoU:\t', IoU)
    print('mIoU:\t', mIoU)
    print('FWmIoU:\t', fwmIoU)

结果

混淆矩阵:
 [[1. 0. 0.]
 [1. 1. 0.]
 [0. 1. 2.]]
pa:	 0.6666666666666666
cpa:	 [1.         0.5        0.66666667]
mpa:	 0.7222222222222222
IoU:	 [0.5        0.33333333 0.66666667]
mIoU:	 0.5
FWmIoU:	 0.5277777777777777