BCEloss和CrossEntropyLoss的区别

1. BCEloss

pytorch官方提供的公式: l ( x , y ) = { l 1 , l 2 , … , l n } T , l(x,y) = \{l_1,l_2,\dots,l_n\}^T, l(x,y)={l1,l2,,ln}T,
l n = − w n [ y n l o g ( x n ) + ( 1 − y n ) l o g ( 1 − x n ) ] l_n=-w_n[y_nlog(x_n)+(1-y_n)log(1-x_n)] ln=wn[ynlog(xn)+(1yn)log(1xn)]
主要用于计算标签只有1或者0时的二分类损失,标签和预测值是一一对应的。需要注意的是,通过nn.BCEloss来计算损失前,需要对预测值进行一次sigmoid计算。sigmoid函数会将预测值映射到0-1之间。如果觉得手动加sigmoid函数麻烦,可以直接调用nn.BCEwithlogitsloss

  1. 使用nn.BCEloss计算损失
import torch
import torch.nn as nn
import torch.nn.functional as F

loss = nn.BCELoss(reduction="none")
target = torch.tensor([1,0,1], dtype=torch.float32)
predict = torch.tensor([0.8, 0.2, 0.3], dtype=torch.float32)
loss(F.sigmoid(predict), target)

#结果计算为:
tensor([0.3711, 0.7981, 0.5544])
  1. 手动实现nn.BCEloss
def myBceloss(predict, target, reduction="none"):

    predict = F.sigmoid(predict)
    if reduction == "none":
        return -(target*torch.log(predict) + (1-target)*torch.log(1-predict))
        
myBceloss(predict, target)

#结果计算为:
tensor([0.3711, 0.7981, 0.5544])

2. CrossEntropyLoss

pytorch官方提供的公式: l ( x , y ) = { l 1 , l 2 , … , l n } T , l(x,y) = \{l_1,l_2,\dots,l_n\}^T, l(x,y)={l1,l2,,ln}T,
l n = − w n l o g ( e x p ( x n , y n ) ∑ c = 1 C e x p ( x n , c ) ) ⋅ 1 l_n=-w_nlog(\frac{exp(x_{n,y_n})}{\sum_{c=1}^Cexp(x_{n,c})}) \cdot 1 ln=wnlog(c=1Cexp(xn,c)exp(xn,yn))1
用于计算多分类任务,一个标签可能对应了预测的多个概率,例如一个任务包含了 C C C个类别,那么预测值就有 C C C个。

  1. 使用nn.CrossEntropyLoss计算损失
loss2 = nn.CrossEntropyLoss(reduction="none")
target2 = torch.tensor([0, 1, 2])
predict2 = torch.tensor([[0.9, 0.2, 0.8], [0.5, 0.2, 0.4], [0.4, 0.2, 0.9]])
loss2(predict2, target2)

#结果计算为:
tensor([0.8761, 1.2729, 0.7434])
  1. 手动实现nn.CrossEntropyLoss
def myCrossEntropyloss(target, predict, reduction="none"):
    if reduction == "none":
        predict = F.softmax(predict, dim=1)
        n = torch.arange(predict.shape[0])
        predict = predict[n, target]
        return -torch.log(predict)
myCrossEntropyloss(target2, predict2)
#结果计算为:
tensor([0.8761, 1.2729, 0.7434])

你可能感兴趣的:(深度学习,深度学习,python,pytorch)