pytorch官方提供的公式: l ( x , y ) = { l 1 , l 2 , … , l n } T , l(x,y) = \{l_1,l_2,\dots,l_n\}^T, l(x,y)={l1,l2,…,ln}T,
l n = − w n [ y n l o g ( x n ) + ( 1 − y n ) l o g ( 1 − x n ) ] l_n=-w_n[y_nlog(x_n)+(1-y_n)log(1-x_n)] ln=−wn[ynlog(xn)+(1−yn)log(1−xn)]
主要用于计算标签只有1或者0时的二分类损失,标签和预测值是一一对应的。需要注意的是,通过nn.BCEloss来计算损失前,需要对预测值进行一次sigmoid计算。sigmoid函数会将预测值映射到0-1之间。如果觉得手动加sigmoid函数麻烦,可以直接调用nn.BCEwithlogitsloss。
import torch
import torch.nn as nn
import torch.nn.functional as F
loss = nn.BCELoss(reduction="none")
target = torch.tensor([1,0,1], dtype=torch.float32)
predict = torch.tensor([0.8, 0.2, 0.3], dtype=torch.float32)
loss(F.sigmoid(predict), target)
#结果计算为:
tensor([0.3711, 0.7981, 0.5544])
def myBceloss(predict, target, reduction="none"):
predict = F.sigmoid(predict)
if reduction == "none":
return -(target*torch.log(predict) + (1-target)*torch.log(1-predict))
myBceloss(predict, target)
#结果计算为:
tensor([0.3711, 0.7981, 0.5544])
pytorch官方提供的公式: l ( x , y ) = { l 1 , l 2 , … , l n } T , l(x,y) = \{l_1,l_2,\dots,l_n\}^T, l(x,y)={l1,l2,…,ln}T,
l n = − w n l o g ( e x p ( x n , y n ) ∑ c = 1 C e x p ( x n , c ) ) ⋅ 1 l_n=-w_nlog(\frac{exp(x_{n,y_n})}{\sum_{c=1}^Cexp(x_{n,c})}) \cdot 1 ln=−wnlog(∑c=1Cexp(xn,c)exp(xn,yn))⋅1
用于计算多分类任务,一个标签可能对应了预测的多个概率,例如一个任务包含了 C C C个类别,那么预测值就有 C C C个。
loss2 = nn.CrossEntropyLoss(reduction="none")
target2 = torch.tensor([0, 1, 2])
predict2 = torch.tensor([[0.9, 0.2, 0.8], [0.5, 0.2, 0.4], [0.4, 0.2, 0.9]])
loss2(predict2, target2)
#结果计算为:
tensor([0.8761, 1.2729, 0.7434])
def myCrossEntropyloss(target, predict, reduction="none"):
if reduction == "none":
predict = F.softmax(predict, dim=1)
n = torch.arange(predict.shape[0])
predict = predict[n, target]
return -torch.log(predict)
myCrossEntropyloss(target2, predict2)
#结果计算为:
tensor([0.8761, 1.2729, 0.7434])