PaddlePaddle动手实现三个版本的CrossEntropy

今天有人对CrossEntropy有疑惑，我做了一个小实验，这里把我的代码分享了出来：

import paddle
import paddle.nn as nn
import paddle.nn.functional as F

y_true=paddle.to_tensor([[1],[2]])
y_pred=paddle.to_tensor([[0.05,0.95,0],[0.1,0.8,0.1]])


one_hot = F.one_hot(y_true,num_classes=3)
res=paddle.sum(paddle.exp(y_pred),axis=1)
res = paddle.reshape(res, [-1, 1])
softmax = paddle.exp(y_pred)/res
logsoftmax = paddle.log(softmax)

nllloss = -paddle.sum(one_hot.squeeze(1)*logsoftmax)/y_true.shape[0]
print(nllloss)

logsoftmax = F.log_softmax(y_pred, axis = 1)

nllloss = F.nll_loss(logsoftmax, y_true)
print(nllloss)

nllloss=paddle.nn.functional.cross_entropy(input=y_pred,label=y_true)
print(nllloss)

输出：

Tensor(shape=[1], dtype=float32, place=CPUPlace, stop_gradient=True,
       [0.98689497])
Tensor(shape=[1], dtype=float32, place=CPUPlace, stop_gradient=True,
       [0.98689508])
Tensor(shape=[1], dtype=float32, place=CPUPlace, stop_gradient=True,
       [0.98689508])

应该是符合预期啦

参考文献

paddle文档
吃透torch.nn.CrossEntropyLoss()