我的 Paddle 版本 和 PyTorch 版本:
>>> paddle.__version__
'2.4.0'
>>> torch.__version__
'1.13.1+cu117'
paddle中的 linear 计算公式:
O u t = X W + b Out = XW + b Out=XW+b
而 Torch 中的计算公式:
y = x A T + b y = x A^T + b y=xAT+b
torch 中的权重会经过转置在做计算,以下是一个 demo
import paddle
import torch
import numpy as np
import paddle.nn.functional as pF
import torch.nn.functional as tF
np.random.seed(1107)
w = np.random.randn(3, 3)
b = np.random.randn(3,)
x = np.random.randn(2, 3)
w_p = paddle.to_tensor(w)
b_p = paddle.to_tensor(b)
x_p = paddle.to_tensor(x)
w_t = torch.tensor(w)
b_t = torch.tensor(b)
x_t = torch.tensor(x)
y_np = np.matmul(x, w) + b
y_p = pF.linear(x_p, w_p, b_p)
y_t = tF.linear(x_t, w_t.T, b_t)
print(y_np)
print(y_p)
print(y_t)
[[ 0.62959419 -1.8686625 1.47863036]
[ 1.31505431 -0.37713307 -1.04742683]]
Tensor(shape=[2, 3], dtype=float64, place=Place(cpu), stop_gradient=True,
[[ 0.62959419, -1.86866250, 1.47863036],
[ 1.31505431, -0.37713307, -1.04742683]])
tensor([[ 0.6296, -1.8687, 1.4786],
[ 1.3151, -0.3771, -1.0474]], dtype=torch.float64)