Pytorch 线性回归 grad清零报错：w.grad.data.zero_() AttributeError: 'NoneType' object has no attribute 'data'

学习了https://github.com/L1aoXingyu/code-of-learn-deep-learning-with-pytorch/blob/master/chapter3_NN/linear-regression-gradient-descend.ipynb   上的Pytorch线性回归这一章，自己复了一遍代码后，再删去了一些人为学习型的输出和plot画图代码后报错 

w.grad.data.zero_()

AttributeError: 'NoneType' object has no attribute 'data'

 

很明显。问题出在梯度清零上。

以下是错误代码

import torch
import numpy as np
from torch.autograd import Variable
import matplotlib.pyplot as plt

torch.manual_seed(2017)


w_target = np.array([0.5, 3, 2.4]) # 定义参数
b_target = np.array([0.9]) # 定义参数

x_sample=np.arange(-3,3.1,0.1)
y_sample=b_target[0] + w_target[0] *x_sample+w_target[1]*x_sample**2 +w_target[2]*x_sample**3


x_train=np.stack([x_sample ** i for i in range(1,4)],axis=1)

x_train=torch.from_numpy(x_train).float()  # 转换成 float tensor
y_train=torch.from_numpy(y_sample).float().unsqueeze(1) 


w=Variable(torch.randn(3,1),requires_grad=True)
b=Variable(torch.zeros(1),requires_grad=True)
x_train=Variable(x_train)
y_tran=Variable(y_train)

def multi_linear(x):
    return torch.mm(x,w)+b

def get_loss(y_, y):
    return torch.mean((y_ - y_train) ** 2)


for e in range(100):
    y_pred = multi_linear(x_train)
    loss = get_loss(y_pred, y_train)
    w.grad.data.zero_()
    b.grad.data.zero_()
    loss.backward()
    
    # 更新参数
    w.data = w.data - 0.001 * w.grad.data
    b.data = b.data - 0.001 * b.grad.data
    if (e + 1) % 20 == 0:
        print('epoch {}, Loss: {:.5f}'.format(e+1, loss.item()))

 

查看教程之后，我发现是grad.data.zero_()函数缺少data值，所以有两种方法

 

法1：

默认一开始梯度为零或者说没有梯度，所以讲梯度清零放在for循环的最后

for e in range(100):
    y_pred = multi_linear(x_train)
    loss = get_loss(y_pred, y_train)
    
    
    loss.backward()
    
    # 更新参数
    w.data = w.data - 0.001 * w.grad.data
    b.data = b.data - 0.001 * b.grad.data
    if (e + 1) % 20 == 0:
        print('epoch {}, Loss: {:.5f}'.format(e+1, loss.item()))
    w.grad.data.zero_()
    b.grad.data.zero_()

 

法2：

在for循环之前走一遍训练

#以下放在for循环之前
y_pred = multi_linear(x_train)
loss = get_loss(y_pred, y_train)
loss.backward()
w.data = w.data - 0.001 * w.grad.data
b.data = b.data - 0.001 * b.grad.data


for e in range(100):
    y_pred = multi_linear(x_train)
    loss = get_loss(y_pred, y_train)
    w.grad.data.zero_()
    b.grad.data.zero_()
    loss.backward()
    
    # 更新参数
    w.data = w.data - 0.001 * w.grad.data
    b.data = b.data - 0.001 * b.grad.data
    if (e + 1) % 20 == 0:
        print('epoch {}, Loss: {:.5f}'.format(e+1, loss.item()))

 

 

ps.该教程最后打印loss时会有报错，同上文代码中一样改成loss.item()就可以了