mindspore的model.train怎么实现pytorch的optimizer.zero_grad()和loss.backward()

问题描述：

【功能模块】

开发环境：Win10

【操作步骤&问题现象】

1、我在使用msp复现我的pytorch代码时，直接用model.train()训练模型，但是损失函数一直是0.1。和我的pytorch代码唯一不一样的是我在msp中没有optimizer.zero_grad()、loss.backward()和optimizer.step()

2、我一直没有找到上诉三个在msp中的实现方法

【截图信息】

pytorch代码：

def train(epoch):

    model.train()

    scheduler.step()

    start_time = time.time()

    total_loss = 0

    for i, (images, labels) in enumerate(data_loader_train):

        optimizer.zero_grad()

        images = Variable(images)

        one_hot = torch.zeros(100, 10).scatter(1, labels.unsqueeze(1), 1)

        labels = Variable(one_hot)



        outputs = model(images)

        loss = cost(outputs, labels)

        total_loss += float(loss)

        loss.backward()

        optimizer.step()



def eval(epoch, if_test):

    model.eval()

    correct = 0

    total = 0

    if if_test:

        for i, (images, labels) in enumerate(data_loader_test):

            images = Variable(images)

            labels = Variable(labels)



            outputs= model(images)

            pred = outputs.max(1)[1]

            total += labels.size(0)

            correct += (pred == labels).sum()



        acc = 100.0 * correct.item() / total

        print('Test correct: %d Accuracy: %.2f%%' % (correct, acc))

        test_scores.append(acc)

        if acc > max(test_scores):

            save_file = str(epoch) + '.pt'

            torch.save(model, os.path.join(save_path, save_file))

    else:

        for i, (images, labels) in enumerate(data_loader_train):

            images = Variable(images)

            labels = Variable(labels)



            outputs = model(images)

            pred = outputs.max(1)[1]

            total += labels.size(0)

            correct += (pred == labels).sum()



        acc = 100.0 * correct.item() / total

        print('Train correct: %d Accuracy: %.2f%%' % (correct, acc))

        train_scores.append(acc)



test_scores = []

train_scores = []

save_path = './LISNN_'

if not os.path.exists(save_path):

    os.mkdir(save_path)

n_epoch = 10

for epoch in range(n_epoch):

    train(epoch)

    if (epoch + 1) % 2 == 0:

        eval(epoch, if_test = True)

    if (epoch + 1) % 20 == 0:

        eval(epoch, if_test = False)

    if (epoch + 1) % 20 == 0:

        print('Best Test Accuracy in %d: %.2f%%' % (epoch + 1, max(test_scores)))

        print('Best Train Accuracy in %d: %.2f%%' % (epoch + 1, max(train_scores)))



mindspore代码：

# 定义训练函数

def train(epoch, ckpoint_cb, sink_mode):

    model.train(epoch,ds_train,callbacks=[ckpoint_cb, LossMonitor(125)], dataset_sink_mode=sink_mode) # 每过200个step就输出训练损失值



# 定义验证函数

def eval():

    acc = model.eval(ds_test, dataset_sink_mode=False)

    print("{}".format(acc))



test_scores = []

train_scores = []

save_path = './LISNN_'

if not os.path.exists(save_path):

    os.mkdir(save_path)

n_epoch = 20

train(n_epoch,ckpoint,False) # 调用训练函数



eval()

解答：

1. PyTorch为Tensor建立了grad属性和backward方法，tensor.grad是通过tensor.backward方法（本质是PyTorch.autograd.backward）计算的，且在计算中进行梯度值累加，因此一般在调用tensor.backward方法前，需要手动将grad属性清零。MindSpore没有为Tensor和grad建立直接联系，在使用时不需要手动清零。

2. 损失函数一直都是0.1，是否在训练之前给网络加入优化器，优化器的用法如下：

from mindspore import context, Tensor, ParameterTuple

from mindspore import nn, Model, ops

import numpy as np

from mindspore import dtype as mstype



class Net(nn.Cell):

  def __init__(self):

    super(Net, self).__init__()

    self.conv = nn.Conv2d(3, 64, 3)

    self.bn = nn.BatchNorm2d(64)

  def construct(self, x):

    x = self.conv(x)

    x = self.bn(x)

    return x



net = Net()

loss = nn.MSELoss()

optimizer = nn.SGD(params=net.trainable_params(), learning_rate=0.01)

# 使用Model接口

model = Model(net, loss_fn=loss, optimizer=optimizer, metrics={"accuracy"})

https://www.mindspore.cn/docs/migration_guide/zh-CN/r1.6/optim.html?highlight=zero_grad()