PyTorch深度学习实践——4.反向传播
PyTorch深度学习实践——4.反向传播
课程链接：《PyTorch深度学习实践》4.反向传播
思路：
1、先算损失loss
2、算反向传播 backward
loss.backward(): dloss \ dw == w.grad （Tensor）
3、更新权重w
w.data = w.data - 0.01 * w.grad.data
4、更新之后必须对权重w的梯度值（w.grad）清零，否则新的梯度值与之前的梯度值相加
w.grad.data.zero_()
import torch
x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]
# 设置初始权重值，要加小括号
# torch.Tensor()生成新的张量
w = torch.Tensor([1.0]) # w值为1.0
w.requires_grad = True # 需要计算梯度
# 是否需要计算梯度？——True
# 设置为需要计算梯度,因为默认的权重不计算梯度
def forward(x):
return x * w # w为Tensor类型，*为Tensor与Tensor之间的数乘，自动把x转换为Tensor类型
def loss(x, y):
y_pred = forward(x) # 求 y_hat
return (y_pred - y) ** 2 # 求损失 loss = (y_hat - y) ^ 2
# item 将数据直接以张量数值的形式取出来
# .item()的作用主要是把数据从tensor取出来，变成python的数据类型
print('Predict (before training)', 4, forward(4).item()) # 训练前 x = 4 时的 y_hat 值 (y_hat = 4 * w)
# 开始设置迭代次数
for epoch in range(100):
l = 0
for x, y in zip(x_data, y_data):
l = loss(x, y) # 前馈过程，只计算loss，loss(l)是Tensor类型的张量
# 将每一次的数据误差存到l中
l.backward() # 调用张量的成员函数 backward()，自动计算所有梯度，存到w里
# 利用反向传播，将误差值传播回去
# 利用PyTorch的反向传播函数求梯度
print('\tgrad:', x, y, w.grad.item()) # 获得梯度 w.grad，用 .item() 将Tensor里的数值拿出来，作为Python的标量
w.data = w.data - 0.01 * w.grad.data # w.grad 是一个Tensor的张量，需要取data数值进行计算，不会建立计算图
# 这里是数值计算，w一定要取data值，tensor做加法运算会构建运算图，消耗内存
# 以学习率为0.01乘以梯度信息，以此来更新权重信息
# 计算时使用的Tensor张量，自动构建计算图，用于求反向传播（梯度）；而更新权重时，需要使用标量
w.grad.data.zero_() # 权重w中梯度的数据全都清零
# 将第一次产生的梯度进行清零，否则第二次的梯度就会变成两个梯度之和的导数
# 每次反向传播的数据要清零
print("progress:", epoch, l.item())
# 输出迭代次数和损失值l的具体数值
print('Predict (after training)', 4, forward(4).item()) # 训练后 x = 4 时的 y_hat 值 (y_hat = 4 * w)
# 将forward(4)的结果以张量的具体数值输出出来
输出结果：
Predict (before training) 4 4.0
grad: 1.0 2.0 -2.0
grad: 2.0 4.0 -7.840000152587891
grad: 3.0 6.0 -16.228801727294922
progress: 0 7.315943717956543
grad: 1.0 2.0 -1.478623867034912
grad: 2.0 4.0 -5.796205520629883
grad: 3.0 6.0 -11.998146057128906
progress: 1 3.9987640380859375
grad: 1.0 2.0 -1.0931644439697266
grad: 2.0 4.0 -4.285204887390137
grad: 3.0 6.0 -8.870372772216797
progress: 2 2.1856532096862793
grad: 1.0 2.0 -0.8081896305084229
grad: 2.0 4.0 -3.1681032180786133
grad: 3.0 6.0 -6.557973861694336
progress: 3 1.1946394443511963
grad: 1.0 2.0 -0.5975041389465332
grad: 2.0 4.0 -2.3422164916992188
grad: 3.0 6.0 -4.848389625549316
progress: 4 0.6529689431190491
grad: 1.0 2.0 -0.4417421817779541
grad: 2.0 4.0 -1.7316293716430664
grad: 3.0 6.0 -3.58447265625
progress: 5 0.35690122842788696
grad: 1.0 2.0 -0.3265852928161621
grad: 2.0 4.0 -1.2802143096923828
grad: 3.0 6.0 -2.650045394897461
progress: 6 0.195076122879982
grad: 1.0 2.0 -0.24144840240478516
grad: 2.0 4.0 -0.9464778900146484
grad: 3.0 6.0 -1.9592113494873047
progress: 7 0.10662525147199631
grad: 1.0 2.0 -0.17850565910339355
grad: 2.0 4.0 -0.699742317199707
grad: 3.0 6.0 -1.4484672546386719
progress: 8 0.0582793727517128
grad: 1.0 2.0 -0.1319713592529297
grad: 2.0 4.0 -0.5173273086547852
grad: 3.0 6.0 -1.070866584777832
progress: 9 0.03185431286692619
grad: 1.0 2.0 -0.09756779670715332
grad: 2.0 4.0 -0.3824653625488281
grad: 3.0 6.0 -0.7917022705078125
progress: 10 0.017410902306437492
grad: 1.0 2.0 -0.07213282585144043
grad: 2.0 4.0 -0.2827606201171875
grad: 3.0 6.0 -0.5853137969970703
progress: 11 0.009516451507806778
grad: 1.0 2.0 -0.053328514099121094
grad: 2.0 4.0 -0.2090473175048828
grad: 3.0 6.0 -0.43272972106933594
progress: 12 0.005201528314501047
grad: 1.0 2.0 -0.039426326751708984
grad: 2.0 4.0 -0.15455150604248047
grad: 3.0 6.0 -0.3199195861816406
progress: 13 0.0028430151287466288
grad: 1.0 2.0 -0.029148340225219727
grad: 2.0 4.0 -0.11426162719726562
grad: 3.0 6.0 -0.23652076721191406
progress: 14 0.0015539465239271522
grad: 1.0 2.0 -0.021549701690673828
grad: 2.0 4.0 -0.08447456359863281
grad: 3.0 6.0 -0.17486286163330078
progress: 15 0.0008493617060594261
grad: 1.0 2.0 -0.01593184471130371
grad: 2.0 4.0 -0.062453269958496094
grad: 3.0 6.0 -0.12927818298339844
progress: 16 0.00046424579340964556
grad: 1.0 2.0 -0.011778593063354492
grad: 2.0 4.0 -0.046172142028808594
grad: 3.0 6.0 -0.09557533264160156
progress: 17 0.0002537401160225272
grad: 1.0 2.0 -0.00870823860168457
grad: 2.0 4.0 -0.03413581848144531
grad: 3.0 6.0 -0.07066154479980469
progress: 18 0.00013869594840798527
grad: 1.0 2.0 -0.006437778472900391
grad: 2.0 4.0 -0.025236129760742188
grad: 3.0 6.0 -0.052239418029785156
progress: 19 7.580435340059921e-05
grad: 1.0 2.0 -0.004759550094604492
grad: 2.0 4.0 -0.018657684326171875
grad: 3.0 6.0 -0.038620948791503906
progress: 20 4.143271507928148e-05
grad: 1.0 2.0 -0.003518819808959961
grad: 2.0 4.0 -0.0137939453125
grad: 3.0 6.0 -0.028553009033203125
progress: 21 2.264650902361609e-05
grad: 1.0 2.0 -0.00260162353515625
grad: 2.0 4.0 -0.010198593139648438
grad: 3.0 6.0 -0.021108627319335938
progress: 22 1.2377059647405986e-05
grad: 1.0 2.0 -0.0019233226776123047
grad: 2.0 4.0 -0.0075397491455078125
grad: 3.0 6.0 -0.0156097412109375
progress: 23 6.768445018678904e-06
grad: 1.0 2.0 -0.0014221668243408203
grad: 2.0 4.0 -0.0055751800537109375
grad: 3.0 6.0 -0.011541366577148438
progress: 24 3.7000872907810844e-06
grad: 1.0 2.0 -0.0010514259338378906
grad: 2.0 4.0 -0.0041217803955078125
grad: 3.0 6.0 -0.008531570434570312
progress: 25 2.021880391112063e-06
grad: 1.0 2.0 -0.0007772445678710938
grad: 2.0 4.0 -0.0030469894409179688
grad: 3.0 6.0 -0.006305694580078125
progress: 26 1.1044940038118511e-06
grad: 1.0 2.0 -0.0005745887756347656
grad: 2.0 4.0 -0.0022525787353515625
grad: 3.0 6.0 -0.0046634674072265625
progress: 27 6.041091182851233e-07
grad: 1.0 2.0 -0.0004248619079589844
grad: 2.0 4.0 -0.0016651153564453125
grad: 3.0 6.0 -0.003444671630859375
progress: 28 3.296045179013163e-07
grad: 1.0 2.0 -0.0003139972686767578
grad: 2.0 4.0 -0.0012311935424804688
grad: 3.0 6.0 -0.0025491714477539062
progress: 29 1.805076408345485e-07
grad: 1.0 2.0 -0.00023221969604492188
grad: 2.0 4.0 -0.0009107589721679688
grad: 3.0 6.0 -0.0018854141235351562
progress: 30 9.874406714516226e-08
grad: 1.0 2.0 -0.00017189979553222656
grad: 2.0 4.0 -0.0006742477416992188
grad: 3.0 6.0 -0.00139617919921875
progress: 31 5.4147676564753056e-08
grad: 1.0 2.0 -0.0001270771026611328
grad: 2.0 4.0 -0.0004978179931640625
grad: 3.0 6.0 -0.00102996826171875
progress: 32 2.9467628337442875e-08
grad: 1.0 2.0 -9.393692016601562e-05
grad: 2.0 4.0 -0.0003681182861328125
grad: 3.0 6.0 -0.0007610321044921875
progress: 33 1.6088051779661328e-08
grad: 1.0 2.0 -6.937980651855469e-05
grad: 2.0 4.0 -0.00027179718017578125
grad: 3.0 6.0 -0.000560760498046875
progress: 34 8.734787115827203e-09
grad: 1.0 2.0 -5.125999450683594e-05
grad: 2.0 4.0 -0.00020122528076171875
grad: 3.0 6.0 -0.0004177093505859375
progress: 35 4.8466972657479346e-09
grad: 1.0 2.0 -3.790855407714844e-05
grad: 2.0 4.0 -0.000148773193359375
grad: 3.0 6.0 -0.000308990478515625
progress: 36 2.6520865503698587e-09
grad: 1.0 2.0 -2.8133392333984375e-05
grad: 2.0 4.0 -0.000110626220703125
grad: 3.0 6.0 -0.0002288818359375
progress: 37 1.4551915228366852e-09
grad: 1.0 2.0 -2.09808349609375e-05
grad: 2.0 4.0 -8.20159912109375e-05
grad: 3.0 6.0 -0.00016880035400390625
progress: 38 7.914877642178908e-10
grad: 1.0 2.0 -1.5497207641601562e-05
grad: 2.0 4.0 -6.103515625e-05
grad: 3.0 6.0 -0.000125885009765625
progress: 39 4.4019543565809727e-10
grad: 1.0 2.0 -1.1444091796875e-05
grad: 2.0 4.0 -4.482269287109375e-05
grad: 3.0 6.0 -9.1552734375e-05
progress: 40 2.3283064365386963e-10
grad: 1.0 2.0 -8.344650268554688e-06
grad: 2.0 4.0 -3.24249267578125e-05
grad: 3.0 6.0 -6.580352783203125e-05
progress: 41 1.2028067430946976e-10
grad: 1.0 2.0 -5.9604644775390625e-06
grad: 2.0 4.0 -2.288818359375e-05
grad: 3.0 6.0 -4.57763671875e-05
progress: 42 5.820766091346741e-11
grad: 1.0 2.0 -4.291534423828125e-06
grad: 2.0 4.0 -1.71661376953125e-05
grad: 3.0 6.0 -3.719329833984375e-05
progress: 43 3.842615114990622e-11
grad: 1.0 2.0 -3.337860107421875e-06
grad: 2.0 4.0 -1.33514404296875e-05
grad: 3.0 6.0 -2.86102294921875e-05
progress: 44 2.2737367544323206e-11
grad: 1.0 2.0 -2.6226043701171875e-06
grad: 2.0 4.0 -1.049041748046875e-05
grad: 3.0 6.0 -2.288818359375e-05
progress: 45 1.4551915228366852e-11
grad: 1.0 2.0 -1.9073486328125e-06
grad: 2.0 4.0 -7.62939453125e-06
grad: 3.0 6.0 -1.430511474609375e-05
progress: 46 5.6843418860808015e-12
grad: 1.0 2.0 -1.430511474609375e-06
grad: 2.0 4.0 -5.7220458984375e-06
grad: 3.0 6.0 -1.1444091796875e-05
progress: 47 3.637978807091713e-12
grad: 1.0 2.0 -1.1920928955078125e-06
grad: 2.0 4.0 -4.76837158203125e-06
grad: 3.0 6.0 -1.1444091796875e-05
progress: 48 3.637978807091713e-12
grad: 1.0 2.0 -9.5367431640625e-07
grad: 2.0 4.0 -3.814697265625e-06
grad: 3.0 6.0 -8.58306884765625e-06
progress: 49 2.0463630789890885e-12
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 50 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 51 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 52 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 53 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 54 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 55 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 56 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 57 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 58 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 59 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 60 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 61 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 62 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 63 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 64 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 65 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 66 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 67 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 68 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 69 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 70 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 71 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 72 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 73 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 74 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 75 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 76 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 77 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 78 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 79 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 80 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 81 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 82 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 83 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 84 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 85 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 86 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 87 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 88 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 89 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 90 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 91 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 92 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 93 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 94 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 95 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 96 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 97 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 98 9.094947017729282e-13
grad: 1.0 2.0 -7.152557373046875e-07
grad: 2.0 4.0 -2.86102294921875e-06
grad: 3.0 6.0 -5.7220458984375e-06
progress: 99 9.094947017729282e-13
Predict (after training) 4 7.999998569488525
torch.rand是均匀分布的随机变量
torch.randn是标准正太分布的随机变量
torch.randn(4)
tensor([-2.1436, 0.9966, 2.3426, -0.6366])
torch.randn(2, 3)
tensor([[1.5954, 2.8929, -1.0923],
[1.1719, -0.4709, -0.1996]])
import torch
x_data = [-3.0, -2.0, -1.0, 0.0, 1.0, 2.0, 3.0]
y_data = [12.0, 4.0, 0.0, 0.0, 4.0, 12.0]
w1 = torch.randn(1) # 标准正太分布的随机变量，一维向量
w1.requires_grad = True
w2 = torch.randn(1)
w2.requires_grad = True
b = torch.Tensor([0.0]) # 偏移量默认为0
b.requires_grad = True
def forward(x):
return w1 * (x ** 2) + w2 * x + b # 预设值为 w1 = 2, w2 = 2, b = 0
# y = 2 x^2 + 2x
def loss(x, y):
y_pred = forward(x)
return (y_pred - y) ** 2
print('Predict (before training)', 4, forward(4).item())
for epoch in range(100):
l = 0
for x, y in zip(x_data, y_data):
l = loss(x, y)
l.backward()
print('\tgrad:', x, y, w1.grad.item(), w2.grad.item(), b.grad.item())
w1.data = w1.data - 0.01 * w1.grad.data
w1.grad.data.zero_() # 必须要清零，否则会一直累加，梯度值变大
w2.data = w2.data - 0.01 * w2.grad.data
w2.grad.data.zero_()
b.data = b.data - 0.01 * b.grad.data
b.grad.data.zero_()
print("progress:", epoch, l.item())
print('Predict (after training)', 4, forward(4).item())
输出结果：
Predict (before training) 4 -8.422724723815918
grad: -3.0 12.0 -273.0215148925781 91.00717163085938 -30.335723876953125
grad: -2.0 4.0 62.68621826171875 -31.343109130859375 15.671554565429688
grad: -1.0 0.0 5.388011455535889 -5.388011455535889 5.388011455535889
grad: 0.0 0.0 0.0 0.0 0.18552318215370178
grad: 1.0 4.0 -6.306151866912842 -6.306151866912842 -6.306151866912842
grad: 2.0 12.0 -54.080047607421875 -27.040023803710938 -13.520011901855469
progress: 0 45.69768142700195
grad: -3.0 12.0 173.32583618164062 -57.7752799987793 19.258426666259766
grad: -2.0 4.0 -17.348819732666016 8.674409866333008 -4.337204933166504
grad: -1.0 0.0 1.5983614921569824 -1.5983614921569824 1.5983614921569824
grad: 0.0 0.0 0.0 0.0 0.24794435501098633
grad: 1.0 4.0 -6.50785493850708 -6.50785493850708 -6.50785493850708
grad: 2.0 12.0 -71.37567901611328 -35.68783950805664 -17.84391975402832
progress: 1 79.60137176513672
grad: -3.0 12.0 -4.573007583618164 1.5243358612060547 -0.5081119537353516
grad: -2.0 4.0 10.890235900878906 -5.445117950439453 2.7225589752197266
grad: -1.0 0.0 2.449244260787964 -2.449244260787964 2.449244260787964
grad: 0.0 0.0 0.0 0.0 0.6367471814155579
grad: 1.0 4.0 -3.773172378540039 -3.773172378540039 -3.773172378540039
grad: 2.0 12.0 -41.495018005371094 -20.747509002685547 -10.373754501342773
progress: 2 26.903696060180664
grad: -3.0 12.0 39.47118377685547 -13.157060623168945 4.385686874389648
grad: -2.0 4.0 1.5013236999511719 -0.7506618499755859 0.37533092498779297
grad: -1.0 0.0 1.7946186065673828 -1.7946186065673828 1.7946186065673828
grad: 0.0 0.0 0.0 0.0 0.7758380174636841
grad: 1.0 4.0 -2.782381534576416 -2.782381534576416 -2.782381534576416
grad: 2.0 12.0 -34.26860427856445 -17.134302139282227 -8.567151069641113
progress: 3 18.34902000427246
grad: -3.0 12.0 11.700010299682617 -3.900003433227539 1.3000011444091797
grad: -2.0 4.0 4.909320831298828 -2.454660415649414 1.227330207824707
grad: -1.0 0.0 1.7312746047973633 -1.7312746047973633 1.7312746047973633
grad: 0.0 0.0 0.0 0.0 0.9021397829055786
grad: 1.0 4.0 -1.7243456840515137 -1.7243456840515137 -1.7243456840515137
grad: 2.0 12.0 -23.89825439453125 -11.949127197265625 -5.9745635986328125
progress: 4 8.923852920532227
grad: -3.0 12.0 12.203630447387695 -4.067876815795898 1.3559589385986328
grad: -2.0 4.0 3.7686080932617188 -1.8843040466308594 0.9421520233154297
grad: -1.0 0.0 1.5178654193878174 -1.5178654193878174 1.5178654193878174
grad: 0.0 0.0 0.0 0.0 0.9617555737495422
grad: 1.0 4.0 -1.08040189743042 -1.08040189743042 -1.08040189743042
grad: 2.0 12.0 -18.039810180664062 -9.019905090332031 -4.509952545166016
progress: 5 5.084918022155762
grad: -3.0 12.0 5.502656936645508 -1.834218978881836 0.6114063262939453
grad: -2.0 4.0 4.105258941650391 -2.0526294708251953 1.0263147354125977
grad: -1.0 0.0 1.3971140384674072 -1.3971140384674072 1.3971140384674072
grad: 0.0 0.0 0.0 0.0 0.9936307668685913
grad: 1.0 4.0 -0.5791754722595215 -0.5791754722595215 -0.5791754722595215
grad: 2.0 12.0 -13.137199401855469 -6.568599700927734 -3.284299850463867
progress: 6 2.6966564655303955
grad: -3.0 12.0 3.1522178649902344 -1.0507392883300781 0.3502464294433594
grad: -2.0 4.0 3.869007110595703 -1.9345035552978516 0.9672517776489258
grad: -1.0 0.0 1.2755769491195679 -1.2755769491195679 1.2755769491195679
grad: 0.0 0.0 0.0 0.0 0.9991660714149475
grad: 1.0 4.0 -0.23718738555908203 -0.23718738555908203 -0.23718738555908203
grad: 2.0 12.0 -9.74737548828125 -4.873687744140625 -2.4368438720703125
progress: 7 1.484552025794983
grad: -3.0 12.0 0.6604156494140625 -0.2201385498046875 0.0733795166015625
grad: -2.0 4.0 3.7885818481445312 -1.8942909240722656 0.9471454620361328
grad: -1.0 0.0 1.1783339977264404 -1.1783339977264404 1.1783339977264404
grad: 0.0 0.0 0.0 0.0 0.9886861443519592
grad: 1.0 4.0 0.007760047912597656 0.007760047912597656 0.007760047912597656
grad: 2.0 12.0 -7.186729431152344 -3.593364715576172 -1.796682357788086
progress: 8 0.8070168495178223
grad: -3.0 12.0 -0.7919940948486328 0.26399803161621094 -0.08799934387207031
grad: -2.0 4.0 3.627819061279297 -1.8139095306396484 0.9069547653198242
grad: -1.0 0.0 1.09141206741333 -1.09141206741333 1.09141206741333
grad: 0.0 0.0 0.0 0.0 0.9664835333824158
grad: 1.0 4.0 0.17557430267333984 0.17557430267333984 0.17557430267333984
grad: 2.0 12.0 -5.3308868408203125 -2.6654434204101562 -1.3327217102050781
progress: 9 0.4440367817878723
grad: -3.0 12.0 -1.8829364776611328 0.6276454925537109 -0.2092151641845703
grad: -2.0 4.0 3.479217529296875 -1.7396087646484375 0.8698043823242188
grad: -1.0 0.0 1.0156728029251099 -1.0156728029251099 1.0156728029251099
grad: 0.0 0.0 0.0 0.0 0.9367715716362
grad: 1.0 4.0 0.2896747589111328 0.2896747589111328 0.2896747589111328
grad: 2.0 12.0 -3.9585723876953125 -1.9792861938476562 -0.9896430969238281
progress: 10 0.244848370552063
grad: -3.0 12.0 -2.576345443725586 0.8587818145751953 -0.28626060485839844
grad: -2.0 4.0 3.3186683654785156 -1.6593341827392578 0.8296670913696289
grad: -1.0 0.0 0.9478564262390137 -0.9478564262390137 0.9478564262390137
grad: 0.0 0.0 0.0 0.0 0.9022102355957031
grad: 1.0 4.0 0.3641386032104492 0.3641386032104492 0.3641386032104492
grad: 2.0 12.0 -2.952484130859375 -1.4762420654296875 -0.7381210327148438
progress: 11 0.13620565831661224
grad: -3.0 12.0 -3.0294971466064453 1.0098323822021484 -0.3366107940673828
grad: -2.0 4.0 3.160045623779297 -1.5800228118896484 0.7900114059448242
grad: -1.0 0.0 0.8868624567985535 -0.8868624567985535 0.8868624567985535
grad: 0.0 0.0 0.0 0.0 0.8648403286933899
grad: 1.0 4.0 0.41031837463378906 0.41031837463378906 0.41031837463378906
grad: 2.0 12.0 -2.2110671997070312 -1.1055335998535156 -0.5527667999267578
progress: 12 0.07638778537511826
grad: -3.0 12.0 -3.2939586639404297 1.0979862213134766 -0.3659954071044922
grad: -2.0 4.0 3.0024261474609375 -1.5012130737304688 0.7506065368652344
grad: -1.0 0.0 0.8313875794410706 -0.8313875794410706 0.8313875794410706
grad: 0.0 0.0 0.0 0.0 0.8260725736618042
grad: 1.0 4.0 0.4361400604248047 0.4361400604248047 0.4361400604248047
grad: 2.0 12.0 -1.6652374267578125 -0.8326187133789062 -0.4163093566894531
progress: 13 0.04332837089896202
grad: -3.0 12.0 -3.4293479919433594 1.1431159973144531 -0.3810386657714844
grad: -2.0 4.0 2.848804473876953 -1.4244022369384766 0.7122011184692383
grad: -1.0 0.0 0.7806005477905273 -0.7806005477905273 0.7806005477905273
grad: 0.0 0.0 0.0 0.0 0.7869192361831665
grad: 1.0 4.0 0.44751834869384766 0.44751834869384766 0.44751834869384766
grad: 2.0 12.0 -1.2624664306640625 -0.6312332153320312 -0.3156166076660156
progress: 14 0.02490346133708954
grad: -3.0 12.0 -3.4713878631591797 1.1571292877197266 -0.3857097625732422
grad: -2.0 4.0 2.69989013671875 -1.349945068359375 0.6749725341796875
grad: -1.0 0.0 0.7338007092475891 -0.7338007092475891 0.7338007092475891
grad: 0.0 0.0 0.0 0.0 0.7480815649032593
grad: 1.0 4.0 0.4487161636352539 0.4487161636352539 0.4487161636352539
grad: 2.0 12.0 -0.964874267578125 -0.4824371337890625 -0.24121856689453125
progress: 15 0.014546599239110947
grad: -3.0 12.0 -3.4488658905029297 1.1496219635009766 -0.3832073211669922
grad: -2.0 4.0 2.556568145751953 -1.2782840728759766 0.6391420364379883
grad: -1.0 0.0 0.6904639005661011 -0.6904639005661011 0.6904639005661011
grad: 0.0 0.0 0.0 0.0 0.7100420594215393
grad: 1.0 4.0 0.4428548812866211 0.4428548812866211 0.4428548812866211
grad: 2.0 12.0 -0.7444839477539062 -0.3722419738769531 -0.18612098693847656
progress: 16 0.008660254999995232
grad: -3.0 12.0 -3.381814956665039 1.1272716522216797 -0.37575721740722656
grad: -2.0 4.0 2.4192352294921875 -1.2096176147460938 0.6048088073730469
grad: -1.0 0.0 0.6501691341400146 -0.6501691341400146 0.6501691341400146
grad: 0.0 0.0 0.0 0.0 0.6731221079826355
grad: 1.0 4.0 0.4321928024291992 0.4321928024291992 0.4321928024291992
grad: 2.0 12.0 -0.580841064453125 -0.2904205322265625 -0.14521026611328125
progress: 17 0.005271505564451218
grad: -3.0 12.0 -3.2849807739257812 1.0949935913085938 -0.36499786376953125
grad: -2.0 4.0 2.288097381591797 -1.1440486907958984 0.5720243453979492
grad: -1.0 0.0 0.6125811338424683 -0.6125811338424683 0.6125811338424683
grad: 0.0 0.0 0.0 0.0 0.6375278234481812
grad: 1.0 4.0 0.418365478515625 0.418365478515625 0.418365478515625
grad: 2.0 12.0 -0.45891571044921875 -0.22945785522460938 -0.11472892761230469
progress: 18 0.0032906816340982914
grad: -3.0 12.0 -3.1689376831054688 1.0563125610351562 -0.35210418701171875
grad: -2.0 4.0 2.163196563720703 -1.0815982818603516 0.5407991409301758
grad: -1.0 0.0 0.5774276852607727 -0.5774276852607727 0.5774276852607727
grad: 0.0 0.0 0.0 0.0 0.6033821105957031
grad: 1.0 4.0 0.4025564193725586 0.4025564193725586 0.4025564193725586
grad: 2.0 12.0 -0.36766815185546875 -0.18383407592773438 -0.09191703796386719
progress: 19 0.0021121855825185776
grad: -3.0 12.0 -3.0413589477539062 1.0137863159179688 -0.33792877197265625
grad: -2.0 4.0 2.044483184814453 -1.0222415924072266 0.5111207962036133
grad: -1.0 0.0 0.5444828867912292 -0.5444828867912292 0.5444828867912292
grad: 0.0 0.0 0.0 0.0 0.5707481503486633
grad: 1.0 4.0 0.38560962677001953 0.38560962677001953 0.38560962677001953
grad: 2.0 12.0 -0.2990264892578125 -0.14951324462890625 -0.07475662231445312
progress: 20 0.0013971381122246385
grad: -3.0 12.0 -2.9077377319335938 0.9692459106445312 -0.32308197021484375
grad: -2.0 4.0 1.9318275451660156 -0.9659137725830078 0.4829568862915039
grad: -1.0 0.0 0.5135593414306641 -0.5135593414306641 0.5135593414306641
grad: 0.0 0.0 0.0 0.0 0.5396474599838257
grad: 1.0 4.0 0.3681297302246094 0.3681297302246094 0.3681297302246094
grad: 2.0 12.0 -0.247039794921875 -0.1235198974609375 -0.06175994873046875
progress: 21 0.0009535728022456169
grad: -3.0 12.0 -2.771799087524414 0.9239330291748047 -0.30797767639160156
grad: -2.0 4.0 1.8250198364257812 -0.9125099182128906 0.4562549591064453
grad: -1.0 0.0 0.484494149684906 -0.484494149684906 0.484494149684906
grad: 0.0 0.0 0.0 0.0 0.5100716352462769
grad: 1.0 4.0 0.3505382537841797 0.3505382537841797 0.3505382537841797
grad: 2.0 12.0 -0.2073974609375 -0.10369873046875 -0.051849365234375
progress: 22 0.0006720891688019037
grad: -3.0 12.0 -2.6364097595214844 0.8788032531738281 -0.2929344177246094
grad: -2.0 4.0 1.723876953125 -0.8619384765625 0.43096923828125
grad: -1.0 0.0 0.4571506083011627 -0.4571506083011627 0.4571506083011627
grad: 0.0 0.0 0.0 0.0 0.4819927215576172
grad: 1.0 4.0 0.33313465118408203 0.33313465118408203 0.33313465118408203
grad: 2.0 12.0 -0.17684173583984375 -0.08842086791992188 -0.04421043395996094
progress: 23 0.0004886406240984797
grad: -3.0 12.0 -2.503389358520508 0.8344631195068359 -0.2781543731689453
grad: -2.0 4.0 1.6281661987304688 -0.8140830993652344 0.4070415496826172
grad: -1.0 0.0 0.4314062297344208 -0.4314062297344208 0.4314062297344208
grad: 0.0 0.0 0.0 0.0 0.4553684890270233
grad: 1.0 4.0 0.3161201477050781 0.3161201477050781 0.3161201477050781
grad: 2.0 12.0 -0.15305328369140625 -0.07652664184570312 -0.03826332092285156
progress: 24 0.0003660204238258302
grad: -3.0 12.0 -2.373973846435547 0.7913246154785156 -0.2637748718261719
grad: -2.0 4.0 1.537628173828125 -0.7688140869140625 0.38440704345703125
grad: -1.0 0.0 0.4071519374847412 -0.4071519374847412 0.4071519374847412
grad: 0.0 0.0 0.0 0.0 0.43014827370643616
grad: 1.0 4.0 0.29962825775146484 0.29962825775146484 0.29962825775146484
grad: 2.0 12.0 -0.1343231201171875 -0.06716156005859375 -0.033580780029296875
progress: 25 0.00028191719320602715
grad: -3.0 12.0 -2.2489871978759766 0.7496623992919922 -0.24988746643066406
grad: -2.0 4.0 1.4520339965820312 -0.7260169982910156 0.3630084991455078
grad: -1.0 0.0 0.38429194688796997 -0.38429194688796997 0.38429194688796997
grad: 0.0 0.0 0.0 0.0 0.4062761068344116
grad: 1.0 4.0 0.28374385833740234 0.28374385833740234 0.28374385833740234
grad: 2.0 12.0 -0.1193389892578125 -0.05966949462890625 -0.029834747314453125
progress: 26 0.00022252803319133818
grad: -3.0 12.0 -2.1288928985595703 0.7096309661865234 -0.2365436553955078
grad: -2.0 4.0 1.3711166381835938 -0.6855583190917969 0.34277915954589844
grad: -1.0 0.0 0.36273613572120667 -0.36273613572120667 0.36273613572120667
grad: 0.0 0.0 0.0 0.0 0.38369300961494446
grad: 1.0 4.0 0.26851749420166016 0.26851749420166016 0.26851749420166016
grad: 2.0 12.0 -0.10720062255859375 -0.053600311279296875 -0.026800155639648438
progress: 27 0.0001795620919438079
grad: -3.0 12.0 -2.0141029357910156 0.6713676452636719 -0.22378921508789062
grad: -2.0 4.0 1.294677734375 -0.6473388671875 0.32366943359375
grad: -1.0 0.0 0.3424070477485657 -0.3424070477485657 0.3424070477485657
grad: 0.0 0.0 0.0 0.0 0.362339049577713
grad: 1.0 4.0 0.25397396087646484 0.25397396087646484 0.25397396087646484
grad: 2.0 12.0 -0.0971832275390625 -0.04859161376953125 -0.024295806884765625
progress: 28 0.00014757155440747738
grad: -3.0 12.0 -1.9045486450195312 0.6348495483398438 -0.21161651611328125
grad: -2.0 4.0 1.2224502563476562 -0.6112251281738281 0.30561256408691406
grad: -1.0 0.0 0.3232286274433136 -0.3232286274433136 0.3232286274433136
grad: 0.0 0.0 0.0 0.0 0.34215420484542847
grad: 1.0 4.0 0.24011802673339844 0.24011802673339844 0.24011802673339844
grad: 2.0 12.0 -0.08881378173828125 -0.044406890869140625 -0.022203445434570312
progress: 29 0.0001232482463819906
grad: -3.0 12.0 -1.8002986907958984 0.6000995635986328 -0.20003318786621094
grad: -2.0 4.0 1.1542205810546875 -0.5771102905273438 0.2885551452636719
grad: -1.0 0.0 0.3051326274871826 -0.3051326274871826 0.3051326274871826
grad: 0.0 0.0 0.0 0.0 0.3230797350406647
grad: 1.0 4.0 0.22694683074951172 0.22694683074951172 0.22694683074951172
grad: 2.0 12.0 -0.08170318603515625 -0.040851593017578125 -0.020425796508789062
progress: 30 0.00010430328984512016
grad: -3.0 12.0 -1.701284408569336 0.5670948028564453 -0.18903160095214844
grad: -2.0 4.0 1.0897712707519531 -0.5448856353759766 0.2724428176879883
grad: -1.0 0.0 0.28805604577064514 -0.28805604577064514 0.28805604577064514
grad: 0.0 0.0 0.0 0.0 0.30505841970443726
grad: 1.0 4.0 0.21444416046142578 0.21444416046142578 0.21444416046142578
grad: 2.0 12.0 -0.07555389404296875 -0.037776947021484375 -0.018888473510742188
progress: 31 8.919360698200762e-05
grad: -3.0 12.0 -1.6074542999267578 0.5358180999755859 -0.1786060333251953
grad: -2.0 4.0 1.0289306640625 -0.51446533203125 0.257232666015625
grad: -1.0 0.0 0.27194106578826904 -0.27194106578826904 0.27194106578826904
grad: 0.0 0.0 0.0 0.0 0.2880347669124603
grad: 1.0 4.0 0.20259380340576172 0.20259380340576172 0.20259380340576172
grad: 2.0 12.0 -0.070159912109375 -0.0350799560546875 -0.01753997802734375
progress: 32 7.691270729992539e-05
grad: -3.0 12.0 -1.5184822082519531 0.5061607360839844 -0.16872024536132812
grad: -2.0 4.0 0.9714698791503906 -0.4857349395751953 0.24286746978759766
grad: -1.0 0.0 0.2567302882671356 -0.2567302882671356 0.2567302882671356
grad: 0.0 0.0 0.0 0.0 0.2719554305076599
grad: 1.0 4.0 0.19136905670166016 0.19136905670166016 0.19136905670166016
grad: 2.0 12.0 -0.06539154052734375 -0.032695770263671875 -0.016347885131835938
progress: 33 6.681333616143093e-05
grad: -3.0 12.0 -1.4342308044433594 0.4780769348144531 -0.15935897827148438
grad: -2.0 4.0 0.9171943664550781 -0.45859718322753906 0.22929859161376953
grad: -1.0 0.0 0.24237218499183655 -0.24237218499183655 0.24237218499183655
grad: 0.0 0.0 0.0 0.0 0.25676965713500977
grad: 1.0 4.0 0.18074512481689453 0.18074512481689453 0.18074512481689453
grad: 2.0 12.0 -0.0611114501953125 -0.03055572509765625 -0.015277862548828125
progress: 34 5.835327101522125e-05
grad: -3.0 12.0 -1.354562759399414 0.4515209197998047 -0.15050697326660156
grad: -2.0 4.0 0.8659515380859375 -0.43297576904296875 0.21648788452148438
grad: -1.0 0.0 0.22881947457790375 -0.22881947457790375 0.22881947457790375
grad: 0.0 0.0 0.0 0.0 0.2424289435148239
grad: 1.0 4.0 0.17069530487060547 0.17069530487060547 0.17069530487060547
grad: 2.0 12.0 -0.0572357177734375 -0.02861785888671875 -0.014308929443359375
progress: 35 5.118636545375921e-05
grad: -3.0 12.0 -1.2791690826416016 0.4263896942138672 -0.14212989807128906
grad: -2.0 4.0 0.8175544738769531 -0.40877723693847656 0.20438861846923828
grad: -1.0 0.0 0.21602535247802734 -0.21602535247802734 0.21602535247802734
grad: 0.0 0.0 0.0 0.0 0.22888696193695068
grad: 1.0 4.0 0.16119384765625 0.16119384765625 0.16119384765625
grad: 2.0 12.0 -0.0537109375 -0.02685546875 -0.013427734375
progress: 36 4.507601261138916e-05
grad: -3.0 12.0 -1.2079639434814453 0.40265464782714844 -0.1342182159423828
grad: -2.0 4.0 0.7718849182128906 -0.3859424591064453 0.19297122955322266
grad: -1.0 0.0 0.2039484828710556 -0.2039484828710556 0.2039484828710556
grad: 0.0 0.0 0.0 0.0 0.21609985828399658
grad: 1.0 4.0 0.15221309661865234 0.15221309661865234 0.15221309661865234
grad: 2.0 12.0 -0.05045318603515625 -0.025226593017578125 -0.012613296508789062
progress: 37 3.9773811295162886e-05
grad: -3.0 12.0 -1.140604019165039 0.3802013397216797 -0.12673377990722656
grad: -2.0 4.0 0.7287483215332031 -0.36437416076660156 0.18218708038330078
grad: -1.0 0.0 0.19254696369171143 -0.19254696369171143 0.19254696369171143
grad: 0.0 0.0 0.0 0.0 0.20402586460113525
grad: 1.0 4.0 0.14372634887695312 0.14372634887695312 0.14372634887695312
grad: 2.0 12.0 -0.04746246337890625 -0.023731231689453125 -0.011865615844726562
progress: 38 3.519820893416181e-05
grad: -3.0 12.0 -1.0770378112792969 0.3590126037597656 -0.11967086791992188
grad: -2.0 4.0 0.6880340576171875 -0.34401702880859375 0.17200851440429688
grad: -1.0 0.0 0.18178413808345795 -0.18178413808345795 0.18178413808345795
grad: 0.0 0.0 0.0 0.0 0.19262570142745972
grad: 1.0 4.0 0.13570880889892578 0.13570880889892578 0.13570880889892578
grad: 2.0 12.0 -0.04467010498046875 -0.022335052490234375 -0.011167526245117188
progress: 39 3.1178409699350595e-05
grad: -3.0 12.0 -1.0168876647949219 0.3389625549316406 -0.11298751831054688
grad: -2.0 4.0 0.6495742797851562 -0.3247871398925781 0.16239356994628906
grad: -1.0 0.0 0.1716221272945404 -0.1716221272945404 0.1716221272945404
grad: 0.0 0.0 0.0 0.0 0.18186180293560028
grad: 1.0 4.0 0.12813472747802734 0.12813472747802734 0.12813472747802734
grad: 2.0 12.0 -0.042083740234375 -0.0210418701171875 -0.01052093505859375
progress: 40 2.767251862678677e-05
grad: -3.0 12.0 -0.9601364135742188 0.32004547119140625 -0.10668182373046875
grad: -2.0 4.0 0.6132659912109375 -0.30663299560546875 0.15331649780273438
grad: -1.0 0.0 0.1620289832353592 -0.1620289832353592 0.1620289832353592
grad: 0.0 0.0 0.0 0.0 0.17169900238513947
grad: 1.0 4.0 0.12098217010498047 0.12098217010498047 0.12098217010498047
grad: 2.0 12.0 -0.0396575927734375 -0.01982879638671875 -0.009914398193359375
progress: 41 2.457382288412191e-05
grad: -3.0 12.0 -0.9065437316894531 0.3021812438964844 -0.10072708129882812
grad: -2.0 4.0 0.5790023803710938 -0.2895011901855469 0.14475059509277344
grad: -1.0 0.0 0.15297260880470276 -0.15297260880470276 0.15297260880470276
grad: 0.0 0.0 0.0 0.0 0.16210374236106873
grad: 1.0 4.0 0.1142263412475586 0.1142263412475586 0.1142263412475586
grad: 2.0 12.0 -0.037384033203125 -0.0186920166015625 -0.00934600830078125
progress: 42 2.1836967789568007e-05
grad: -3.0 12.0 -0.8558692932128906 0.2852897644042969 -0.09509658813476562
grad: -2.0 4.0 0.546630859375 -0.2733154296875 0.13665771484375
grad: -1.0 0.0 0.1444217413663864 -0.1444217413663864 0.1444217413663864
grad: 0.0 0.0 0.0 0.0 0.15304438769817352
grad: 1.0 4.0 0.10784530639648438 0.10784530639648438 0.10784530639648438
grad: 2.0 12.0 -0.03527069091796875 -0.017635345458984375 -0.008817672729492188
progress: 43 1.943783718161285e-05
grad: -3.0 12.0 -0.8081302642822266 0.2693767547607422 -0.08979225158691406
grad: -2.0 4.0 0.5160980224609375 -0.25804901123046875 0.12902450561523438
grad: -1.0 0.0 0.13635045289993286 -0.13635045289993286 0.13635045289993286
grad: 0.0 0.0 0.0 0.0 0.1444912850856781
grad: 1.0 4.0 0.1018218994140625 0.1018218994140625 0.1018218994140625
grad: 2.0 12.0 -0.03326416015625 -0.016632080078125 -0.0083160400390625
progress: 44 1.7289130482822657e-05
grad: -3.0 12.0 -0.7629146575927734 0.2543048858642578 -0.08476829528808594
grad: -2.0 4.0 0.4872398376464844 -0.2436199188232422 0.1218099594116211
grad: -1.0 0.0 0.1287284940481186 -0.1287284940481186 0.1287284940481186
grad: 0.0 0.0 0.0 0.0 0.13641594350337982
grad: 1.0 4.0 0.09613227844238281 0.09613227844238281 0.09613227844238281
grad: 2.0 12.0 -0.0313873291015625 -0.01569366455078125 -0.007846832275390625
progress: 45 1.5393194189528003e-05
grad: -3.0 12.0 -0.7203598022460938 0.24011993408203125 -0.08003997802734375
grad: -2.0 4.0 0.46002197265625 -0.230010986328125 0.1150054931640625
grad: -1.0 0.0 0.12153422832489014 -0.12153422832489014 0.12153422832489014
grad: 0.0 0.0 0.0 0.0 0.1287919282913208
grad: 1.0 4.0 0.09076213836669922 0.09076213836669922 0.09076213836669922
grad: 2.0 12.0 -0.0296173095703125 -0.01480865478515625 -0.007404327392578125
progress: 46 1.3706016034120694e-05
grad: -3.0 12.0 -0.6800537109375 0.2266845703125 -0.0755615234375
grad: -2.0 4.0 0.4342994689941406 -0.2171497344970703 0.10857486724853516
grad: -1.0 0.0 0.1147407814860344 -0.1147407814860344 0.1147407814860344
grad: 0.0 0.0 0.0 0.0 0.12159384787082672
grad: 1.0 4.0 0.08568954467773438 0.08568954467773438 0.08568954467773438
grad: 2.0 12.0 -0.0279541015625 -0.01397705078125 -0.006988525390625
progress: 47 1.2209871783852577e-05
grad: -3.0 12.0 -0.6420822143554688 0.21402740478515625 -0.07134246826171875
grad: -2.0 4.0 0.4100341796875 -0.20501708984375 0.102508544921875
grad: -1.0 0.0 0.10832815617322922 -0.10832815617322922 0.10832815617322922
grad: 0.0 0.0 0.0 0.0 0.11479806900024414
grad: 1.0 4.0 0.080902099609375 0.080902099609375 0.080902099609375
grad: 2.0 12.0 -0.0263824462890625 -0.01319122314453125 -0.006595611572265625
progress: 48 1.0875523003051057e-05
grad: -3.0 12.0 -0.6062049865722656 0.20206832885742188 -0.06735610961914062
grad: -2.0 4.0 0.3871192932128906 -0.1935596466064453 0.09677982330322266
grad: -1.0 0.0 0.10227359086275101 -0.10227359086275101 0.10227359086275101
grad: 0.0 0.0 0.0 0.0 0.10838203132152557
grad: 1.0 4.0 0.07638072967529297 0.07638072967529297 0.07638072967529297
grad: 2.0 12.0 -0.02490234375 -0.012451171875 -0.0062255859375
progress: 49 9.689480066299438e-06
grad: -3.0 12.0 -0.5723018646240234 0.1907672882080078 -0.06358909606933594
grad: -2.0 4.0 0.365478515625 -0.1827392578125 0.09136962890625
grad: -1.0 0.0 0.0965571478009224 -0.0965571478009224 0.0965571478009224
grad: 0.0 0.0 0.0 0.0 0.10232453793287277
grad: 1.0 4.0 0.0721120834350586 0.0721120834350586 0.0721120834350586
grad: 2.0 12.0 -0.0235137939453125 -0.01175689697265625 -0.005878448486328125
progress: 50 8.639039151603356e-06
grad: -3.0 12.0 -0.5403213500976562 0.18010711669921875 -0.06003570556640625
grad: -2.0 4.0 0.3450431823730469 -0.17252159118652344 0.08626079559326172
grad: -1.0 0.0 0.09116020798683167 -0.09116020798683167 0.09116020798683167
grad: 0.0 0.0 0.0 0.0 0.09660567343235016
grad: 1.0 4.0 0.06808185577392578 0.06808185577392578 0.06808185577392578
grad: 2.0 12.0 -0.02219390869140625 -0.011096954345703125 -0.0055484771728515625
progress: 51 7.696399734413717e-06
grad: -3.0 12.0 -0.5101432800292969 0.17004776000976562 -0.056682586669921875
grad: -2.0 4.0 0.32576751708984375 -0.16288375854492188 0.08144187927246094
grad: -1.0 0.0 0.08606553822755814 -0.08606553822755814 0.08606553822755814
grad: 0.0 0.0 0.0 0.0 0.091206394135952
grad: 1.0 4.0 0.06427764892578125 0.06427764892578125 0.06427764892578125
grad: 2.0 12.0 -0.02094268798828125 -0.010471343994140625 -0.0052356719970703125
progress: 52 6.853065315226559e-06
grad: -3.0 12.0 -0.4815959930419922 0.16053199768066406 -0.05351066589355469
grad: -2.0 4.0 0.30754852294921875 -0.15377426147460938 0.07688713073730469
grad: -1.0 0.0 0.08125440031290054 -0.08125440031290054 0.08125440031290054
grad: 0.0 0.0 0.0 0.0 0.08610881119966507
grad: 1.0 4.0 0.060685157775878906 0.060685157775878906 0.060685157775878906
grad: 2.0 12.0 -0.01976776123046875 -0.009883880615234375 -0.0049419403076171875
progress: 53 6.1056935010128655e-06
grad: -3.0 12.0 -0.45476531982421875 0.15158843994140625 -0.05052947998046875
grad: -2.0 4.0 0.2903785705566406 -0.1451892852783203 0.07259464263916016
grad: -1.0 0.0 0.07671407610177994 -0.07671407610177994 0.07671407610177994
grad: 0.0 0.0 0.0 0.0 0.08129618316888809
grad: 1.0 4.0 0.05729389190673828 0.05729389190673828 0.05729389190673828
grad: 2.0 12.0 -0.0186614990234375 -0.00933074951171875 -0.004665374755859375
progress: 54 5.441430403152481e-06
grad: -3.0 12.0 -0.4292736053466797 0.14309120178222656 -0.04769706726074219
grad: -2.0 4.0 0.27413177490234375 -0.13706588745117188 0.06853294372558594
grad: -1.0 0.0 0.07242576032876968 -0.07242576032876968 0.07242576032876968
grad: 0.0 0.0 0.0 0.0 0.07675246149301529
grad: 1.0 4.0 0.054091453552246094 0.054091453552246094 0.054091453552246094
grad: 2.0 12.0 -0.01763153076171875 -0.008815765380859375 -0.0044078826904296875
progress: 55 4.857357453147415e-06
grad: -3.0 12.0 -0.40530967712402344 0.1351032257080078 -0.04503440856933594
grad: -2.0 4.0 0.2588157653808594 -0.1294078826904297 0.06470394134521484
grad: -1.0 0.0 0.06837771087884903 -0.06837771087884903 0.06837771087884903
grad: 0.0 0.0 0.0 0.0 0.0724627897143364
grad: 1.0 4.0 0.05106925964355469 0.05106925964355469 0.05106925964355469
grad: 2.0 12.0 -0.016632080078125 -0.0083160400390625 -0.00415802001953125
progress: 56 4.322282620705664e-06
grad: -3.0 12.0 -0.38265037536621094 0.1275501251220703 -0.04251670837402344
grad: -2.0 4.0 0.24434661865234375 -0.12217330932617188 0.06108665466308594
grad: -1.0 0.0 0.06455598026514053 -0.06455598026514053 0.06455598026514053
grad: 0.0 0.0 0.0 0.0 0.06841279566287994
grad: 1.0 4.0 0.04821491241455078 0.04821491241455078 0.04821491241455078
grad: 2.0 12.0 -0.0157012939453125 -0.00785064697265625 -0.003925323486328125
progress: 57 3.8520411180797964e-06
grad: -3.0 12.0 -0.3612957000732422 0.12043190002441406 -0.04014396667480469
grad: -2.0 4.0 0.23070526123046875 -0.11535263061523438 0.05767631530761719
grad: -1.0 0.0 0.060948655009269714 -0.060948655009269714 0.060948655009269714
grad: 0.0 0.0 0.0 0.0 0.06458912789821625
grad: 1.0 4.0 0.04551982879638672 0.04551982879638672 0.04551982879638672
grad: 2.0 12.0 -0.0148162841796875 -0.00740814208984375 -0.003704071044921875
progress: 58 3.4300355764571577e-06
grad: -3.0 12.0 -0.34105682373046875 0.11368560791015625 -0.03789520263671875
grad: -2.0 4.0 0.21780014038085938 -0.10890007019042969 0.054450035095214844
grad: -1.0 0.0 0.05754180997610092 -0.05754180997610092 0.05754180997610092
grad: 0.0 0.0 0.0 0.0 0.060979101806879044
grad: 1.0 4.0 0.042975425720214844 0.042975425720214844 0.042975425720214844
grad: 2.0 12.0 -0.0139923095703125 -0.00699615478515625 -0.003498077392578125
progress: 59 3.0591363611165434e-06
grad: -3.0 12.0 -0.3220024108886719 0.10733413696289062 -0.035778045654296875
grad: -2.0 4.0 0.20562744140625 -0.102813720703125 0.0514068603515625
grad: -1.0 0.0 0.05432556942105293 -0.05432556942105293 0.05432556942105293
grad: 0.0 0.0 0.0 0.0 0.05757088586688042
grad: 1.0 4.0 0.0405731201171875 0.0405731201171875 0.0405731201171875
grad: 2.0 12.0 -0.013214111328125 -0.0066070556640625 -0.00330352783203125
progress: 60 2.7283240342512727e-06
grad: -3.0 12.0 -0.304046630859375 0.101348876953125 -0.033782958984375
grad: -2.0 4.0 0.19414520263671875 -0.09707260131835938 0.04853630065917969
grad: -1.0 0.0 0.05128989741206169 -0.05128989741206169 0.05128989741206169
grad: 0.0 0.0 0.0 0.0 0.05435321480035782
grad: 1.0 4.0 0.038306236267089844 0.038306236267089844 0.038306236267089844
grad: 2.0 12.0 -0.01247406005859375 -0.006237030029296875 -0.0031185150146484375
progress: 61 2.431283974146936e-06
grad: -3.0 12.0 -0.2870006561279297 0.09566688537597656 -0.03188896179199219
grad: -2.0 4.0 0.18327713012695312 -0.09163856506347656 0.04581928253173828
grad: -1.0 0.0 0.04842260107398033 -0.04842260107398033 0.04842260107398033
grad: 0.0 0.0 0.0 0.0 0.05131533741950989
grad: 1.0 4.0 0.036164283752441406 0.036164283752441406 0.036164283752441406
grad: 2.0 12.0 -0.01178741455078125 -0.005893707275390625 -0.0029468536376953125
progress: 62 2.170986590499524e-06
grad: -3.0 12.0 -0.2709674835205078 0.09032249450683594 -0.030107498168945312
grad: -2.0 4.0 0.17303466796875 -0.086517333984375 0.0432586669921875
grad: -1.0 0.0 0.045716237276792526 -0.045716237276792526 0.045716237276792526
grad: 0.0 0.0 0.0 0.0 0.04844733327627182
grad: 1.0 4.0 0.03414344787597656 0.03414344787597656 0.03414344787597656
grad: 2.0 12.0 -0.0111236572265625 -0.00556182861328125 -0.002780914306640625
progress: 63 1.933371095219627e-06
grad: -3.0 12.0 -0.2558784484863281 0.08529281616210938 -0.028430938720703125
grad: -2.0 4.0 0.16338348388671875 -0.08169174194335938 0.04084587097167969
grad: -1.0 0.0 0.04316173866391182 -0.04316173866391182 0.04316173866391182
grad: 0.0 0.0 0.0 0.0 0.04573959857225418
grad: 1.0 4.0 0.032235145568847656 0.032235145568847656 0.032235145568847656
grad: 2.0 12.0 -0.01049041748046875 -0.005245208740234375 -0.0026226043701171875
progress: 64 1.7195134205394424e-06
grad: -3.0 12.0 -0.24151039123535156 0.08050346374511719 -0.026834487915039062
grad: -2.0 4.0 0.15422821044921875 -0.07711410522460938 0.03855705261230469
grad: -1.0 0.0 0.04074845835566521 -0.04074845835566521 0.04074845835566521
grad: 0.0 0.0 0.0 0.0 0.043183133006095886
grad: 1.0 4.0 0.03043365478515625 0.03043365478515625 0.03043365478515625
grad: 2.0 12.0 -0.009918212890625 -0.0049591064453125 -0.00247955322265625
progress: 65 1.5370460459962487e-06
grad: -3.0 12.0 -0.22806930541992188 0.07602310180664062 -0.025341033935546875
grad: -2.0 4.0 0.14562606811523438 -0.07281303405761719 0.036406517028808594
grad: -1.0 0.0 0.038471825420856476 -0.038471825420856476 0.038471825420856476
grad: 0.0 0.0 0.0 0.0 0.04076964408159256
grad: 1.0 4.0 0.028733253479003906 0.028733253479003906 0.028733253479003906
grad: 2.0 12.0 -0.0093536376953125 -0.00467681884765625 -0.002338409423828125
progress: 66 1.3670396583620459e-06
grad: -3.0 12.0 -0.21526336669921875 0.07175445556640625 -0.02391815185546875
grad: -2.0 4.0 0.13747787475585938 -0.06873893737792969 0.034369468688964844
grad: -1.0 0.0 0.036321260035037994 -0.036321260035037994 0.036321260035037994
grad: 0.0 0.0 0.0 0.0 0.0384909063577652
grad: 1.0 4.0 0.027126312255859375 0.027126312255859375 0.027126312255859375
grad: 2.0 12.0 -0.0088348388671875 -0.00441741943359375 -0.002208709716796875
progress: 67 1.219599653268233e-06
grad: -3.0 12.0 -0.2032642364501953 0.06775474548339844 -0.022584915161132812
grad: -2.0 4.0 0.12979888916015625 -0.06489944458007812 0.03244972229003906
grad: -1.0 0.0 0.034291259944438934 -0.034291259944438934 0.034291259944438934
grad: 0.0 0.0 0.0 0.0 0.036339618265628815
grad: 1.0 4.0 0.025610923767089844 0.025610923767089844 0.025610923767089844
grad: 2.0 12.0 -0.00833892822265625 -0.004169464111328125 -0.0020847320556640625
progress: 68 1.086526935978327e-06
grad: -3.0 12.0 -0.19188308715820312 0.06396102905273438 -0.021320343017578125
grad: -2.0 4.0 0.12253189086914062 -0.06126594543457031 0.030632972717285156
grad: -1.0 0.0 0.03237421065568924 -0.03237421065568924 0.03237421065568924
grad: 0.0 0.0 0.0 0.0 0.03430856019258499
grad: 1.0 4.0 0.024179458618164062 0.024179458618164062 0.024179458618164062
grad: 2.0 12.0 -0.00787353515625 -0.003936767578125 -0.0019683837890625
progress: 69 9.686336852610111e-07
grad: -3.0 12.0 -0.18118858337402344 0.06039619445800781 -0.020132064819335938
grad: -2.0 4.0 0.115692138671875 -0.0578460693359375 0.02892303466796875
grad: -1.0 0.0 0.03056499734520912 -0.03056499734520912 0.03056499734520912
grad: 0.0 0.0 0.0 0.0 0.03239104896783829
grad: 1.0 4.0 0.022828102111816406 0.022828102111816406 0.022828102111816406
grad: 2.0 12.0 -0.00743865966796875 -0.003719329833984375 -0.0018596649169921875
progress: 70 8.645884008728899e-07
grad: -3.0 12.0 -0.17102622985839844 0.05700874328613281 -0.019002914428710938
grad: -2.0 4.0 0.10921859741210938 -0.05460929870605469 0.027304649353027344
grad: -1.0 0.0 0.028856372460722923 -0.028856372460722923 0.028856372460722923
grad: 0.0 0.0 0.0 0.0 0.03058069944381714
grad: 1.0 4.0 0.021552085876464844 0.021552085876464844 0.021552085876464844
grad: 2.0 12.0 -0.00701904296875 -0.003509521484375 -0.0017547607421875
progress: 71 7.697963155806065e-07
grad: -3.0 12.0 -0.1614990234375 0.0538330078125 -0.0179443359375
grad: -2.0 4.0 0.10312271118164062 -0.05156135559082031 0.025780677795410156
grad: -1.0 0.0 0.02724394202232361 -0.02724394202232361 0.02724394202232361
grad: 0.0 0.0 0.0 0.0 0.028871532529592514
grad: 1.0 4.0 0.02034759521484375 0.02034759521484375 0.02034759521484375
grad: 2.0 12.0 -0.006622314453125 -0.0033111572265625 -0.00165557861328125
progress: 72 6.852351361885667e-07
grad: -3.0 12.0 -0.15246963500976562 0.050823211669921875 -0.016941070556640625
grad: -2.0 4.0 0.09736251831054688 -0.04868125915527344 0.02434062957763672
grad: -1.0 0.0 0.025721391662955284 -0.025721391662955284 0.025721391662955284
grad: 0.0 0.0 0.0 0.0 0.02725784108042717
grad: 1.0 4.0 0.0192108154296875 0.0192108154296875 0.0192108154296875
grad: 2.0 12.0 -0.006256103515625 -0.0031280517578125 -0.00156402587890625
progress: 73 6.11544237472117e-07
grad: -3.0 12.0 -0.1439208984375 0.0479736328125 -0.0159912109375
grad: -2.0 4.0 0.09191131591796875 -0.045955657958984375 0.022977828979492188
grad: -1.0 0.0 0.024283340200781822 -0.024283340200781822 0.024283340200781822
grad: 0.0 0.0 0.0 0.0 0.025734348222613335
grad: 1.0 4.0 0.018136978149414062 0.018136978149414062 0.018136978149414062
grad: 2.0 12.0 -0.0059051513671875 -0.00295257568359375 -0.001476287841796875
progress: 74 5.448564479593188e-07
grad: -3.0 12.0 -0.13590431213378906 0.04530143737792969 -0.015100479125976562
grad: -2.0 4.0 0.0867767333984375 -0.04338836669921875 0.021694183349609375
grad: -1.0 0.0 0.022926142439246178 -0.022926142439246178 0.022926142439246178
grad: 0.0 0.0 0.0 0.0 0.024296050891280174
grad: 1.0 4.0 0.01712322235107422 0.01712322235107422 0.01712322235107422
grad: 2.0 12.0 -0.0055694580078125 -0.00278472900390625 -0.001392364501953125
progress: 75 4.846697265747935e-07
grad: -3.0 12.0 -0.12833404541015625 0.04277801513671875 -0.01425933837890625
grad: -2.0 4.0 0.08193588256835938 -0.04096794128417969 0.020483970642089844
grad: -1.0 0.0 0.021645106375217438 -0.021645106375217438 0.021645106375217438
grad: 0.0 0.0 0.0 0.0 0.022938115522265434
grad: 1.0 4.0 0.01616668701171875 0.01616668701171875 0.01616668701171875
grad: 2.0 12.0 -0.00525665283203125 -0.002628326416015625 -0.0013141632080078125
progress: 76 4.3175623432034627e-07
grad: -3.0 12.0 -0.121124267578125 0.040374755859375 -0.013458251953125
grad: -2.0 4.0 0.07735061645507812 -0.03867530822753906 0.01933765411376953
grad: -1.0 0.0 0.02043512649834156 -0.02043512649834156 0.02043512649834156
grad: 0.0 0.0 0.0 0.0 0.021656014025211334
grad: 1.0 4.0 0.015262603759765625 0.015262603759765625 0.015262603759765625
grad: 2.0 12.0 -0.00496673583984375 -0.002483367919921875 -0.0012416839599609375
progress: 77 3.8544476410606876e-07
grad: -3.0 12.0 -0.11434364318847656 0.03811454772949219 -0.012704849243164062
grad: -2.0 4.0 0.07302474975585938 -0.03651237487792969 0.018256187438964844
grad: -1.0 0.0 0.019292693585157394 -0.019292693585157394 0.019292693585157394
grad: 0.0 0.0 0.0 0.0 0.02044559270143509
grad: 1.0 4.0 0.014409065246582031 0.014409065246582031 0.014409065246582031
grad: 2.0 12.0 -0.00469207763671875 -0.002346038818359375 -0.0011730194091796875
progress: 78 3.439936335780658e-07
grad: -3.0 12.0 -0.10797500610351562 0.035991668701171875 -0.011997222900390625
grad: -2.0 4.0 0.06895065307617188 -0.03447532653808594 0.01723766326904297
grad: -1.0 0.0 0.01821470446884632 -0.01821470446884632 0.01821470446884632
grad: 0.0 0.0 0.0 0.0 0.0193028561770916
grad: 1.0 4.0 0.013604164123535156 0.013604164123535156 0.013604164123535156
grad: 2.0 12.0 -0.004425048828125 -0.0022125244140625 -0.00110626220703125
progress: 79 3.0595401767641306e-07
grad: -3.0 12.0 -0.10193252563476562 0.033977508544921875 -0.011325836181640625
grad: -2.0 4.0 0.06509017944335938 -0.03254508972167969 0.016272544860839844
grad: -1.0 0.0 0.017196521162986755 -0.017196521162986755 0.017196521162986755
grad: 0.0 0.0 0.0 0.0 0.018223974853754044
grad: 1.0 4.0 0.012844085693359375 0.012844085693359375 0.012844085693359375
grad: 2.0 12.0 -0.00417327880859375 -0.002086639404296875 -0.0010433197021484375
progress: 80 2.721290002227761e-07
grad: -3.0 12.0 -0.09625053405761719 0.03208351135253906 -0.010694503784179688
grad: -2.0 4.0 0.06145477294921875 -0.030727386474609375 0.015363693237304688
grad: -1.0 0.0 0.016235247254371643 -0.016235247254371643 0.016235247254371643
grad: 0.0 0.0 0.0 0.0 0.017205392941832542
grad: 1.0 4.0 0.012125968933105469 0.012125968933105469 0.012125968933105469
grad: 2.0 12.0 -0.003936767578125 -0.0019683837890625 -0.00098419189453125
progress: 81 2.421584213152528e-07
grad: -3.0 12.0 -0.09086036682128906 0.030286788940429688 -0.010095596313476562
grad: -2.0 4.0 0.058017730712890625 -0.029008865356445312 0.014504432678222656
grad: -1.0 0.0 0.015327902510762215 -0.015327902510762215 0.015327902510762215
grad: 0.0 0.0 0.0 0.0 0.016243716701865196
grad: 1.0 4.0 0.011447906494140625 0.011447906494140625 0.011447906494140625
grad: 2.0 12.0 -0.00372314453125 -0.001861572265625 -0.0009307861328125
progress: 82 2.1659070625901222e-07
grad: -3.0 12.0 -0.08579635620117188 0.028598785400390625 -0.009532928466796875
grad: -2.0 4.0 0.054779052734375 -0.0273895263671875 0.01369476318359375
grad: -1.0 0.0 0.014471316710114479 -0.014471316710114479 0.014471316710114479
grad: 0.0 0.0 0.0 0.0 0.015335836447775364
grad: 1.0 4.0 0.010808944702148438 0.010808944702148438 0.010808944702148438
grad: 2.0 12.0 -0.00351715087890625 -0.001758575439453125 -0.0008792877197265625
progress: 83 1.9328672351548448e-07
grad: -3.0 12.0 -0.08098983764648438 0.026996612548828125 -0.008998870849609375
grad: -2.0 4.0 0.051715850830078125 -0.025857925415039062 0.012928962707519531
grad: -1.0 0.0 0.013662590645253658 -0.013662590645253658 0.013662590645253658
grad: 0.0 0.0 0.0 0.0 0.014478673227131367
grad: 1.0 4.0 0.010204315185546875 0.010204315185546875 0.010204315185546875
grad: 2.0 12.0 -0.00331878662109375 -0.001659393310546875 -0.0008296966552734375
progress: 84 1.7209913494298235e-07
grad: -3.0 12.0 -0.07644081115722656 0.025480270385742188 -0.008493423461914062
grad: -2.0 4.0 0.048816680908203125 -0.024408340454101562 0.012204170227050781
grad: -1.0 0.0 0.012898378074169159 -0.012898378074169159 0.012898378074169159
grad: 0.0 0.0 0.0 0.0 0.013669424690306187
grad: 1.0 4.0 0.009634017944335938 0.009634017944335938 0.009634017944335938
grad: 2.0 12.0 -0.003143310546875 -0.0015716552734375 -0.00078582763671875
progress: 85 1.5438126865774393e-07
grad: -3.0 12.0 -0.07220077514648438 0.024066925048828125 -0.008022308349609375
grad: -2.0 4.0 0.046100616455078125 -0.023050308227539062 0.011525154113769531
grad: -1.0 0.0 0.012177936732769012 -0.012177936732769012 0.012177936732769012
grad: 0.0 0.0 0.0 0.0 0.012905457057058811
grad: 1.0 4.0 0.009096145629882812 0.009096145629882812 0.009096145629882812
grad: 2.0 12.0 -0.00295257568359375 -0.001476287841796875 -0.0007381439208984375
progress: 86 1.362141119898297e-07
grad: -3.0 12.0 -0.06813240051269531 0.022710800170898438 -0.0075702667236328125
grad: -2.0 4.0 0.04351043701171875 -0.021755218505859375 0.010877609252929688
grad: -1.0 0.0 0.011496606282889843 -0.011496606282889843 0.011496606282889843
grad: 0.0 0.0 0.0 0.0 0.01218410860747099
grad: 1.0 4.0 0.008586883544921875 0.008586883544921875 0.008586883544921875
grad: 2.0 12.0 -0.0027923583984375 -0.00139617919921875 -0.000698089599609375
progress: 87 1.2183227227069438e-07
grad: -3.0 12.0 -0.06433868408203125 0.02144622802734375 -0.00714874267578125
grad: -2.0 4.0 0.041080474853515625 -0.020540237426757812 0.010270118713378906
grad: -1.0 0.0 0.010854286141693592 -0.010854286141693592 0.010854286141693592
grad: 0.0 0.0 0.0 0.0 0.011503136716783047
grad: 1.0 4.0 0.008107185363769531 0.008107185363769531 0.008107185363769531
grad: 2.0 12.0 -0.0026397705078125 -0.00131988525390625 -0.000659942626953125
progress: 88 1.0888106771744788e-07
grad: -3.0 12.0 -0.060802459716796875 0.020267486572265625 -0.006755828857421875
grad: -2.0 4.0 0.0388031005859375 -0.01940155029296875 0.009700775146484375
grad: -1.0 0.0 0.010248169302940369 -0.010248169302940369 0.010248169302940369
grad: 0.0 0.0 0.0 0.0 0.010860266163945198
grad: 1.0 4.0 0.007655143737792969 0.007655143737792969 0.007655143737792969
grad: 2.0 12.0 -0.00247955322265625 -0.001239776611328125 -0.0006198883056640625
progress: 89 9.606537787476555e-08
grad: -3.0 12.0 -0.057369232177734375 0.019123077392578125 -0.006374359130859375
grad: -2.0 4.0 0.03662872314453125 -0.018314361572265625 0.009157180786132812
grad: -1.0 0.0 0.009675176814198494 -0.009675176814198494 0.009675176814198494
grad: 0.0 0.0 0.0 0.0 0.010253195650875568
grad: 1.0 4.0 0.0072269439697265625 0.0072269439697265625 0.0072269439697265625
grad: 2.0 12.0 -0.002349853515625 -0.0011749267578125 -0.00058746337890625
progress: 90 8.627830538898706e-08
grad: -3.0 12.0 -0.054141998291015625 0.018047332763671875 -0.006015777587890625
grad: -2.0 4.0 0.034572601318359375 -0.017286300659179688 0.008643150329589844
grad: -1.0 0.0 0.009134188294410706 -0.009134188294410706 0.009134188294410706
grad: 0.0 0.0 0.0 0.0 0.009680111892521381
grad: 1.0 4.0 0.0068225860595703125 0.0068225860595703125 0.0068225860595703125
grad: 2.0 12.0 -0.0022125244140625 -0.00110626220703125 -0.000553131103515625
progress: 91 7.648850441910326e-08
grad: -3.0 12.0 -0.051120758056640625 0.017040252685546875 -0.005680084228515625
grad: -2.0 4.0 0.032642364501953125 -0.016321182250976562 0.008160591125488281
grad: -1.0 0.0 0.008623672649264336 -0.008623672649264336 0.008623672649264336
grad: 0.0 0.0 0.0 0.0 0.009139036759734154
grad: 1.0 4.0 0.0064411163330078125 0.0064411163330078125 0.0064411163330078125
grad: 2.0 12.0 -0.0020904541015625 -0.00104522705078125 -0.000522613525390625
progress: 92 6.828122423030436e-08
grad: -3.0 12.0 -0.04825401306152344 0.016084671020507812 -0.0053615570068359375
grad: -2.0 4.0 0.03081512451171875 -0.015407562255859375 0.0077037811279296875
grad: -1.0 0.0 0.008141590282320976 -0.008141590282320976 0.008141590282320976
grad: 0.0 0.0 0.0 0.0 0.008628210984170437
grad: 1.0 4.0 0.006081581115722656 0.006081581115722656 0.006081581115722656
grad: 2.0 12.0 -0.00197601318359375 -0.000988006591796875 -0.0004940032958984375
progress: 93 6.10098140896298e-08
grad: -3.0 12.0 -0.04554176330566406 0.015180587768554688 -0.0050601959228515625
grad: -2.0 4.0 0.029083251953125 -0.0145416259765625 0.00727081298828125
grad: -1.0 0.0 0.0076859937980771065 -0.0076859937980771065 0.0076859937980771065
grad: 0.0 0.0 0.0 0.0 0.00814596377313137
grad: 1.0 4.0 0.005741119384765625 0.005741119384765625 0.005741119384765625
grad: 2.0 12.0 -0.00186920166015625 -0.000934600830078125 -0.0004673004150390625
progress: 94 5.459241947392002e-08
grad: -3.0 12.0 -0.04305267333984375 0.01435089111328125 -0.00478363037109375
grad: -2.0 4.0 0.027477264404296875 -0.013738632202148438 0.006869316101074219
grad: -1.0 0.0 0.007257451303303242 -0.007257451303303242 0.007257451303303242
grad: 0.0 0.0 0.0 0.0 0.0076907058246433735
grad: 1.0 4.0 0.005420684814453125 0.005420684814453125 0.005420684814453125
grad: 2.0 12.0 -0.0017547607421875 -0.00087738037109375 -0.000438690185546875
progress: 95 4.8112269723787904e-08
grad: -3.0 12.0 -0.04059791564941406 0.013532638549804688 -0.0045108795166015625
grad: -2.0 4.0 0.025928497314453125 -0.012964248657226562 0.006482124328613281
grad: -1.0 0.0 0.0068511334247887135 -0.0068511334247887135 0.0068511334247887135
grad: 0.0 0.0 0.0 0.0 0.007260804530233145
grad: 1.0 4.0 0.0051174163818359375 0.0051174163818359375 0.0051174163818359375
grad: 2.0 12.0 -0.0016632080078125 -0.00083160400390625 -0.000415802001953125
progress: 96 4.322282620705664e-08
grad: -3.0 12.0 -0.03836631774902344 0.012788772583007812 -0.0042629241943359375
grad: -2.0 4.0 0.0244903564453125 -0.01224517822265625 0.006122589111328125
grad: -1.0 0.0 0.006468663923442364 -0.006468663923442364 0.006468663923442364
grad: 0.0 0.0 0.0 0.0 0.006854990031570196
grad: 1.0 4.0 0.0048313140869140625 0.0048313140869140625 0.0048313140869140625
grad: 2.0 12.0 -0.00156402587890625 -0.000782012939453125 -0.0003910064697265625
progress: 97 3.822151484200731e-08
grad: -3.0 12.0 -0.03620338439941406 0.012067794799804688 -0.0040225982666015625
grad: -2.0 4.0 0.023113250732421875 -0.011556625366210938 0.005778312683105469
grad: -1.0 0.0 0.006106880493462086 -0.006106880493462086 0.006106880493462086
grad: 0.0 0.0 0.0 0.0 0.006471832282841206
grad: 1.0 4.0 0.004561424255371094 0.004561424255371094 0.004561424255371094
grad: 2.0 12.0 -0.00148773193359375 -0.000743865966796875 -0.0003719329833984375
progress: 98 3.4583536034915596e-08
grad: -3.0 12.0 -0.03414344787597656 0.011381149291992188 -0.0037937164306640625
grad: -2.0 4.0 0.021816253662109375 -0.010908126831054688 0.005454063415527344
grad: -1.0 0.0 0.005765250883996487 -0.005765250883996487 0.005765250883996487
grad: 0.0 0.0 0.0 0.0 0.006110093556344509
grad: 1.0 4.0 0.004305839538574219 0.004305839538574219 0.004305839538574219
grad: 2.0 12.0 -0.00140380859375 -0.000701904296875 -0.0003509521484375
progress: 99 3.079185262322426e-08
Predict (after training) 4 39.99186325073242
