【进阶实战】用PaddlePaddle检测空气质量
In[1]
#请先运行此代码块获得代码
!unzip -o /home/aistudio/data/data2365/testfluid.zip -d /home/aistudio/
!NEW_NAME=$(find -name *[0-9].ipynb) && NEW_NAME=${NEW_NAME%.*} && NEW_NAME=${NEW_NAME#./} && cp -rf testfluid/. . && mv fluid_LinearRegression_complete.ipynb ${NEW_NAME}"_complete.ipynb"Archive: /home/aistudio/data/data2365/testfluid.zip creating: /home/aistudio/testfluid/datasets/ creating: /home/aistudio/testfluid/datasets/.ipynb_checkpoints/ inflating: /home/aistudio/testfluid/datasets/data.txt mv: cannot stat 'fluid_LinearRegression_complete.ipynb': No such file or directory
根据PaddlePaddle实现线性回归预测PM2.5与AQI(空气质量指数关系)
引用库
- numpy：一个python的基本库，用于科学计算
- matplotlib.pyplot：用于生成图，在验证模型准确率和展示成本变化趋势时会使用到
- paddle.fluid：PaddlePaddle其中一种深度学习框架
- pandas:一种基于NumPy的工具，高效处理数据
In[2]
import numpy as np
import paddle
import paddle.fluid as fluid
import matplotlib.pyplot as plt
import pandas as pd
from paddle.utils.plot import Ploter
from __future__ import print_function
from paddle.fluid.contrib.trainer import *
from paddle.fluid.contrib.inferencer import *
%matplotlib inline数据预处理
数据集介绍
- 包含武汉市2017年6月至2018年11月PM2.5与AQI(空气质量指数关系)监测数据由网页(https://www.aqistudy.cn/historydata/) 手动获取。为了简化模型，假设影响AQI的因素只有PM2.5，因此数据集只有两列，以txt的形式储存。
归一化处理
In[3]
colnames = ['PM2.5']+['AQI']
print_data = pd.read_csv('/home/aistudio/testfluid/datasets/data.txt',names = colnames)
print_data.head(10) #观察数据| PM2.5 | AQI | |
|---|---|---|
| 0 | 179 | 225 |
| 1 | 183 | 229 |
| 2 | 93 | 122 |
| 3 | 83 | 110 |
| 4 | 63 | 87 |
| 5 | 75 | 112 |
| 6 | 83 | 119 |
| 7 | 44 | 81 |
| 8 | 44 | 78 |
| 9 | 46 | 74 |
** 归一化 **
观察一下数据的分布特征，一般而言，如果样本有多个属性，那么各维属性的取值范围差异会很大，这就要用到一个常见的操作-归一化（normalization）了。归一化的目标是把各位属性的取值范围放缩到差不多的区间，例如[-0.5, 0.5]。这里我们使用一种很常见的操作方法：减掉均值，然后除以原取值范围。
In[4]
# coding = utf-8 #
global x_raw,train_data,test_data
data = np.loadtxt('/home/aistudio/testfluid/datasets/data.txt',delimiter = ',')
x_raw = data.T[0].copy()
maximums,minimums,avgs = data.max(axis=0),data.min(axis=0),data.sum(axis=0)/data.shape[0]
feature_num = 2
for i in range(feature_num-1):
data[:,i] = (data[:,i] - avgs[i])/(maximums[i] - minimums[i])
print ("归一化后的数据data :")
print (data)归一化后的数据data : [[ 6.55009836e-01 2.25000000e+02] [ 6.74811816e-01 2.29000000e+02] [ 2.29267262e-01 1.22000000e+02] ... [-8.35650067e-03 1.16000000e+02] [ 4.11484498e-02 8.80000000e+01] [ 4.11484498e-02 7.60000000e+01]]
** 数据集分割 **
In[5]
ratio = 0.8
offset = int(data.shape[0]*ratio)
train_data = data[:offset].copy()
test_data = data[offset:].copy() 定义reader
构造read_data()函数，来读取训练数据集train_set或者测试数据集test_set。它的具体实现是在read_data()函数内部构造一个reader()，使用yield关键字来让reader()成为一个Generator（生成器），注意，yield关键字的作用和使用方法类似return关键字，不同之处在于yield关键字可以构造生成器（Generator）。虽然我们可以直接创建一个包含所有数据的列表，但是由于内存限制，我们不可能创建一个无限大的或者巨大的列表，并且很多时候在创建了一个百万数量级别的列表之后，我们却只需要用到开头的几个或几十个数据，这样造成了极大的浪费，而生成器的工作方式是在每次循环时计算下一个值，不断推算出后续的元素，不会创建完整的数据集列表，从而节约了内存使用。 获取训练数据集和测试数据集
In[6]
def read_data(data_set):
def reader():
for data in data_set:
yield data[:-1], data[-1:]
return reader
def train():
global train_data
return read_data(train_data)
def test():
global test_data
return read_data(test_data)测试reader：
In[7]
test_array = ([10,100],[20,200])
print("test_array for read_data:")
for value in read_data(test_array)():
print(value)test_array for read_data: ([10], [100]) ([20], [200])
训练过程
完成了数据的预处理工作并构造了read_data()来读取数据，接下来将进入模型的训练过程，使用PaddlePaddle来定义构造可训练的线性回归模型，关键步骤如下：
-
初始化
-
配置网络结构和设置参数
- 配置网络结构
- 定义损失函数cost
- 定义program
- 定义优化器optimizer
-
模型训练
-
预测
-
绘制拟合图像
** （1）定义运算场所 ** 首先进行最基本的运算场所定义，在 fluid 中使用 place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace() 来进行初始化：
- place 表示fluid program的执行设备，常见的有 fluid.CUDAPlace(0) 和 fluid.CPUPlace()
- use_cuda = False 表示不使用 GPU 进行加速训练
In[8]
#使用CPU训练
#use_cuda = False
use_cuda = True
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()** （2）配置网络结构和设置参数 **
** 配置网络结构 **
线性回归的模型其实就是一个采用线性激活函数（linear activation）的全连接层（fully-connected layer，fc_layer），因此在Peddlepeddle中利用全连接层模型构造线性回归，这样一个全连接层就可以看做是一个简单的神经网络，只包含输入层和输出层即可。本次的模型由于只有一个影响参数，因此输入只含一个X0X_0X0。
接下来就让我们利用PaddlePaddle提供的接口，搭建我们自己的网络吧！
输入层
我们可以用x = fluid.layers.data(name='x', shape=[1], dtype='float32')来表示数据的一个输入层，其中名称为"x"，数据类型为一维向量。
输出层
用y_predict = fluid.layers.fc(input=x, size=1, act=None)来表示输出层：其中paddle.layer.fc表示全连接层，input=x表示该层出入数据为x，size=1表示该层有一个神经元，act=None表示激活函数为线性激活函数。
标签层
用y = fluid.layers.data(name='y', shape=[1], dtype='float32')来表示标签数据，名称为y，数据类型为一维向量
定义损失函数
本次使用的是均方差损失函数，可以调用fluid.layers.square_error_cost(input= ,laybel= )实现方差计算，通过fluid.layers.mean(loss)对方差求平均。将输入定义为房价预测值，label定义为标签数据，计算损失值。
封装训练参数
将设计完成的网络参数写入train_program()函数，便于训练时调用
In[9]
def train_program():
# 输入层，fluid.layers.data表示数据层,name=’x’：名称为x,输出类型为tensor
x = fluid.layers.data(name='x', shape=[1], dtype='float32')
# 标签数据，fluid.layers.data表示数据层,name=’y’：名称为y,输出类型为tensor
y = fluid.layers.data(name='y', shape=[1], dtype='float32')
# 输出层，fluid.layers.fc表示全连接层，input=x: 该层输入数据为x
y_predict = fluid.layers.fc(input=x, size=1, act=None)
loss =fluid.layers.square_error_cost(input=y_predict, label=y)
avg_loss = fluid.layers.mean(loss)
return avg_loss** 优化方法 **
损失函数定义确定后，需要定义参数优化方法。为了改善模型的训练速度以及效果，学术界先后提出了很多优化算法，包括： Momentum、RMSProp、Adam 等，已经被封装在fluid内部，读者可直接调用。本次可以用sgd_optimizer = fluid.optimizer.SGD(learning_rate= )使用随机梯度下降的方法优化，其中learning_rate可自己尝试修改。
In[10]
# 创建optimizer，更多优化算子可以参考 fluid.optimizer()
def optimizer_program():
return fluid.optimizer.SGD(learning_rate=0.01)** 其它配置 **
feed_order=['x', 'y']是数据层名称和数组索引的映射，用于定义数据的读取顺序。params_dirname用于定义模型保存路径。
最后定义事件处理器event_handler_plot(event)用于打印训练进程，在fulid中有两种形式的事件处理器，分别是文字形式与图像形式。本次使用的是图像形式的处理器，当训练开始时调用该函数，可以得到loss变化图像，便于使用者进行参数调整。
In[11]
# 数据层和数组索引映射，用于trainer训练时喂数据
feed_order=['x', 'y']
# 保存模型
params_dirname = "easy_fit_a_line.inference.model"
# Plot data
from paddle.utils.plot import Ploter
train_title = "Train cost"
test_title = "Test cost"
plot_cost = Ploter(train_title, test_title)
step = 0
# 事件处理
def event_handler_plot(event):
global step
if isinstance(event, EndStepEvent):
if event.step % 10 == 0:
#plot_cost.append(train_title, step, event.metrics[0])
#plot_cost.plot()
print("%s,Step %d, Cost %f" %(train_title, step, event.metrics[0]))
if event.step % 100 == 0: # 每10个batch,记录cost
test_metrics = trainer.test(
reader=test_reader, feed_order=feed_order)
#plot_cost.append(test_title, step, test_metrics[0])
#plot_cost.plot()
print("%s, Step %d, Cost %f" %(test_title, step, test_metrics[0]))
if test_metrics[0] < 0.01:
# 如果准确率达到阈值，则停止训练
# print('loss is less than 0.01, stop')
print('loss is less than 0.001, stop')
trainer.stop()
# 将参数存储，用于预测使用
if params_dirname is not None:
trainer.save_params(params_dirname)
step += 1** （3）训练模型 **
上述内容进行了模型初始化、网络结构的配置并创建了训练函数、硬件位置、优化方法，接下来利用上述配置进行模型训练。
首先定义执行器，fulid使用了一个C++类Executor用于运行一个程序，Executor类似一个解释器，Fluid将会使用这样一个解析器来训练和测试模型。 一个随机梯度下降trainer，配置三个参数cost、parameters、update_equation，它们分别表示损失函数、参数和更新公式。
再利用trainer.train()即可开始真正的模型训练，我们可以设置参数如下：
** 定义执行器(参数随机初始化) **
用户配置完模型后，参数初始化操作会被写入到fluid.default_startup_program() 中。使用 fluid.Executor() 运行 这一程序，即可在全局 fluid.global_scope() 中随机初始化参数。
In[12]
# 创建执行器，palce在程序初始化时设定
exe = fluid.Executor(place)
# 初始化执行器
exe.run(fluid.default_startup_program())[]
** 设置训练参数 **
同学们可以试着调整一下参数值，看会有什么变化。
In[13]
BATCH_SIZE = 20
# 设置训练reader
train_reader = paddle.batch(
paddle.reader.shuffle(
train(), buf_size=400),
batch_size=BATCH_SIZE)
#设置测试reader
test_reader = paddle.batch(
paddle.reader.shuffle(
test(), buf_size=400),
batch_size=BATCH_SIZE)** 创建训练器 **
训练器会读入一个配置好的训练程序和一些必要的其他参数
In[14]
trainer =Trainer(
train_func=train_program,
place=place,
optimizer_func=optimizer_program)** 开始训练 **
In[15]
trainer.train(
reader=train_reader,
### START CODE HERE ### (≈ 1 lines of code)
num_epochs=100,
### END CODE HERE ###
event_handler=event_handler_plot,
feed_order=feed_order)Train cost,Step 2, Cost 10559.277344 Test cost, Step 2, Cost 7391.766950 Train cost,Step 22, Cost 6246.083496 Train cost,Step 42, Cost 3965.115967 Train cost,Step 54, Cost 4575.041504 Test cost, Step 54, Cost 3715.424090 Train cost,Step 74, Cost 2897.070801 Train cost,Step 94, Cost 2043.355835 Train cost,Step 106, Cost 1452.470581 Test cost, Step 106, Cost 2464.979239 Train cost,Step 126, Cost 1706.162476 Train cost,Step 146, Cost 1311.699707 Train cost,Step 158, Cost 1852.690063 Test cost, Step 158, Cost 2045.962892 Train cost,Step 178, Cost 1393.618896 Train cost,Step 198, Cost 1382.461670 Train cost,Step 210, Cost 3816.041748 Test cost, Step 210, Cost 1959.052793 Train cost,Step 230, Cost 2606.771240 Train cost,Step 250, Cost 1119.502930 Train cost,Step 262, Cost 612.618469 Test cost, Step 262, Cost 1938.794571 Train cost,Step 282, Cost 1062.165161 Train cost,Step 302, Cost 1138.468628 Train cost,Step 314, Cost 850.107056 Test cost, Step 314, Cost 1985.267896 Train cost,Step 334, Cost 1342.018555 Train cost,Step 354, Cost 1079.512695 Train cost,Step 366, Cost 2942.920410 Test cost, Step 366, Cost 1986.087712 Train cost,Step 386, Cost 1086.931641 Train cost,Step 406, Cost 924.745117 Train cost,Step 418, Cost 1110.824951 Test cost, Step 418, Cost 1954.340401 Train cost,Step 438, Cost 2516.216797 Train cost,Step 458, Cost 913.026978 Train cost,Step 470, Cost 1868.921509 Test cost, Step 470, Cost 2190.663182 Train cost,Step 490, Cost 408.938049 Train cost,Step 510, Cost 865.810669 Train cost,Step 522, Cost 441.908600 Test cost, Step 522, Cost 2091.344683 Train cost,Step 542, Cost 1039.553223 Train cost,Step 562, Cost 724.524658 Train cost,Step 574, Cost 1279.106079 Test cost, Step 574, Cost 1966.642781 Train cost,Step 594, Cost 758.582703 Train cost,Step 614, Cost 800.177002 Train cost,Step 626, Cost 718.616150 Test cost, Step 626, Cost 2119.009195 Train cost,Step 646, Cost 925.540161 Train cost,Step 666, Cost 1208.964111 Train cost,Step 678, Cost 988.833130 Test cost, Step 678, Cost 2015.971549 Train cost,Step 698, Cost 1089.924805 Train cost,Step 718, Cost 1195.520874 Train cost,Step 730, Cost 528.684021 Test cost, Step 730, Cost 2001.963571 Train cost,Step 750, Cost 1634.816406 Train cost,Step 770, Cost 717.188293 Train cost,Step 782, Cost 1090.921021 Test cost, Step 782, Cost 2081.599771 Train cost,Step 802, Cost 741.314148 Train cost,Step 822, Cost 715.717224 Train cost,Step 834, Cost 1004.869751 Test cost, Step 834, Cost 2057.661717 Train cost,Step 854, Cost 2225.604248 Train cost,Step 874, Cost 901.660156 Train cost,Step 886, Cost 1362.283081 Test cost, Step 886, Cost 2003.605373 Train cost,Step 906, Cost 1364.846069 Train cost,Step 926, Cost 1357.100220 Train cost,Step 938, Cost 308.381622 Test cost, Step 938, Cost 1958.902252 Train cost,Step 958, Cost 807.769165 Train cost,Step 978, Cost 1066.846436 Train cost,Step 990, Cost 1785.702026 Test cost, Step 990, Cost 2008.073508 Train cost,Step 1010, Cost 638.823914 Train cost,Step 1030, Cost 932.781067 Train cost,Step 1042, Cost 485.031219 Test cost, Step 1042, Cost 2011.930660 Train cost,Step 1062, Cost 1915.102417 Train cost,Step 1082, Cost 607.985779 Train cost,Step 1094, Cost 1450.252808 Test cost, Step 1094, Cost 1965.785932 Train cost,Step 1114, Cost 1154.989136 Train cost,Step 1134, Cost 631.671021 Train cost,Step 1146, Cost 1244.287720 Test cost, Step 1146, Cost 1927.756825 Train cost,Step 1166, Cost 1483.890625 Train cost,Step 1186, Cost 1069.428711 Train cost,Step 1198, Cost 1595.059692 Test cost, Step 1198, Cost 1965.174983 Train cost,Step 1218, Cost 921.366699 Train cost,Step 1238, Cost 931.738403 Train cost,Step 1250, Cost 1242.236450 Test cost, Step 1250, Cost 1965.442509 Train cost,Step 1270, Cost 746.811707 Train cost,Step 1290, Cost 773.015259 Train cost,Step 1302, Cost 589.658997 Test cost, Step 1302, Cost 2020.146532 Train cost,Step 1322, Cost 419.204010 Train cost,Step 1342, Cost 981.260376 Train cost,Step 1354, Cost 306.530182 Test cost, Step 1354, Cost 2018.669643 Train cost,Step 1374, Cost 1511.610962 Train cost,Step 1394, Cost 1054.686890 Train cost,Step 1406, Cost 1237.014893 Test cost, Step 1406, Cost 2028.482043 Train cost,Step 1426, Cost 678.446472 Train cost,Step 1446, Cost 730.978333 Train cost,Step 1458, Cost 1594.279175 Test cost, Step 1458, Cost 1899.966585 Train cost,Step 1478, Cost 823.310059 Train cost,Step 1498, Cost 766.319763 Train cost,Step 1510, Cost 1220.811401 Test cost, Step 1510, Cost 1914.998263 Train cost,Step 1530, Cost 546.997742 Train cost,Step 1550, Cost 581.843933 Train cost,Step 1562, Cost 379.570404 Test cost, Step 1562, Cost 1912.717294 Train cost,Step 1582, Cost 645.285339 Train cost,Step 1602, Cost 638.192322 Train cost,Step 1614, Cost 1223.254272 Test cost, Step 1614, Cost 1961.235195 Train cost,Step 1634, Cost 531.333801 Train cost,Step 1654, Cost 1002.861938 Train cost,Step 1666, Cost 710.517456 Test cost, Step 1666, Cost 1928.238913 Train cost,Step 1686, Cost 629.154846 Train cost,Step 1706, Cost 593.340698 Train cost,Step 1718, Cost 501.806702 Test cost, Step 1718, Cost 1944.706857 Train cost,Step 1738, Cost 997.933289 Train cost,Step 1758, Cost 670.551392 Train cost,Step 1770, Cost 976.966614 Test cost, Step 1770, Cost 1976.321764 Train cost,Step 1790, Cost 469.286346 Train cost,Step 1810, Cost 654.507935 Train cost,Step 1822, Cost 719.815735 Test cost, Step 1822, Cost 1910.986751 Train cost,Step 1842, Cost 1209.124390 Train cost,Step 1862, Cost 696.789856 Train cost,Step 1874, Cost 1319.474976 Test cost, Step 1874, Cost 1921.132586 Train cost,Step 1894, Cost 847.336609 Train cost,Step 1914, Cost 792.540283 Train cost,Step 1926, Cost 1144.486206 Test cost, Step 1926, Cost 1949.780308 Train cost,Step 1946, Cost 995.823364 Train cost,Step 1966, Cost 616.997253 Train cost,Step 1978, Cost 1182.751221 Test cost, Step 1978, Cost 1921.443669 Train cost,Step 1998, Cost 1294.922485 Train cost,Step 2018, Cost 662.101562 Train cost,Step 2030, Cost 1004.771118 Test cost, Step 2030, Cost 1904.647705 Train cost,Step 2050, Cost 658.294678 Train cost,Step 2070, Cost 665.050110 Train cost,Step 2082, Cost 617.468445 Test cost, Step 2082, Cost 1917.511928 Train cost,Step 2102, Cost 1274.968506 Train cost,Step 2122, Cost 715.062256 Train cost,Step 2134, Cost 609.610596 Test cost, Step 2134, Cost 1908.047324 Train cost,Step 2154, Cost 508.122650 Train cost,Step 2174, Cost 667.684937 Train cost,Step 2186, Cost 595.823364 Test cost, Step 2186, Cost 1930.728354 Train cost,Step 2206, Cost 458.775299 Train cost,Step 2226, Cost 673.518494 Train cost,Step 2238, Cost 353.338562 Test cost, Step 2238, Cost 1991.154946 Train cost,Step 2258, Cost 721.771729 Train cost,Step 2278, Cost 576.594116 Train cost,Step 2290, Cost 205.122223 Test cost, Step 2290, Cost 1977.390769 Train cost,Step 2310, Cost 975.067993 Train cost,Step 2330, Cost 622.460083 Train cost,Step 2342, Cost 1112.011353 Test cost, Step 2342, Cost 1950.411176 Train cost,Step 2362, Cost 529.334656 Train cost,Step 2382, Cost 333.433655 Train cost,Step 2394, Cost 695.670837 Test cost, Step 2394, Cost 2038.681601 Train cost,Step 2414, Cost 1037.829224 Train cost,Step 2434, Cost 456.196777 Train cost,Step 2446, Cost 1660.180908 Test cost, Step 2446, Cost 1971.238508 Train cost,Step 2466, Cost 745.344666 Train cost,Step 2486, Cost 626.252319 Train cost,Step 2498, Cost 2029.284424 Test cost, Step 2498, Cost 1922.448347 Train cost,Step 2518, Cost 713.415405 Train cost,Step 2538, Cost 483.916321 Train cost,Step 2550, Cost 577.169067 Test cost, Step 2550, Cost 1899.213283 Train cost,Step 2570, Cost 284.223663 Train cost,Step 2590, Cost 654.132446 Train cost,Step 2602, Cost 563.590515 Test cost, Step 2602, Cost 1882.858697 Train cost,Step 2622, Cost 476.700775 Train cost,Step 2642, Cost 447.382782 Train cost,Step 2654, Cost 1313.885376 Test cost, Step 2654, Cost 1951.092429 Train cost,Step 2674, Cost 900.858582 Train cost,Step 2694, Cost 535.888306 Train cost,Step 2706, Cost 370.347870 Test cost, Step 2706, Cost 1987.435211 Train cost,Step 2726, Cost 588.094971 Train cost,Step 2746, Cost 487.342041 Train cost,Step 2758, Cost 468.601898 Test cost, Step 2758, Cost 1931.544150 Train cost,Step 2778, Cost 1198.925659 Train cost,Step 2798, Cost 574.686707 Train cost,Step 2810, Cost 465.802643 Test cost, Step 2810, Cost 1923.721915 Train cost,Step 2830, Cost 524.074341 Train cost,Step 2850, Cost 661.897278 Train cost,Step 2862, Cost 576.249390 Test cost, Step 2862, Cost 1936.073735 Train cost,Step 2882, Cost 174.004135 Train cost,Step 2902, Cost 740.146667 Train cost,Step 2914, Cost 423.182617 Test cost, Step 2914, Cost 1890.983294 Train cost,Step 2934, Cost 696.061890 Train cost,Step 2954, Cost 309.280914 Train cost,Step 2966, Cost 797.509399 Test cost, Step 2966, Cost 14443.882459 Train cost,Step 2986, Cost 910.742371 Train cost,Step 3006, Cost 578.689636 Train cost,Step 3018, Cost 627.917419 Test cost, Step 3018, Cost 1982.179683 Train cost,Step 3038, Cost 230.554108 Train cost,Step 3058, Cost 565.949036 Train cost,Step 3070, Cost 254.459625 Test cost, Step 3070, Cost 2050.827249 Train cost,Step 3090, Cost 466.467773 Train cost,Step 3110, Cost 400.398712 Train cost,Step 3122, Cost 625.268188 Test cost, Step 3122, Cost 1884.919552 Train cost,Step 3142, Cost 702.799011 Train cost,Step 3162, Cost 371.007141 Train cost,Step 3174, Cost 584.925293 Test cost, Step 3174, Cost 1886.684876 Train cost,Step 3194, Cost 895.600037 Train cost,Step 3214, Cost 541.050537 Train cost,Step 3226, Cost 346.957977 Test cost, Step 3226, Cost 1911.858817 Train cost,Step 3246, Cost 440.069000 Train cost,Step 3266, Cost 581.938904 Train cost,Step 3278, Cost 445.007294 Test cost, Step 3278, Cost 1887.050915 Train cost,Step 3298, Cost 404.389374 Train cost,Step 3318, Cost 563.310425 Train cost,Step 3330, Cost 732.573425 Test cost, Step 3330, Cost 1974.778429 Train cost,Step 3350, Cost 356.416748 Train cost,Step 3370, Cost 544.706360 Train cost,Step 3382, Cost 311.886383 Test cost, Step 3382, Cost 1941.634038 Train cost,Step 3402, Cost 701.907532 Train cost,Step 3422, Cost 631.762024 Train cost,Step 3434, Cost 1019.215454 Test cost, Step 3434, Cost 1891.084478 Train cost,Step 3454, Cost 765.299500 Train cost,Step 3474, Cost 596.936340 Train cost,Step 3486, Cost 178.929565 Test cost, Step 3486, Cost 1910.588946 Train cost,Step 3506, Cost 1052.194336 Train cost,Step 3526, Cost 656.392334 Train cost,Step 3538, Cost 753.606873 Test cost, Step 3538, Cost 1975.384718 Train cost,Step 3558, Cost 724.063660 Train cost,Step 3578, Cost 602.949829 Train cost,Step 3590, Cost 319.996552 Test cost, Step 3590, Cost 1925.980905 Train cost,Step 3610, Cost 833.619080 Train cost,Step 3630, Cost 619.450684 Train cost,Step 3642, Cost 722.689575 Test cost, Step 3642, Cost 1956.713527 Train cost,Step 3662, Cost 524.150574 Train cost,Step 3682, Cost 430.775208 Train cost,Step 3694, Cost 354.953094 Test cost, Step 3694, Cost 1895.001779 Train cost,Step 3714, Cost 386.799713 Train cost,Step 3734, Cost 445.899323 Train cost,Step 3746, Cost 733.504395 Test cost, Step 3746, Cost 1923.313224 Train cost,Step 3766, Cost 305.943634 Train cost,Step 3786, Cost 396.759277 Train cost,Step 3798, Cost 609.446716 Test cost, Step 3798, Cost 1966.921452 Train cost,Step 3818, Cost 231.538864 Train cost,Step 3838, Cost 466.861176 Train cost,Step 3850, Cost 433.019775 Test cost, Step 3850, Cost 1923.174077 Train cost,Step 3870, Cost 839.345154 Train cost,Step 3890, Cost 332.134491 Train cost,Step 3902, Cost 708.837158 Test cost, Step 3902, Cost 1923.087577 Train cost,Step 3922, Cost 531.642822 Train cost,Step 3942, Cost 353.531097 Train cost,Step 3954, Cost 929.086243 Test cost, Step 3954, Cost 2059.055398 Train cost,Step 3974, Cost 477.027924 Train cost,Step 3994, Cost 351.008789 Train cost,Step 4006, Cost 433.647278 Test cost, Step 4006, Cost 1924.174242 Train cost,Step 4026, Cost 537.472168 Train cost,Step 4046, Cost 592.184021 Train cost,Step 4058, Cost 615.912231 Test cost, Step 4058, Cost 1900.048793 Train cost,Step 4078, Cost 313.236816 Train cost,Step 4098, Cost 515.148926 Train cost,Step 4110, Cost 387.657532 Test cost, Step 4110, Cost 1949.317950 Train cost,Step 4130, Cost 278.425385 Train cost,Step 4150, Cost 512.334167 Train cost,Step 4162, Cost 463.260956 Test cost, Step 4162, Cost 1937.945108 Train cost,Step 4182, Cost 195.640396 Train cost,Step 4202, Cost 297.414764 Train cost,Step 4214, Cost 420.895721 Test cost, Step 4214, Cost 1900.783858 Train cost,Step 4234, Cost 380.936707 Train cost,Step 4254, Cost 285.449066 Train cost,Step 4266, Cost 189.405792 Test cost, Step 4266, Cost 1944.665628 Train cost,Step 4286, Cost 249.444244 Train cost,Step 4306, Cost 400.050690 Train cost,Step 4318, Cost 387.928253 Test cost, Step 4318, Cost 1932.568717 Train cost,Step 4338, Cost 496.759186 Train cost,Step 4358, Cost 424.341400 Train cost,Step 4370, Cost 475.920807 Test cost, Step 4370, Cost 1939.604462 Train cost,Step 4390, Cost 412.863037 Train cost,Step 4410, Cost 434.611633 Train cost,Step 4422, Cost 410.197845 Test cost, Step 4422, Cost 1988.271868 Train cost,Step 4442, Cost 887.196899 Train cost,Step 4462, Cost 300.795227 Train cost,Step 4474, Cost 1213.234497 Test cost, Step 4474, Cost 1979.970891 Train cost,Step 4494, Cost 370.056244 Train cost,Step 4514, Cost 573.498657 Train cost,Step 4526, Cost 426.365295 Test cost, Step 4526, Cost 1901.347081 Train cost,Step 4546, Cost 154.763428 Train cost,Step 4566, Cost 560.311707 Train cost,Step 4578, Cost 352.273010 Test cost, Step 4578, Cost 1940.301043 Train cost,Step 4598, Cost 512.810059 Train cost,Step 4618, Cost 298.849945 Train cost,Step 4630, Cost 266.844604 Test cost, Step 4630, Cost 14917.804487 Train cost,Step 4650, Cost 338.598419 Train cost,Step 4670, Cost 659.610168 Train cost,Step 4682, Cost 333.382568 Test cost, Step 4682, Cost 1939.766968 Train cost,Step 4702, Cost 306.020020 Train cost,Step 4722, Cost 279.022430 Train cost,Step 4734, Cost 232.477615 Test cost, Step 4734, Cost 1986.254591 Train cost,Step 4754, Cost 320.879700 Train cost,Step 4774, Cost 367.728149 Train cost,Step 4786, Cost 994.452454 Test cost, Step 4786, Cost 1914.187932 Train cost,Step 4806, Cost 582.139587 Train cost,Step 4826, Cost 306.927673 Train cost,Step 4838, Cost 518.740234 Test cost, Step 4838, Cost 1923.731367 Train cost,Step 4858, Cost 655.760498 Train cost,Step 4878, Cost 264.207428 Train cost,Step 4890, Cost 495.894989 Test cost, Step 4890, Cost 1960.732378 Train cost,Step 4910, Cost 1200.270142 Train cost,Step 4930, Cost 329.419128 Train cost,Step 4942, Cost 416.390930 Test cost, Step 4942, Cost 2147.829437 Train cost,Step 4962, Cost 190.092148 Train cost,Step 4982, Cost 393.490326 Train cost,Step 4994, Cost 327.246185 Test cost, Step 4994, Cost 2034.657360 Train cost,Step 5014, Cost 638.247925 Train cost,Step 5034, Cost 416.215088 Train cost,Step 5046, Cost 399.653259 Test cost, Step 5046, Cost 15021.685464 Train cost,Step 5066, Cost 396.323639 Train cost,Step 5086, Cost 327.943024 Train cost,Step 5098, Cost 365.311127 Test cost, Step 5098, Cost 1974.118177 Train cost,Step 5118, Cost 1146.107056 Train cost,Step 5138, Cost 723.379089 Train cost,Step 5150, Cost 354.521423 Test cost, Step 5150, Cost 1953.950317 Train cost,Step 5170, Cost 712.080688 Train cost,Step 5190, Cost 498.009979
预测
** （1）设定预测程序 **
类似于trainer.train，预测器需要一个预测程序来做预测。我们可以稍加修改我们的训练程序来把预测值包含进来。
In[16]
# 设置预测程序
def inference_program():
x = fluid.layers.data(name='x', shape=[1], dtype='float32')
y_predict = fluid.layers.fc(input=x, size=1, act=None)
return y_predict** (2)预测 **
预测器会从params_dirname中读取已经训练好的模型，来对从未遇见过的数据进行预测。
- tensor_x:生成batch_size个[0,1]区间的随机数，以tensor的格式储存
- results：
- raw_x:由于数据处理时我们做了归一化操作，为了更直观的判断预测是否准确，将数据进行反归一化，得到随机数对应的原始数据。
In[17]
inferencer = Inferencer(
infer_func=inference_program, param_path=params_dirname, place=place)
batch_size = 2
tensor_x = np.random.uniform(0, 1, [batch_size, 1]).astype("float32")
results = inferencer.infer({'x': tensor_x})
raw_x = tensor_x*(maximums[i]-minimums[i])+avgs[i]
a=(results[0][1]-results[0][0])/(4.8*(raw_x[1]-raw_x[0]))+1
b=(results[0][1]-a*raw_x[1])/128
print(a,b)[1.1560305] [0.00785673]
得到预测结果，大概函数为 y=1.1x+19.4
** （5）绘制拟合图像 **
通过训练，本次线性回归模型输出了一条拟合的直线，想要直观的判断模型好坏可将拟合直线与数据的图像绘制出来。
In[18]
import numpy as np
import matplotlib.pyplot as plt
def plot_data(data):
x = data[:,0]
y = data[:,1]
y_predict = x*a + b
plt.scatter(x,y,marker='.',c='r',label='True')
plt.title('AQI Distributions')
plt.xlabel('PM2.5 ')
plt.ylabel('AQI ')
plt.xlim(0,150)
plt.ylim(0,150)
predict = plt.plot(x,y_predict,label='Predict')
plt.legend(loc='upper left')
plt.savefig('result1.png')
plt.show()
data = np.loadtxt('/home/aistudio/testfluid/datasets/data.txt',delimiter = ',')
plot_data(data)
到此，可以得出结论：
从输出结果图可以看出，预测数据落在直线上，通过观察可以清楚地看到真实数据大部分散布在预测数据周围，说明预测结果是比较可靠的。