Tensorflow2.0:(三)神经网络搭建八股

第三章 神经网络搭建八股

本次Tensorflow2.0学习笔记参考北京大学曹健老师《人工智能实践:Tensorflow笔记》课程。
课程链接:https://www.icourse163.org/course/PKU-1002536002
B站链接:https://www.bilibili.com/video/BV1B7411L7Qt?p=1
课程代码及自制数据集下载:https://github.com/jlff/tf2_notes
该课程适合无基础的初学者入门,在保证主线完整的前提下,各章节略有增删改简。

学习目的

  • 神经网络搭建八股
  • iris数据集识别鸢尾花代码复现
  • MNIST数据集识别手写数字代码复现
  • FASHION数据集识别衣裤鞋包代码复现

一、六步法

  1. keras简介
  • 什么是keras?keras 是一个用 Python 编写的高级神经网络 API,它能够以 TensorFlow, CNTK, 或者 Theano 作为后端运行。它的开发由Google支持,支持GPU和CPU,主要包括Models API、Layers API、Callbacks API、Data preprocessing、Optimizers、Metrics、Losses、Built-in small datasets、Keras Applications、Utilities等10个模块包,参见文档:https://keras.io/api/
  • keras vs. tf.keras? TensorFlow 2.0发布后, keras 正式成为 TensorFlow 的官方高级 API。2019年随着 keras 2.3.0 的发布,keras的创建者和首席维护者Francois Chollet 声明: keras v2.3.0 是首个与 tf.keras 同步的版本,也将是最后一个支持除 TensorFlow 以外的后端(即 Theano,CNTK 等)的最终版本。
  1. keras搭建神经网络六步法
  • 第一步:import
    import相关模块,如import tensorflow as tf
  • 第二步:train,test
    指定输入网络的训练集和测试集
  • 第三步:model = tf.keras.models.Sequential / class MyModel(Model) model=MyModel
    逐层搭建网络结构
model = tf.keras.models.Sequential([网络结构]) #描述各层网络
"""
例如:
拉直层:tf.keras.layers.Flatten()
全连接层:tf.keras.layers.Dense(神经元个数, activation="激活函数“ ,kernel_regularizer=哪种正则化)
activation(字符串给出)可选: relu、softmax、sigmoid 、tanh
kernel_regularizer可选:tf.keras.regularizers.l1()、tf.keras.regularizers.l2()
卷积层:tf.keras.layers.Conv2D(filters = 卷积核个数, kernel_size= 卷积核尺寸,
strides = 卷积步长,padding = " valid" or "same")
LSTM层:tf.keras.layers.LSTM()
"""

使用Sequential可以快速搭建网络结构,但是如果网络包含跳连等其他复杂网络结构,Sequential就无法表示了。这时就需要使用class来自定义网络结构。

class MyModel(Model):
	def __init__(self):
	# 定义网络结构块
		super(MyModel, self).__init__()
	def call(self, x):
	# 调用网络结构块,实现前向传播
	return y
	
model= MyModel()
  • 第四步:model.compile
    配置训练方法:优化器、损失函数和最终评价指标。优化器、损失函数在第二章中有介绍。
model.compile(optimizer = 优化器, loss = 损失函数, metrics = [“准确率”])

"""
Optimizer可选:
‘sgd’ or tf.keras.optimizers.SGD(lr=学习率,momentum=动量参数)
‘adagrad’ or tf.keras.optimizers.Adagrad(lr=学习率,decay=学习率衰减率)
‘adadelta’ or tf.keras.optimizers.Adadelta(lr=学习率,decay=学习率衰减率)
‘adam’ or tf.keras.optimizers.Adam(lr=学习率, beta_1=0.9, beta_2=0.999)
loss可选:
‘mse’ or tf.keras.losses.MeanSquaredError()
‘sparse_categorical_crossentropy’ or tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)
Metrics可选:
‘accuracy’ :y_和y都是数值,如y_=[1] y=[1]
‘categorical_accuracy’ :y_和y都是独热码(概率分布),如y_=[0,1,0] y=[0.256,0.695,0.048]
‘sparse_categorical_accuracy’ :y_是数值,y是独热码(概率分布),如y_=[1] y=[0.256,0.695,0.048]
"""
  • 第五步:model.fit
    执行训练过程
model.fit(训练集的输入特征, 训练集的标签, batch_size= , epochs= ,
validation_data=(测试集的输入特征,测试集的标签),
validation_split=从训练集划分多少比例给测试集,validation_freq= 多少次epoch测试一次)
  • 第六步:model.summary
    打印网络结构,统计参数数目
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 3)                 15        
=================================================================
Total params: 15
Trainable params: 15
Non-trainable params: 0
_________________________________________________________________

二、iris数据集识别鸢尾花

使用Sequential

# 第一步import
import tensorflow as tf
from sklearn import datasets
import numpy as np

# 第二步train test
x_train = datasets.load_iris().data # 测试集的输入特征x_test和标签y_test可以像x_train和y_train一样直接从数据集获取,也可以如上述在fit中按比例从训练集中划分,本例选择从训练集中划分,所以只需加载x_train,y_train即可
y_train = datasets.load_iris().target

np.random.seed(116)
np.random.shuffle(x_train) # 将数据集乱序
np.random.seed(116)
np.random.shuffle(y_train)
tf.random.set_seed(116)

# 第三步models.Sequential
model = tf.keras.models.Sequential([
    tf.keras.layers.Dense(3, activation='softmax', kernel_regularizer=tf.keras.regularizers.l2())
]) # 使用单层全连接网络,第一个参数表示神经元个数,第二个参数表示网络所使用的激活函数,第三个参数表示选用的正则化方法

# 第四步model.compile
model.compile(optimizer=tf.keras.optimizers.SGD(lr=0.1),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy']) # 使用SGD优化器,并将学习率设置为0.1,选择SparseCategoricalCrossentrop作为损失函数,输出为概率分布,所以metrics需要设置为sparse_categorical_accuracy

# 第五步model.fit
model.fit(x_train, y_train, batch_size=32, epochs=500, validation_split=0.2, validation_freq=20) # batch_size表示神经网络进行一次训练样本数,epochs表示所有样本进行迭代的次数validation_split表示数据集中验证集的划分比例,validation_freq表示每迭代20次在测试集上测试一次准确率。

# 第六步model.summary()
model.summary()
...
Epoch 499/500
4/4 [==============================] - 0s 2ms/step - loss: 0.3691 - sparse_categorical_accuracy: 0.9306
Epoch 500/500
4/4 [==============================] - 0s 14ms/step - loss: 0.3634 - sparse_categorical_accuracy: 0.9304 - val_loss: 0.3516 - val_sparse_categorical_accuracy: 0.8667
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 3)                 15        
=================================================================
Total params: 15
Trainable params: 15
Non-trainable params: 0
_________________________________________________________________

使用自定义Class IrisModel(Model)

# 第一步import
import tensorflow as tf
from tensorflow.keras.layers import Dense
from tensorflow.keras import Model
from sklearn import datasets
import numpy as np

# 第二步train test
x_train = datasets.load_iris().data
y_train = datasets.load_iris().target

np.random.seed(116)
np.random.shuffle(x_train)
np.random.seed(116)
np.random.shuffle(y_train)
tf.random.set_seed(116)

# 第三步class IrisModel
class IrisModel(Model):
    def __init__(self):
        super(IrisModel, self).__init__()
        self.d1 = Dense(3, activation='sigmoid', kernel_regularizer=tf.keras.regularizers.l2())

    def call(self, x):
        y = self.d1(x)
        return y


model = IrisModel()

# 第四步model.compile
model.compile(optimizer=tf.keras.optimizers.SGD(lr=0.1),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

# 第五步model.fit
model.fit(x_train, y_train, batch_size=32, epochs=500, validation_split=0.2, validation_freq=20)

# 第六步model.summary()
model.summary()
...
Epoch 499/500
4/4 [==============================] - 0s 2ms/step - loss: 0.3691 - sparse_categorical_accuracy: 0.9306
Epoch 500/500
4/4 [==============================] - 0s 16ms/step - loss: 0.3634 - sparse_categorical_accuracy: 0.9304 - val_loss: 0.3516 - val_sparse_categorical_accuracy: 0.8667
Model: "iris_model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              multiple                  15        
=================================================================
Total params: 15
Trainable params: 15
Non-trainable params: 0
_________________________________________________________________

三、MNIST数据集识别手写数字

查看数据格式

import tensorflow as tf
from matplotlib import pyplot as plt

mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()

# 可视化训练集输入特征的第一个元素
plt.imshow(x_train[0], cmap='gray')  # 绘制灰度图
plt.show()

# 打印出训练集输入特征的第一个元素
print("x_train[0]:\n", x_train[0])
# 打印出训练集标签的第一个元素
print("y_train[0]:\n", y_train[0])

# 打印出整个训练集输入特征形状
print("x_train.shape:\n", x_train.shape)
# 打印出整个训练集标签的形状
print("y_train.shape:\n", y_train.shape)
# 打印出整个测试集输入特征的形状
print("x_test.shape:\n", x_test.shape)
# 打印出整个测试集标签的形状
print("y_test.shape:\n", y_test.shape)

Tensorflow2.0:(三)神经网络搭建八股_第1张图片

x_train[0]:
 [[  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   3  18  18  18 126 136 175 26  166 255 247 127   0   0   0   0]
 [   0   0   0   0   0   0   0   0  30  36  94 154 170 253 253 253 253 253 225 172 253 242 195  64   0   0   0   0]
 [   0   0   0   0   0   0   0  49 238 253 253 253 253 253 253 253 253 251 93  82  82  56   39   0   0   0   0   0]
 [   0   0   0   0   0   0   0  18 219 253 253 253 253 253 198 182 247 241   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0  80 156 107 253 253 205  11   0  43 154   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0  14   1 154 253  90   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0 139 253 190   2   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0  11 190 253  70   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0  35 241 225 160 108   1   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0  81 240 253 253 119  25   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0  45 186 253 253 150  27   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  16  93 252 253 187   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 249 253 249  64   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0  46 130 183 253 253 207   2   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0  39 148 229 253 253 253 250 182   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0  24 114 221 253 253 253 253 201  78   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0  23  66 213 253 253 253 253 198  81   2   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0  18 171 219 253 253 253 253 195  80   9   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0  55 172 226 253 253 253 253 244 133  11   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0 136 253 253 253 212 135 132  16   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]]
y_train[0]:
 5
x_train.shape:
 (60000, 28, 28)
y_train.shape:
 (60000,)
x_test.shape:
 (10000, 28, 28)
y_test.shape:
 (10000,)

使用Sequential训练模型

import tensorflow as tf

mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0 # 将输入特征的灰度值归一化到[0,1]区间,这可以使网络更快收敛

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(), # 输入全连接网络时需要先将数据拉直为一维数组,把784个像素点的灰度值作为输入特征输入神经网络
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

model.fit(x_train, y_train, batch_size=32, epochs=5, validation_data=(x_test, y_test), validation_freq=1)
model.summary()
...
Epoch 4/5
1875/1875 [==============================] - 3s 1ms/step - loss: 0.0554 - sparse_categorical_accuracy: 0.9830 - val_loss: 0.0746 - val_sparse_categorical_accuracy: 0.9769
Epoch 5/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.0402 - sparse_categorical_accuracy: 0.9885 - val_loss: 0.0766 - val_sparse_categorical_accuracy: 0.9780
Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_1 (Flatten)          (None, 784)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 128)               100480    
_________________________________________________________________
dense_3 (Dense)              (None, 10)                1290      
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________

使用自定义Class MnistModel(Model)

import tensorflow as tf
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras import Model

mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0


class MnistModel(Model):
    def __init__(self):
        super(MnistModel, self).__init__()
        self.flatten = Flatten()
        self.d1 = Dense(128, activation='relu')
        self.d2 = Dense(10, activation='softmax')

    def call(self, x):
        x = self.flatten(x)
        x = self.d1(x)
        y = self.d2(x)
        return y


model = MnistModel()

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

model.fit(x_train, y_train, batch_size=32, epochs=5, validation_data=(x_test, y_test), validation_freq=1)
model.summary()
...
Epoch 4/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.0559 - sparse_categorical_accuracy: 0.9827 - val_loss: 0.0786 - val_sparse_categorical_accuracy: 0.9763
Epoch 5/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.0443 - sparse_categorical_accuracy: 0.9869 - val_loss: 0.0774 - val_sparse_categorical_accuracy: 0.9755
Model: "mnist_model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_2 (Flatten)          multiple                  0         
_________________________________________________________________
dense_4 (Dense)              multiple                  100480    
_________________________________________________________________
dense_5 (Dense)              multiple                  1290      
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________

四、FASHION数据集识别衣裤鞋包

使用Sequential训练模型

import tensorflow as tf

fashion = tf.keras.datasets.fashion_mnist
(x_train, y_train),(x_test, y_test) = fashion.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

model.fit(x_train, y_train, batch_size=32, epochs=5, validation_data=(x_test, y_test), validation_freq=1)
model.summary()
...
Epoch 4/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.0559 - sparse_categorical_accuracy: 0.9827 - val_loss: 0.0786 - val_sparse_categorical_accuracy: 0.9763
Epoch 5/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.0443 - sparse_categorical_accuracy: 0.9869 - val_loss: 0.0774 - val_sparse_categorical_accuracy: 0.9755
Model: "mnist_model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_2 (Flatten)          multiple                  0         
_________________________________________________________________
dense_4 (Dense)              multiple                  100480    
_________________________________________________________________
dense_5 (Dense)              multiple                  1290      
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________

使用自定义Class MnistModel(Model)

import tensorflow as tf
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras import Model

fashion = tf.keras.datasets.fashion_mnist
(x_train, y_train),(x_test, y_test) = fashion.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0


class MnistModel(Model):
    def __init__(self):
        super(MnistModel, self).__init__()
        self.flatten = Flatten()
        self.d1 = Dense(128, activation='relu')
        self.d2 = Dense(10, activation='softmax')

    def call(self, x):
        x = self.flatten(x)
        x = self.d1(x)
        y = self.d2(x)
        return y


model = MnistModel()

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

model.fit(x_train, y_train, batch_size=32, epochs=5, validation_data=(x_test, y_test), validation_freq=1)
model.summary()
...
Epoch 4/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.3151 - sparse_categorical_accuracy: 0.8848 - val_loss: 0.3678 - val_sparse_categorical_accuracy: 0.8659
Epoch 5/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2976 - sparse_categorical_accuracy: 0.8902 - val_loss: 0.3643 - val_sparse_categorical_accuracy: 0.8717
Model: "mnist_model_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_4 (Flatten)          multiple                  0         
_________________________________________________________________
dense_8 (Dense)              multiple                  100480    
_________________________________________________________________
dense_9 (Dense)              multiple                  1290      
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________

五、参考

【1】keras官方文档:https://keras.io/zh/
【2】tf.keras官方文档:https://tensorflow.google.cn/api_docs/python/tf/keras
【3】keras vs. tf.keras:https://www.pyimagesearch.com/2019/10/21/keras-vs-tf-keras-whats-the-difference-in-tensorflow-2-0/

你可能感兴趣的:(深度学习,tensorflow,神经网络,深度学习,python)