卷积神经网络

2D Convolution

Kernel size

• 矩阵卷积

Padding & Stride

• 步长2

Channels

For instance

• x: [b,28,28,3]
• one k: [3,3,3]
• multi-k: [16,3,3,3]
• stride: 1
• padding: [1,1,1,1]
• bias: [16]
• out: [b,28,28,16]

LeNet-5

Pyramid Architecture

• 从底层的边缘颜色到高层抽象的概念（轮子、车窗）

layers.Conv2D

import tensorflow as tf
from tensorflow.keras import layers

x = tf.random.normal([1, 32, 32, 3])

# padding='valid'：输入和输出维度不同
layer = layers.Conv2D(4, kernel_size=5, strides=1, padding='valid')
out = layer(x)
out.shape

TensorShape([1, 28, 28, 4])

# padding='same'：输入和输出维度相同
layer = layers.Conv2D(4, kernel_size=5, strides=1, padding='same')
out = layer(x)
out.shape

TensorShape([1, 32, 32, 4])

layer = layers.Conv2D(4, kernel_size=5, strides=2, padding='same')
out = layer(x)
out.shape

TensorShape([1, 16, 16, 4])

layer.call(x).shape

TensorShape([1, 16, 16, 4])

weight & bias

layer = layers.Conv2D(4, kernel_size=5, strides=2, padding='same')
out = layer(x)
out.shape

TensorShape([1, 16, 16, 4])

# 5，5--》size，3--》通道数，4--》核数量
layer.kernel.shape

TensorShape([5, 5, 3, 4])

layer.bias

nn.conv2d

w = tf.random.normal([5, 5, 3, 4])
b = tf.zeros([4])
x.shape

TensorShape([1, 32, 32, 3])

out = tf.nn.conv2d(x, w, strides=1, padding='VALID')
out.shape

TensorShape([1, 28, 28, 4])

out = out + b
out.shape

TensorShape([1, 28, 28, 4])

out = tf.nn.conv2d(x, w, strides=2, padding='VALID')
out.shape

TensorShape([1, 14, 14, 4])

Gradient?

\[ \frac{\partial{Loss}}{\partial{w}} \]

For instance