吴恩达深度学习课程-Course 2 改善深层神经网络 第三周 TensorFlow入门编程作业
虽然申请书的ddl临近，但还是先写写编程作业…
编程作业的代码用的是tf1，而我的环境为tf2，所以
TensorFlow入门
- TensorFlow教程
- 1 - 探索Tensorflow库
-
- 1.1 - 线性函数
- 1.2 - 计算Sigmoid函数
- 1.3 -计算成本(Cost)
- 1.4 - 使用One Hot编码
- 1.5 -用0和1初始化
- 2 - 用Tensorflow构建你的第一个神经网络
-
- 2.0 - 问题声明:SIGNS数据集
- 2.1 -创建占位符placeholders
- 2.2 - 初始化参数
- 2.3 - TensorFlow中的前向传播
- 2.4 - 计算成本
- 2.5 - 反向传播和参数更新
- 2.6 -建立模型
TensorFlow教程
欢迎来到本周的编程作业。到目前为止，您一直使用numpy来构建神经网络。现在我们将逐步介绍一个深度学习框架，它将让你更容易地构建神经网络。像TensorFlow, PaddlePaddle, Torch, Caffe, Keras等机器学习框架可以大大加快机器学习的发展。所有这些框架都有大量文档，您可以随意阅读。在这个作业中，你将学习在TensorFlow中做以下事情:
- 初始化变量
- 开始你自己的会话
- 训练算法
- 实现神经网络
编程框架不仅可以缩短您的编码时间，有时还可以执行优化来加快您的代码。
1 - 探索Tensorflow库
首先，你需要导入库：
import math
import numpy as np
import h5py
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow.python.framework import ops
from tf_utils import load_dataset, random_mini_batches, convert_to_one_hot, predict
%matplotlib inline
np.random.seed(1)
既然已经导入了库，我们将介绍它的不同应用。你将从一个例子开始，我们会为你计算一个训练例子的损失。
l o s s = L ( y ^ , y ) = ( y ^ ( i ) − y ( i ) ) 2 (1) loss = \mathcal{L}(\hat{y}, y) = (\hat y^{(i)} - y^{(i)})^2 \tag{1} loss=L(y^,y)=(y^(i)−y(i))2(1)
y_hat = tf.constant(36, name='y_hat') # Define y_hat constant. Set to 36.
y = tf.constant(39, name='y') # Define y. Set to 39
loss = tf.Variable((y - y_hat)**2, name='loss') # Create a variable for the loss
init = tf.global_variables_initializer() # When init is run later (session.run(init)),
# the loss variable will be initialized and ready to be computed
with tf.Session() as session: # Create a session and print the output
session.run(init) # Initializes the variables
print(session.run(loss)) # Prints the loss
在TensorFlow中编写和运行程序有以下步骤:
- 创建尚未执行/求值的张量(变量)。
- 这些张量之间的写操作。
- 初始化你的张量。
- 创建一个会话。
- 运行会话。这将运行您在上面编写的操作。
因此，当我们为损失创建一个变量时，我们只是将损失定义为其他量的函数，而没有对其值进行评估。为了计算它，我们必须运行init=tf. global_variable_initializer()。这初始化了loss变量，在最后一行，我们终于能够计算loss的值并打印它的值。
现在让我们看一个简单的例子。运行下面的单元格：
a = tf.constant(2)
b = tf.constant(10)
c = tf.multiply(a,b)
print(c)
运行结果为
Tensor("Mul:0", shape=(), dtype=int32)
正如所料，您不会看到20!你得到一个张量，它的结果是一个没有shape属性的张量，类型是int32。你所做的只是把它放到“计算图”中，但是你还没有运行这个计算。为了将这两个数字相乘，您必须创建一个会话并运行它。
sess = tf.Session()
print(sess.run(c))
输出结果为
20
太棒了！总之，请记住初始化变量、创建会话并在会话中运行操作。
接下来，您还必须了解占位符(placeholders)。占位符是一个对象，它的值只能在以后指定。要为占位符指定值，可以通过使用“feed dictionary”(feed_dict变量)传递值。下面，我们为 x x x创建了一个占位符。这允许我们在稍后运行会话时传入一个数字。
# Change the value of x in the feed_dict
x = tf.placeholder(tf.int64, name = 'x')
print(sess.run(2 * x, feed_dict = {x:3}))
sess.close()
输出结果为
6
当你第一次定义 x x x时，你不需要为它指定一个值。占位符只是一个变量，您将在以后运行会话时将数据分配给它。我们说，你在运行会话时向这些占位符提供数据。
当你指定一个计算所需的操作时，你正在告诉TensorFlow如何构造一个计算图。计算图可以有一些占位符，这些占位符的值将在以后指定。最后，当您运行会话时，您将告诉TensorFlow执行计算图。
1.1 - 线性函数
让我们从以下公式开始编程练习： Y = W X + b Y = WX + b Y=WX+b，其中 W W W和 X X X是随机矩阵，b是随机向量。
练习:计算 W X + b WX + b WX+b，其中 W W W、 X X X和 b b b为随机正态分布。 W W W的形状是(4,3)， X X X是(3,1)， b b b是(4,1)。作为一个例子，下面是你如何定义一个具有形状(3,1)的常数 X X X：
X = tf.constant(np.random.randn(3, 1), name = 'X')
你可能会发现以下函数很有用:
tf.matmul(…，…)来做矩阵乘法tf.add(…，…)来做加法np.random.randn(…)随机初始化
# GRADED FUNCTION: linear_function
def linear_function():
"""
Implements a linear function:
Initializes W to be a random tensor of shape (4,3)
Initializes X to be a random tensor of shape (3,1)
Initializes b to be a random tensor of shape (4,1)
Returns:
result -- runs the session for Y = WX + b
"""
np.random.seed(1)
### START CODE HERE ### (4 lines of code)
X = tf.constant(np.random.randn(3,1), name = "X")
W = tf.constant(np.random.randn(4,3), name = "W")
b = tf.constant(np.random.randn(4,1), name = "b")
Y = tf.add(tf.matmul(W,X), b)
### END CODE HERE ###
# Create the session using tf.Session() and run it with sess.run(...) on the variable you want to calculate
### START CODE HERE ###
sess = tf.Session()
result = sess.run(Y)
### END CODE HERE ###
# close the session
sess.close()
return result
print( "result = " + str(linear_function()))
输出结果为
result = [[-2.15657382]
[ 2.95891446]
[-1.08926781]
[-0.84538042]]
1.2 - 计算Sigmoid函数
太棒了！你实现了一个线性函数。Tensorflow提供了多种常用的神经网络函数，如tf.sigmoid和tf.softmax。在这个练习中，我们要计算输入的sigmoid函数。
您将使用一个占位符变量x来完成这个练习。当运行会话时，您应该使用feed dictionary来传递输入z。
在这个练习中，你必须
- 创建一个占位符
x， - 使用
tf.sigmoid定义计算sigmoid函数所需的操作 - 运行会话
练习:实现下面的sigmoid函数。你应该使用以下方法:
tf.placeholder(tf.float32, name = "...")tf.sigmoid(...)sess.run(..., feed_dict = {x: z})
注意，在tensorflow中有两种典型的创建和使用会话的方式:
Method 1:
sess = tf.Session()
# Run the variables initialization (if needed), run the operations
result = sess.run(..., feed_dict = {...})
sess.close() # Close the session
Method 2:
with tf.Session() as sess:
# run the variables initialization (if needed), run the operations
result = sess.run(..., feed_dict = {...})
# This takes care of closing the session for you :)
# GRADED FUNCTION: sigmoid
def sigmoid(z):
"""
Computes the sigmoid of z
Arguments:
z -- input value, scalar or vector
Returns:
results -- the sigmoid of z
"""
### START CODE HERE ### ( approx. 4 lines of code)
# Create a placeholder for x. Name it 'x'.
x = tf.placeholder(tf.float32, name = "x")
# compute sigmoid(x)
sigmoid = tf.sigmoid(x)
# Create a session, and run it. Please use the method 2 explained above.
# You should use a feed_dict to pass z's value to x.
with tf.Session() as sess:
# Run session and call the output "result"
result = sess.run(sigmoid, feed_dict = {x:z})
### END CODE HERE ###
return result
print ("sigmoid(0) = " + str(sigmoid(0)))
print ("sigmoid(12) = " + str(sigmoid(12)))
输出结果为
sigmoid(0) = 0.5
sigmoid(12) = 0.9999939
1.3 -计算成本(Cost)
你也可以使用一个内置函数来计算你的神经网络的成本。所以不需要写代码来计算这个函数 a [ 2 ] ( i ) a^{[2](i)} a[2](i)和 y ( i ) y^{(i)} y(i)。对于i=1…m:
J = − 1 m ∑ i = 1 m ( y ( i ) log a [ 2 ] ( i ) + ( 1 − y ( i ) ) log ( 1 − a [ 2 ] ( i ) ) ) (2) J = - \frac{1}{m} \sum_{i = 1}^m \large ( \small y^{(i)} \log a^{ [2] (i)} + (1-y^{(i)})\log (1-a^{ [2] (i)} )\large )\small\tag{2} J=−m1i=1∑m(y(i)loga[2](i)+(1−y(i))log(1−a[2](i)))(2)
你可以在tensorflow的一行代码中完成!
练习:实现交叉熵损失。你将使用的函数是:
tf.nn.sigmoid_cross_entropy_with_logits(logits = ..., labels = ...)
你的代码应该输入’ z '，计算sigmoid(to get ’ a ')，然后计算交叉熵的代价 J J J。所有这些都可以通过调用tf.nn.sigmoid_cross_entropy_with_logits来完成，它计算了
− 1 m ∑ i = 1 m ( y ( i ) log σ ( z [ 2 ] ( i ) ) + ( 1 − y ( i ) ) log ( 1 − σ ( z [ 2 ] ( i ) ) ) (2) - \frac{1}{m} \sum_{i = 1}^m \large ( \small y^{(i)} \log \sigma(z^{[2](i)}) + (1-y^{(i)})\log (1-\sigma(z^{[2](i)})\large )\small\tag{2} −m1i=1∑m(y(i)logσ(z[2](i))+(1−y(i))log(1−σ(z[2](i)))(2)
# GRADED FUNCTION: cost
def cost(logits, labels):
"""
Computes the cost using the sigmoid cross entropy
Arguments:
logits -- vector containing z, output of the last linear unit (before the final sigmoid activation)
labels -- vector of labels y (1 or 0)
Note: What we've been calling "z" and "y" in this class are respectively called "logits" and "labels"
in the TensorFlow documentation. So logits will feed into z, and labels into y.
Returns:
cost -- runs the session of the cost (formula (2))
"""
### START CODE HERE ###
# Create the placeholders for "logits" (z) and "labels" (y) (approx. 2 lines)
z = tf.placeholder(tf.float32, name = "z")
y = tf.placeholder(tf.float32, name = 'y')
# Use the loss function (approx. 1 line)
cost = tf.nn.sigmoid_cross_entropy_with_logits(logits = z, labels = y)
# Create a session (approx. 1 line). See method 1 above.
sess = tf.Session()
# Run the session (approx. 1 line).
cost = sess.run(cost, feed_dict={z:logits, y:labels})
# Close the session (approx. 1 line). See method 1 above.
sess.close()
### END CODE HERE ###
return cost
logits = sigmoid(np.array([0.2,0.4,0.7,0.9]))
cost = cost(logits, np.array([0,0,1,1]))
print ("cost = " + str(cost))
输出结果为
cost = [1.0053873 1.0366409 0.41385436 0.39956617]
1.4 - 使用One Hot编码
在深度学习中，经常会有一个y向量，其范围从0到C-1，其中C是类的数量。例如，如果C是4，那么你可能有以下的y向量，你将需要转换如下:
这被称为“one hot”编码，因为在转换后的表示中，每一列的一个元素恰好是“hot”(意思是设置为1)。要在numpy中进行这种转换，您可能需要编写几行代码。在tensorflow中，你可以使用一行代码:
- tf.one_hot(labels, depth, axis)
练习：实现下面的函数，取一个标签向量和类的总数 C C C，并返回one hot编码。使用tf.one_hot()来完成此操作。
# GRADED FUNCTION: one_hot_matrix
def one_hot_matrix(labels, C):
"""
Creates a matrix where the i-th row corresponds to the ith class number and the jth column
corresponds to the jth training example. So if example j had a label i. Then entry (i,j)
will be 1.
Arguments:
labels -- vector containing the labels
C -- number of classes, the depth of the one hot dimension
Returns:
one_hot -- one hot matrix
"""
### START CODE HERE ###
# Create a tf.constant equal to C (depth), name it 'C'. (approx. 1 line)
C = tf.constant(C, name = "C")
# Use tf.one_hot, be careful with the axis (approx. 1 line)
one_hot_matrix = tf.one_hot(labels, C, axis = 0)
# Create the session (approx. 1 line)
sess = tf.Session()
# Run the session (approx. 1 line)
one_hot = sess.run(one_hot_matrix)
# Close the session (approx. 1 line). See method 1 above.
sess.close()
### END CODE HERE ###
return one_hot
labels = np.array([1,2,3,0,2,1])
one_hot = one_hot_matrix(labels, C = 4)
print ("one_hot = " + str(one_hot))
输出结果为
one_hot = [[0. 0. 0. 1. 0. 0.]
[1. 0. 0. 0. 0. 1.]
[0. 1. 0. 0. 1. 0.]
[0. 0. 1. 0. 0. 0.]]
1.5 -用0和1初始化
现在您将学习如何初始化一个由0和1组成的向量。您将调用的函数是tf.ones()。要用零初始化，可以使用tf.zeros()代替。这些函数采用一个形状，并返回一个分别充满0和1的维形状数组。
练习：实现下面的函数以获取一个形状并返回一个数组(该形状的维数为1)。
tf.ones(shape)
# GRADED FUNCTION: ones
def ones(shape):
"""
Creates an array of ones of dimension shape
Arguments:
shape -- shape of the array you want to create
Returns:
ones -- array containing only ones
"""
### START CODE HERE ###
# Create "ones" tensor using tf.ones(...). (approx. 1 line)
ones = tf.ones(shape)
# Create the session (approx. 1 line)
sess = tf.Session()
# Run the session to compute 'ones' (approx. 1 line)
ones = sess.run(ones)
# Close the session (approx. 1 line). See method 1 above.
sess.close()
### END CODE HERE ###
return ones
print ("ones = " + str(ones([3])))
输出结果为
ones = [1. 1. 1.]
2 - 用Tensorflow构建你的第一个神经网络
在这部分作业中，你将使用Tensorflow构建一个神经网络。记住，实现Tensorflow模型有两个部分:
- 创建计算图
- 运行图
让我们深入研究你想解决的问题吧！
2.0 - 问题声明:SIGNS数据集
一天下午，我们和一些朋友决定教我们的电脑破译手语。我们花了几个小时在一堵白墙前拍照，并得出了以下数据集。现在你的工作是建立一个算法，来促进语言障碍的人与不懂手语的人之间的交流。
- 训练集:1080张符号图像(64 * 64像素)代表0到5的数字(180张图片每个数字)。
- 测试集:120幅符号图像(64 × 64像素)代表0到5的数字(每个数字20幅)。
注意，这是SIGNS数据集的一个子集。完整的数据集包含了更多的符号。
下面是每个数字的例子，以及如何解释我们如何表示标签。这些是在我们将图像分辨率降低到64x64像素之前的原始图片。
运行以下代码加载数据集。
# Loading the dataset
X_train_orig, Y_train_orig, X_test_orig, Y_test_orig, classes = load_dataset()
更改下面的index并运行单元格以显示数据集中的一些样本。
# Example of a picture
index = 0
plt.imshow(X_train_orig[index])
print ("y = " + str(np.squeeze(Y_train_orig[:, index])))
和往常一样，您可以将图像数据集平坦化，然后通过除以255对其进行归一化。在此基础上，将每个标签转换为one-hot向量，如图1所示。运行下面的单元格来执行此操作。
# Flatten the training and test images
X_train_flatten = X_train_orig.reshape(X_train_orig.shape[0], -1).T
X_test_flatten = X_test_orig.reshape(X_test_origin.shape[0], -1).T
# Normalize image vectors
X_train = X_train_flatten/255.
X_test = X_test_flatten/255.
# Convert training and test labels to one hot matrices
Y_train = convert_to_one_hot(Y_train_orig, 6)
Y_test = convert_to_one_hot(Y_test_orig, 6)
print ("number of training examples = " + str(X_train.shape[1]))
print ("number of test examples = " + str(X_test.shape[1]))
print ("X_train shape: " + str(X_train.shape))
print ("Y_train shape: " + str(Y_train.shape))
print ("X_test shape: " + str(X_test.shape))
print ("Y_test shape: " + str(Y_test.shape))
输出结果
number of training examples = 1080
number of test examples = 120
X_train shape: (12288, 1080)
Y_train shape: (6, 1080)
X_test shape: (12288, 120)
Y_test shape: (6, 120)
注意，12288来自64×64×364×64×3。每个图像是正方形，64 × 64像素，3是RGB颜色。在继续之前，请确保所有这些形状对您来说都可以理解。
你的目标是建立一个算法，能够识别一个符号的高准确性。为此，您将构建一个tensorflow模型，该模型与您之前在numpy中构建的用于识别猫的模型几乎相同(但现在使用softmax输出)。这是比较numpy实现和tensorflow实现的好时机。
模型为LINEAR -> RELU -> LINEAR -> RELU -> LINEAR -> SOFTMAX。SIGMOID输出层已经转换为SOFTMAX。当有两个以上的类时，SOFTMAX层将SIGMOID一般化。
2.1 -创建占位符placeholders
您的第一个任务是为X和Y创建占位符。这将允许您稍后在运行会话时传递训练数据。
练习：实现下面的函数以在tensorflow中创建占位符。
# GRADED FUNCTION: create_placeholders
def create_placeholders(n_x, n_y):
"""
Creates the placeholders for the tensorflow session.
Arguments:
n_x -- scalar, size of an image vector (num_px * num_px = 64 * 64 * 3 = 12288)
n_y -- scalar, number of classes (from 0 to 5, so -> 6)
Returns:
X -- placeholder for the data input, of shape [n_x, None] and dtype "float"
Y -- placeholder for the input labels, of shape [n_y, None] and dtype "float"
Tips:
- You will use None because it let's us be flexible on the number of examples you will for the placeholders.
In fact, the number of examples during test/train is different.
"""
### START CODE HERE ### (approx. 2 lines)
X = tf.placeholder(shape=[n_x, None],dtype=tf.float32)
Y = tf.placeholder(shape=[n_y, None],dtype=tf.float32)
### END CODE HERE ###
return X, Y
X, Y = create_placeholders(12288, 6)
print ("X = " + str(X))
print ("Y = " + str(Y))
输出结果为
X = Tensor("X_7:0", dtype=float32)
Y = Tensor("Y_3:0", dtype=float32)
2.2 - 初始化参数
第二个任务是初始化tensorflow中的参数。
实现下面的函数来初始化tensorflow中的参数。你将使用Xavier初始化的权重和零初始化的偏差。形状如下所示。例如，对于 W 1 W1 W1和 b 1 b1 b1，你可以使用:
W1 = tf.get_variable("W1", [25,12288], initializer = tf.contrib.layers.xavier_initializer(seed = 1))注意tf2这里已经改了，要写成tf.get_variable("W1", [25, 12288], initializer = tf.keras.initializers.glorot_normal(seed = 1))b1 = tf.get_variable("b1", [25,1], initializer = tf.zeros_initializer())
请使用seed = 1以确保您的结果与我们的匹配。
# GRADED FUNCTION: initialize_parameters
def initialize_parameters():
"""
Initializes parameters to build a neural network with tensorflow. The shapes are:
W1 : [25, 12288]
b1 : [25, 1]
W2 : [12, 25]
b2 : [12, 1]
W3 : [6, 12]
b3 : [6, 1]
Returns:
parameters -- a dictionary of tensors containing W1, b1, W2, b2, W3, b3
"""
tf.set_random_seed(1) # so that your "random" numbers match ours
### START CODE HERE ### (approx. 6 lines of code)
W1 = tf.get_variable("W1", [25, 12288], initializer = tf.keras.initializers.glorot_normal(seed = 1))
b1 = tf.get_variable("b1", [25,1],initializer = tf.zeros_initializer())
W2 = tf.get_variable("W2", [12, 25], initializer = tf.keras.initializers.glorot_normal(seed = 1))
b2 = tf.get_variable("b2", [12,1],initializer = tf.zeros_initializer())
W3 = tf.get_variable("W3", [6, 12], initializer = tf.keras.initializers.glorot_normal(seed = 1))
b3 = tf.get_variable("b3", [6,1],initializer = tf.zeros_initializer())
### END CODE HERE ###
parameters = {"W1": W1,
"b1": b1,
"W2": W2,
"b2": b2,
"W3": W3,
"b3": b3}
return parameters
tf.reset_default_graph()
with tf.Session() as sess:
parameters = initialize_parameters()
print("W1 = " + str(parameters["W1"]))
print("b1 = " + str(parameters["b1"]))
print("W2 = " + str(parameters["W2"]))
print("b2 = " + str(parameters["b2"]))
输出结果为
W1 = <tf.Variable 'W1:0' shape=(25, 12288) dtype=float32_ref>
b1 = <tf.Variable 'b1:0' shape=(25, 1) dtype=float32_ref>
W2 = <tf.Variable 'W2:0' shape=(25, 12288) dtype=float32_ref>
b2 = <tf.Variable 'b2:0' shape=(25, 1) dtype=float32_ref>
正如预期的那样，参数还没有被求值。
2.3 - TensorFlow中的前向传播
现在将在tensorflow中实现前向传播模块。该函数将接受一个包含参数的字典，并完成前向传播。你将使用的函数是:
tf.add(…，…)来做加法tf.matmul(…，…)来做矩阵乘法tf.nn.relu(…)应用ReLU激活
问题：实现神经网络的前向传播。我们为您注释了numpy的等价代码，以便您可以将tensorflow实现与numpy进行比较。重要的是要注意向前传播在z3停止。原因是在TensorFlow中，最后的线性层输出作为计算损失的函数的输入。因此，您不需要a3！
# GRADED FUNCTION: forward_propagation
def forward_propagation(X, parameters):
"""
Implements the forward propagation for the model: LINEAR -> RELU -> LINEAR -> RELU -> LINEAR -> SOFTMAX
Arguments:
X -- input dataset placeholder, of shape (input size, number of examples)
parameters -- python dictionary containing your parameters "W1", "b1", "W2", "b2", "W3", "b3"
the shapes are given in initialize_parameters
Returns:
Z3 -- the output of the last LINEAR unit
"""
# Retrieve the parameters from the dictionary "parameters"
W1 = parameters['W1']
b1 = parameters['b1']
W2 = parameters['W2']
b2 = parameters['b2']
W3 = parameters['W3']
b3 = parameters['b3']
### START CODE HERE ### (approx. 5 lines) # Numpy Equivalents:
Z1 = tf.add(tf.matmul(W1, X), b1) # Z1 = np.dot(W1, X) + b1
A1 = tf.nn.relu(Z1) # A1 = relu(Z1)
Z2 = tf.add(tf.matmul(W2, A1), b2) # Z2 = np.dot(W2, a1) + b2
A2 = tf.nn.relu(Z2) # A2 = relu(Z2)
Z3 = tf.add(tf.matmul(W3, A2), b3) # Z3 = np.dot(W3,Z2) + b3
### END CODE HERE ###
return Z3
tf.reset_default_graph()
with tf.Session() as sess:
X, Y = create_placeholders(12288, 6)
parameters = initialize_parameters()
Z3 = forward_propagation(X, parameters)
print("Z3 = " + str(Z3))
输出结果为
Z3 = Tensor("Add_2:0", dtype=float32)
您可能已经注意到向前传播不输出任何缓存。当我们讲到反向传播时，你就会明白为什么。
2.4 - 计算成本
如前所述，使用以下方法计算成本是非常容易的:
tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = ..., labels = ...))
问题：执行下面的成本函数。
-
了解
tf.nn.softmax_cross_entropy_with_logits的输入"logits“和”labels"的期望形状为(number of examples, num_classes)是很重要的。我们把 Z 3 Z3 Z3和 Y Y Y进行了转置。 -
此外
tf.reduce_mean基本上是对这些样本求和。
# GRADED FUNCTION: compute_cost
def compute_cost(Z3, Y):
"""
Computes the cost
Arguments:
Z3 -- output of forward propagation (output of the last LINEAR unit), of shape (6, number of examples)
Y -- "true" labels vector placeholder, same shape as Z3
Returns:
cost - Tensor of the cost function
"""
# to fit the tensorflow requirement for tf.nn.softmax_cross_entropy_with_logits(...,...)
logits = tf.transpose(Z3)
labels = tf.transpose(Y)
### START CODE HERE ### (1 line of code)
cost = cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = logits, labels = labels))
### END CODE HERE ###
return cost
tf.reset_default_graph()
with tf.Session() as sess:
X, Y = create_placeholders(12288, 6)
parameters = initialize_parameters()
Z3 = forward_propagation(X, parameters)
cost = compute_cost(Z3, Y)
print("cost = " + str(cost))
输出结果为
WARNING:tensorflow:From D:\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\util\dispatch.py:206: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.
See `tf.nn.softmax_cross_entropy_with_logits_v2`.
cost = Tensor("softmax_cross_entropy_with_logits_sg/Reshape_2:0", dtype=float32)
2.5 - 反向传播和参数更新
这就是你感激编程框架的地方。所有的反向传播和参数更新都在一行代码中完成。很容易将这一行合并到模型中。
在你计算出代价函数之后。您将创建一个“optimizer”对象。在运行tf.session时，您必须调用这个对象以及cost。当调用时，它将使用所选的方法和学习率对给定的代价进行优化。
例如，对于梯度下降，优化器将是:
optimizer = tf.train.GradientDescentOptimizer(learning_rate = learning_rate).minimize(cost)
要进行优化，你需要做:
_ , c = sess.run([optimizer, cost], feed_dict={X: minibatch_X, Y: minibatch_Y})
这是通过反向计算tensorflow图来计算反向传播的。从cost到input。
注意：在编写代码时，我们经常使用_ 作为“一次性”变量来存储我们以后不需要使用的值。这里，_ 取优化器的评估值，这是我们不需要的(和c取cost变量的值)。
2.6 -建立模型
现在，你要把一切都做好!
练习:实现模型。您将调用之前实现的函数。
def model(X_train, Y_train, X_test, Y_test, learning_rate = 0.0001,
num_epochs = 1500, minibatch_size = 32, print_cost = True):
"""
Implements a three-layer tensorflow neural network: LINEAR->RELU->LINEAR->RELU->LINEAR->SOFTMAX.
Arguments:
X_train -- training set, of shape (input size = 12288, number of training examples = 1080)
Y_train -- test set, of shape (output size = 6, number of training examples = 1080)
X_test -- training set, of shape (input size = 12288, number of training examples = 120)
Y_test -- test set, of shape (output size = 6, number of test examples = 120)
learning_rate -- learning rate of the optimization
num_epochs -- number of epochs of the optimization loop
minibatch_size -- size of a minibatch
print_cost -- True to print the cost every 100 epochs
Returns:
parameters -- parameters learnt by the model. They can then be used to predict.
"""
ops.reset_default_graph() # to be able to rerun the model without overwriting tf variables
tf.set_random_seed(1) # to keep consistent results
seed = 3 # to keep consistent results
(n_x, m) = X_train.shape # (n_x: input size, m : number of examples in the train set)
n_y = Y_train.shape[0] # n_y : output size
costs = [] # To keep track of the cost
# Create Placeholders of shape (n_x, n_y)
### START CODE HERE ### (1 line)
X, Y = create_placeholders(n_x, n_y)
### END CODE HERE ###
# Initialize parameters
### START CODE HERE ### (1 line)
parameters = initialize_parameters()
### END CODE HERE ###
# Forward propagation: Build the forward propagation in the tensorflow graph
### START CODE HERE ### (1 line)
Z3 = forward_propagation(X, parameters)
### END CODE HERE ###
# Cost function: Add cost function to tensorflow graph
### START CODE HERE ### (1 line)
cost = compute_cost(Z3, Y)
### END CODE HERE ###
# Backpropagation: Define the tensorflow optimizer. Use an AdamOptimizer.
### START CODE HERE ### (1 line)
optimizer = tf.train.AdamOptimizer(learning_rate = learning_rate).minimize(cost)
### END CODE HERE ###
# Initialize all the variables
init = tf.global_variables_initializer()
# Start the session to compute the tensorflow graph
with tf.Session() as sess:
# Run the initialization
sess.run(init)
# Do the training loop
for epoch in range(num_epochs):
epoch_cost = 0. # Defines a cost related to an epoch
num_minibatches = int(m / minibatch_size) # number of minibatches of size minibatch_size in the train set
seed = seed + 1
minibatches = random_mini_batches(X_train, Y_train, minibatch_size, seed)
for minibatch in minibatches:
# Select a minibatch
(minibatch_X, minibatch_Y) = minibatch
# IMPORTANT: The line that runs the graph on a minibatch.
# Run the session to execute the "optimizer" and the "cost", the feedict should contain a minibatch for (X,Y).
### START CODE HERE ### (1 line)
_ , minibatch_cost = sess.run([optimizer, cost], feed_dict={X: minibatch_X, Y: minibatch_Y})
### END CODE HERE ###
epoch_cost += minibatch_cost / num_minibatches
# Print the cost every epoch
if print_cost == True and epoch % 100 == 0:
print ("Cost after epoch %i: %f" % (epoch, epoch_cost))
if print_cost == True and epoch % 5 == 0:
costs.append(epoch_cost)
# plot the cost
plt.plot(np.squeeze(costs))
plt.ylabel('cost')
plt.xlabel('iterations (per tens)')
plt.title("Learning rate =" + str(learning_rate))
plt.show()
# lets save the parameters in a variable
parameters = sess.run(parameters)
print ("Parameters have been trained!")
# Calculate the correct predictions
correct_prediction = tf.equal(tf.argmax(Z3), tf.argmax(Y))
# Calculate accuracy on the test set
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
print ("Train Accuracy:", accuracy.eval({X: X_train, Y: Y_train}))
print ("Test Accuracy:", accuracy.eval({X: X_test, Y: Y_test}))
return parameters
跑了一次。。结果有点烂啊
Cost after epoch 0: 1.882638
Cost after epoch 100: 1.355498
Cost after epoch 200: 1.134945
Cost after epoch 300: 0.941005
Cost after epoch 400: 0.807623
Cost after epoch 500: 0.745780
Cost after epoch 600: 0.643747
Cost after epoch 700: 0.584362
Cost after epoch 800: 0.536316
Cost after epoch 900: 0.490915
Cost after epoch 1000: 0.472785
Cost after epoch 1100: 0.424659
Cost after epoch 1200: 0.400054
Cost after epoch 1300: 0.403058
Cost after epoch 1400: 0.382891
