cs231n assignment 3 Q2 Image Captioning with Vanilla RNNs

文章目录

嫌啰嗦直接看代码
Q2 Image Captioning with Vanilla RNNs
- 一个给的工具代码里的bug
- - 问题展示
  - 问题解决思路
  - 解决办法
- rnn_step_forward
- - 题面
  - 解析
  - 代码
  - 输出
- rnn_step_backward
- - 题面
  - 解析
  - 代码
  - 输出
- rnn_forward
- - 题面
  - 解析
  - 代码
  - 输出
- rnn_backward
- - 题面
  - 解析
  - 代码
  - 输出
- word_embedding_forward
- - word embedding 技术解释
  - 题面
  - 解析
  - 代码
  - 输出
- word_embedding_backward
- - 题面
  - 解析
  - 代码
  - 输出
- CaptioningRNN .loss
- - 题面
  - 解析
  - 代码
  - 输出
- CaptioningRNN.sample
- - 题面
  - 解析
  - 代码
  - 输出
- 结语

嫌啰嗦直接看代码

Q2 Image Captioning with Vanilla RNNs

一个给的工具代码里的bug

image_from_url 里的报错

[WinError 32] 另一个程序正在使用此文件，进程无法访问。: 'C:\\Users\\Leezed\\AppData\\Local\\Temp\\tmp7r3fjusu'

问题展示

我在运行这段代码的时候就报错了另一个进程正在使用此文件，文件无法访问

问题解决思路

我一开始以为是img = imread(fname) 里的问题导致文件还在被占用，所以无法释放文件的所有权，导致os.remove(fname)无法删除。就是我以为img = imread(fname) 是另开了一个线程去读取图片，然后直接运行了os.remove,但是图片还没有读取完，导致占用没有被释放，所以删除失败
所以我一开始加了延时函数，time.sleep(5),但是还是同样的问题，正常情况下一张图片5秒肯定能读完了，我不死心再检查了一下，我直接吧img = imread(fname) 改成了 img =None ,结果还是报了同样的错，我就知道肯定不是它的问题了
我后来甚至开始怀疑是ff没有close的问题了，但是with as 语句会自动关闭ff，这就很奇怪了
最后我一步步排查觉得是tempfile.mkstemp的问题
查阅相关文档，这个函数返回的是两个参数，一个是fd，一个是fname，fd是文件描述符，fname是指生成的文件的绝对路径。

那么文件描述符是啥呢

 内核（kernel）利用文件描述符（file descriptor）来访问文件。文件描述符是非负整数。
 打开现存文件或新建文件时，内核会返回一个文件描述符。
 读写文件也需要使用文件描述符来指定待读写的文件。

文件描述符在同一进程下与文件是对应的，一个描述符只指向一个文件，但是一个文件可以被多个文件描述符关联。
同一进程下，文件描述符是不可重复的。但是不同进程可以有一样的文件描述符。它们也可以指向不同的文件。

因此如果直接使用os.remove(fname)删除文件的话，文件描述符还在我们的内存里，而文件描述符还在内存里说明文件还被占用着，所以无法删除

那我们应该怎么删除临时文件呢

首先需要使用os.close(fd) 方法用于关闭指定的文件描述符 fd,
然后再使用os.remove(fname)删除临时文件。

解决办法

在os.remove(fname)之前加一句代码 os.close(_)就好了，如下图所示

rnn_step_forward

题面

让我们完成循环神经网络的前向一步

解析

看课吧，我觉得课程里讲的很详细了，或者看代码注释

代码

def rnn_step_forward(x, prev_h, Wx, Wh, b):
    """Run the forward pass for a single timestep of a vanilla RNN using a tanh activation function.

    The input data has dimension D, the hidden state has dimension H,
    and the minibatch is of size N.

    Inputs:
    - x: Input data for this timestep, of shape (N, D)
    - prev_h: Hidden state from previous timestep, of shape (N, H)
    - Wx: Weight matrix for input-to-hidden connections, of shape (D, H)
    - Wh: Weight matrix for hidden-to-hidden connections, of shape (H, H)
    - b: Biases of shape (H,)

    Returns a tuple of:
    - next_h: Next hidden state, of shape (N, H)
    - cache: Tuple of values needed for the backward pass.
    """
    next_h, cache = None, None
    ##############################################################################
    # TODO: Implement a single forward step for the vanilla RNN. Store the next  #
    # hidden state and any values you need for the backward pass in the next_h   #
    # and cache variables respectively.                                          #
    ##############################################################################
    # *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****

    next_h = x @ Wx + prev_h @ Wh + b
    next_h = np.tanh(next_h)
    cache = (x, prev_h, Wx, Wh, b, next_h)

    # *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
    ##############################################################################
    #                               END OF YOUR CODE                             #
    ##############################################################################
    return next_h, cache

输出

rnn_step_backward

题面

让我们完成循环神经网络的后向一步

解析

不赘述了，看代码吧

tanh函数求导公式可以看下面这个连接的文章
激活函数tanh(x)求导

代码

def rnn_step_backward(dnext_h, cache):
    """Backward pass for a single timestep of a vanilla RNN.

    Inputs:
    - dnext_h: Gradient of loss with respect to next hidden state, of shape (N, H)
    - cache: Cache object from the forward pass

    Returns a tuple of:
    - dx: Gradients of input data, of shape (N, D)
    - dprev_h: Gradients of previous hidden state, of shape (N, H)
    - dWx: Gradients of input-to-hidden weights, of shape (D, H)
    - dWh: Gradients of hidden-to-hidden weights, of shape (H, H)
    - db: Gradients of bias vector, of shape (H,)
    """
    dx, dprev_h, dWx, dWh, db = None, None, None, None, None
    ##############################################################################
    # TODO: Implement the backward pass for a single step of a vanilla RNN.      #
    #                                                                            #
    # HINT: For the tanh function, you can compute the local derivative in terms #
    # of the output value from tanh.                                             #
    ##############################################################################
    # *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****

    x, prev_h, Wx, Wh, b, next_h = cache
    # 求导 tanh(x) = (1 - tanh(x)^2) * dx
    dnext_h = dnext_h * (1 - next_h ** 2)
    dx = dnext_h @ Wx.T
    dprev_h = dnext_h @ Wh.T
    dWx = x.T @ dnext_h
    dWh = prev_h.T @ dnext_h
    db = np.sum(dnext_h, axis=0)

    # *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
    ##############################################################################
    #                               END OF YOUR CODE                             #
    ##############################################################################
    return dx, dprev_h, dWx, dWh, db

输出

rnn_forward

题面

解析

看代码吧

代码

def rnn_forward(x, h0, Wx, Wh, b):
    """Run a vanilla RNN forward on an entire sequence of data.
    
    We assume an input sequence composed of T vectors, each of dimension D. The RNN uses a hidden
    size of H, and we work over a minibatch containing N sequences. After running the RNN forward,
    we return the hidden states for all timesteps.

    Inputs:
    - x: Input data for the entire timeseries, of shape (N, T, D)
    - h0: Initial hidden state, of shape (N, H)
    - Wx: Weight matrix for input-to-hidden connections, of shape (D, H)
    - Wh: Weight matrix for hidden-to-hidden connections, of shape (H, H)
    - b: Biases of shape (H,)

    Returns a tuple of:
    - h: Hidden states for the entire timeseries, of shape (N, T, H)
    - cache: Values needed in the backward pass
    """
    h, cache = None, None
    ##############################################################################
    # TODO: Implement forward pass for a vanilla RNN running on a sequence of    #
    # input data. You should use the rnn_step_forward function that you defined  #
    # above. You can use a for loop to help compute the forward pass.            #
    ##############################################################################
    # *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****

    # 获取维度
    N, T, _ = x.shape
    _, H = h0.shape
    # 初始化
    h = np.zeros((N, T, H))
    cache = []
    # 前向传播
    for i in range(T):
        if i == 0:
            h[:, i, :], cache_i = rnn_step_forward(x[:, i, :], h0, Wx, Wh, b)
        else:
            h[:, i, :], cache_i = rnn_step_forward(x[:, i, :], h[:, i - 1, :], Wx, Wh, b)
        cache.append(cache_i)

    cache = tuple(cache)

    # *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
    ##############################################################################
    #                               END OF YOUR CODE                             #
    ##############################################################################
    return h, cache

输出

rnn_backward

题面

解析

看代码吧，认真听课的话肯定能理解的

代码

def rnn_backward(dh, cache):
    """Compute the backward pass for a vanilla RNN over an entire sequence of data.

    Inputs:
    - dh: Upstream gradients of all hidden states, of shape (N, T, H)
    
    NOTE: 'dh' contains the upstream gradients produced by the 
    individual loss functions at each timestep, *not* the gradients
    being passed between timesteps (which you'll have to compute yourself
    by calling rnn_step_backward in a loop).

    Returns a tuple of:
    - dx: Gradient of inputs, of shape (N, T, D)
    - dh0: Gradient of initial hidden state, of shape (N, H)
    - dWx: Gradient of input-to-hidden weights, of shape (D, H)
    - dWh: Gradient of hidden-to-hidden weights, of shape (H, H)
    - db: Gradient of biases, of shape (H,)
    """
    dx, dh0, dWx, dWh, db = None, None, None, None, None
    ##############################################################################
    # TODO: Implement the backward pass for a vanilla RNN running an entire      #
    # sequence of data. You should use the rnn_step_backward function that you   #
    # defined above. You can use a for loop to help compute the backward pass.   #
    ##############################################################################
    # *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****

    # 获取维度
    N, T, H = dh.shape
    D, _ = cache[0][2].shape
    # 初始化
    dx = np.zeros((N, T, D))
    dh0 = np.zeros((N, H))
    dWx = np.zeros((D, H))
    dWh = np.zeros((H, H))
    db = np.zeros((H,))
    # 反向传播
    for i in range(T - 1, -1, -1):
        dx[:, i, :], dh0, dWx_i, dWh_i, db_i = rnn_step_backward(dh[:, i, :] + dh0, cache[i])
        dWx += dWx_i
        dWh += dWh_i
        db += db_i

    # *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
    ##############################################################################
    #                               END OF YOUR CODE                             #
    ##############################################################################
    return dx, dh0, dWx, dWh, db

输出

word_embedding_forward

word embedding 技术解释

题面

解析