[CS231n Assignment 3 #00] 第三次作业介绍

1.作业介绍

• 作业主页：Assignment #3
• 作业任务：在这次作业中，你将实现循环网络，并将它们应用于微软COCO数据集上的图像字幕。您还将探索在ImageNet上可视化预训练模型的特征的方法，以及实现风格迁移Style Transfer模型。最后，您将训练一个生成式对抗网络来生成看起来像训练数据集的图像!
• 作业目的：
  • 理解递归神经网络(RNNs)的结构，以及它们如何通过随时间共享权值对序列进行操作；
  • 理解并实现普通的RNNs和长短期记忆(LSTM)网络；
  • 了解如何结合卷积神经网络和递归网络来实现一个图像字幕系统；
  • 探索图像梯度的各种应用，包括显著性映射（saliency maps），欺骗图像（fooling images），类可视化（class visualizations）。
  • 理解并实现图像风格转换技术。
  • 了解如何训练和实现生成式对抗网络(GAN)，以生成类似于数据集样本的图像。

2.作业内容

• 00 下载数据
  运行cd cs231n/datasets ./get_assignment3_data.sh下载 the COCO captioning data, pretrained SqueezeNet model (TensorFlow-only), and a few ImageNet validation images。（Windows用户也可以自己去对应的文件找相关连接手动下载）

• 01 使用简单的RNN完成图像字幕(Image Captioning with Vanilla RNNs )
  The Jupyter notebook RNN_Captioning.ipynb will walk you through the implementation of an image captioning system on MS-COCO using vanilla recurrent networks.

• 02 使用LSTM完成图像字幕(Image Captioning with LSTMs )
  The Jupyter notebook LSTM_Captioning.ipynb will walk you through the implementation of Long-Short Term Memory (LSTM) RNNs, and apply them to image captioning on MS-COCO.

• 03 网络可视化:显著映射、类可视化和欺骗图像(Network Visualization: Saliency maps, Class Visualization, and Fooling Images)
  The Jupyter notebooks NetworkVisualization-TensorFlow.ipynb /NetworkVisualization-PyTorch.ipynb will introduce the pretrained SqueezeNet model, compute gradients with respect to images, and use them to produce saliency maps and fooling images.

• 04 风格迁移(Style Transfer)
  In the Jupyter notebooks StyleTransfer-TensorFlow.ipynb/StyleTransfer-PyTorch.ipynb you will learn how to create images with the content of one image but the style of another.

• 05 生成对抗网络(Generative Adversarial Networks)
  In the Jupyter notebooks GANS-TensorFlow.ipynb/GANS-PyTorch.ipynb you will learn how to generate images that match a training dataset, and use these models to improve classifier performance when training on a large amount of unlabeled data and a small amount of labeled data.