深度学习——GAN的基础理解
一.GAN的基本要素
1.真实数据集，初始化虚假数据集（噪音）
2.生成器，鉴别器：
生成器：
输入：原始数据的维数（一条数据）
输出：原始数据的维数（一条数据）
除了最后一层都要经sigmoid（）
鉴别器：
输入：原始数据的维数（一个batch的数据）
输出：一维（以判别结果的真假）
经sigmoid,在（0,1）范围（切确地说输入层，隐藏层，输出层都要经过）
3.训练环（每一次epoch）：
每一次eopch里面，D和G都要训练，并且各自训练多次。（在代码中见到的情况是先D后G)
生成器训练周期：
真实数据训练：
（1）判别器出来的结果向“1”靠近
（2）反向传播
生成器产生数据训练：
（1）冻结生成器并产生数据
（2）判别器出来的结果向“0”靠近
（3）反向传播
鉴别器训练周期：
（1）不冻结生成器并产生数据（喂如D前转置了以下）
（2）判别器出来的结果向“1”靠近
（3）反向传播
二.GAN的损失函数
![\large min_Gmax_D E _r_e_a_l[logD(x)]+E_f_a_k_e[log(1-D(z))]](http://img.e-com-net.com/image/info8/50a70e11dcb14aa19539a8e37f40da8c.gif){kind=small}
解释：
对于判别器来说：损失函数的值越大越好
包括式子的两项
对于生成器来说：损失函数的值越小越好
只包括式子的后一项
三.公式和代码的转换技巧
1.损失函数的max,min，“+”，“-”等等：基本只体现在，代码的criterion()函数里。在代码里面的思路很简单：criterion()就是为了度量你想要的和真实训练出来的差距;反向传播的过程就是改变网络的参数，使criterion()的值越来越小（即网络训练出来的会更加靠近你想要的）。
PS:所以看文章的时候要小心，损失函数等max,还是min（老师和师兄都落坑了啊）。
2.网络的结构框图：基本只体现在代码的训练周期里。
3.看代码，运行代码真的是和刷题一样重要！！！论文摘要就像教科书里的重点，论文文本基本是玄学。只看课本文字不刷题，想想后果就知道，基本学不会本质的东西并且还浪费时间。
四.文章所辅助的代码
#!/usr/bin/env python
# Generative Adversarial Networks (GAN) example in PyTorch. Tested with PyTorch 0.4.1, Python 3.6.7 (Nov 2018)
# See related blog post at https://medium.com/@devnag/generative-adversarial-networks-gans-in-50-lines-of-code-pytorch-e81b79659e3f#.sch4xgsa9
####################################################导入包################################################################
import numpy as np
import os
import torch
import torch.nn as nn
import torch.optim as optim
from torch.autograd import Variable
matplotlib_is_available = True
try:
from matplotlib import pyplot as plt
except ImportError:
print("Will skip plotting; matplotlib is not available.")
matplotlib_is_available = False
####################################################导入包################################################################
####################################################获取数据################################################################
# 使用第二张GPU卡
os.environ["CUDA_VISIBLE_DEVICES"] = "1"
# Data params
data_mean = 4
data_stddev = 1.25
# ### Uncomment only one of these to define what data is actually sent to the Discriminator
#(name, preprocess, d_input_func) = ("Raw data", lambda data: data, lambda x: x)
#(name, preprocess, d_input_func) = ("Data and variances", lambda data: decorate_with_diffs(data, 2.0), lambda x: x * 2)
#(name, preprocess, d_input_func) = ("Data and diffs", lambda data: decorate_with_diffs(data, 1.0), lambda x: x * 2)
(name, preprocess, d_input_func) = ("Only 4 moments", lambda data: get_moments(data), lambda x: 4)
print("Using data [%s]" % (name))
# ##### DATA: Target data and generator input data
def get_distribution_sampler(mu, sigma):
return lambda n: torch.Tensor(np.random.normal(mu, sigma, (1, n))) # Gaussian
def get_generator_input_sampler():
return lambda m, n: torch.rand(m, n) # Uniform-dist data into generator, _NOT_ Gaussian
####################################################获取数据################################################################
# ##### MODELS: Generator model and discriminator model
####################################################生成器和判别器################################################################
class Generator(nn.Module):
def __init__(self, input_size, hidden_size, output_size, f):
super(Generator, self).__init__()
self.map1 = nn.Linear(input_size, hidden_size)
self.map2 = nn.Linear(hidden_size, hidden_size)
self.map3 = nn.Linear(hidden_size, output_size)
self.f = f
def forward(self, x):
x = self.map1(x)
x = self.f(x)
x = self.map2(x)
x = self.f(x)
x = self.map3(x)
return x
class Discriminator(nn.Module):
def __init__(self, input_size, hidden_size, output_size, f):
super(Discriminator, self).__init__()
self.map1 = nn.Linear(input_size, hidden_size)
self.map2 = nn.Linear(hidden_size, hidden_size)
self.map3 = nn.Linear(hidden_size, output_size)
self.f = f
def forward(self, x):
x = self.f(self.map1(x))
x = self.f(self.map2(x))
return self.f(self.map3(x))
####################################################生成器和判别器################################################################
def extract(v):
return v.data.storage().tolist()
def stats(d):
return [np.mean(d), np.std(d)]
def get_moments(d):
# Return the first 4 moments of the data provided
mean = torch.mean(d) #生成的高斯分布求均值
diffs = d - mean
var = torch.mean(torch.pow(diffs, 2.0))
std = torch.pow(var, 0.5) #生成的高斯分布求标准差元素与
zscores = diffs / std
skews = torch.mean(torch.pow(zscores, 3.0))
kurtoses = torch.mean(torch.pow(zscores, 4.0)) - 3.0 # excess kurtosis, should be 0 for Gaussian
final = torch.cat((mean.reshape(1,), std.reshape(1,), skews.reshape(1,), kurtoses.reshape(1,))) #一个向量，有四个元素，如代码
return final
def decorate_with_diffs(data, exponent, remove_raw_data=False):
mean = torch.mean(data.data, 1, keepdim=True)
mean_broadcast = torch.mul(torch.ones(data.size()), mean.tolist()[0][0])
diffs = torch.pow(data - Variable(mean_broadcast), exponent)
if remove_raw_data:
return torch.cat([diffs], 1)
else:
return torch.cat([data, diffs], 1)
def train():
# Model parameters
g_input_size = 1 # Random noise dimension coming into generator, per output vector
g_hidden_size = 5 # Generator complexity
g_output_size = 1 # Size of generated output vector
d_input_size = 500 # Minibatch size - cardinality of distributions
d_hidden_size = 10 # Discriminator complexity
d_output_size = 1 # Single dimension for 'real' vs. 'fake' classification
minibatch_size = d_input_size
d_learning_rate = 1e-3
g_learning_rate = 1e-3
sgd_momentum = 0.9
num_epochs = 5000
print_interval = 100
d_steps = 20
g_steps = 20
dfe, dre, ge = 0, 0, 0
d_real_data, d_fake_data, g_fake_data = None, None, None
discriminator_activation_function = torch.sigmoid
generator_activation_function = torch.tanh
d_sampler = get_distribution_sampler(data_mean, data_stddev)
gi_sampler = get_generator_input_sampler()
G = Generator(input_size=g_input_size,
hidden_size=g_hidden_size,
output_size=g_output_size,
f=generator_activation_function)
D = Discriminator(input_size=d_input_func(d_input_size),
hidden_size=d_hidden_size,
output_size=d_output_size,
f=discriminator_activation_function)
criterion = nn.BCELoss() # Binary cross entropy: http://pytorch.org/docs/nn.html#bceloss
d_optimizer = optim.SGD(D.parameters(), lr=d_learning_rate, momentum=sgd_momentum)
g_optimizer = optim.SGD(G.parameters(), lr=g_learning_rate, momentum=sgd_momentum)
####################################################训练################################################################
for epoch in range(num_epochs):
for d_index in range(d_steps): #在每一个训练期里面，D训练20次
# 1. Train D on real+fake
D.zero_grad() #将梯度都置为0,注意是梯度不是网络参数
# 1A: Train D on real
d_real_data = Variable(d_sampler(d_input_size)) #产生高斯分布的数据，是随机散乱排列的
d_real_decision = D(preprocess(d_real_data)) #这里直接调用forward（）？？
d_real_error = criterion(d_real_decision, Variable(torch.ones([1,1]))) # ones = true
d_real_error.backward() # compute/store gradients, but don't change params
# 1B: Train D on fake
d_gen_input = Variable(gi_sampler(minibatch_size, g_input_size))
d_fake_data = G(d_gen_input).detach() # detach to avoid training G on these labels
d_fake_decision = D(preprocess(d_fake_data.t()))
d_fake_error = criterion(d_fake_decision, Variable(torch.zeros([1,1]))) # zeros = fake
d_fake_error.backward()
d_optimizer.step() # Only optimizes D's parameters; changes based on stored gradients from backward()
dre, dfe = extract(d_real_error)[0], extract(d_fake_error)[0] #每一次的损失
for g_index in range(g_steps): #在每一个训练器里面G训练20次
# 2. Train G on D's response (but DO NOT train D on these labels)
G.zero_grad()
gen_input = Variable(gi_sampler(minibatch_size, g_input_size)) #随机产生（0,1）间的数，注意不是高斯分布
g_fake_data = G(gen_input) #不明感觉输出维度不对
dg_fake_decision = D(preprocess(g_fake_data.t()))
g_error = criterion(dg_fake_decision, Variable(torch.ones([1,1]))) # Train G to pretend it's genuine
g_error.backward()
g_optimizer.step() # Only optimizes G's parameters
ge = extract(g_error)[0]
if epoch % print_interval == 0:
print("Epoch %s: D (%s real_err, %s fake_err) G (%s err); Real Dist (%s), Fake Dist (%s) " %
(epoch, dre, dfe, ge, stats(extract(d_real_data)), stats(extract(d_fake_data))))
####################################################训练################################################################
####################################################画图################################################################
if matplotlib_is_available:
print("Plotting the generated distribution...")
values = extract(g_fake_data) #G产生出来的数据
print(" Values: %s" % (str(values)))
plt.hist(values, bins=50) #画直方图，参数一：输入数据，参数二:条的个数
plt.xlabel('Value')
plt.ylabel('Count')
plt.title('Histogram of Generated Distribution')
plt.grid(True) #生成网格
plt.show()
####################################################画图################################################################
train()
结果：
代码输出图像近似接近于高斯分布。
