Improved Techniques for Training GANs 代码执行中的坑 python3.6 tf环境
improved-gan-master：中的代码：
**Status:** Archive (code is provided as-is, no updates expected)
# improved-gan
code for the paper "Improved Techniques for Training GANs"
MNIST, SVHN, CIFAR10 experiments in the mnist_svhn_cifar10 folder
imagenet experiments in the imagenet folder
# MNIST/SVHN/CIFAR-10 experiments
This part of the code is built using Theano and Lasagne. Any recent version of these packages should work for running the code.
The experiments are run using the train*.py files. All experiments perform semi-supervised learning with a set of labeled examples and a set of unlabeled examples. There are two kinds of models: the "feature matching" models that achieve the best predictive performance, and the "minibatch discrimination" models that achieve the best image quality.
The provided train*.py files each train a single model for a single random labeled/unlabeled data split and a single random parameter initialization. To reproduce our results using ensembling / averaging over random seeds, you can run these files multiple times using different inputs for the "seed" and "seed_data" arguments, and then combine the results.
This code is still being developed and subject to change.
实验室电脑遇到问题：
Traceback (most recent call last):
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py", line 4, in
import theano as th
ModuleNotFoundError: No module named 'theano'
Process finished with exit code 1
尽量用conda安装theano，尤其是在windows：conda install theano pygpu 安装
安装后：
Traceback (most recent call last):
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py", line 4, in
import theano as th
File "/home/gis/anaconda3/lib/python3.6/site-packages/theano/__init__.py", line 88, in
from theano.configdefaults import config
File "/home/gis/anaconda3/lib/python3.6/site-packages/theano/configdefaults.py", line 137, in
in_c_key=False)
File "/home/gis/anaconda3/lib/python3.6/site-packages/theano/configparser.py", line 287, in AddConfigVar
configparam.__get__(root, type(root), delete_key=True)
File "/home/gis/anaconda3/lib/python3.6/site-packages/theano/configparser.py", line 335, in __get__
self.__set__(cls, val_str)
File "/home/gis/anaconda3/lib/python3.6/site-packages/theano/configparser.py", line 346, in __set__
self.val = self.filter(val)
File "/home/gis/anaconda3/lib/python3.6/site-packages/theano/configdefaults.py", line 116, in filter
'You are tring to use the old GPU back-end. '
ValueError: You are tring to use the old GPU back-end. It was removed from Theano. Use device=cuda* now. See https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29 for more information.
Process finished with exit code 1
gedit ~/.theanorc # 如果不存在该文件则会建立,不需要使用sudo命令！
1
然后在 .theanorc 中添加配置如下：
[global]
floatX=float32
device=cuda1
root=/usr/local/cuda-9.0 # 这个是自己cuda的安装路径
[gpuarray]
preallocate = 1
---------------------
原来的是这个内容：去掉
[global]
device=gpu
floatX=float32
[nvcc]
fastmath=True
[cuda]
root=/usr/local/cuda/bin/
其实我说的坑就是这里，因为网上好多设置GPU的时候将这里device=gpu ，这样的话就会导致错误比如类似这种，说是old GPU back-end。另外还有一些设置nvcc变量的。
...
ValueError: You are tring to use the old GPU back-end. It was removed from Theano. Use device=cuda* now. See for more information.
(py27) iMacvanQuinten:AI_Writer quintendewilde$ python -c "from theano.sandbox.cuda.dnn import dnn_available as d; print(d() or d.msg)"
Traceback (most recent call last)
...
其实哩，不能说是上面的device=gpu 错误，只能说Theano版本更新了，以前的不支持了。其实我们可以看一下Theano的发行版本就知道了，2017/08/09已经移除了device=gpu了。
2017/11/15: Release of Theano 1.0.0. Everybody is encouraged to update.
2017/10/30: Release of Theano 1.0.0rc1, new features and many bugfixes, final release to coming.
2017/10/16: Release of Theano 0.10.0beta4, new features and many bugfixes, release candidate to coming.
2017/09/28: IMPORTANT: MILA will stop developing Theano and the next release (renamed to 1.0) will be the last main release.
2017/09/20: Release of Theano 0.10.0beta3, new features and many bugfixes, release candidate to coming.
2017/09/07: Release of Theano 0.10.0beta2, new features and many bugfixes, release candidate to coming.
2017/08/09: Release of Theano 0.10.0beta1, many improvements and bugfixes, release candidate to coming.
Removed support for the old (device=gpu) backend. Use the new backend (device=cuda) for gpu computing. See Converting to the new gpu back end(gpuarray) for help with conversion.
2017/03/20: Release of Theano 0.9.0. Everybody is encouraged to update.
2017/03/13: Release of Theano 0.9.0rc4, with crash fixes and bug fixes.
2017/03/06: Release of Theano 0.9.0rc3, with crash fixes, bug fixes and improvements.
2017/02/27: Release of Theano 0.9.0rc2, with crash fixes, bug fixes and improvements.
2017/02/20: Release of Theano 0.9.0rc1, many improvements and bugfixes, final release to coming.
2017/01/24: Release of Theano 0.9.0beta1, many improvements and bugfixes, release candidate to coming.
2016/05/09: New technical report on Theano: Theano: A Python framework for fast computation of mathematical expressions. This is the new preferred reference.
2016/04/21: Release of Theano 0.8.2, adding support for CuDNN v5
...
那么我们的解决方式可以参考官网的API介绍，里面一段话说的很清楚了
你可以使用以下设备：
String value: either 'cpu', 'cuda', 'cuda0', 'cuda1', 'opencl0:0', 'opencl0:1', ...
Default device for computations. If 'cuda*, change the default to try to move computation to the GPU using CUDA libraries. If 'opencl*', the OpenCL libraries will be used. To let the driver select the device, use 'cuda' or 'opencl'. If we are not able to use the GPU, either we fall back on the CPU, or an error is raised, depending on the force_device flag.
另外有一些Theano config keys 也是被废除了， Converting to the new gpu back end(gpuarray)
The following Theano config keys sections don't have any effect on the new backend and should be removed:
1. nvcc.*
2. cuda.root
3. lib.cnmem (replace by gpuarray.preallocate)
Important: The default changed to be faster, but cause more memory fragmentation. To keep the speed and remove the fragmentation, use the flag gpuarray.preallocate=1 (or any value greater then 0, see the dot. To have the old default of Theano, use the flag: gpuarray.preallocate=-1
然而这样配置完了之后，我运行了theano的代码，并没与调用gpu. 我们还需要将CUDA的路径添加到环境变量中。参考这两篇文章Linux下非root用户安装Theano并配置GPU(以及集群上安装自己所需要的python依赖， Checking cuda for theano errors.. #839
vim ~/.bashrc
然后添加代码：
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/
export PATH=$PATH:/usr/local/cuda/bin
这样我们的环境算是搭建好了。
2. 测试
为了测试theano在cpu和gpu分别的表现，我们按照网上推荐的代码，测试Theano使用GPU并行计算，以验证环境搭建是否成功
首先我们创建一个test.py文件
添加如下内容：
from theano import function, config, shared, sandbox
import theano.tensor as T
import numpy
import time
vlen = 10 * 30 * 768 # 10 x #cores x # threads per core
iters = 1000
rng = numpy.random.RandomState(22)
x = shared(numpy.asarray(rng.rand(vlen), config.floatX))
f = function([], T.exp(x))
print(f.maker.fgraph.toposort())
t0 = time.time()
for i in range(iters):
r = f()
t1 = time.time()
print("Looping %d times took %f seconds" % (iters, t1 - t0))
print("Result is %s" % (r,))
if numpy.any([isinstance(x.op, T.Elemwise) for x in f.maker.fgraph.toposort()]):
print('Used the cpu')
else:
print('Used the gpu')
然后保存文件。
2.1 使用cpu跑这个代码
首先我们更改 .theanorc 这个文件，将其中device=cpu，
然后我们在终端输入命令： python test.py 测试结果如下：
需要时间2.53s
2.2 使用gpu跑这个代码
首先我们更改 .theanorc 这个文件，将其中device=cuda0，
然后我们在终端输入命令： python test.py 测试结果如下：
需要时间0.27s
3. 展望
好像theano在去年已经不进行更新了，MILA will stop developing Theano，如何看待 Yoshua Bengio 宣布停止 Theano 维护与开发？经历了十个年头，已经完成了他的使命了，好多开发人员都转去开发其他框架了，比如tensorflow了。其实我只是取跑一个实验，咋就写了这么多了～～
提示错误：
Traceback (most recent call last):
File "/home/gis/anaconda3/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 227, in
use(config.device)
File "/home/gis/anaconda3/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 214, in use
init_dev(device, preallocate=preallocate)
File "/home/gis/anaconda3/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 140, in init_dev
context.free_gmem / MB))
RuntimeError: Trying to preallocate 10618 MB of GPU memory while only 95 MB are available.
Traceback (most recent call last):
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py", line 7, in
import lasagne
ModuleNotFoundError: No module named 'lasagne' 人脸识别库
Process finished with exit code 1
Lasagne
安装
Lasagne是写在Theano之上的库包,可以使用户更简单的使用深层学习训练 这里将要安装的是Lasagne 0.2.dev1版本,直接执行 >conda install -c http://conda.anaconda.org/toli lasagne
教程
这里有Lasagne tutorial 和 Lasagne tutorial2的简单ipython教程
需要下载的东西比较多，折腾半天。
重新执行：又有错误：
Using cuDNN version 7005 on context None
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
File "/home/gis/anaconda3/lib/python3.5/site-packages/theano/gpuarray/__init__.py", line 227, in
use(config.device)
File "/home/gis/anaconda3/lib/python3.5/site-packages/theano/gpuarray/__init__.py", line 214, in use
init_dev(device, preallocate=preallocate)
File "/home/gis/anaconda3/lib/python3.5/site-packages/theano/gpuarray/__init__.py", line 140, in init_dev
context.free_gmem / MB))
RuntimeError: Trying to preallocate 10618 MB of GPU memory while only 95 MB are available.
Traceback (most recent call last):
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py", line 7, in
import lasagne
File "/home/gis/anaconda3/lib/python3.5/site-packages/lasagne/__init__.py", line 19, in
from . import layers
File "/home/gis/anaconda3/lib/python3.5/site-packages/lasagne/layers/__init__.py", line 7, in
from .pool import *
File "/home/gis/anaconda3/lib/python3.5/site-packages/lasagne/layers/pool.py", line 6, in
from theano.tensor.signal import downsample
ImportError: cannot import name 'downsample'
Process finished with exit code 1
在学习http://blog.csdn.net/u012162613/article/details/43277187时，运行代码，发现无法加载downsample模型
from theano.tensor.signal import downsample
1
ImportError: cannot import name ‘downsample’
查找资料发现downsample已经改为pool
将上面代码改为from theano.tensor.signal import pool
代码中运用到downsample的地方也要改掉
# 子采样
pooled_out = downsample.max_pool_2d(
input=conv_out,
ds=poolsize,
ignore_border=True
)
1
2
3
4
5
6
这里直接将downsample改为pool会出错，因为里面相应的函数有变化
查看pool.py的源码，发现downsample.max_pool_2d()与pool. pool_2d()函数功能相同，用ws代替ds
# 子采样
pooled_out = pool.pool_2d(
input=conv_out,
ws=poolsize,
ignore_border=True
)
1
2
3
4
5
6
运行后，发现问题已经解决。
---------------------
作者：陌筱北
来源：CSDN
原文：https://blog.csdn.net/moxiaobeiMM/article/details/75015408
版权声明：本文为博主原创文章，转载请附上博文链接！
后来
#from lasagne.layers import dnn
#import nn
两行一直有问题，注释了。
再运行提示：
出现这个错误，ImportError: No module named 'cPickle' 因为 python3不支持，改成了pickle即可。
然后又出错：
/home/gis/anaconda3/bin/python /home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py
Using cuDNN version 7005 on context None
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
File "/home/gis/anaconda3/lib/python3.5/site-packages/theano/gpuarray/__init__.py", line 227, in
use(config.device)
File "/home/gis/anaconda3/lib/python3.5/site-packages/theano/gpuarray/__init__.py", line 214, in use
init_dev(device, preallocate=preallocate)
File "/home/gis/anaconda3/lib/python3.5/site-packages/theano/gpuarray/__init__.py", line 140, in init_dev
context.free_gmem / MB))
RuntimeError: Trying to preallocate 10618 MB of GPU memory while only 97 MB are available.
Namespace(batch_size=100, count=400, data_dir='./data/cifar-10-python', learning_rate=0.0003, seed=1, seed_data=1, unlabeled_weight=1.0)
Traceback (most recent call last):
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py", line 35, in
trainx, trainy = cifar10_data.load(args.data_dir, subset='train')
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/cifar10_data.py", line 34, in load
train_data = [unpickle(os.path.join(data_dir,'cifar-10-batches-py/data_batch_' + str(i))) for i in range(1,6)]
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/cifar10_data.py", line 34, in
train_data = [unpickle(os.path.join(data_dir,'cifar-10-batches-py/data_batch_' + str(i))) for i in range(1,6)]
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/cifar10_data.py", line 27, in unpickle
d = pickle.load(fo)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 6: ordinal not in range(128)
Process finished with exit code 1
也就是说pickle.load()默认解码是以encoding=”ASCII”解码的,而我们要载入的文件并不是以”ASCII”形式存储的，所以要改变参数encoding=” ”
解决方式如下：
pickle.load(f,encoding='bytes')
1
这样编码可以用“字节”来读取这些8位字符串实例作为字节对象。
/home/gis/anaconda3/bin/python /home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py
Using cuDNN version 7005 on context None
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
File "/home/gis/anaconda3/lib/python3.5/site-packages/theano/gpuarray/__init__.py", line 227, in
use(config.device)
File "/home/gis/anaconda3/lib/python3.5/site-packages/theano/gpuarray/__init__.py", line 214, in use
init_dev(device, preallocate=preallocate)
File "/home/gis/anaconda3/lib/python3.5/site-packages/theano/gpuarray/__init__.py", line 140, in init_dev
context.free_gmem / MB))
RuntimeError: Trying to preallocate 10618 MB of GPU memory while only 97 MB are available.
Namespace(batch_size=100, count=400, data_dir='./data/cifar-10-python', learning_rate=0.0003, seed=1, seed_data=1, unlabeled_weight=1.0)
Traceback (most recent call last):
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py", line 35, in
trainx, trainy = cifar10_data.load(args.data_dir, subset='train')
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/cifar10_data.py", line 34, in load
train_data = [unpickle(os.path.join(data_dir,'cifar-10-batches-py/data_batch_' + str(i))) for i in range(1,6)]
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/cifar10_data.py", line 34, in
train_data = [unpickle(os.path.join(data_dir,'cifar-10-batches-py/data_batch_' + str(i))) for i in range(1,6)]
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/cifar10_data.py", line 29, in unpickle
return {'x': np.cast[np.float32]((-127.5 + d['data'].reshape((10000,3,32,32)))/128.), 'y': np.array(d['labels']).astype(np.uint8)}
KeyError: 'data'
Process finished with exit code 1
因为这个问题就是说字典datadict里面没有data这个字段罢了，所以百度了也没有办法。后来进行了debug，观察了datadict的数据发现，在key值data和labels 的前面都有一个b，后来了我就加了一个b在data和labels 前面。如下
Y = datadict[b'labels']
X = datadict[b'data']
然后正常了……
百度之后才发现这个是python2和python3的问题，我使用了官方给的一个代码库，不过这个是用python2写的，而官方的训练数据也是使用python2生成的。结果我使用的python3去运行，就出现了这个错误。
这里是百度之后的结果：
b:bytes
python3.x里默认的str是unicode, bytes是py2.x的str, b''前缀代表的就是bytes
python2.x里, b前缀没什么具体意义，只是为了兼容python3.x的这种写法
1
2
3
参考博客
所以我这里加上b''前缀就可以让python2的代码兼容python3了。
最后再说一句：坑的一匹的python2和3，简直坑小白神器
---------------------
/home/gis/anaconda3/bin/python /home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py
Using cuDNN version 7005 on context None
Preallocating 10618/11177 Mb (0.950000) on cuda
Mapped name None to device cuda: GeForce GTX 1080 Ti (0000:01:00.0)
Traceback (most recent call last):
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py", line 10, in
from lasagne.layers import dnn
File "/home/gis/anaconda3/lib/python3.5/site-packages/lasagne/layers/dnn.py", line 2, in
from theano.sandbox.cuda import dnn
File "/home/gis/anaconda3/lib/python3.5/site-packages/theano/sandbox/cuda/__init__.py", line 6, in
"You are importing theano.sandbox.cuda. This is the old GPU back-end and "
unittest.case.SkipTest: You are importing theano.sandbox.cuda. This is the old GPU back-end and is removed from Theano. Use Theano 0.9 to use it. Even better, transition to the new GPU back-end! See https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
Process finished with exit code 1
一、搭建环境
安装：pip install theano==0.9.0
卸载：pip uninstall theano
查看版本号：import theano
theano.__version__
设置gpu\cpu：
方法1：vim /root/.theanorc
[global]
model=FAST_RUN (该模式运行速度快)
device=cuda1 (或者gpu或cpu)
floatX=float32
[blas]
ldflags=-L/usr/lib/libblas.so
方法2：
THEANO_FLAGS=mode=FAST_RUN,device=cuda,floatX=float32 python test_new.py
备注：theano0.9以上版本，使用gpu新后端，device=gpu或cpu或cuda(新后端)
#test_theano_gpu.py测试gpu/cpu，https://www.cnblogs.com/shouhuxianjian/p/4590224.html
THEANO_FLAGS=mode=FAST_RUN,device=cuda,floatX=float32 python test_theano_gpu.py
THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 python test_theano_gpu.py
THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python test_theano_gpu.py
tensorflow和theano同时要使用GPU，如何设置？
如果.theanorc中device设置为gpu，那么tensorflow将无法使用GPU；
如果.theanorc中device设置为cuda，那么theano在第二次调用中将无法使用GPU；
因此，如果tensorflow和theano同时要使用GPU，.theanorc中device必须设置为cuda，而且指明哪一个cuda。如上面的device=cuda1。
与此同时，tensorflow不需要特别指定GPU。
pygpu安装：
git clone https://github.com/Theano/libgpuarray.git
cd libgpuarray
mkdir Build
cd Build
cmake .. -DCMAKE_BUILD_TYPE=Release
make
make install
cd ..
python setup.py build
python setup.py install
sudo ldconfig
备注：有测试过pygpu==0.6.5与theano==0.9.0 或者pygpu==0.7.5与theano==1.0.0 OK.
pygpu版本如果为0.65，那么theano必须为0.9。如果为1.0，不兼容。无法成功import theano。
pygpu版本如果为0.75，那么theano必须为1.0。如果为0.9，不兼容。无法正常初始化pygpu。
例子：DeepAlgnmentNetwork： theano==0.9.0
error:immporting theano: AttributeError: 'module' object has no attribute 'find_graphviz'
解决方案：sudo pip uninstall -y pydot 或者 pip install pydot-ng
安装lasagne：https://github.com/Lasagne/Lasagne (深度框架)
pip install https://github.com/Lasagne/Lasagne/archive/master.zip
pip install theano==0.9.0
安装pygpu
二、基本用法：
theano.tensor常用数据类型:
有double、int、uchar、float等,float是因为GPU一般是float32类型.
数值：iscalar(int32)、fscalar(float32)、wscalar(int16）、bscalar(int8)、lscalar(int64)
a = T.scalar() #print (a.dtype) #float32
一维向量：ivector(int 类型的向量)、fvector(float类型的向量)、
二维矩阵：fmatrix(float类型矩阵)、imatrix（int类型的矩阵）
三维float类型矩阵：ftensor3
四维float类型矩阵：ftensor4 #tensor5、tensor6、tensor7
例子：theano.tensor.tensor3(name=None, dtype=config.floatX) #数据类型最好一致，不然会出错
theano.tensor常用函数:
x=theano.tensor.iscalar('x',dtype='int32') #声明一个int类型的变量x
y=theano.tensor.pow(x,3) #定义y=x^3
y1= 1 / (1 + theano.tensor.exp(-x))
f=theano.function([x],y) #定义函数的自变量为x（输入），因变量为y（输出）
print (f(2)) #8
dx=theano.grad(y,x) #偏导数函数
共享变量：
共享变量是多线程编程中的一个名词，故名思议就是各线程，公共拥有的变量，这个是为了多线程高效计算、
访问而使用的变量。
w= theano.shared(1) #定义一个共享变量w，其初始值为1
print (x.get_value()) #取值
x.set_value(2) #设置数值
theano.tensor的层及函数：import theano.tensor as T
T.nnet：conv2d、softmax、
T: T.mean、log、pow、exp、dot、argmax、tanh、grad、
T.signal.downsample.max_pool_2d：池化操作
保存、加载模型：#import pickle
#save model
with open("model.pickle", "wb") as file:
model = [w.get_value(), b.get_value()] #或写成字典的形式
pickle.dump( model, file )
print (model[0][:10]) #打印w的前10个数值
#load model
with open("model.pickle", "rb") as file:
model = pickle.load( file )
w.set_value( model[0] )
b.set_value( model[1] )
print ( w.get_value()[:10] ) #打印w的前10个数值
lasagne:https://github.com/Lasagne/Lasagne (theano自己的深度框架)
http://lasagne.readthedocs.io/en/latest/index.html (手册)
pip install -r https://raw.githubusercontent.com/Lasagne/Lasagne/master/requirements.txt (卸载)
pip install https://github.com/Lasagne/Lasagne/archive/master.zip (安装)
备注：batch_norm()函数存储的参数是beta、gamma、mean、inv_std共四个参数。
# normalize
normalized = (input - mean) * (gamma * inv_std) + beta
#层：
lasagne.layers:
#DenseLayer、DropoutLayer、InputLayer、Conv2DLayer、MaxPool2DLayer、
get_all_params、get_output、set_all_param_values、get_all_param_values
lasagne.updates
lasagne.init
lasagne.nonlinearities
lasagne.objectives
lasagne.regularization
lasagne.random
lasagne.utils
lasagne.layers.DenseLayer(DropoutLayer、InputLayer、Conv2DLayer、MaxPool2DLayer、
get_all_params、get_output、set_all_param_values、get_all_param_values)
lasagne.nonlinearities.rectify(softmax、tanh、relu) #激活函数
W=lasagne.init.GlorotUniform() #权值初始化
loss = lasagne.objectives.categorical_crossentropy(prediction, target_var)
手写字识别例子：
可以参考lasagne 源码给的mnist.py例子，主要代码如下：
import numpy as np
import theano
import theano.tensor as T
import lasagne
import sys，os
def load_dataset():
if sys.version_info[0] == 2: # Python 2
from urllib import urlretrieve
else: # Python 3
from urllib.request import urlretrieve
def download(filename, source='http://yann.lecun.com/exdb/mnist/'):
print("Downloading %s" % filename)
urlretrieve(source + filename, filename)
import gzip
def load_mnist_images(filename): #下载图片
if not os.path.exists(filename):
download(filename)
with gzip.open(filename, 'rb') as f:
data = np.frombuffer(f.read(), np.uint8, offset=16)
data = data.reshape(-1, 1, 28, 28) #(None, channels, rows, columns)
return data / np.float32(256)
def load_mnist_labels(filename): #下载标签
if not os.path.exists(filename):
download(filename)
with gzip.open(filename, 'rb') as f:
data = np.frombuffer(f.read(), np.uint8, offset=8)
return data
#下载数据
X_train = load_mnist_images('train-images-idx3-ubyte.gz')
y_train = load_mnist_labels('train-labels-idx1-ubyte.gz')
X_test = load_mnist_images('t10k-images-idx3-ubyte.gz')
y_test = load_mnist_labels('t10k-labels-idx1-ubyte.gz')
#最后10000个用于预测
X_train, X_val = X_train[:-10000], X_train[-10000:]
y_train, y_val = y_train[:-10000], y_train[-10000:]
return X_train, y_train, X_val, y_val, X_test, y_test
#构建网络模型
def build_cnn(input_var=None):
network = lasagne.layers.InputLayer(shape=(None, 1, 28, 28),input_var=input_var)
network = lasagne.layers.Conv2DLayer(network, num_filters=32, filter_size=(5, 5),
nonlinearity=lasagne.nonlinearities.rectify,W=lasagne.init.GlorotUniform())
network = lasagne.layers.MaxPool2DLayer(network, pool_size=(2, 2))
network = lasagne.layers.Conv2DLayer(network, num_filters=32, filter_size=(5, 5),
nonlinearity=lasagne.nonlinearities.rectify)
network = lasagne.layers.MaxPool2DLayer(network, pool_size=(2, 2))
network = lasagne.layers.DenseLayer(lasagne.layers.dropout(network, p=.5),
num_units=256,nonlinearity=lasagne.nonlinearities.rectify)
network = lasagne.layers.DenseLayer(lasagne.layers.dropout(network, p=.5),
num_units=10,nonlinearity=lasagne.nonlinearities.softmax)
return network
def main(num_epochs=500):
#加载数据
X_train, y_train, X_val, y_val, X_test, y_test = load_dataset()
# 定义 Theano variables for inputs and targets
input_var = T.tensor4('inputs')
target_var = T.ivector('targets')
network = build_cnn(input_var) #构建网络模型
prediction = lasagne.layers.get_output(network) #网络返回结果
#定义交叉商
loss = lasagne.objectives.categorical_crossentropy(prediction, target_var)
loss = loss.mean() # batch_size个数据的均值
#要学习的网络参数
params = lasagne.layers.get_all_params(network, trainable=True)
#网络学习过程中梯度下降的方式
updates = lasagne.updates.nesterov_momentum(loss, params, learning_rate=0.01, momentum=0.9)
#预测，deterministic=True是进行一次前向传播，禁用dropout
test_prediction = lasagne.layers.get_output(network, deterministic=True)
test_loss = lasagne.objectives.categorical_crossentropy(test_prediction,target_var)
test_loss = test_loss.mean()
test_acc = T.mean(T.eq(T.argmax(test_prediction, axis=1), target_var),dtype=theano.config.floatX)
#主要函数
train_fn = theano.function([input_var, target_var], loss, updates=updates)
val_fn = theano.function([input_var, target_var], [test_loss, test_acc])
#开始训练
for epoch in range(num_epochs):
train_err = 0
train_batches = 0 #iterate_minibatches()函数要自己写
for batch in iterate_minibatches(X_train, y_train, 500, shuffle=True):
inputs, targets = batch
train_err += train_fn(inputs, targets)
train_batches += 1
#测试
test_err = 0
test_acc = 0
test_batches = 0
for batch in iterate_minibatches(X_test, y_test, 500, shuffle=False):
inputs, targets = batch
err, acc = val_fn(inputs, targets)
test_err += err
test_acc += acc
test_batches += 1
#保存模型参数
np.savez('model.npz', *lasagne.layers.get_all_param_values(network))
#加载模型参数
with np.load('model.npz') as f:
param_values = [f['arr_%d' % i] for i in range(len(f.files))]
lasagne.layers.set_all_param_values(network, param_values)
备注：
参数存储：
net = {}
net['input'] = lasagne.layers.InputLayer(shape=(None,nChannels=1,h=112,w=112), input_var=self.data)
print("Input shape: {0}".format(net['input'].output_shape)) #(None, 1, 112, 112)
net['s1_conv1_1'] = batch_norm(Conv2DLayer(net['input'], 64, 3, pad='same', W=GlorotUniform('relu')))
#(None, 64, 112, 112)
0 (64, 1, 3, 3) #'s1_conv1_1'，1是上一层的卷积核个数
1 (64,) #以下4个是存储batch_norm中的beta、gamma、mean、std
2 (64,)
3 (64,)
4 (64,)
# batch_norm normalize
normalized = (input - mean) * (gamma * inv_std) + beta
numpy(save、load):
如果你想将多个数组保存到一个文件中的话，可以使用numpy.savez函数。
savez函数的第一个参数是文件名，其后的参数都是需要保存的数组，
也可以使用关键字参数为数组起一个名字，非关键字参数传递的数组会自动起名为arr_0,
arr_1, …。savez函数输出的是一个压缩文件(扩展名为npz)，其中每个文件都是一个
save函数保存的npy文件，文件名对应于数组名。load函数自动识别npz文件，并且返回一个
类似于字典的对象，可以通过数组名作为关键字获取数组的内容：
C=np.array([1,0,1,0])
np.savez("files.npz",A,B,C_array=C)
D=np.load("files.npz")
>>D['arr_0']
>>D['arr_1']
>>D['C_array']
官方给出安装环境建议是：
1）conda工具安装
2）python版本支持2.7、3.4、3.5
theano官方参考网站
2.创建theano环境
根据官方要求，conda中创建theano环境是设置python=3.6
conda update -n base conda //update最新版本的conda
conda create -n theano python=3.6 //创建theano环境
conda activate theano //开启theano环境
conda deactivate //关闭环境
3.安装theano
conda activate theano
anaconda search -t conda theano
1
2
可以看到theano 1.0.3版本支持python=3.6
//查看thean版本
anaconda show conda-forge/theano
//安装theano=1.0.3
conda install --channel https://conda.anaconda.org/conda-forge theano=1.0.3
安装过程图
1）theano=1.0.3
2）libgpuarray=0.7.6
3）pygpu=0.7.6
4）numpy=1.13.3
---------------------
配置虚拟环境运行：theano，
然后提示出错：
from theano.tensor.signal import downsample ImportError: cannot import name 'downsample'
mportError: cannot import name ‘downsample’
查找资料发现downsample已经改为pool
将上面代码改为from theano.tensor.signal import pool
代码中运用到downsample的地方也要改掉
# 子采样
pooled_out = downsample.max_pool_2d(
input=conv_out,
ds=poolsize,
ignore_border=True
)
这里直接将downsample改为pool会出错，因为里面相应的函数有变化
查看pool.py的源码，发现downsample.max_pool_2d()与pool. pool_2d()函数功能相同，用ws代替ds
# 子采样
pooled_out = pool.pool_2d(
input=conv_out,
ws=poolsize,
ignore_border=True
)
运行后，发现问题已经解决。
-
又有问题：
from nose.plugins.skip import SkipTest
ModuleNotFoundError: No module named 'nose'
pip install nose
Using cuDNN version 7005 on context None ERROR (theano.gpuarray): Could not initialize pygpu, support disabled Traceback (most recent call last):提示0.9.0版本支持。
虚拟环境下重新卸载pygpu，重新 安装
conda remove pygpu #删除原来的pygpu重新安装指定版本：
conda install -c conda-forge theano=0.9.0 安装这个版本
/home/gis/anaconda3/envs/theano/bin/python /home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py
Traceback (most recent call last):
File "/home/gis/PycharmProjects/guo/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py", line 4, in
import theano as th
File "/home/gis/anaconda3/envs/theano/lib/python3.6/site-packages/theano/__init__.py", line 80, in
from theano.scan_module import (scan, map, reduce, foldl, foldr, clone,
File "/home/gis/anaconda3/envs/theano/lib/python3.6/site-packages/theano/scan_module/__init__.py", line 41, in
from theano.scan_module import scan_opt
File "/home/gis/anaconda3/envs/theano/lib/python3.6/site-packages/theano/scan_module/scan_opt.py", line 60, in
from theano import tensor, scalar
File "/home/gis/anaconda3/envs/theano/lib/python3.6/site-packages/theano/tensor/__init__.py", line 9, in
from theano.tensor.subtensor import *
File "/home/gis/anaconda3/envs/theano/lib/python3.6/site-packages/theano/tensor/subtensor.py", line 27, in
from cutils_ext.cutils_ext import inplace_increment
ImportError: cannot import name 'inplace_increment'
Deleting the cache manually worked.
rm -rf ~/.theano又有错误：
raise ImportError("dnn not available") # pragma: no cover
ImportError: dnn not available
修改代码行：
#from lasagne.layers import dnn
from theano.sandbox.cuda import dnn
nn.py中也修改。
import matplotlib
ModuleNotFoundError: No module named 'matplotlib'
Process finished with exit code 1
安装模块：conda install matplotlib
Traceback (most recent call last):
File "train_cifar_feature_matching.py", line 15, in
import cifar10_data
File "/home/gden/PycharmProjects/improved-gan-master/mnist_svhn_cifar10/cifar10_data.py", line 1, in
import cPickle
ImportError: No module named 'cPickle'
在python3.x下使用如下代码：
import cPickle as pk
会报如下错误：
ImportError: No module named 'cPickle'
原因：python2有cPickle，但是在python3下，是没有cPickle的；
解决办法：将cPickle改为pickle即可，代码如下：
cifar10_data.py中cpkicle都改为pickle
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 6: ordinal not in range(128)
d = pickle.load(fo)
改为:
d = pickle.load(fo,encoding='bytes')
Namespace(batch_size=100, count=400, data_dir='./data/cifar-10-python', learning_rate=0.0003, seed=1, seed_data=1, unlabeled_weight=1.0)
Traceback (most recent call last):
File "/home/gden/PycharmProjects/improved-gan-master/mnist_svhn_cifar10/train_cifar_feature_matching.py", line 36, in
trainx, trainy = cifar10_data.load(args.data_dir, subset='train')
File "/home/gden/PycharmProjects/improved-gan-master/mnist_svhn_cifar10/cifar10_data.py", line 34, in load
train_data = [unpickle(os.path.join(data_dir,'cifar-10-batches-py/data_batch_' + str(i))) for i in range(1,6)]
File "/home/gden/PycharmProjects/improved-gan-master/mnist_svhn_cifar10/cifar10_data.py", line 34, in
train_data = [unpickle(os.path.join(data_dir,'cifar-10-batches-py/data_batch_' + str(i))) for i in range(1,6)]
File "/home/gden/PycharmProjects/improved-gan-master/mnist_svhn_cifar10/cifar10_data.py", line 29, in unpickle
return {'x': np.cast[np.float32]((-127.5 + d['data'].reshape((10000,3,32,32)))/128.), 'y': np.array(d['labels']).astype(np.uint8)}
KeyError: 'data'
KeyError:’data’
这里是在实现李飞飞-深度学习与机器视觉课程的课后作业的时候遇到的问题。使用的代码是官方给出的代码，自己只是照着敲一遍，但是却出现了KeyError:’data’。
经过查询也没有发现问题所在。出错的代码是这一段
def load_CIFAR_batch(filename):
""" load single batch of cifar """
with open(filename, 'rb') as f:
datadict = pickle.load(f, encoding='bytes')
Y = datadict['labels']
X = datadict['data']
X = X.reshape(10000, 3, 32, 32).transpose(0,2,3,1).astype("float")
Y = np.array(Y)
return X, Y
因为这个问题就是说字典datadict里面没有data这个字段罢了，所以百度了也没有办法。后来进行了debug，观察了datadict的数据发现，在key值data和labels 的前面都有一个b，后来了我就加了一个b在data和labels 前面。如下
Y = datadict[b'labels']
X = datadict[b'data']
1
2
然后正常了……
百度之后才发现这个是python2和python3的问题，我使用了官方给的一个代码库，不过这个是用python2写的，而官方的训练数据也是使用python2生成的。结果我使用的python3去运行，就出现了这个错误。
这里是百度之后的结果：
b:bytes
python3.x里默认的str是unicode, bytes是py2.x的str, b''前缀代表的就是bytes
python2.x里, b前缀没什么具体意义，只是为了兼容python3.x的这种写法
-
cifar10_data.py 29行：data labels前加b
return {'x': np.cast[np.float32]((-127.5 + d[b'data'].reshape((10000,3,32,32)))/128.), 'y': np.array(d[b'labels']).astype(np.uint8)}