论文：Remote Sensing Image Scene Classification 代码执行过程各种坑
论文：Remote Sensing Image Scene Classification 代码
Md. Arafat Hussain, Emon Kumar Dey*
github上的
Remote Sensing Image Scene Classification Using Deep Learning
Here we classified scene images of NWPU-RESISC45 dataset
The dataset can be downloaded from http://www.escience.cn/people/JunweiHan/NWPU-RESISC45.html
We applied Convolutional Neural Network on this dataset to classifiy images. We trained the dataset from scratch. Also appied transfer learning using pre-trained VGG16 abd ResNet50.
We used training_from_scratch.py to train the dataset from scratch. We used python 2.7, keras 2.05 and Theano backend 1.0,OpenCV 3.30.
Applied transfer learning using VGG16_Transfer_Learning.py and RestNet50_Transfer_Learning.py. Here we used python 3.5, keras, 2.05 using Tensorfloe backend 1.0 and openCV 3.30
/usr/bin/python3.6 /home/project/rsisc/training_from_scratch.py
Traceback (most recent call last):
File "/home/project/rsisc/training_from_scratch.py", line 10, in
import os,cv2
ModuleNotFoundError: No module named 'cv2'
Process finished with exit code 1
gden@gden-dl:/media/gden/6C72D37472D3420E$ pip install opencv-python
Collecting opencv-python
Downloading https://files.pythonhosted.org/packages/18/7f/c836c44ab30074a8486e30f8ea6adc8e6ac02332851ab6cc069e2ac35b84/opencv_python-3.4.3.18-cp36-cp36m-manylinux1_x86_64.whl (25.0MB)
100% |████████████████████████████████| 25.0MB 135kB/s
Requirement already satisfied: numpy>=1.11.3 in /home/gden/anaconda3/lib/python3.6/site-packages (from opencv-python) (1.14.3)
distributed 1.21.8 requires msgpack, which is not installed.
Installing collected packages: opencv-python
Could not install packages due to an EnvironmentError: [Errno 13] 权限不够: '/home/gden/anaconda3/lib/python3.6/site-packages/cv2'
Consider using the `--user` option or check the permissions.
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
gden@gden-dl:/media/gden/6C72D37472D3420E$ pip install --upgrade pip
Collecting pip
Using cached https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl
distributed 1.21.8 requires msgpack, which is not installed.
Installing collected packages: pip
Found existing installation: pip 10.0.1
Uninstalling pip-10.0.1:
Could not install packages due to an EnvironmentError: [Errno 13] 权限不够: '/home/gden/anaconda3/bin/pip'
Consider using the `--user` option or check the permissions.
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
gden@gden-dl:/media/gden/6C72D37472D3420E$ sudo pip install opencv-python --user
[sudo] gden 的密码：
sudo: pip：找不到命令
gden@gden-dl:/media/gden/6C72D37472D3420E$ pip install opencv-python --user
Collecting opencv-python
Using cached https://files.pythonhosted.org/packages/18/7f/c836c44ab30074a8486e30f8ea6adc8e6ac02332851ab6cc069e2ac35b84/opencv_python-3.4.3.18-cp36-cp36m-manylinux1_x86_64.whl
Requirement already satisfied: numpy>=1.11.3 in /home/gden/anaconda3/lib/python3.6/site-packages (from opencv-python) (1.14.3)
distributed 1.21.8 requires msgpack, which is not installed.
Installing collected packages: opencv-python
Successfully installed opencv-python-3.4.3.18
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
2\
/usr/bin/python3.6 /home/project/rsisc/training_from_scratch.py
ImportError: numpy.core.multiarray failed to import
Traceback (most recent call last):
File "/home/project/rsisc/training_from_scratch.py", line 10, in
import os,cv2
File "/home/gden/.local/lib/python3.6/site-packages/cv2/__init__.py", line 3, in
from .cv2 import *
ImportError: numpy.core.multiarray failed to import
Process finished with exit code 1
需要更新numpy版本：
gden@gden-dl:/media/gden/6C72D37472D3420E$ pip install -U numpy
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError("bad handshake: SysCallError(104, 'ECONNRESET')",),)': /simple/numpy/
Collecting numpy
Using cached https://files.pythonhosted.org/packages/22/02/bae88c4aaea4256d890adbf3f7cf33e59a443f9985cf91cd08a35656676a/numpy-1.15.2-cp36-cp36m-manylinux1_x86_64.whl
distributed 1.21.8 requires msgpack, which is not installed.
Installing collected packages: numpy
Found existing installation: numpy 1.14.3
Uninstalling numpy-1.14.3:
Could not install packages due to an EnvironmentError: [Errno 13] 权限不够: 'matlib.py'
Consider using the `--user` option or check the permissions.
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
gden@gden-dl:/media/gden/6C72D37472D3420E$ pip install -U numpy --user
Collecting numpy
Using cached https://files.pythonhosted.org/packages/22/02/bae88c4aaea4256d890adbf3f7cf33e59a443f9985cf91cd08a35656676a/numpy-1.15.2-cp36-cp36m-manylinux1_x86_64.whl
distributed 1.21.8 requires msgpack, which is not installed.
Installing collected packages: numpy
Successfully installed numpy-1.15.2
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
3
/usr/bin/python3.6 /home/project/rsisc/training_from_scratch.py
Traceback (most recent call last):
File "/home/project/rsisc/training_from_scratch.py", line 12, in
import matplotlib.pyplot as plt
ModuleNotFoundError: No module named 'matplotlib'
Process finished with exit code 1
折腾半天：
/home/gden/.conda/envs/tensorflow-gpu/bin/python /home/project/rsisc/training_from_scratch.py
Traceback (most recent call last):
File "/home/project/rsisc/training_from_scratch.py", line 12, in
import matplotlib.pyplot as plt
File "/home/gden/.local/lib/python3.6/site-packages/matplotlib/__init__.py", line 141, in
from . import cbook, rcsetup
File "/home/gden/.local/lib/python3.6/site-packages/matplotlib/rcsetup.py", line 25, in
from matplotlib.fontconfig_pattern import parse_fontconfig_pattern
File "/home/gden/.local/lib/python3.6/site-packages/matplotlib/fontconfig_pattern.py", line 19, in
from pyparsing import (Literal, ZeroOrMore, Optional, Regex, StringEnd,
ModuleNotFoundError: No module named 'pyparsing'
Process finished with exit code 1
后来执行：
conda list发现tensorflow没有配好，重新安装，参照上面的一篇博文，注意细节，要把对应的包安装到tensorflow开启的环境下。
最后终于搞定，但提示 58行代码 内存错误。折腾了半天发现是数据集太大，不能一次调取，需要分批。换了小数据集继续进行。一切ok。
然后运行后发现380行有错误；
test_image = cv2.imread('data/runway/runway08.tif')
test_image=cv2.cvtColor(test_image, cv2.COLOR_BGR2GRAY) #380行原来上一行的路径图片不存在，进行修改继续运行
rennet50运行出错：
/home/gden/.conda/envs/tensorflow/bin/python3.6 /home/project/rsisc/resnet50.py
Using TensorFlow backend.
Traceback (most recent call last):
File "/home/project/rsisc/resnet50.py", line 31, in
from keras.applications.imagenet_utils import _obtain_input_shape
ImportError: cannot import name '_obtain_input_shape'
Process finished with exit code 1
/home/gden/.conda/envs/tensorflow/bin/python3.6 /home/project/rsisc/resnet50.py
Using TensorFlow backend.
Traceback (most recent call last):
File "/home/project/rsisc/resnet50.py", line 277, in
model = ResNet50(include_top=True, weights='imagenet') #参数个数发生变化
File "/home/project/rsisc/resnet50.py", line 181, in ResNet50
include_top=include_top)
TypeError: _obtain_input_shape() got an unexpected keyword argument 'include_top'
Process finished with exit code 1
在运行其他开源keras项目时，遇到了一些问题：
1、在导入_obtain_input_shape时
from keras.applications.imagenet_utils import _obtain_input_shape
1
出现错误如下：
ImportError: cannot import name '_obtain_input_shape'
1
原因是在keras 2.2.2中，keras.applications.imagenet_utils模块不再有_obtain_input_shape方法。解决方法：
将导入语句修改如下
1from keras_applications.imagenet_utils import _obtain_input_shape
重点：使用keras_applications代替keras.applications
2、在使用_obtain_input_shape方法时出现错误：
_obtain_input_shape() got an unexpected keyword argument 'include_top'
1
原因是我们的调用中使用了
input_shape = _obtain_input_shape(input_shape,
default_size=224,
min_size=32,
data_format=K.image_data_format(),
include_top=include_top or weights)
1
2
3
4
5
而在keras 2.2.2中函数 _obtain_input_shape() 的形式为：
def _obtain_input_shape(input_shape,
default_size,
min_size,
data_format,
require_flatten,
weights=None):
自然没有include_top属性。因此把调用中的include_top改为require_flatten即可
---------------------
作者：sssaltyfish
来源：CSDN
原文：https://blog.csdn.net/weixin_43344725/article/details/83187692
版权声明：本文为博主原创文章，转载请附上博文链接！
File "/home/project/rsisc/resnet50.py", line 280, in
img = image.load_img(img_path, target_size=(224, 224))
File "/home/gden/.local/lib/python3.6/site-packages/keras_preprocessing/image.py", line 498, in load_img
img = pil_image.open(path)
File "/home/gden/.conda/envs/tensorflow/lib/python3.6/site-packages/PIL/Image.py", line 2477, in open
fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'elephant.jpg'
Process finished with exit code 1
img_path = 'elephant.jpg'这一行的文件不存在，所以放一个存在的在路径下就行了。
中间用的ubuntu命令：
解压rar文件，安装rar工具：sudo apt-get install rar 卸载 sudo apt-get remove rar 解压 rar x 压缩文件名
解压zip文件 unzip 压缩文件名
权限不够的情况 sudo 命令 最后加--user
mv 文件夹 新文件夹 作用移动文件夹，重命名
cp -r 文件夹 路径 复制文件夹中所有文件到新路径
rm -rf 文件或文件夹 删除命令
下面文章可以参考：https://blog.csdn.net/lilong117194/article/details/79138265
我的问题是：
在使用Jupyter Notebook跑tensorflow程序时，出现报错ImportError: No module named 'matplotlib'，然而在spyder（非tensorflow环境）下运行import matplotlib.pyplot as plt就没有问题。然后在网上查了很多资料，大部分都是说：
（1）用终端“sudo apt-get install python-matplotlib ”，安装matplotlib包
（2）或者是在tensorflow环境下进行“sudo apt-get install python-matplotlib ”包的安装。
以上我都试了，然后打开Jupyter Notebook跑tensorflow程序还是报错。
最后经过多次尝试，终于搞定。
首先我的系统是ubuntu 14.04 的，然后安装的是Anaconda 2和python2.7，以上是一些基础信息。这里安装matplotlib包其实是这样的：
（1）首先进入tensorflow的环境：$ source activate tensorflow
（2）通过命令安装matplotlib包：conda install matplotlib
（3）然后就是根据提示选择“y”，确认安装
再次运行tensorflow程序中的import matplotlib.pyplot as plt 就没问题了。
这里要说明的是：
（1）Anaconda
是一个集成许多第三方科学计算库的 Python 科学计算环境，用 conda 作为自己的包管理工具，同时具有自己的计算环境，类似 Virtualenv。
（2）Conda
Conda 是一个开源的软件包管理系统和环境管理系统，用于安装多个版本的软件包及其依赖关系，并在它们之间轻松切换。
常用命令：
conda list 列出当前 conda 环境所链接的软件包
conda create 创建一个 conda 环境，名称为 tf
conda create -n 环境名 -c 镜像源
创建conda执行环境 ：
Create a conda environment called tensorflow:
Python 2.7
$ conda create -n tensorflow python=2.7
Python 3.4
$ conda create -n tensorflow python=3.4
Python 3.5
$ conda create -n tensorflow python=3.5
1
2
3
4
5
6
7
8
激活\关闭conda执行环境：
source activate tensorflow
source deactivate tensorflow
1
2
参考：
https://stackoverflow.com/questions/42356396/importerror-no-module-named-matplotlib-using-anaconda-tensorflow-environme
http://blog.csdn.net/cs_leebo/article/details/64444243
---------------------
作者：lilong117194
来源：CSDN
原文：https://blog.csdn.net/lilong117194/article/details/79138265
版权声明：本文为博主原创文章，转载请附上博文链接！
进入mobaXterm工具，输入用户名登陆：
然后输入：
Downloads/pycharm-community-2017.2/bin/pycharm.sh
启动工具。
Remote Sensing Image Scene Classification Using Deep Learning
Here we classified scene images of NWPU-RESISC45 dataset
The dataset can be downloaded from http://www.escience.cn/people/JunweiHan/NWPU-RESISC45.html
We applied Convolutional Neural Network on this dataset to classifiy images. We trained the dataset from scratch. Also appied transfer learning using pre-trained VGG16 abd ResNet50.
We used training_from_scratch.py to train the dataset from scratch. We used python 2.7, keras 2.05 and Theano backend 1.0,OpenCV 3.30.
Applied transfer learning using VGG16_Transfer_Learning.py and RestNet50_Transfer_Learning.py. Here we used python 3.5, keras, 2.05 using Tensorfloe backend 1.0 and openCV 3.30
运行程序：
gis@gis:~/PycharmProjects/guo/rsisc$ python training_from_scratch.py
Traceback (most recent call last):
File "training_from_scratch.py", line 8, in
import os,cv2
ModuleNotFoundError: No module named 'cv2'
在安装opevncv时会出现 ImportError: No module named cv2 的错误，找不到cv2的包。
这时候安装扩展包即可：
pip install opencv-python
结果：
Collecting opencv-python
Downloading https://files.pythonhosted.org/packages/18/7f/c836c44ab30074a8486e30f8ea6adc8e6ac02332851ab6cc069e2ac35b84/opencv_python-3.4.3.18-cp36-cp36m-manylinux1_x86_64.whl (25.0MB)
100% |████████████████████████████████| 25.0MB 50kB/s
Requirement already satisfied: numpy>=1.11.3 in /home/gis/anaconda3/lib/python3.6/site-packages (from opencv-python) (1.14.3)
Installing collected packages: opencv-python
Successfully installed opencv-python-3.4.3.18
然后重新运行：
is@gis:~/PycharmProjects/guo/rsisc$ python training_from_scratch.py
/home/gis/anaconda3/lib/python3.6/site-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)
Traceback (most recent call last):
File "training_from_scratch.py", line 15, in
from keras import backend as K
ModuleNotFoundError: No module named 'keras'
缺少包 keras
执行：pip3 install --upgrade keras
Collecting keras
Downloading https://files.pythonhosted.org/packages/34/7d/b1dedde8af99bd82f20ed7e9697aac0597de3049b1f786aa2aac3b9bd4da/Keras-2.2.2-py2.py3-none-any.whl (299kB)
100% |████████████████████████████████| 307kB 27kB/s
Collecting keras-applications==1.0.4 (from keras)
Downloading https://files.pythonhosted.org/packages/54/90/8f327deaa37a71caddb59b7b4aaa9d4b3e90c0e76f8c2d1572005278ddc5/Keras_Applications-1.0.4-py2.py3-none-any.whl (43kB)
100% |████████████████████████████████| 51kB 7.6kB/s
Requirement already satisfied, skipping upgrade: numpy>=1.9.1 in /home/gis/anaconda3/lib/python3.6/site-packages (from keras) (1.14.3)
Requirement already satisfied, skipping upgrade: h5py in /home/gis/anaconda3/lib/python3.6/site-packages (from keras) (2.7.1)
Requirement already satisfied, skipping upgrade: scipy>=0.14 in /home/gis/anaconda3/lib/python3.6/site-packages (from keras) (1.1.0)
Collecting keras-preprocessing==1.0.2 (from keras)
Downloading https://files.pythonhosted.org/packages/71/26/1e778ebd737032749824d5cba7dbd3b0cf9234b87ab5ec79f5f0403ca7e9/Keras_Preprocessing-1.0.2-py2.py3-none-any.whl
Requirement already satisfied, skipping upgrade: pyyaml in /home/gis/anaconda3/lib/python3.6/site-packages (from keras) (3.12)
Requirement already satisfied, skipping upgrade: six>=1.9.0 in /home/gis/anaconda3/lib/python3.6/site-packages (from keras) (1.11.0)
Installing collected packages: keras-applications, keras-preprocessing, keras
Successfully installed keras-2.2.2 keras-applications-1.0.4 keras-preprocessing-1.0.2
重新运行：gis@gis:~/PycharmProjects/guo/rsisc$ python training_from_scratch.py
一切ok
gis@gis:~/PycharmProjects/guo/rsisc$ python training_from_scratch.py
/home/gis/anaconda3/lib/python3.6/site-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)
/home/gis/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
Loaded the images of dataset-meadow
Loaded the images of dataset-circular_farmland
。。。。。。。。。
二、
# MARTA-GAN
This is the code for [MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification](https://arxiv.org/abs/1612.08879). An multiple-layer feature-matching generative adversarial networks (MARTA GANs) to learn a representation using only unlabeled data.
## Prepare data
Download and unzip dataset from [BaiDuYun](https://pan.baidu.com/s/1i5zQNdj) or [Google Drive](https://drive.google.com/open?id=0B1Evui8Soh85ZXM3cDNvbGdOamc).
## Dependencies
NVIDIA GPU + CUDA CuDNN (CPU mode and CUDA without CuDNN mode are also available but significantly slower)
- tensorflow
- tensorlayer
- sklearn
## Usage
Training GAN
```
python train_marta_gan.py
```
Extract features
```
python extract_feature.py
```
Training SVM
```
python train_svm.py
```
## Citation
If you find this code useful for your research, please cite:
```
@article{lin2017marta,
title={MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification},
author={Lin, Daoyu and Fu, Kun and Wang, Yang and Xu, Guangluan and Sun, Xian},
journal={IEEE Geoscience and Remote Sensing Letters},
year={2017},
publisher={IEEE}
}
```
首先运行程序：
gis@gis:~/PycharmProjects/guo/MARTA-GAN-master$ python train_marta_gan.py
File "train_marta_gan.py", line 112
print net_g_name, net_d_name
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(net_g_name, net_d_name)?
112 print 少括号，3.6版本下
gis@gis:~/PycharmProjects/guo/MARTA-GAN-master$ python train_marta_gan.py
File "train_marta_gan.py", line 142
print sample_images.shape
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(sample_images.shape)?
142，186行一样少括号，改完后在运行：
gis@gis:~/PycharmProjects/guo/MARTA-GAN-master$ python train_marta_gan.py
/home/gis/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "train_marta_gan.py", line 8, in
import tensorlayer as tl
ModuleNotFoundError: No module named 'tensorlayer'
提示少tensorlayer包：TensorLayer 是为研究人员和工程师设计的一款基于Google TensorFlow开发的深度学习与强化学习库。 它提供高级别的（Higher-Level）深度学习API，这样不仅可以加快研究人员的实验速度，也能够减少工程师在实际开发当中的重复工作。 TensorLayer非常易于修改和扩展，这使它可以同时用于机器学习的研究与应用。 此外，TensorLayer 提供了大量示例和教程来帮助初学者理解深度学习，并提供大量的官方例子程序方便开发者快速找到适合自己项目的例子。
TensorLayer 是建立在 Google TensorFlow 上的深度学习(Deep Learning)与增强学习(Reinforcement Learning) 软件库。它的优势在于既能提供高级 API 以方便开发者快速搭建网络模型，也能保证与 TensorFlow 的全透明衔接。源码简洁，非常容易修改与拓展，同时适合于算法研究人员和产品开发者，加快新算法产品化。 官方例子涵盖 Stacked Denoising Autoencoder, Dropout, CNN, LSTM, Word Embedding, Language Modelling, Machine Translation, Deep Reinforcement Learning等等。官方网站 不仅仅描述了如何使用该API，它还包含所有官方例子的算法教程。
个人强烈推荐 TensorLayer 的原因是它不仅仅是一个功能齐全的工具，它与 TensorFlow 全透明的衔接方式是 Keras、Tflearn 没法相提并论的；对研究者而言，可以非常方便地修改训练的过程，而不会像其他库用简单的一行 fit() 搞定，根本没法让人修改内部运作，然而对研究者而言最重要的是它的源码真的写得太棒了，我觉得它是我见过最简洁的python代码，我们的算法可以很容易拓展进去，然后和它已有的功能一起使用！！！ 对普通开发者而言，它的好处是例子多，例子的代码风格保持一致，不像TensorFlow官方代码那样每个例子的代码风格完全不同。。一致的代码风格有助于开发人员快速找到合适的模型来开发产品。
最后，它的开发者是帝国理工计算机系的博士，陆陆续续有帝国的大神支撑，不像其他框架只是个人在开发。我们可不希望辛辛苦苦学一个框架，然后这个框架最后不再维护了。
此外，值得注意的是 TuneLayer 的前身是 TensorLayer 它是为科学家和工程师而设计的一款基于Google TensorFlow开发的深度学习与增强学习库。使用 TLayer 可以大大加快算法实验和产品开发速度。它非常容易拓展和修改，并提供大量的官方例子程序，方便开发者快速找到适合自己项目的例子。
官方文档 不仅仅描述如何使用 TuneLayer API，还包含了大量的教程，覆盖不同的神经网络类型、深度增强学习和自然语言处理等等。不过，与其它基于TensorFlow开发的傻瓜式API不同，TuneLayer需要使用者有基本的神经网络知识。了解TensorFlow的基础，可以让用非常熟练地使用它。
设计理念
TuneLayer 的发展需要和 TensorFlow 紧密结合，因此 TLayer 的设计具有很强的可塑性，它追随如下的设计理念：
Transparency（透明）：不会去隐藏 TensorFlow，而是尽可能地依赖 TensorFlow 的方法，追随 TensorFlow 的惯例。不会去隐藏训练过程，所有迭代、初始化都可以被用户管理。
Tensor （张量）：神经网络通过高纬度数据表达。
TPU（张量处理单元）：张量处理单元是 Google 设计的专门执行机器学习的定制化 ASIC，除了 TPU 我们将不断兼容更多的机器学习 ASIC。
Distribution（分布式）：分布式机器学习是 TensorFlow 的基本功能，TLayer 在 TensorFlow 的基础上结合 Spark ，让分布式计算更好地服务于机器学习。
Compatibility（兼容性）：网络被抽象为规则函数、损失函数和每层输出。容易与其它 TensorFlow 库结合。
Simplicity（简化）：容易使用，容易拓展与修改，加快研究成果的产品化进度。
High-Speed（高速）：在使用GPU时，运行速度和纯TensorFlow代码一样，不会因为简化代码而牺牲性能。
安装 tensorlayer
2018年08月27日 13:45:47 luoganttcc 阅读数：59更多
个人分类： tensorlayer
版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/luoganttcc/article/details/82111149
pip install --user tensorlayer
gis@gis:~/PycharmProjects/guo/MARTA-GAN-master$ pip install --user tensorlayer
Collecting tensorlayer
Using cached https://files.pythonhosted.org/packages/c4/e2/736458723564163cfd87e3a9719a9fdece9011429bf556fb910d3691352e/tensorlayer-1.10.1-py2.py3-none-any.whl
Collecting scikit-image<0.15,>=0.14 (from tensorlayer)
Downloading https://files.pythonhosted.org/packages/34/79/cefff573a53ca3fb4c390739d19541b95f371e24d2990aed4cd8837971f0/scikit_image-0.14.0-cp36-cp36m-manylinux1_x86_64.whl (25.3MB)
安装sklearn包
gis@gis:~/PycharmProjects/guo/MARTA-GAN-master$ pip install -U scikit-learn
Collecting scikit-learn
Downloading https://files.pythonhosted.org/packages/f9/c8/8db4108aba5e2166cd2ea4eafa1a4b82f89240a1fa85733029cc2358ad1f/scikit_learn-0.19.2-cp36-cp36m-manylinux1_x86_64.whl (4.9MB)
100% |████████████████████████████████| 4.9MB 335kB/s
tensorlayer 1.10.1 has requirement numpy<1.16,>=1.14, but you'll have numpy 1.13.3 which is incompatible.
Installing collected packages: scikit-learn
Found existing installation: scikit-learn 0.19.1
Uninstalling scikit-learn-0.19.1:
Successfully uninstalled scikit-learn-0.19.1
Successfully installed scikit-learn-0.19.2
gis@gis:~/PycharmProjects/guo/MARTA-GAN-master$ python extract_feature.py
/home/gis/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 166, in _parse
return self.parser.parse(argument)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_argument_parser.py", line 152, in parse
val = self.convert(argument)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_argument_parser.py", line 268, in convert
type(argument)))
TypeError: Expect argument to be a string or int, found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "extract_feature.py", line 27, in
flags.DEFINE_integer("train_size", np.inf, "The size of train images [np.inf]")
File "/home/gis/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/flags.py", line 58, in wrapper
return original_function(*args, **kwargs)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_defines.py", line 315, in DEFINE_integer
DEFINE(parser, name, default, help, flag_values, serializer, **args)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_defines.py", line 81, in DEFINE
DEFINE_flag(_flag.Flag(parser, serializer, name, default, help, **args),
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 107, in __init__
self._set_default(default)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 196, in _set_default
self.default = self._parse(value)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 169, in _parse
'flag --%s=%s: %s' % (self.name, argument, e))
absl.flags._exceptions.IllegalFlagValueError: flag --train_size=inf: Expect argument to be a string or int, found
gis@gis:~/PycharmProjects/guo/MARTA-GAN-master$ pip install numpy==1.13.3
Collecting numpy==1.13.3
Downloading https://files.pythonhosted.org/packages/57/a7/e3e6bd9d595125e1abbe162e323fd2d06f6f6683185294b79cd2cdb190d5/numpy-1.13.3-cp36-cp36m-manylinux1_x86_64.whl (17.0MB)
100% |████████████████████████████████| 17.0MB 81kB/s
tensorlayer 1.10.1 has requirement numpy<1.16,>=1.14, but you'll have numpy 1.13.3 which is incompatible.
Installing collected packages: numpy
Found existing installation: numpy 1.14.3
Uninstalling numpy-1.14.3:
Successfully uninstalled numpy-1.14.3
Successfully installed numpy-1.13.3
gis@gis:~/PycharmProjects/guo/MARTA-GAN-master$ python train_marta_gan.py
Traceback (most recent call last):
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 166, in _parse
return self.parser.parse(argument)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_argument_parser.py", line 152, in parse
val = self.convert(argument)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_argument_parser.py", line 268, in convert
type(argument)))
TypeError: Expect argument to be a string or int, found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train_marta_gan.py", line 26, in
flags.DEFINE_integer("train_size", np.inf, "The size of train images [np.inf]")
File "/home/gis/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/flags.py", line 58, in wrapper
return original_function(*args, **kwargs)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_defines.py", line 315, in DEFINE_integer
DEFINE(parser, name, default, help, flag_values, serializer, **args)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_defines.py", line 81, in DEFINE
DEFINE_flag(_flag.Flag(parser, serializer, name, default, help, **args),
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 107, in __init__
self._set_default(default)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 196, in _set_default
self.default = self._parse(value)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 169, in _parse
'flag --%s=%s: %s' % (self.name, argument, e))
absl.flags._exceptions.IllegalFlagValueError: flag --train_size=inf: Expect argument to be a string or int, found
1.错误：
D:\Program Files\python3.6\lib\site-packages\h5py\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`. from ._conv import register_converters as _register_converters
2.原因：（参考：https://wuwb.me/2018/03/h5py-futurewarning-problem-fix.html）
h5py 和 numpy 版本冲突，h5py 官方已修复合并到 master 分支，但是还没发新版，在发版之前可以用降级 numpy 的方法跳过这个问题。降级命令如下：
pip install numpy==1.13.3(tensorflow-gpu 1.6.0 匹配numpy 1.13.3）
/home/gis/anaconda3/bin/python /home/gis/PycharmProjects/guo/rsisc/resnet50_Transfer_Learning.py
Using TensorFlow backend.
Traceback (most recent call last):
File "/home/gis/PycharmProjects/guo/rsisc/resnet50_Transfer_Learning.py", line 5, in
from resnet50 import ResNet50
File "/home/gis/PycharmProjects/guo/rsisc/resnet50.py", line 31, in
from keras.applications.imagenet_utils import _obtain_input_shape
ImportError: cannot import name '_obtain_input_shape'
down vote
This issue occured because of the version of keras.
In my case, I was downgrade keras 2.2.2 to 2.2.0, and the problem was solved.
shareimprove this answer
answered Aug 1 at 8:56
user5803658
6113
how do you upgraded kerns 2.2.2 to version less than 2.2.2 – W. Sam Aug 13 at 1:14
Use 'pip install keras==2.2.0' – Raein Hashemi Aug 15 at 17:02
1
@W.Sam you don't have to downgrade Keras, see my answer below. – Geeocode Aug 30 at 16:11
_obtain_input_shape() got an unexpected keyword argument 'include_top'
keras.__version__ == 2.0.9 中，函数 _obtain_input_shape() 的形式：
def _obtain_input_shape(input_shape,
default_size,
min_size,
data_format,
require_flatten,
weights=None):
2）deep-learning-models 案例中，调用 _obtain_input_shape() 函数的方式如下：
# Determine proper input shape
input_shape = _obtain_input_shape(input_shape,
default_size=299,
min_size=71,
data_format=K.image_data_format(),
include_top=include_top)
显然，本文使用的环境中，函数 _obtain_input_shape() 的形参中，没有关键字 include_top 而是 require_flatten。而案例中的代码使用的是关键字 include_top，这显然是不可取的。综上，只要把案例中的 inlcude_top 关键字换成 require_flatten 即可。如下所示：
【keras-DeepLearning_Models】_obtain_input_shape() got an unexpected keyword argument 'include_top'
2017年11月11日 16:34:04 Houchaoqun_XMU 阅读数：3199 标签： _obtain_input_shape keras Ubuntu inception 更多
个人分类： 【深度学习】
前言：
最近想跑一些主流的网络感受感受。从github上找到了 deep-learning-models 提供的几个模型，包括：inception-v2, inception-v3, resnet50, vgg16, vgg19 等等。这些代码都是基于 keras 框架，正好我最近有在学 tensorflow 和 keras，所以很想跑跑这些代码。
心动不如行动，万事俱备，只欠把代码跑起来。此时，出现了一些常见的问题，也正好借此机会整理下来。问题如下：
1）_obtain_input_shape() got an unexpected keyword argument 'include_top'
2）Exception: URL fetch failure on https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels.h5: None -- [Errno 110] Connection timed out
本文主要分析和整理了第一个问题的解决方案。
第二个问题就是：在下载模型参数文件的过程中，可能url对应的地址被墙了，导致下载不了。解决办法就是另想办法把 resnet50_weights_tf_dim_ordering_tf_kernels.h5 这个文件下载下来。如果有需要的话，可在本文留言。
Reference：
1）github 源码：https://github.com/fchollet/deep-learning-models
2）模型参数资源：https://github.com/fchollet/deep-learning-models/releases
3）相关博客：http://blog.csdn.net/sinat_26917383/article/details/72982230
4）本文使用的测试数据如下所示：
elephant.jpg
本文使用的 tensorflow 和 keras 的版本：
- tensorflow：
>>> import tensorflow as tf
>>> tf.__version__
'1.1.0'
- keras：
import keras
>>> print keras.__version__
2.0.9
本文实践步骤如下，以 "resnet50.py" 为例：
1）下载 github 源码，源码中使用一张名为“elephant.jpg”的图像作为测试。
2）下载测试数据集，上文有提供链接。
3）在源码的目录下执行如下命令：
python resnet50.py
程序报错，如下所示：
Traceback (most recent call last):
File "resnet50.py", line 289, in
model = ResNet50(include_top=True, weights='imagenet')
File "resnet50.py", line 193, in ResNet50
include_top=include_top)
TypeError: _obtain_input_shape() got an unexpected keyword argument 'include_top'
导致程序报错的原因分析：
1）keras.__version__ == 2.0.9 中，函数 _obtain_input_shape() 的形式：
def _obtain_input_shape(input_shape,
default_size,
min_size,
data_format,
require_flatten,
weights=None):
2）deep-learning-models 案例中，调用 _obtain_input_shape() 函数的方式如下：
# Determine proper input shape
input_shape = _obtain_input_shape(input_shape,
default_size=299,
min_size=71,
data_format=K.image_data_format(),
include_top=include_top)
显然，本文使用的环境中，函数 _obtain_input_shape() 的形参中，没有关键字 include_top 而是 require_flatten。而案例中的代码使用的是关键字 include_top，这显然是不可取的。综上，只要把案例中的 inlcude_top 关键字换成 require_flatten 即可。如下所示：
# Determine proper input shape
input_shape = _obtain_input_shape(input_shape,
default_size=224,
min_size=197,
data_format=K.image_data_format(),
require_flatten=include_top)
再次执行命令，就可以成功运行案例代码，如下图所示：
本文的第二个问题：无法正常下载模型参数
模型参数“resnet50_weights_tf_dim_ordering_tf_kernels.h5”下载地址如下：
链接：http://pan.baidu.com/s/1dE1Lh5J 密码：puke
需修改的代码如下：
# WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels.h5'
# load weights
if weights == 'imagenet':
# if include_top:
# weights_path = get_file('resnet50_weights_tf_dim_ordering_tf_kernels.h5',
# WEIGHTS_PATH,
# cache_subdir='models',
# md5_hash='a7b3fe01876f51b976af0dea6bc144eb')
# else:
# weights_path = get_file('resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5',
# WEIGHTS_PATH_NO_TOP,
# cache_subdir='models',
# md5_hash='a268eb855778b3df3c7506639542a6af')
weights_path = './resnet50_weights_tf_dim_ordering_tf_kernels.h5'
model.load_weights(weights_path)
运行结果如下所示：
Predicted: [[(u'n01871265', u'tusker', 0.65325415), (u'n02504458', u'African_elephant', 0.29492217), (u'n02504013', u'Indian_elephant', 0.048155606), (u'n02422106', u'hartebeest', 0.001847562), (u'n02397096', u'warthog', 0.00034257883)]]
由结果可知：p(tusker) = 0.65, p(African_elephant) = 0.29, p(Indian_elephant) = 0.048 ... ... 其中 tusker 的概率是最高的，所以识别结果为 tusker（有长牙的动物，如：象，野猪等）