编译记录
在anaconda中创建虚拟环境
conda create --name tf_gpu_env2 python=3.6
查看所有的虚拟环境
conda info --envs
连接环境
source activate tf_gpu_env2
关闭环境
source deactivate
查看tensorflow各个版本：（查看会发现有一大堆TensorFlow源，但是不能随便选，选择可以用查找命令定位）
anaconda search -t conda tensorflow-gpu
查看tensorflow版本信息
anaconda show anaconda/tensorflow-gpu
安装指定版本
conda install --channel https://conda.anaconda.org/anaconda tensorflow-gpu=1.12.0
错误1：
Traceback (most recent call last):
File “setup.py”, line 4, in
from Cython.Build import cythonize
ModuleNotFoundError: No module named 'Cython’
mv: cannot stat ‘build//.so’: No such file or directory
解决方法：在虚拟环境中 python -m pip install cython
错误2：
Traceback (most recent call last):
File “./main/demo.py”, line 7, in
import cv2
ModuleNotFoundError: No module named 'cv2’
解决方法：pip install opencv-python
错误3（警告？）
/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,)type’.
_np_qint8 = np.dtype([(“qint8”, np.int8, 1)])
/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,)type’.
numpy1-17-0版本过高，使用numpy-1.16-0版本即可？
解决方法：
重新安装numpy-1.16-0：
pip install numpy==1.16.0
可以直接覆盖
pip show numpy查看版本
以上可以成功执行demo检测图片
错误4：
Traceback (most recent call last):
File “./main/train.py”, line 11, in
from utils.dataset import data_provider as data_provider
File “/home/cheoywoo/CTPN_train/train_myself/text-detection-ctpn/utils/dataset/data_provider.py”, line 6, in
import matplotlib.pyplot as plt
ModuleNotFoundError: No module named 'matplotlib’
解决方法：python -m pip install matplotlib
错误5：
File “/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/absl/flags/_flag.py”, line 181, in _parse
return self.parser.parse(argument)
File “/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/absl/flags/_argument_parser.py”, line 152, in parse
val = self.convert(argument)
File “/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/absl/flags/_argument_parser.py”, line 268, in convert
type(argument)))
TypeError: Expect argument to be a string or int, found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “./main/train.py”, line 16, in
tf.app.flags.DEFINE_integer(‘decay_rate’, 0.1, ‘’)
File “/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/tensorflow/python/platform/flags.py”, line 58, in wrapper
return original_function(*args, **kwargs)
File “/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/absl/flags/_defines.py”, line 315, in DEFINE_integer
DEFINE(parser, name, default, help, flag_values, serializer, **args)
File “/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/absl/flags/_defines.py”, line 81, in DEFINE
DEFINE_flag(_flag.Flag(parser, serializer, name, default, help, **args),
File “/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/absl/flags/_flag.py”, line 110, in init
self._set_default(default)
File “/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/absl/flags/_flag.py”, line 216, in _set_default
self.default = self._parse(value)
File “/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/absl/flags/_flag.py”, line 184, in _parse
‘flag --%s=%s: %s’ % (self.name, argument, e))
absl.flags._exceptions.IllegalFlagValueError: flag --decay_rate=0.1: Expect argument to be a string or int, found
错误原因：大概率是因为python3和python2不兼容的问题
（原因是np.inf返回的是float,然而在chapter_8的main.py中,"train_size"定义时使用的是DEFINE_integer, 所以将DEFINE_integer改成DEFINE_float即可）
解决方案：
flags.DEFINE_integer(“train_size”, np.inf, “The size of train images [np.inf]”)改成
flags.DEFINE_float(“train_size”, np.inf, “The size of train images [np.inf]”)
错误6：tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for data/vgg_16.ckpt
未找到预训练模型。下载好vgg_16.ckpt放入即可。
错误7：
/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:112: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
WARNING:tensorflow:Variable Conv/weights missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable Conv/biases missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/bidirectional_rnn/fw/lstm_cell/kernel missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/bidirectional_rnn/fw/lstm_cell/bias missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/bidirectional_rnn/bw/lstm_cell/kernel missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/bidirectional_rnn/bw/lstm_cell/bias missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/weights missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/biases missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable bbox_pred/weights missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable bbox_pred/biases missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable cls_pred/weights missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable cls_pred/biases missing in checkpoint data/vgg_16.ckpt
2020-08-22 13:28:21.974242: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2020-08-22 13:28:22.074152: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-22 13:28:22.074405: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: GeForce GTX 1650 major: 7 minor: 5 memoryClockRate(GHz): 1.56
pciBusID: 0000:01:00.0
totalMemory: 3.81GiB freeMemory: 3.22GiB
2020-08-22 13:28:22.074419: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2020-08-22 13:28:22.284743: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-08-22 13:28:22.284776: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2020-08-22 13:28:22.284782: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2020-08-22 13:28:22.284893: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3708 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1650, pci bus id: 0000:01:00.0, compute capability: 7.5)
Traceback (most recent call last):
File “./main/train.py”, line 117, in
tf.app.run()
File “/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/tensorflow/python/platform/app.py”, line 125, in run
sys.exit(main(argv))
File “./main/train.py”, line 80, in main
restore_step = int(ckpt.split(’.’)[0].split(’’)[-1])
AttributeError: ‘NoneType’ object has no attribute 'split’
可能存在两个错误：
解决方法：
https://github.com/eragonruan/text-detection-ctpn/issues/353
tf.app.flags.DEFINE_boolean(‘restore’, True, ‘’) set the ‘True’ to ‘False’,and you can run
（https://github.com/eragonruan/text-detection-ctpn/issues/161
也许路径不正确，检查点文件的路径是demo.py文件的上层文件夹。
我通过更改路径解决了这个问题。）？
tf.app.flags.DEFINE_string（‘test_data_path’，’…/data/demo/’，’’）
tf.app.flags.DEFINE_string（‘output_path’，’…/data/res/’，’’）
tf.app.flags.DEFINE_string（‘checkpoint_path’，’…/checkpoints_mlt/’，’’）
标题
遇到net问题
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting nets
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/9c/4c/dc299e96dd82f0ea3b74843d300505c08d1fde74cd1c509609dedf4b2ff0/nets-0.0.2.tar.gz (5.9 MB)
|████████████████████████████████| 5.9 MB 1.8 MB/s
Collecting torch
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/38/53/914885a93a44b96c0dd1c36f36ff10afe341f091230aad68f7228d61db1e/torch-1.6.0-cp36-cp36m-manylinux1_x86_64.whl (748.8 MB)
|████████████████████████████████| 748.8 MB 25 kB/s
Requirement already satisfied: numpy in /home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages (from nets) (1.16.0)
Collecting pandas
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a1/c6/9ac4ae44c24c787a1738e5fb34dd987ada6533de5905a041aa6d5bea4553/pandas-1.1.1-cp36-cp36m-manylinux1_x86_64.whl (10.5 MB)
|████████████████████████████████| 10.5 MB 15.4 MB/s
Collecting sklearn
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/1e/7a/dbb3be0ce9bd5c8b7e3d87328e79063f8b263b2b1bfa4774cb1147bfcd3f/sklearn-0.0.tar.gz (1.1 kB)
Requirement already satisfied: scipy in /home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages (from nets) (1.5.0)
Collecting future
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/45/0b/38b06fd9b92dc2b68d58b75f900e97884c45bedd2ff83203d933cf5851c9/future-0.18.2.tar.gz (829 kB)
|████████████████████████████████| 829 kB 11.7 MB/s
Collecting pytz>=2017.2
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/4f/a4/879454d49688e2fad93e59d7d4efda580b783c745fd2ec2a3adf87b0808d/pytz-2020.1-py2.py3-none-any.whl (510 kB)
|████████████████████████████████| 510 kB 11.4 MB/s
Requirement already satisfied: python-dateutil>=2.7.3 in /home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages (from pandas->nets) (2.8.1)
Collecting scikit-learn
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/5c/a1/273def87037a7fb010512bbc5901c31cfddfca8080bc63b42b26e3cc55b3/scikit_learn-0.23.2-cp36-cp36m-manylinux1_x86_64.whl (6.8 MB)
|████████████████████████████████| 6.8 MB 9.3 MB/s
Requirement already satisfied: six>=1.5 in /home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages (from python-dateutil>=2.7.3->pandas->nets) (1.15.0)
Collecting threadpoolctl>=2.0.0
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f7/12/ec3f2e203afa394a149911729357aa48affc59c20e2c1c8297a60f33f133/threadpoolctl-2.1.0-py3-none-any.whl (12 kB)
Collecting joblib>=0.11
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/51/dd/0e015051b4a27ec5a58b02ab774059f3289a94b0906f880a3f9507e74f38/joblib-0.16.0-py3-none-any.whl (300 kB)
|████████████████████████████████| 300 kB 11.8 MB/s
Building wheels for collected packages: nets, sklearn, future
Building wheel for nets (setup.py) … done
Created wheel for nets: filename=nets-0.0.2-py3-none-any.whl size=56607 sha256=351ab979c4173718b9d26ff92c3650c19efd7ceb1cba0235c98dfd987759200d
Stored in directory: /home/cheoywoo/.cache/pip/wheels/75/11/99/0e3ee00e2963c4f9c79d69444371307c903f40f921f5086f8f
Building wheel for sklearn (setup.py) … done
Created wheel for sklearn: filename=sklearn-0.0-py2.py3-none-any.whl size=1315 sha256=f77d51deb724435403501f853ff1d4d18abc64d79f3f62d7c7492ac16e326f8d
Stored in directory: /home/cheoywoo/.cache/pip/wheels/1f/e0/8d/744e7e074d3d78f60c949a50b3c8ce3922e29cf92c8d78660d
Building wheel for future (setup.py) … done
Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491058 sha256=8a25958a0537f9220a7ec8a0e373d5402e97bd6ef9572eded52ee2989a847aba
Stored in directory: /home/cheoywoo/.cache/pip/wheels/f3/f6/70/8b2bba44d8bf6ee5b7eac561416d9ef5c28dd2320490fc0f1f
Successfully built nets sklearn future
Installing collected packages: future, torch, pytz, pandas, threadpoolctl, joblib, scikit-learn, sklearn, nets
Successfully installed future-0.18.2 joblib-0.16.0 nets-0.0.2 pandas-1.1.1 pytz-2020.1 scikit-learn-0.23.2 sklearn-0.0 threadpoolctl-2.1.0 torch-1.6.0
超出内存
/home/cheoywoo/anaconda3/envs/tf_gpu_env2/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:112: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
WARNING:tensorflow:Variable Conv/weights missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable Conv/biases missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/bidirectional_rnn/fw/lstm_cell/kernel missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/bidirectional_rnn/fw/lstm_cell/bias missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/bidirectional_rnn/bw/lstm_cell/kernel missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/bidirectional_rnn/bw/lstm_cell/bias missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/weights missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable BiLSTM/biases missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable bbox_pred/weights missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable bbox_pred/biases missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable cls_pred/weights missing in checkpoint data/vgg_16.ckpt
WARNING:tensorflow:Variable cls_pred/biases missing in checkpoint data/vgg_16.ckpt
2020-08-22 13:31:29.047865: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2020-08-22 13:31:29.132162: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-22 13:31:29.132415: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: GeForce GTX 1650 major: 7 minor: 5 memoryClockRate(GHz): 1.56
pciBusID: 0000:01:00.0
totalMemory: 3.81GiB freeMemory: 3.21GiB
2020-08-22 13:31:29.132431: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2020-08-22 13:31:29.336321: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-08-22 13:31:29.336351: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2020-08-22 13:31:29.336360: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2020-08-22 13:31:29.336436: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3708 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1650, pci bus id: 0000:01:00.0, compute capability: 7.5)
Find 3422 images
3422 training images in data/dataset/mlt/
Find 3422 images
3422 training images in data/dataset/mlt/
Find 3422 images
3422 training images in data/dataset/mlt/
Find 3422 images
3422 training images in data/dataset/mlt/
2020-08-22 13:31:32.746306: E tensorflow/stream_executor/cuda/cuda_driver.cc:806] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2020-08-22 13:31:33.405261: W tensorflow/core/common_runtime/bfc_allocator.cc:211] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.19GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
2020-08-22 13:31:33.406240: W tensorflow/core/common_runtime/bfc_allocator.cc:211] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.19GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
2020-08-22 13:31:34.561947: W tensorflow/core/common_runtime/bfc_allocator.cc:211] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.14GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
2020-08-22 13:31:34.561993: W tensorflow/core/common_runtime/bfc_allocator.cc:211] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.14GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
Segmentation fault (core dumped)
new
查看所有的虚拟环境
conda info --envs
连接环境
source activate tf_gpu_env1
关闭环境
source deactivate
首先安装pytorch1.3.1
pip3 install torch==1.3.1
然后安装需要的环境
pip3 install lmdb pillow torchvision nltk natsort
注意，最好分开安装，安装torchvision时会自动安装对应的pytorch版本，因此需要指定版本
或者尝试pip3 install lmdb pillow torchvision==0.4.2 nltk natsort
pytorch和torchvision的对应版本
（https://blog.csdn.net/qq_40263477/article/details/106577790）
项目https://github.com/clovaai/deep-text-recognition-benchmark
执行
CUDA_VISIBLE_DEVICES=0 python3 demo.py
–sensitive
–Transformation TPS --FeatureExtraction ResNet --SequenceModeling BiLSTM --Prediction Attn
–image_folder demo_image/
–saved_model TPS-ResNet-BiLSTM-Attn-case-sensitive.pth