AI challenger 场景分类 train test softmax
与前文Ai challenger 场景分类： train softmax using tfrecord的区别见代码前面的changes说明。
目前tfrecord坑很多，参见 [Enhancement] Redesigning TensorFlow’s input pipelines #7951
目前赤裸的softmax过拟合严重：0.7 vs 0.18
# -*- coding: utf-8 -*-
"""
Created on Wed Sep 20 16:05:02 2017
@author: wayne
FEELINGS
目前tfrecord的坑还是挺多的，未来的1.4版本和2版本特性参见
https://github.com/tensorflow/tensorflow/issues/7902
和
https://github.com/tensorflow/tensorflow/issues/7951
CHANGES
- 训练和测试的一体化，以方便加入统一的数据预处理：注意目前是直接将验证集作为测试集来使用！！！注意数据增强只在训练时使用。
train_flag = False (测试模式)
- 将测试集的结果写入提交格式submit.json，供官方提供的scene_eval.py 使用：
https://github.com/AIChallenger/AI_Challenger/tree/master/AI_Challenger_eval_public
- image = tf.image.per_image_standardization(image) 修改到tf.image.resize_images后
- 其他小细节的改进
TODO
【看着很复杂，分解后逐步实现比较容易（注意需要尽可能考虑程序未来的可扩展性，以降低重构的工作量），最后可以再考虑进一步优化程序的架构等等，先跑通必要的功能】
- NEXT (train_flag = True): 增加每训练一段时间显示一次验证准确率，即train_flag = True时需要load train和val.
https://stackoverflow.com/questions/44270198/when-using-tfrecord-how-can-i-run-intermediate-validation-check-a-better-way
https://github.com/tensorflow/tensorflow/issues/7902
训练结束显示整个训练集上的准确率？
- NEXT: finetune基于imagenet的inception-resnet v2, senet等
- NEXT: 调参和数据增强，模型复杂度， use log file, use input args 模块化等
REFERENCES
输入数据
https://stackoverflow.com/questions/44054656/creating-tfrecords-from-a-list-of-strings-and-feeding-a-graph-in-tensorflow-afte
https://indico.io/blog/tensorflow-data-inputs-part1-placeholders-protobufs-queues/
https://indico.io/blog/tensorflow-data-input-part2-extensions/
整个架构
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/how_tos/reading_data/fully_connected_reader.py
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/udacity/2_fullyconnected.ipynb
模型的存储和调用
http://blog.csdn.net/u014595019/article/details/53912710
http://blog.csdn.net/u012436149/article/details/52883747 (restore变量的子集)
https://github.com/SymphonyPy/Valified_Code_Classify/tree/master/Classified
"""
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import time
import json
def read_and_decode(tfrecords_file, batch_size, num_epochs):
filename_queue = tf.train.string_input_producer([tfrecord_file], num_epochs = num_epochs)
reader = tf.TFRecordReader()
_, serialized_example = reader.read(filename_queue)
img_features = tf.parse_single_example(
serialized_example,
features={
'label': tf.FixedLenFeature([], tf.int64),
'h': tf.FixedLenFeature([], tf.int64),
'w': tf.FixedLenFeature([], tf.int64),
'c': tf.FixedLenFeature([], tf.int64),
'image': tf.FixedLenFeature([], tf.string),
})
h = tf.cast(img_features['h'], tf.int32)
w = tf.cast(img_features['w'], tf.int32)
c = tf.cast(img_features['c'], tf.int32)
image = tf.decode_raw(img_features['image'], tf.uint8)
image = tf.reshape(image, [h, w, c])
label = tf.cast(img_features['label'],tf.int32)
#label = tf.reshape(label, [1])
##########################################################
'''data augmentation here'''
# distorted_image = tf.random_crop(images, [530, 530, img_channel])
# distorted_image = tf.image.random_flip_left_right(distorted_image)
# distorted_image = tf.image.random_brightness(distorted_image, max_delta=63)
# distorted_image = tf.image.random_contrast(distorted_image, lower=0.2, upper=1.8)
image = tf.image.resize_images(image, (image_size,image_size))
image = tf.image.per_image_standardization(image)
image = tf.reshape(image, [image_size * image_size * 3])
#image, label = tf.train.batch([image, label], batch_size= batch_size)
##########################################################
'''shuffle here'''
image_batch, label_batch = tf.train.shuffle_batch([image, label],
batch_size= batch_size,
num_threads= 64, # 注意多线程有可能改变图片顺序
capacity = 10240,
min_after_dequeue= 256
)
#print(type(label_batch))
return image_batch, label_batch # tf.reshape(label_batch, [batch_size])
def read_and_decode_test(tfrecords_file, batch_size, num_epochs):
filename_queue = tf.train.string_input_producer([tfrecord_file], num_epochs = num_epochs)
reader = tf.TFRecordReader()
_, serialized_example = reader.read(filename_queue)
img_features = tf.parse_single_example(
serialized_example,
features={
'label': tf.FixedLenFeature([], tf.int64),
'h': tf.FixedLenFeature([], tf.int64),
'w': tf.FixedLenFeature([], tf.int64),
'c': tf.FixedLenFeature([], tf.int64),
'image': tf.FixedLenFeature([], tf.string), #https://stackoverflow.com/questions/41921746/tensorflow-varlenfeature-vs-fixedlenfeature
'image_id': tf.FixedLenFeature([], tf.string)
})
h = tf.cast(img_features['h'], tf.int32)
w = tf.cast(img_features['w'], tf.int32)
c = tf.cast(img_features['c'], tf.int32)
image_id = img_features['image_id']
image = tf.decode_raw(img_features['image'], tf.uint8)
image = tf.reshape(image, [h, w, c])
label = tf.cast(img_features['label'],tf.int32)
#label = tf.reshape(label, [1])
##########################################################
'''no data augmentation'''
image = tf.image.resize_images(image, (image_size,image_size))
image = tf.image.per_image_standardization(image)
image = tf.reshape(image, [image_size * image_size * 3])
#image, label = tf.train.batch([image, label], batch_size= batch_size)
image_batch, label_batch, image_id_batch= tf.train.batch([image, label, image_id],
batch_size= batch_size,
num_threads= 64, # 注意多线程有可能改变图片顺序
capacity = 2000)
#print(type(label_batch))
return image_batch, label_batch, image_id_batch
def batch_to_list_of_dicts(indices2, image_id_batch2):
result = [] #[{"image_id":"a0563eadd9ef79fcc137e1c60be29f2f3c9a65ea.jpg","label_id": [5,18,32]}]
dict_ = {}
for item in range(batch_size):
dict_ ['image_id'] = image_id_batch2[item].decode()
dict_['label_id'] = indices2[item,:].tolist()
result.append(dict_)
dict_ = {}
return result
def read_tfrecord2(tfrecord_file, batch_size, train_flag):
weights = tf.Variable(
tf.truncated_normal([image_size * image_size * 3, num_labels]))
biases = tf.Variable(tf.zeros([num_labels]))
#因为test有image_id，否则和train共用输入函数就行了。另外read_and_decode训练中会加入data augmentation，因此验证集和测试集均用第二个函数
if train_flag:
train_batch, train_label_batch = read_and_decode(tfrecord_file, batch_size, num_epochs)
# val_test_batch, val_test_label_batch, image_id_batch= read_and_decode_test(tfrecord_file_val, batch_size, 1) #每次用val的时候整个数据过一遍，下次又用怎么办？
# Variables.
# Training computation.
logits = tf.matmul(train_batch, weights) + biases
# https://gxnotes.com/article/29754.html : 张量流tf.nn.softmax和tf.nn.softmax_cross_entropy_with_logits之间的差异
loss = tf.reduce_mean(
tf.nn.sparse_softmax_cross_entropy_with_logits(labels=train_label_batch, logits=logits))
# Optimizer.
optimizer = tf.train.GradientDescentOptimizer(0.5).minimize(loss)
# Predictions for the training, validation, and test data.
train_prediction = tf.nn.softmax(logits)
'''minibatch accuracy, non-streaming'''
accuracy = tf.reduce_mean(tf.cast(tf.nn.in_top_k(predictions = logits, targets=train_label_batch, k=3),tf.float32))
else:
val_test_batch, val_test_label_batch, image_id_batch= read_and_decode_test(tfrecord_file, batch_size, num_epochs)
val_test_logits = tf.matmul(val_test_batch, weights) + biases
val_test_prediction = tf.nn.softmax(val_test_logits)
'''Useless minibatch accuracy, non-streaming'''
#http://blog.csdn.net/ib_h20/article/details/72782581: correct = tf.nn.in_top_k(logits, labels, k)
#http://blog.csdn.net/uestc_c2_403/article/details/73187915: tf.nn.in_top_k的用法
val_test_accuracy_batch = tf.reduce_mean(tf.cast(tf.nn.in_top_k(predictions = val_test_logits, targets=val_test_label_batch, k=3),tf.float32))
'''不是minibatch accuracy'''
val_test_accuracy, val_test_accuracy_update= tf.metrics.mean(tf.cast(tf.nn.in_top_k(predictions = val_test_logits, targets=val_test_label_batch, k=3),tf.float32))
# https://github.com/tensorflow/tensorflow/issues/9498
# Implementing non streaming accuracy is simple, ex:
# tf.reduce_mean(tf.to_float32(predictions == labels))
values, indices = tf.nn.top_k(val_test_logits, 3)
saver = tf.train.Saver() # 生成saver
with tf.Session() as sess:
# https://github.com/tensorflow/tensorflow/issues/1045
sess.run(tf.group(tf.global_variables_initializer(), tf.local_variables_initializer()))
print("Initialized")
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(coord=coord)
if train_flag:
try:
step = 0
start_time = time.time()
while not coord.should_stop():
_, l, predictions, logits2, acc= sess.run([optimizer, loss, train_prediction, logits, accuracy])
duration = time.time() - start_time
if (step % 10 == 0):
print("Minibatch loss at step %d: %.6f (%.3f sec)" % (step, l, duration))
print("Minibatch accuracy: %.6f" % acc)
#if (step % 100 == 0):
#Validating accuracy
step += 1
except tf.errors.OutOfRangeError:
print('Done training for %d epochs, %d steps.' % (num_epochs, step))
#Final Training accuracy
#Final Validating accuracy
saver.save(sess, "save_path/model.ckpt")
finally:
coord.request_stop()
else:
# # read a batch of test set to verify the input function
# val_test_batch22, val_test_label_batch22, image_id_batch22 = sess.run([val_test_batch, val_test_label_batch, image_id_batch])
# print(val_test_batch22.shape) #(8, 43200)
# print(val_test_label_batch22.shape) #(8,)
# print(image_id_batch22)
# print(type(image_id_batch22[0])) # bytes
# print(type(image_id_batch22[0].decode())) # str
# coord.request_stop()
saver.restore(sess, "save_path/model.ckpt") #会将已经保存的变量值resotre到 变量中。
results = []
try:
step = 0
start_time = time.time()
while not coord.should_stop():
val_test_predictions2, val_test_logits2, val_test_acc2_batch, val_test_acc2, val_test_acc2_update,image_id_batch2, indices2, values2= sess.run([val_test_prediction, val_test_logits, val_test_accuracy_batch, val_test_accuracy, val_test_accuracy_update, image_id_batch, indices, values])
step += 1
results += batch_to_list_of_dicts(indices2, image_id_batch2)
if (step % 10 == 0):
print('Useless minibatch testing accuracy at step %d: %.6f' % (step, val_test_acc2_batch))
#print(val_test_logits2[0])
#print(indices2[0])
#print(values2[0])
#print(val_test_predictions2[0])
#print(val_test_acc2)
#print('Useless streaming testing accuracy at step %d: %.6f' % (step, val_test_acc2))
except tf.errors.OutOfRangeError:
print('Done testing in, %d steps.' % (step))
print('FInal Testing accuracy: %.6f' % (val_test_acc2_update))
'''Writing JSON data'''
#results = [{"image_id":"a0563eadd9ef79fcc137e1c60be29f2f3c9a65ea.jpg","label_id": [5,18,32]}]
print(len(results))
print(results[0:20])
with open('submit.json', 'w') as f:
json.dump(results, f)
finally:
coord.request_stop()
coord.join(threads)
train_flag = False
image_size = 120
num_labels = 80
if train_flag:
tfrecord_file = '../ai_challenger_scene_train_20170904/train.tfrecord'
# tfrecord_file_val = '../ai_challenger_scene_train_20170904/val.tfrecord' # validate while training
batch_size = 128
num_epochs = 10
read_tfrecord2(tfrecord_file, batch_size, train_flag)
else:
tfrecord_file = '../ai_challenger_scene_train_20170904/val.tfrecord' #test
batch_size = 16 # 要求metric能累加起来, 除不尽的话最后不足的，不够一个batch的部分不会被使用！！！
num_epochs = 1
read_tfrecord2(tfrecord_file, batch_size, train_flag)
# with open('submit.json', 'r') as file1:
# submit_data = json.load(file1)
# with open('scene_validation_annotations_20170908.json', 'r') as file2:
# ref_data1 = json.load(file2)
# with open('ref.json', 'r') as file2:
# ref_data2 = json.load(file2)
# with open('submit0.json', 'r') as file3:
# submit0_data = json.load(file3)
# 53879 7120
训练
Minibatch accuracy: 0.734375
Minibatch loss at step 4150: 39.479721 (7128.005 sec)
Minibatch accuracy: 0.781250
Minibatch loss at step 4160: 63.868481 (7146.708 sec)
Minibatch accuracy: 0.750000
Minibatch loss at step 4170: 38.228550 (7165.086 sec)
Minibatch accuracy: 0.820312
Minibatch loss at step 4180: 55.918961 (7183.481 sec)
Minibatch accuracy: 0.695312
Minibatch loss at step 4190: 51.741051 (7201.407 sec)
Minibatch accuracy: 0.757812
Minibatch loss at step 4200: 40.578758 (7219.511 sec)
Minibatch accuracy: 0.750000
2017-09-21 23:30:22.727027: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 128, current size 38)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
2017-09-21 23:30:22.727050: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 128, current size 38)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
Done training for 10 epochs, 4209 steps.
wayne@wayne-GE60-2OC-2OD-2OE:~/python/kaggle/Ai_challenger/classification/me_udacity$ python task1_train_val.py
测试（用的是验证集）
Useless minibatch testing accuracy at step 390: 0.125000
Useless minibatch testing accuracy at step 400: 0.250000
Useless minibatch testing accuracy at step 410: 0.062500
Useless minibatch testing accuracy at step 420: 0.062500
Useless minibatch testing accuracy at step 430: 0.000000
Useless minibatch testing accuracy at step 440: 0.187500
2017-09-22 07:33:42.005287: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: FIFOQueue '_1_batch/fifo_queue' is closed and has insufficient elements (requested 16, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT32, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/fifo_queue, batch/n)]]
Done testing in, 445 steps.
FInal Testing accuracy: 0.183427
和官网的验证脚本结果一致
wayne@wayne-GE60-2OC-2OD-2OE:~/python/kaggle/Ai_challenger/classification/me_udacity$ python scene_eval.py --submit ./submit.json --ref ./scene_validation_annotations_20170908.json
Evaluation time of your result: 3.187874 s
{'error': [], 'warning': [], 'score': '0.18342696629213484'}
wayne@wayne-GE60-2OC-2OD-2OE:~/python/kaggle/Ai_challenger/classification/me_udacity$