Caffe - 训练日志 log 可视化分析
Caffe - 训练日志 log 可视化分析
在采用 shell 脚本进行 caffe 训练时，可以输出训练过程到log 文件，如
$CAFFE_ROOT/build/tools/caffe train \
--solver=solver.prototxt \
--gpu 0 \
2>&1 | tee train.logCaffe 提供了对输出 log 文件的解析工具 - parse_log.py：
$CAFFE_ROOT/tools/extra/parse_log.py train.log ./输出两个解析文件：
train.log.train
train.log.test
其内容格式如：
NumIters,Seconds,LearningRate,loss
0.0,0.366678,0.05,4.30619
10.0,3.210073,0.05,2.73271
20.0,6.03005,0.05,8.48341
......NumIters,Seconds,LearningRate,acc/top-1,acc/top-5,loss
7000.0,2266.206901,0.05,0.240812,0.591906,2.67359
14000.0,4538.298707,0.05,0.42175,0.780375,2.01819
21000.0,6798.336418,0.05,0.491844,0.832719,1.7494
......根据解析的结果，即可绘制 train loss，test loss 和 accuracy 的变化曲线，如：
#
import pandas as pd
import matplotlib.pyplot as plt
train_log = pd.read_csv("train.log.train")
test_log = pd.read_csv("train.log.test")
_, ax1 = plt.subplots()
ax1.set_title("train loss and test loss")
ax1.plot(train_log["NumIters"], train_log["loss"], alpha=0.5)
ax1.plot(test_log["NumIters"], test_log["loss"], 'g')
ax1.set_xlabel('iteration')
ax1.set_ylabel('train loss')
plt.legend(loc='upper left')
ax2 = ax1.twinx()
ax2.plot(test_log["NumIters"], test_log["acc/top-1"], 'r')
ax2.plot(test_log["NumIters"], test_log["acc/top-5"], 'm')
ax2.set_ylabel('test accuracy')
plt.legend(loc='upper right')
plt.show()
print 'Done.'