深度学习（六十八）darknet使用

这几天因为要对yolo进行重新训练，需要用到imagenet pretrain，由于网络是自己设计的网络，所以需要先在darknet上训练imagenet，由于网上都没有相关的说明教程，特别是图片路径是怎么和类别标签对应起来的，让我百思不得其解，所以最后就自己去查看了darknet的源码，发现原来作者是用了字符串匹配，来查找图片路径字符串中是否有与类别标签字符串匹配的子字符串，以此判断该类别标签的。

1、darknet对于图片分类训练、验证命令为：

./darknet classifier train cfg/imagenet1k.data cfg/extraction.cfg extraction.weights

./darknet classifier valid cfg/imagenet1k.data cfg/extraction.cfg extraction.weights

2、数据格式：数据路径配置主要读取自：cfg/imagenet1k.data 

classes=1000
train  = imagenet/darknet_train.txt
valid  = imagenet/darknet_val.txt
backup = backup/
labels = data/imagenet.labels.list
names  = data/imagenet.shortnames.list
top=5

darknet_train.txt，darknet_val.txt的训练格式只有图片路径，比如：

/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10026.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10027.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10029.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10040.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10042.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10043.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10048.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10066.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10074.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_1009.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10095.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10108.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10110.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10120.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10124.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10150.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10159.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10162.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10183.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10194.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10211.JPEG
/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/n01440764/n01440764_10218.JPEG

那么darknet是怎么知道每一行图片路径，对应的类别标签的。其主要是从：

data/imagenet.labels.list

读取标签字符串，然后用类别标签字符串，匹配上面每一行的图片路径，查找是否有子字符串，以此确定类别标签，所以训练的时候，一定要确保图片路径包含了类别标签，比如：n01440764等就是类别标签。

3、由于imagenet的val图片是放在一起的，路径不包含标签，所以需要读取val标签.xml文件，把val的图片根据标签，重新存过一遍，放在对应的类别标签文件：

#coding=utf-8
import os
import shutil
from BeautifulSoup import BeautifulSoup
#train.txt可通过运行脚本caffe/data/get_ilsvrc_aux.sh下载获得
'''with open("../imagenet/train.txt") as f:
    with open("../imagenet/darknet_train.txt",'w') as w:
        for l in f.readlines():
            w.writelines('/home/research/disk1/imagenet/ILSVRC2015/Data/CLS-LOC/train/'+l.split()[0]+'\n')'''


#val
dataroot='/home/research/disk1/imagenet/ILSVRC2015/'
vallabel=dataroot+'Annotations/CLS-LOC/val'
valimage=dataroot+'Data/CLS-LOC/val'
with open("../imagenet/darknet_val.txt",'w') as w:
    for l in os.listdir(vallabel):

        xml = ""
        with open(os.path.join(vallabel,l)) as f:
            xml = f.readlines()
        xml = ''.join([line.strip('\t') for line in xml])

        label=BeautifulSoup(xml).find('name').string
        filename=BeautifulSoup(xml).find('filename').string+'.JPEG'

        saveroot='../temp/'+label
        if os.path.exists(saveroot) is False:
            os.makedirs(saveroot)
        shutil.copy(os.path.join(valimage,filename),os.path.join(saveroot,filename))
        w.writelines('/home/research/disk1/compress_yolo/temp/' + filename+ '\n')