python项目应用实例(二)制作数据集相关: label更改与批量化图片处理

背景:制作数据集时,时常需要对图片进行重命名、格式转换、以及批量处理等等。python可以方便的实现这些操作。我们对这种程序进行汇总,以便以后查阅。先汇总程序,然后汇总各种模块化的功能如何实现。

博主代码地址:https://github.com/Xingxiangrui/Batch_data_preprocess

目录

一、字符串相关

1.1 ground truth格式转换

1.2 预测生成bounding box

1.3 图片及路径写入txt文件

1.4 split分开为不同元素

二、图片处理相关

2.1 图片重命名写入写出

2.2查看图像任一点坐标

2.3 批量重命名

三、序列乱序(拆训练集与验证集)

序列乱序

list拆为两个list

图像与标签写入

图像读出与写入

标签的生成

四、字典元素更改


一、字符串相关

程序不难看懂。

1.1 ground truth格式转换

转换背景:COCO与YOLO的数据集的bounding box的标注方式是不同的。其中一个是[ left,top,right,bottom ],另一个是 [ x,y , width, height ],这就要涉及转换,及写入。这个程序就是解决这个问题。

除了背景转换之外,还需要将图片读出并根据图片的名称创建相应的标注文档 txt。

"""
author:xing xiangrui
time  :2018.9.26   10:34
copy image and change name to the new directory
write image name and labels to the list.txt
"""
import os
import shutil
oldDir = "data"
newDir = "head_data"
width = 352 
height = 288
fw = open('list.txt','w')
for root,dirs,files in os.walk(oldDir):
    if 'images' in root:
        continue
    for file in files:
        if 'jpg' in file:
            # copy jpg
            oldPic = root + os.path.sep + file
            newPic = newDir + os.path.sep + root.split(os.path.sep)[-2] + "_" + root.split(os.path.sep)[-1] + "_" + file
            shutil.copyfile(oldPic,newPic)
            
            fw.write(newPic)
            # write txt
            oldTxt = root + os.path.sep + file.replace('jpg','txt')
            fo = open(oldTxt,'r')
            lines = fo.readlines()
            for line in lines:
                line = line.split()
                cx, cy, w, h = float(line[1])*width, float(line[2])*height, float(line[3])*width, float(line[4])*height
                #[x, y, w, h] = [float(line[i]) for i in range(1,5)]
                x1 = round(cx - w/2, 2)
                y1 = round(cy - h/2, 2)
                x2 = round(cx + w/2, 2)
                y2 = round(cy + h/2, 2)
                print(x1,y1,x2,y2)
                fw.write(" " + str(x1) + " " + str(y1) + " " + str(x2) + " " + str(y2))
            fw.write("\n")
            fo.close()
            #print("fine")
print("done")
fw.close()

1.2 预测生成bounding box

程序目的: 将网络预测结果按照mAP测试的格式写入文件,然后用系统调用mAP运行的程序运行测得mAP

                for tmp_file in jpg_list:
                    img=cv2.imread(tmp_file)
                    # add ROI region
                    ROI=img[ROI_idx[0]:ROI_idx[1],ROI_idx[2]:ROI_idx[3]]
                    ROI_temp=ROI.copy()
                    img[:,:,:]=0
                    img[ROI_idx[0]:ROI_idx[1],ROI_idx[2]:ROI_idx[3]]=ROI_temp
                    #create txt file
                    tmp_file=tmp_file.replace("jpg","txt")
                    txt_filename=tmp_file.replace("images","predicted")
                    print("LOACTION!!!predict:"+tmp_file)
                    
#                    start_time = time.time()
                    #print("LOCATION!!!detect_face function start"+"\n")
                    rectangles, points = detect_face(img, args.minsize,
                                                     pnet_fun, rnet_fun, onet_fun,
                                                     args.threshold, args.factor)
                    #print("LOCATION!!!idetect_face function done"+"\n")
#                    duration = time.time() - start_time
    
#                    print("duration:"+str(duration))
                    #print(type(rectangles))
                    points = np.transpose(points)
                    #print("LOCATION!!!loop rectangles"+"\n")
                    with open(txt_filename,'w') as result_file:
                        for rectangle in rectangles:
                            result_file.write("head" + " " + str(rectangle[4]) + " " + str(rectangle[0]) + " " + str(rectangle[1]) + " " + str(rectangle[2]) +  " " + str(rectangle[3])+"\n")
                    #print("LOCATION!!!Write done!"+"\n")
                print(ROI_idx)
                os.chdir("mAP/")
                os.system("python main.py -na")

更改文件夹,用os.chdir即可。然后运行程序直接 os.system("需要的命令行")

1.3 图片及路径写入txt文件

运用见这篇文章。就是将文件夹之中所有jpg格式的文件路径写入train.txt 方便后续训练。

 YOLOv3:Darknet代码解析(四)结构更改与训练

"""
Created on Wed Dec 13, 15:14:53 2017
批量写入 train.txt
@auther xingxiangrui
"""        
import os
#root = os.getcwd()
#picpath= '\\0\\1_f\\'
#filename=os.listdir(root+picpath)
#for file in filename:
#    if 'jpg' in file:
#        print(file)
openfile=open('train.txt','w')

for root,dirs,files in os.walk(os.getcwd())
    for item in files:
        if 'jpg' in item:
            openfile.write(os.path.join(root,item)+'\n')
        if '.mat' in item or '.m' in item
            os.remove(os.path.join(root,item))
openfile.close()

1.4 split分开为不同元素

split的用法,通过符号对字符串进行分割--gpu_ids 0,1,2,3

        # set gpu ids
        str_ids = opt.gpu_ids.split(',')
        opt.gpu_ids = []
        for str_id in str_ids:
            id = int(str_id)
            if id >= 0:
                opt.gpu_ids.append(id)
        if len(opt.gpu_ids) > 0:
            torch.cuda.set_device(opt.gpu_ids[0])

二、图片处理相关

2.1 图片重命名写入写出

# -*- coding: utf-8 -*
"""
Created by Xingxiangrui on 2019.4.15
This code is created to copy and rename images from the source_dir
 to the target_dir
"""

#import matplotlib.pyplot as plt
import numpy as np
from PIL import Image
import os

#image source directory
#---------------------here need to be changed---------dir location---------
source_dir="/Users/baidu/Desktop/trainingSet/pired-images/lambda_10_idt_0.5/tear/"
target_dir="/Users/baidu/Desktop/trainingSet/pired-images/lambda_10_idt_0.5/tear/augmented-tear/"
if not os.path.isdir(target_dir):
    os.makedirs(target_dir)

#images list in source directory
source_imgs_list = os.listdir(source_dir)
print source_imgs_list

#for each image in the images list
for source_img_name in source_imgs_list:
    if '.jpg' in source_img_name:
        print(source_img_name)

        #read images
        path_source_img = os.path.join(source_dir, source_img_name)
        src_img = Image.open(path_source_img) 
        #src_img.show()

        #---------------------here need to be changed---------rename--------
        target_img_name=source_img_name.replace('.jpg','_real_A.png')
        path_target_img=os.path.join(target_dir, target_img_name)
        print(path_target_img)
        #save new reanmed img to the target dir
        src_img.save(path_target_img)

判断文件夹是否存在,如果不存在;则创建文件夹:

if not os.path.isdir(target_dir):
    os.makedirs(target_dir)

2.2查看图像任一点坐标

import matplotlib.pyplot as plt
import matplotlib as mpl

mpl.rcParams['font.family'] = 'sans-serif'
mpl.rcParams['font.sans-serif'] = 'NSimSun,Times New Roman'
path = input('please enter your pic path: ')
img = plt.imread(path)
plt.imshow(img)
plt.show()

2.3 批量重命名

https://blog.csdn.net/weixin_42854038/article/details/81332779

#coding=gbk
import os
import sys
def rename():
    path=input("请输入路径(例如D:\\\\picture):")
    name=input("请输入开头名:")
    startNumber=input("请输入开始数:")
    fileType=input("请输入后缀名(如 .jpg、.txt等等):")
    print("正在生成以"+name+startNumber+fileType+"迭代的文件名")
    count=0
    filelist=os.listdir(path)
    for files in filelist:
        Olddir=os.path.join(path,files)
        if os.path.isdir(Olddir):
            continue
        Newdir=os.path.join(path,name+str(count+int(startNumber))+fileType)
        os.rename(Olddir,Newdir)
        count+=1
    print("一共修改了"+str(count)+"个文件")

rename() 

三、序列乱序(拆训练集与验证集)

https://blog.csdn.net/matrix_google/article/details/72803741

序列乱序

参考:https://blog.csdn.net/amy_wumanfang/article/details/64483340

https://blog.csdn.net/matrix_google/article/details/72803741

直接 random.shuffle(list),直接讲list乱序后存入自己,很方便。

例如:

# -*- coding: utf-8 -*-
import random
# 对list洗牌,在原list上做改变
list = range(10)
print list
random.shuffle(list)
print "随机排序列表 : ",  list

list拆为两个list

拆为训练集和验证集,分别1/4和3/4

# train list and val list
source_train_list=[]
source_val_list=[]
for idx in range(len(source_image_list)):
    if idx

图像与标签写入

图像读出与写入

图像存于src_img之中,图像重命名用后用save写入。

    # read dource images and rename
    path_source_img = os.path.join(source_image_dir, source_image_name)
    src_img = Image.open(path_source_img)
    full_image_name=prefix+"_train_"+source_image_name
    print(full_image_name)
    # save renamed image to the target dir
    target_image_path=os.path.join(target_image_dir, full_image_name)
    src_img.save(target_image_path)

标签的生成

创建文件:a表示追加写入

# create label_file or write label file
txt_file_train_name="train.txt"
txt_file_val_name="val.txt"
txt_file_train_path=os.path.join(txt_file_dir, txt_file_train_name)
txt_file_val_path=os.path.join(txt_file_dir, txt_file_val_name)
train_txt_file= open(txt_file_train_path,"a")
val_txt_file= open(txt_file_val_path,"a")

有必要对每行加一个"\n"进行结尾

    # write image names and labels
    line_strings= full_image_name+"\t"+str(class_label)+"\n"
    train_txt_file.write(line_strings)

四、字典元素更改

https://www.cnblogs.com/volcao/p/8780725.html

字典key_value容易更改,确定key_value更改key不太容易更改

可以用pop方法进行更改key的值:

思路:先删除原键值对,保存值,然后以新键插入字典

格式:dict[newkey] = dict.pop(key)

d = {'a':1, 'aa':11}
d['b'] = d.pop('a')
d['bb'] = d.pop('aa')
print(d)
#输出:{'b': 1, 'bb': 11}

#执行语句:d.pop('a'),删除 'a' 所对应的键值对,返回 'a' 对应的值;d['b'],相当于给字典新添加一个key,其value为d.pop('a')返回的值。

d = {'a':1, 'aa':11}
c = d.pop('a')
print(c)
#输出:1

 

 

 

你可能感兴趣的:(python)