调了很多模型,包括目标检测的yolo,分类的cnn,efficientnet,这次做分割的unet和resunet。但是在调模型的时候,我发现很难真正学到什么经验,我只是在改输入输出路径,调bug,对输出的东西处理。最难的应该是调unet,花了很多时间在统一数据集的格式和输出的东西上,因为我不太理解gif,png,jpg存储的区别,cv2和Image读取图片的区别,位深1,8(灰度图和彩色图的区别),二值图(true,false)和普通灰度图(0-255)的区别,导致总是报错。
真正构建模型时,只是去调了参数,包括学习率,epoch,loss,但是经常会出现调了半天还不如第一次的情况,并且越跑越低;有时候跑的训练还可以,在测试上的结果又很低了。
在这里做一个总结。
在图像分割时用到的指标,应该是和混淆矩阵中的F1 score是一回事,但是计算方式会不一样,我的问题是,对于预测的mask和label的mask进行对比计算,通过混淆矩阵计算的值有0.9以上,但是用交并比计算的值只有0.6.
发现问题出现在,0和1弄反了,用代码计算出的混淆矩阵并不等于书上对应的TP,FP,TN,FN.计算出来的混淆矩阵是这样:
但实际上,我想算的是正类,白色,1的概率,对我来说1是tp,是第一列第一行的,而算法算的是第二行的。
混淆矩阵的求法:(对于二维图像,要用flatten展开)
from sklearn.metrics import confusion_matrix
imgpred = imgPredict.flatten()
imglab = imgLabel.flatten()
C=confusion_matrix(imgpred, imglab)
Dice score的求法:第一种是用medpy,第二种是直接求交并比。注意的点是label和pred的灰度、通道什么的都一致。
# 1
from medpy.metric import binary
def dice(seg, gt):
img1 = cv2.imread(seg)
img2 = cv2.imread(gt)
# transf = transforms.ToTensor()
img_tensor1 = np.array(img1)
img_tensor2 = np.array(img2)
return binary.dc(img_tensor1, img_tensor2)
# 2
def calculate_dice(seg_path,pred_path):
imgPredict = cv2.imread(seg_path)
imgLabel = cv2.imread(pred_path)
imgPredict = np.array(cv2.cvtColor(imgPredict, cv2.COLOR_BGR2GRAY) / 255., dtype=np.uint8)
imgLabel = np.array(cv2.cvtColor(imgLabel, cv2.COLOR_BGR2GRAY) / 255., dtype=np.uint8)
label = [1]
dice = []
for i in range(len(label)):
intersection = np.sum((imgLabel == label[i])*(imgPredict == label[i]))
# print("intersection:{}".format(intersection))
union = np.sum(imgPredict == label[i])+np.sum(imgLabel == label[i])
# print("union:{}".format(union))
dice.append(2*intersection/union)
return np.mean(dice)
这个里面内容还挺多的,也学不完的,这次主要是学了Datasets类怎么做。
Datasets类在dataset.py文件中,它的模板:
class Data_Loader(Dataset):
def __init__(self, data_path):
self.data_path = data_path
self.imgs_path = glob.glob(os.path.join(data_path, 'image/*.png'))
def augment(self, image, flipCode):
flip = cv2.flip(image, flipCode)
return flip
def __getitem__(self, index):
image_path = self.imgs_path[index]
label_path = image_path.replace('image', 'label')
image = cv2.imread(image_path)
# image.resize(256,256)
label = cv2.imread(label_path)
image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
label = cv2.cvtColor(label, cv2.COLOR_BGR2GRAY)
image = image.reshape(1, image.shape[0], image.shape[1])
label = label.reshape(1, label.shape[0], label.shape[1])
if label.max() > 1:
label = label / 255
# add some enhancement trick
# flipCode = random.choice([-1, 0, 1, 2])
# if flipCode != 2:
# image = self.augment(image, flipCode)
# label = self.augment(label, flipCode)
return image, label
def __len__(self):
return len(self.imgs_path)
记录在数据处理中遇到的一个问题,就说二值图和灰度图的统一化
二值图的矩阵只有true,false,灰度图是0-255. 我处理二值图的方式:先用cv2读,如果是nonetype,就是二值图,再用Image 转。
for f in fname:
fpath = './ori_data/label/' + f
# print(fpath)
img = cv2.imread(fpath)
if img is None: # is binary img
image = Image.open(fpath)
image = np.asarray(image)
# print(image.shape)
matrix = image.astype(np.uint8)
else:
image = Image.open(fpath)
matrix = np.asarray(img).astype(np.uint8)
# print(matrix.max())
matrix = cv2.resize(matrix,(512,512))
cv2.imwrite('./data/train/label/'+f[:-4] + '.png', matrix)
待学习的问题:
dataloader, dataset 和transfromer的关系。
随便画的,没调
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
# import seaborn as
# tips = pd.read_csv('tips.csv')
# tips['tip_pct'] = tips['tip'] / (tips['total_bill'] - tips['tip'])
# tips = {}
# tips['diceloss']=[0.957192,0.967573,0.966401,0.948316,0.966088]
# # tips =
# fig,axes = plt.subplots()
# tips.plot(kind='box',ax=axes)
# axes.set_ylabel('values of tip_pct')
# fig.savefig('p1.png') # 将绘制的图形保存为p1.png
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import pylab
data = [0.957192,0.967573,0.966401,0.948316,0.966088]
data2 = [0.954232,0.965143,0.962576,0.943105,0.96407]
df = pd.DataFrame()
# ------- Dice score ---------
df['U-Net (Cross-entropy loss'] = [0.654768,0.630031,0.645995,0.691208,0.6956]
df['U-Net (Dice loss)'] = [0.674023,0.666306,0.666497,0.713796,0.684237]
df['Gauss stretch'] = [0.429391,0.372636,0.400701,0.429882,0.43245]
df['ResUNet++'] = [0.53323,0.4572,0.52324,0.6425,0.5893]
print(df)
colors = ['pink', 'lightblue','lightgreen']
figure,axes=plt.subplots(figsize= (8, 10)) #得到画板、轴
bplot = axes.boxplot(df,patch_artist=True) #描点上色
for patch, color in zip(bplot['boxes'], colors):
patch.set_facecolor(color)
plt.grid(linestyle="--", alpha=0.3)
plt.savefig('ds.png')
一个额外的问题。
为什么会有梯度爆炸?
因为深度神经网络在更新w,b时,需要计算相应的梯度,其中梯度的表达式中就包含激活函数的梯度,当激活函数的导数>1时,并且隐藏层很多时,越靠近输入层的权重w的梯度就会越大(因为根据链式法则,越靠近隐藏层的权重w的梯度会包含更多的激活函数的导数的乘积),这样会导致梯度爆炸
表现?
梯度爆炸的表现就是模型没办法收敛,loss变化比较大,或者loss变成NAN
如何解决?
sigmoid或者Tanh会容易爆炸,隐藏层用relu比较好。这只是缓解梯度爆炸,解决得用梯度裁剪。