评价一个用深度学习框架搭建的神经网络模型,除了精确度(比如目标检测中常用的map)指标之外,模型复杂度也必须要考虑,通常用正向推理的计算量(FLOPs)和参数个数(Parameters)来描述模型的复杂度。
YOLOv5s模型的参数信息如下:
from n params module arguments
0 -1 1 3520 models.common.Conv [3, 32, 6, 2, 2]
1 -1 1 18560 models.common.Conv [32, 64, 3, 2]
2 -1 1 18816 models.common.C3 [64, 64, 1]
3 -1 1 73984 models.common.Conv [64, 128, 3, 2]
4 -1 2 115712 models.common.C3 [128, 128, 2]
5 -1 1 295424 models.common.Conv [128, 256, 3, 2]
6 -1 3 625152 models.common.C3 [256, 256, 3]
7 -1 1 1180672 models.common.Conv [256, 512, 3, 2]
8 -1 1 1182720 models.common.C3 [512, 512, 1]
9 -1 1 656896 models.common.SPPF [512, 512, 5]
10 -1 1 131584 models.common.Conv [512, 256, 1, 1]
11 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
12 [-1, 6] 1 0 models.common.Concat [1]
13 -1 1 361984 models.common.C3 [512, 256, 1, False]
14 -1 1 33024 models.common.Conv [256, 128, 1, 1]
15 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
16 [-1, 4] 1 0 models.common.Concat [1]
17 -1 1 90880 models.common.C3 [256, 128, 1, False]
18 -1 1 147712 models.common.Conv [128, 128, 3, 2]
19 [-1, 14] 1 0 models.common.Concat [1]
20 -1 1 296448 models.common.C3 [256, 256, 1, False]
21 -1 1 590336 models.common.Conv [256, 256, 3, 2]
22 [-1, 10] 1 0 models.common.Concat [1]
23 -1 1 1182720 models.common.C3 [512, 512, 1, False]
24 [17, 20, 23] 1 229245 models.yolo.Detect [80, [[10, 13, 16, 30, 33, 23], [30, 61, 62, 45, 59, 119], [116, 90, 156, 198, 373, 326]], [128, 256, 512]]
FLOPS
FLOPs
GFLOPs
YOLOv5-6.x版本中,torch_utils.py
文件中的model_info
函数,负责计算并打印模型的参数信息(parameter、grandient、FLOPs):
# 打印模型的参数、FLOPs等信息
def model_info(model, verbose=False, img_size=640):
# Model information. img_size may be int or list, i.e. img_size=640 or img_size=[640, 320]
n_p = sum(x.numel() for x in model.parameters()) # number parameters
# 训练时: parameters = gradients, 验证时: gradients = 0
n_g = sum(x.numel() for x in model.parameters() if x.requires_grad) # number gradients
if verbose:
print(f"{'layer':>5} {'name':>40} {'gradient':>9} {'parameters':>12} {'shape':>20} {'mu':>10} {'sigma':>10}")
for i, (name, p) in enumerate(model.named_parameters()):
name = name.replace('module_list.', '')
print('%5g %40s %9s %12g %20s %10.3g %10.3g' %
(i, name, p.requires_grad, p.numel(), list(p.shape), p.mean(), p.std()))
# 调用thop库中的profile计算FLOPs
try: # FLOPs
from thop import profile
stride = max(int(model.stride.max()), 32) if hasattr(model, 'stride') else 32
# input
# img = torch.zeros((1, model.yaml.get('ch', 3), stride * 8, stride * 8), device=next(model.parameters()).device) # 帮助理解如何计算FLOPs的尝试
img = torch.zeros((1, model.yaml.get('ch', 3), stride, stride), device=next(model.parameters()).device) # input
flops = profile(deepcopy(model), inputs=(img,), verbose=False)[0] / 1E9 * 2 # stride GFLOPs
img_size = img_size if isinstance(img_size, list) else [img_size, img_size] # expand if int/float
fs = ', %.1f GFLOPs' % (flops * img_size[0] / stride * img_size[1] / stride) # 640x640 GFLOPs
# fs = ', %.1f GFLOPs' % (flops * img_size[0] / (stride * 8) * img_size[1] / (stride * 8)) # 640x640 GFLOPs # 帮助理解如何计算FLOPs的尝试
except (ImportError, Exception):
fs = ''
LOGGER.info(f"Model Summary: {len(list(model.modules()))} layers, {n_p} parameters, {n_g} gradients{fs}")
Model Summary: 270 layers, 7235389 parameters, 7235389 gradients, 16.5 GFLOPs
Fusing layers...
Model Summary: 213 layers, 7225885 parameters, 0 gradients
torch_utils.py
文件中的fuse_conv_and_bn
函数gradients = parameters
gradients = 0
深度学习中GPU和显存分析
分享一个FLOPs计算神器
Pytorch中计算自己模型的FLOPs | thop.profile() 方法 | yolov5s 网络模型参数量、计算量统计
Pytorch:model.train()和model.eval()用法和区别,以及model.eval()和torch.no_grad()的区别
【YOLOV5-5.x 源码解读】torch_utils.py