【记录】为什么在CNN的Batch Norm中对C个channel进行归一化

下面是cs231n-assignment2中的Spatial Batch Norm的介绍中对于此处空间归一化的解释，为什么对shape为NCHW的图像数据进行Batch Norm时需要计算NHW维度的mean和var

If the feature map was produced using convolutions, then we expect every feature channel’s statistics e.g. mean, variance to be relatively consistent both between different images, and different locations within the same image – after all, every feature channel is produced by the same convolutional filter! Therefore spatial batch normalization computes a mean and variance for each of the C feature channels by computing statistics over the minibatch dimension N as well the spatial dimensions H and W.

如果使用卷积生成feature map，那么我们期望每个feature channel的统计数据，例如不同图像之间，以及同一图像中的不同位置之间的均值、方差相对一致 - 毕竟，每个feature channel都是由相同的filter卷积产生的！因此，空间批量归一化通过计算小批量维度N以及空间维度H和W的统计量来计算每个C feature channel的均值和方差。