CNN网络量化--Two-Step Quantization for Low-bit Neural Networks

Two-Step Quantization for Low-bit Neural Networks
CVPR2018
https://github.com/wps712/Two-Step-Quantization-AlexNet
https://github.com/HolmesShuan/Optimal-Ternary-Weights-Approximation

本文的思路很简单：将CNN网络模型的 weight quantization 和 activation quantization 分开做。

这里将 activations 的量化看做一个 encodings 过程；weights 量化 看做一个 learn transformations。 针对 activations 量化，我们提出 sparse quantization method for code learning。 weight 量化 可以看做一个 a non-linear least square regression problem with low-bit constraints

这里的 sparse quantization 就是 只量化重要的值，不重要的 activations 全部清零。这里我们假定大的 activations 比较重要。

only quantize important values while set other unimportant values to zeros Here we explore the sparsity of activations，we assume big activations are more important than small activations. Thus the sparsity is introduced by setting all activations below a threshold to zeros.

对于 weight 量化，这里看做一个 non-linear least square regression problem with low-bit constraints 使用 低精度网络来近似高精度网络，看做一个函数逼近问题，将量化问题变为一个数学迭代优化问题

实验部分 只用了 AlexNet and VGG-16 网络，没有使用残差网络

11