Pocketflow 使用

Pocketflow的简介：https://pocketflow.github.io/

由两个modules组成：Learner module 负责利用随机hyperparameters压缩并fine-tuning，optimizer module 负责依据随机参数对应的结果确定最佳的压缩参数。Learn module 遍历的hyperparameter来自于：

Name	Description
ChannelPrunedLearner	channel pruning with LASSO-based channel selection (He et al., 2017)
DisChnPrunedLearner	discrimination-aware channel pruning (Zhuang et al., 2018)
WeightSparseLearner	weight sparsification with dynamic pruning schedule (Zhu & Gupta, 2017)
UniformQuantLearner	weight quantization with uniform reconstruction levels (Jacob et al., 2018)
UniformQuantTFLearner	weight quantization with uniform reconstruction levels and TensorFlow APIs
NonUniformQuantLearner	weight quantization with non-uniform reconstruction levels (Han et al., 2016)

压缩后可以快速训练收敛。

optimizer module 的策略来自于

Gaussian Processes (GP, Mockus, 1975), Tree-structured Parzen Estimator (TPE, Bergstra et al., 2013), and Deterministic Deep Policy Gradients (DDPG, Lillicrap et al., 2016) 以及当前hyperparameters的reward

安装Pocketflow：https://pocketflow.github.io/installation/

本地安装或在docker环境中安装

本地安装：

配置文件 path.conf 说明数据集路径(.tfrecord)以及预训练模型的路径

首先安装 anaconda 和 tensorflow，注意python 版本不能高于3.6

一句话就可训练：./scripts/run_local.sh nets/resnet_at_cifar10_run.py

docker环境安装：

官方推荐  horovod docker ，因其支持多GPU分布

一句话训练：./scripts/run_docker.sh nets/resnet_at_cifar10_run.py

 

压缩方法：

参考https://pocketflow.github.io/tutorial/

输出：*.tflite 文件

Step 1: 准备数据成TFRecord格式

Step 2: 根据上面的步骤训练或直接下载 .ckpt 模型 保存在 models

Step 3: 选择压缩方法 开始调优压缩后的模型 

./scripts/run_local.sh nets/resnet_at_ilsvrc12_run.py \ --learner dis-chn-pruned

压缩方法与arg对应：

Learner name	Learner class	Note
full-prec	FullPrecLearner	No model compression
channel	ChannelPrunedLearner	Channel pruning with LASSO-based channel selection (He et al., 2017)
dis-chn-pruned	DisChnPrunedLearner	Discrimination-aware channel pruning (Zhuang et al., 2018)
weight-sparse	WeightSparseLearner	Weight sparsification with dynamic pruning schedule (Zhu & Gupta, 2017)
uniform	UniformQuantLearner	Weight quantization with uniform reconstruction levels (Jacob et al., 2018)
uniform-tf	UniformQuantTFLearner	Weight quantization with uniform reconstruction levels and TensorFlow APIs
non-uniform	NonUniformQuantLearner	Weight quantization with non-uniform reconstruction levels (Han et al., 2016)

保存在models_dcp_eval文件夹

Step 3: 转化成 tflite 格式：

python tools/conversion/export_pb_tflite_models.py \ --model_dir models_dcp_eval

保存在 models_dcp_eval