基于tensorflow2的多种模型定点化方式
Copyright 2020 The TensorFlow Authors.
#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Quantization aware training comprehensive guide
| View on TensorFlow.org | Run in Google Colab | View source on GitHub | Download notebook |
Welcome to the comprehensive guide for Keras quantization aware training.
This page documents various use cases and shows how to use the API for each one. Once you know which APIs you need, find the parameters and the low-level details in the
API docs.
- If you want to see the benefits of quantization aware training and what’s supported, see the overview.
- For a single end-to-end example, see the quantization aware training example.
The following use cases are covered:
- Deploy a model with 8-bit quantization with these steps.
- Define a quantization aware model.
- For Keras HDF5 models only, use special checkpointing and
deserialization logic. Training is otherwise standard. - Create a quantized model from the quantization aware one.
- Experiment with quantization.
- Anything for experimentation has no supported path to deployment.
- Custom Keras layers fall under experimentation.
Setup
For finding the APIs you need and understanding purposes, you can run but skip reading this section.
! pip uninstall -y tensorflow
! pip install -q tf-nightly
! pip install -q tensorflow-model-optimization
import tensorflow as tf
import numpy as np
import tensorflow_model_optimization as tfmot
import tempfile
input_shape = [20]
x_train = np.random.randn(1, 20).astype(np.float32)
y_train = tf.keras.utils.to_categorical(np.random.randn(1), num_classes=20)
def setup_model():
model = tf.keras.Sequential([
tf.keras.layers.Dense(20, input_shape=input_shape),
tf.keras.layers.Flatten()
])
return model
def setup_pretrained_weights():
model= setup_model()
model.compile(
loss=tf.keras.losses.categorical_crossentropy,
optimizer='adam',
metrics=['accuracy']
)
model.fit(x_train, y_train)
_, pretrained_weights = tempfile.mkstemp('.tf')
model.save_weights(pretrained_weights)
return pretrained_weights
def setup_pretrained_model():
model = setup_model()
pretrained_weights = setup_pretrained_weights()
model.load_weights(pretrained_weights)
return model
setup_model()
pretrained_weights = setup_pretrained_weights()
WARNING: Skipping tensorflow as it is not installed.
1/1 [==============================] - 0s 1000us/step - loss: 1.1921e-07 - accuracy: 0.0000e+00
##Define quantization aware model
By defining models in the following ways, there are available paths to deployment to backends listed in the overview page. By default, 8-bit quantization is used.
Note: a quantization aware model is not actually quantized. Creating a quantized model is a separate step.
Quantize whole model
Your use case:
- Subclassed models are not supported.
Tips for better model accuracy:
- Try “Quantize some layers” to skip quantizing the layers that reduce accuracy the most.
- It’s generally better to finetune with quantization aware training as opposed to training from scratch.
To make the whole model aware of quantization, apply tfmot.quantization.keras.quantize_model to the model.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
quant_aware_model = tfmot.quantization.keras.quantize_model(base_model)
quant_aware_model.summary()
Model: "sequential_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
quantize_layer (QuantizeLaye (None, 20) 3
_________________________________________________________________
quant_dense_2 (QuantizeWrapp (None, 20) 425
_________________________________________________________________
quant_flatten_2 (QuantizeWra (None, 20) 1
=================================================================
Total params: 429
Trainable params: 420
Non-trainable params: 9
_________________________________________________________________
Quantize some layers
Quantizing a model can have a negative effect on accuracy. You can selectively quantize layers of a model to explore the trade-off between accuracy, speed, and model size.
Your use case:
- To deploy to a backend that only works well with fully quantized models (e.g. EdgeTPU v1, most DSPs), try “Quantize whole model”.
Tips for better model accuracy:
- It’s generally better to finetune with quantization aware training as opposed to training from scratch.
- Try quantizing the later layers instead of the first layers.
- Avoid quantizing critical layers (e.g. attention mechanism).
In the example below, quantize only the Dense layers.
# Create a base model
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
# Helper function uses `quantize_annotate_layer` to annotate that only the
# Dense layers should be quantized.
def apply_quantization_to_dense(layer):
if isinstance(layer, tf.keras.layers.Dense):
return tfmot.quantization.keras.quantize_annotate_layer(layer)
return layer
# Use `tf.keras.models.clone_model` to apply `apply_quantization_to_dense`
# to the layers of the model.
annotated_model = tf.keras.models.clone_model(
base_model,
clone_function=apply_quantization_to_dense,
)
# Now that the Dense layers are annotated,
# `quantize_apply` actually makes the model quantization aware.
quant_aware_model = tfmot.quantization.keras.quantize_apply(annotated_model)
quant_aware_model.summary()
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
Model: "sequential_3"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
quantize_layer_1 (QuantizeLa (None, 20) 3
_________________________________________________________________
quant_dense_3 (QuantizeWrapp (None, 20) 425
_________________________________________________________________
flatten_3 (Flatten) (None, 20) 0
=================================================================
Total params: 428
Trainable params: 420
Non-trainable params: 8
_________________________________________________________________
While this example used the type of the layer to decide what to quantize, the easiest way to quantize a particular layer is to set its name property, and look for that name in the clone_function.
print(base_model.layers[0].name)
dense_3
More readable but potentially lower model accuracy
This is not compatible with finetuning with quantization aware training, which is why it may be less accurate than the above examples.
Functional example
# Use `quantize_annotate_layer` to annotate that the `Dense` layer
# should be quantized.
i = tf.keras.Input(shape=(20,))
x = tfmot.quantization.keras.quantize_annotate_layer(tf.keras.layers.Dense(10))(i)
o = tf.keras.layers.Flatten()(x)
annotated_model = tf.keras.Model(inputs=i, outputs=o)
# Use `quantize_apply` to actually make the model quantization aware.
quant_aware_model = tfmot.quantization.keras.quantize_apply(annotated_model)
# For deployment purposes, the tool adds `QuantizeLayer` after `InputLayer` so that the
# quantized model can take in float inputs instead of only uint8.
quant_aware_model.summary()
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 20)] 0
_________________________________________________________________
quantize_layer_2 (QuantizeLa (None, 20) 3
_________________________________________________________________
quant_dense_4 (QuantizeWrapp (None, 10) 215
_________________________________________________________________
flatten_4 (Flatten) (None, 10) 0
=================================================================
Total params: 218
Trainable params: 210
Non-trainable params: 8
_________________________________________________________________
Sequential example
# Use `quantize_annotate_layer` to annotate that the `Dense` layer
# should be quantized.
annotated_model = tf.keras.Sequential([
tfmot.quantization.keras.quantize_annotate_layer(tf.keras.layers.Dense(20, input_shape=input_shape)),
tf.keras.layers.Flatten()
])
# Use `quantize_apply` to actually make the model quantization aware.
quant_aware_model = tfmot.quantization.keras.quantize_apply(annotated_model)
quant_aware_model.summary()
Model: "sequential_4"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
quantize_layer_3 (QuantizeLa (None, 20) 3
_________________________________________________________________
quant_dense_5 (QuantizeWrapp (None, 20) 425
_________________________________________________________________
flatten_5 (Flatten) (None, 20) 0
=================================================================
Total params: 428
Trainable params: 420
Non-trainable params: 8
_________________________________________________________________
Checkpoint and deserialize
Your use case: this code is only needed for the HDF5 model format (not HDF5 weights or other formats).
# Define the model.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
quant_aware_model = tfmot.quantization.keras.quantize_model(base_model)
# Save or checkpoint the model.
_, keras_model_file = tempfile.mkstemp('.h5')
quant_aware_model.save(keras_model_file)
# `quantize_scope` is needed for deserializing HDF5 models.
with tfmot.quantization.keras.quantize_scope():
loaded_model = tf.keras.models.load_model(keras_model_file)
loaded_model.summary()
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
WARNING:tensorflow:No training configuration found in the save file, so the model was *not* compiled. Compile it manually.
Model: "sequential_5"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
quantize_layer_4 (QuantizeLa (None, 20) 3
_________________________________________________________________
quant_dense_6 (QuantizeWrapp (None, 20) 425
_________________________________________________________________
quant_flatten_6 (QuantizeWra (None, 20) 1
=================================================================
Total params: 429
Trainable params: 420
Non-trainable params: 9
_________________________________________________________________
Create and deploy quantized model
In general, reference the documentation for the deployment backend that you
will use.
This is an example for the TFLite backend.
base_model = setup_pretrained_model()
quant_aware_model = tfmot.quantization.keras.quantize_model(base_model)
# Typically you train the model here.
converter = tf.lite.TFLiteConverter.from_keras_model(quant_aware_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
quantized_tflite_model = converter.convert()
1/1 [==============================] - 0s 1000us/step - loss: 1.5351 - accuracy: 0.0000e+00
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
WARNING:tensorflow:From d:\python_virtualenv\tf2.1_gpu\lib\site-packages\tensorflow\python\keras\backend.py:465: set_learning_phase (from tensorflow.python.keras.backend) is deprecated and will be removed after 2020-10-11.
Instructions for updating:
Simply pass a True/False value to the `training` argument of the `__call__` method of your layer or model.
WARNING:tensorflow:From d:\python_virtualenv\tf2.1_gpu\lib\site-packages\tensorflow\python\training\tracking\tracking.py:105: Network.state_updates (from tensorflow.python.keras.engine.network) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
INFO:tensorflow:Assets written to: C:\Users\ADMINI~1\AppData\Local\Temp\tmp4nz2ufyn\assets
Experiment with quantization
Your use case: using the following APIs means that there is no
supported path to deployment. The features are also experimental and not
subject to backward compatibility.
tfmot.quantization.keras.QuantizeConfigtfmot.quantization.keras.quantizers.Quantizertfmot.quantization.keras.quantizers.LastValueQuantizertfmot.quantization.keras.quantizers.MovingAverageQuantizer
Setup: DefaultDenseQuantizeConfig
Experimenting requires using tfmot.quantization.keras.QuantizeConfig, which describes how to quantize the weights, activations, and outputs of a layer.
Below is an example that defines the same QuantizeConfig used for the Dense layer in the API defaults.
During the forward propagation in this example, the LastValueQuantizer returned in get_weights_and_quantizers is called with layer.kernel as the input, producing an output. The output replaces layer.kernel
in the original forward propagation of the Dense layer, via the logic defined in set_quantize_weights. The same idea applies to the activations and outputs.
LastValueQuantizer = tfmot.quantization.keras.quantizers.LastValueQuantizer
MovingAverageQuantizer = tfmot.quantization.keras.quantizers.MovingAverageQuantizer
class DefaultDenseQuantizeConfig(tfmot.quantization.keras.QuantizeConfig):
# Configure how to quantize weights.
def get_weights_and_quantizers(self, layer):
return [(layer.kernel, LastValueQuantizer(num_bits=8, symmetric=True, narrow_range=False, per_axis=False))]
# Configure how to quantize activations.
def get_activations_and_quantizers(self, layer):
return [(layer.activation, MovingAverageQuantizer(num_bits=8, symmetric=False, narrow_range=False, per_axis=False))]
def set_quantize_weights(self, layer, quantize_weights):
# Add this line for each item returned in `get_weights_and_quantizers`
# , in the same order
layer.kernel = quantize_weights[0]
def set_quantize_activations(self, layer, quantize_activations):
# Add this line for each item returned in `get_activations_and_quantizers`
# , in the same order.
layer.activation = quantize_activations[0]
# Configure how to quantize outputs (may be equivalent to activations).
def get_output_quantizers(self, layer):
return []
def get_config(self):
return {}
Quantize custom Keras layer
This example uses the DefaultDenseQuantizeConfig to quantize the CustomLayer.
Applying the configuration is the same across
the “Experiment with quantization” use cases.
- Apply
tfmot.quantization.keras.quantize_annotate_layerto theCustomLayerand pass in theQuantizeConfig. - Use
tfmot.quantization.keras.quantize_annotate_modelto continue to quantize the rest of the model with the API defaults.
quantize_annotate_layer = tfmot.quantization.keras.quantize_annotate_layer
quantize_annotate_model = tfmot.quantization.keras.quantize_annotate_model
quantize_scope = tfmot.quantization.keras.quantize_scope
class CustomLayer(tf.keras.layers.Dense):
pass
model = quantize_annotate_model(tf.keras.Sequential([
quantize_annotate_layer(CustomLayer(20, input_shape=(20,)), DefaultDenseQuantizeConfig()),
tf.keras.layers.Flatten()
]))
# `quantize_apply` requires mentioning `DefaultDenseQuantizeConfig` with `quantize_scope`
# as well as the custom Keras layer.
with quantize_scope(
{'DefaultDenseQuantizeConfig': DefaultDenseQuantizeConfig,
'CustomLayer': CustomLayer}):
# Use `quantize_apply` to actually make the model quantization aware.
quant_aware_model = tfmot.quantization.keras.quantize_apply(model)
quant_aware_model.summary()
Model: "sequential_8"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
quantize_layer_6 (QuantizeLa (None, 20) 3
_________________________________________________________________
quant_custom_layer (Quantize (None, 20) 425
_________________________________________________________________
quant_flatten_9 (QuantizeWra (None, 20) 1
=================================================================
Total params: 429
Trainable params: 420
Non-trainable params: 9
_________________________________________________________________
Modify quantization parameters
Common mistake: quantizing the bias to fewer than 32-bits usually harms model accuracy too much.
This example modifies the Dense layer to use 4-bits for its weights instead
of the default 8-bits. The rest of the model continues to use API defaults.
quantize_annotate_layer = tfmot.quantization.keras.quantize_annotate_layer
quantize_annotate_model = tfmot.quantization.keras.quantize_annotate_model
quantize_scope = tfmot.quantization.keras.quantize_scope
class ModifiedDenseQuantizeConfig(DefaultDenseQuantizeConfig):
# Configure weights to quantize with 4-bit instead of 8-bits.
def get_weights_and_quantizers(self, layer):
return [(layer.kernel, LastValueQuantizer(num_bits=4, symmetric=True, narrow_range=False, per_axis=False))]
Applying the configuration is the same across
the “Experiment with quantization” use cases.
- Apply
tfmot.quantization.keras.quantize_annotate_layerto theDenselayer and pass in theQuantizeConfig. - Use
tfmot.quantization.keras.quantize_annotate_modelto continue to quantize the rest of the model with the API defaults.
model = quantize_annotate_model(tf.keras.Sequential([
# Pass in modified `QuantizeConfig` to modify this Dense layer.
quantize_annotate_layer(tf.keras.layers.Dense(20, input_shape=(20,)), ModifiedDenseQuantizeConfig()),
tf.keras.layers.Flatten()
]))
# `quantize_apply` requires mentioning `ModifiedDenseQuantizeConfig` with `quantize_scope`:
with quantize_scope(
{'ModifiedDenseQuantizeConfig': ModifiedDenseQuantizeConfig}):
# Use `quantize_apply` to actually make the model quantization aware.
quant_aware_model = tfmot.quantization.keras.quantize_apply(model)
quant_aware_model.summary()
Model: "sequential_9"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
quantize_layer_7 (QuantizeLa (None, 20) 3
_________________________________________________________________
quant_dense_9 (QuantizeWrapp (None, 20) 425
_________________________________________________________________
quant_flatten_10 (QuantizeWr (None, 20) 1
=================================================================
Total params: 429
Trainable params: 420
Non-trainable params: 9
_________________________________________________________________
Modify parts of layer to quantize
This example modifies the Dense layer to skip quantizing the activation. The rest of the model continues to use API defaults.
quantize_annotate_layer = tfmot.quantization.keras.quantize_annotate_layer
quantize_annotate_model = tfmot.quantization.keras.quantize_annotate_model
quantize_scope = tfmot.quantization.keras.quantize_scope
class ModifiedDenseQuantizeConfig(DefaultDenseQuantizeConfig):
def get_activations_and_quantizers(self, layer):
# Skip quantizing activations.
return []
def set_quantize_activations(self, layer, quantize_activations):
# Empty since `get_activaations_and_quantizers` returns
# an empty list.
return
Applying the configuration is the same across
the “Experiment with quantization” use cases.
- Apply
tfmot.quantization.keras.quantize_annotate_layerto theDenselayer and pass in theQuantizeConfig. - Use
tfmot.quantization.keras.quantize_annotate_modelto continue to quantize the rest of the model with the API defaults.
model = quantize_annotate_model(tf.keras.Sequential([
# Pass in modified `QuantizeConfig` to modify this Dense layer.
quantize_annotate_layer(tf.keras.layers.Dense(20, input_shape=(20,)), ModifiedDenseQuantizeConfig()),
tf.keras.layers.Flatten()
]))
# `quantize_apply` requires mentioning `ModifiedDenseQuantizeConfig` with `quantize_scope`:
with quantize_scope(
{'ModifiedDenseQuantizeConfig': ModifiedDenseQuantizeConfig}):
# Use `quantize_apply` to actually make the model quantization aware.
quant_aware_model = tfmot.quantization.keras.quantize_apply(model)
quant_aware_model.summary()
Model: "sequential_10"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
quantize_layer_8 (QuantizeLa (None, 20) 3
_________________________________________________________________
quant_dense_10 (QuantizeWrap (None, 20) 423
_________________________________________________________________
quant_flatten_11 (QuantizeWr (None, 20) 1
=================================================================
Total params: 427
Trainable params: 420
Non-trainable params: 7
_________________________________________________________________
Use custom quantization algorithm
The tfmot.quantization.keras.quantizers.Quantizer class is a callable that
can apply any algorithm to its inputs.
In this example, the inputs are the weights, and we apply the math in the
FixedRangeQuantizer __call__ function to the weights. Instead of the original
weights values, the output of the
FixedRangeQuantizer is now passed to whatever would have used the weights.
quantize_annotate_layer = tfmot.quantization.keras.quantize_annotate_layer
quantize_annotate_model = tfmot.quantization.keras.quantize_annotate_model
quantize_scope = tfmot.quantization.keras.quantize_scope
class FixedRangeQuantizer(tfmot.quantization.keras.quantizers.Quantizer):
"""Quantizer which forces outputs to be between -1 and 1."""
def build(self, tensor_shape, name, layer):
# Not needed. No new TensorFlow variables needed.
return {}
def __call__(self, inputs, training, weights, **kwargs):
return tf.keras.backend.clip(inputs, -1.0, 1.0)
def get_config(self):
# Not needed. No __init__ parameters to serialize.
return {}
class ModifiedDenseQuantizeConfig(DefaultDenseQuantizeConfig):
# Configure weights to quantize with 4-bit instead of 8-bits.
def get_weights_and_quantizers(self, layer):
# Use custom algorithm defined in `FixedRangeQuantizer` instead of default Quantizer.
return [(layer.kernel, FixedRangeQuantizer())]
Applying the configuration is the same across
the “Experiment with quantization” use cases.
- Apply
tfmot.quantization.keras.quantize_annotate_layerto theDenselayer and pass in theQuantizeConfig. - Use
tfmot.quantization.keras.quantize_annotate_modelto continue to quantize the rest of the model with the API defaults.
model = quantize_annotate_model(tf.keras.Sequential([
# Pass in modified `QuantizeConfig` to modify this `Dense` layer.
quantize_annotate_layer(tf.keras.layers.Dense(20, input_shape=(20,)), ModifiedDenseQuantizeConfig()),
tf.keras.layers.Flatten()
]))
# `quantize_apply` requires mentioning `ModifiedDenseQuantizeConfig` with `quantize_scope`:
with quantize_scope(
{'ModifiedDenseQuantizeConfig': ModifiedDenseQuantizeConfig}):
# Use `quantize_apply` to actually make the model quantization aware.
quant_aware_model = tfmot.quantization.keras.quantize_apply(model)
quant_aware_model.summary()
Model: "sequential_11"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
quantize_layer_9 (QuantizeLa (None, 20) 3
_________________________________________________________________
quant_dense_11 (QuantizeWrap (None, 20) 423
_________________________________________________________________
quant_flatten_12 (QuantizeWr (None, 20) 1
=================================================================
Total params: 427
Trainable params: 420
Non-trainable params: 7
_________________________________________________________________