<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Quantization aware training in Keras example
Overview
Welcome to an end-to-end example for quantization aware training.
Learning Objectives
Train a tf.keras model for MNIST from scratch.
Fine tune the model by applying the quantization aware training API, see the accuracy, and export a quantization aware model.
Use the model to create an actually quantized model for the TFLite backend.
See the persistence of accuracy in TFLite and a 4x smaller model. To see the latency benefits on mobile, try out the TFLite examples in the TFLite app repository.
Introduction
Quantization aware training emulates inference-time quantization, creating a model that downstream tools will use to produce actually quantized models. The quantized models use lower-precision (e.g. 8-bit instead of 32-bit float), leading to benefits during deployment.
Each learning objective will correspond to a #TODO in the student lab notebook -- try to complete that notebook first before reviewing this solution notebook.
Setup
Step1:  This notebook uses TF2.x.
Please check your tensorflow version using the cell below.
Step2:  Train a model for MNIST without quantization aware training
Step3:  Clone and fine-tune pre-trained model with quantization aware training
Define the model
You will apply quantization aware training to the whole model and see this in the model summary. All layers are now prefixed by "quant".
Note that the resulting model is quantization aware but not quantized (e.g. the weights are float32 instead of int8). The sections after show how to create a quantized model from the quantization aware one.
In the comprehensive guide, you can see how to quantize some layers for model accuracy improvements.
Step4:  Train and evaluate the model against baseline
To demonstrate fine tuning after training the model for just an epoch, fine tune with quantization aware training on a subset of the training data.
Step5:  For this example, there is minimal to no loss in test accuracy after quantization aware training, compared to the baseline.
Step6:  Create quantized model for TFLite backend
After this, you have an actually quantized model with int8 weights and uint8 activations.
Step7:  See persistence of accuracy from TF to TFLite
Define a helper function to evaluate the TF Lite model on the test dataset.
Step8:  You evaluate the quantized model and see that the accuracy from TensorFlow persists to the TFLite backend.
Step9:  See 4x smaller model from quantization
You create a float TFLite model and then see that the quantized TFLite model
is 4x smaller.
<ASSISTANT_TASK:>
Python Code:
! pip uninstall -y tensorflow
! pip install -q tensorflow-model-optimization
! pip install --upgrade tensorflow==2.6
import tempfile
import os
import tensorflow as tf
from tensorflow import keras
Explanation: Quantization aware training in Keras example
Overview
Welcome to an end-to-end example for quantization aware training.
Learning Objectives
Train a tf.keras model for MNIST from scratch.
Fine tune the model by applying the quantization aware training API, see the accuracy, and export a quantization aware model.
Use the model to create an actually quantized model for the TFLite backend.
See the persistence of accuracy in TFLite and a 4x smaller model. To see the latency benefits on mobile, try out the TFLite examples in the TFLite app repository.
Introduction
Quantization aware training emulates inference-time quantization, creating a model that downstream tools will use to produce actually quantized models. The quantized models use lower-precision (e.g. 8-bit instead of 32-bit float), leading to benefits during deployment.
Each learning objective will correspond to a #TODO in the student lab notebook -- try to complete that notebook first before reviewing this solution notebook.
Setup
End of explanation
# Show the currently installed version of TensorFlow
print("TensorFlow version: ",tf.version.VERSION)
Explanation: This notebook uses TF2.x.
Please check your tensorflow version using the cell below.
End of explanation
# Load MNIST dataset
mnist = keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
# Normalize the input image so that each pixel value is between 0 to 1.
train_images = train_images / 255.0
test_images = test_images / 255.0
# Define the model architecture.
# TODO
model = keras.Sequential([
  keras.layers.InputLayer(input_shape=(28, 28)),
  keras.layers.Reshape(target_shape=(28, 28, 1)),
  keras.layers.Conv2D(filters=12, kernel_size=(3, 3), activation='relu'),
  keras.layers.MaxPooling2D(pool_size=(2, 2)),
  keras.layers.Flatten(),
  keras.layers.Dense(10)
])
# Train the digit classification model
# TODO
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])
model.fit(
  train_images,
  train_labels,
  epochs=1,
  validation_split=0.1,
)
Explanation: Train a model for MNIST without quantization aware training
End of explanation
import tensorflow_model_optimization as tfmot
quantize_model = tfmot.quantization.keras.quantize_model
# q_aware stands for for quantization aware.
q_aware_model = quantize_model(model)
# `quantize_model` requires a recompile.
q_aware_model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])
q_aware_model.summary()
Explanation: Clone and fine-tune pre-trained model with quantization aware training
Define the model
You will apply quantization aware training to the whole model and see this in the model summary. All layers are now prefixed by "quant".
Note that the resulting model is quantization aware but not quantized (e.g. the weights are float32 instead of int8). The sections after show how to create a quantized model from the quantization aware one.
In the comprehensive guide, you can see how to quantize some layers for model accuracy improvements.
End of explanation
train_images_subset = train_images[0:1000] # out of 60000
train_labels_subset = train_labels[0:1000]
q_aware_model.fit(train_images_subset, train_labels_subset,
                  batch_size=500, epochs=1, validation_split=0.1)
Explanation: Train and evaluate the model against baseline
To demonstrate fine tuning after training the model for just an epoch, fine tune with quantization aware training on a subset of the training data.
End of explanation
_, baseline_model_accuracy = model.evaluate(
    test_images, test_labels, verbose=0)
_, q_aware_model_accuracy = q_aware_model.evaluate(
   test_images, test_labels, verbose=0)
print('Baseline test accuracy:', baseline_model_accuracy)
print('Quant test accuracy:', q_aware_model_accuracy)
Explanation: For this example, there is minimal to no loss in test accuracy after quantization aware training, compared to the baseline.
End of explanation
converter = tf.lite.TFLiteConverter.from_keras_model(q_aware_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
quantized_tflite_model = converter.convert()
Explanation: Create quantized model for TFLite backend
After this, you have an actually quantized model with int8 weights and uint8 activations.
End of explanation
import numpy as np
def evaluate_model(interpreter):
  input_index = interpreter.get_input_details()[0]["index"]
  output_index = interpreter.get_output_details()[0]["index"]
  # Run predictions on every image in the "test" dataset.
  prediction_digits = []
  for i, test_image in enumerate(test_images):
    if i % 1000 == 0:
      print('Evaluated on {n} results so far.'.format(n=i))
    # Pre-processing: add batch dimension and convert to float32 to match with
    # the model's input data format.
    # TODO
    test_image = np.expand_dims(test_image, axis=0).astype(np.float32)
    interpreter.set_tensor(input_index, test_image)
    # Run inference.
    interpreter.invoke()
    # Post-processing: remove batch dimension and find the digit with highest
    # probability.
    output = interpreter.tensor(output_index)
    digit = np.argmax(output()[0])
    prediction_digits.append(digit)
  print('\n')
  # Compare prediction results with ground truth labels to calculate accuracy.
  prediction_digits = np.array(prediction_digits)
  accuracy = (prediction_digits == test_labels).mean()
  return accuracy
Explanation: See persistence of accuracy from TF to TFLite
Define a helper function to evaluate the TF Lite model on the test dataset.
End of explanation
interpreter = tf.lite.Interpreter(model_content=quantized_tflite_model)
interpreter.allocate_tensors()
test_accuracy = evaluate_model(interpreter)
print('Quant TFLite test_accuracy:', test_accuracy)
print('Quant TF test accuracy:', q_aware_model_accuracy)
Explanation: You evaluate the quantized model and see that the accuracy from TensorFlow persists to the TFLite backend.
End of explanation
# Create float TFLite model.
# TODO
float_converter = tf.lite.TFLiteConverter.from_keras_model(model)
float_tflite_model = float_converter.convert()
# Measure sizes of models.
_, float_file = tempfile.mkstemp('.tflite')
_, quant_file = tempfile.mkstemp('.tflite')
with open(quant_file, 'wb') as f:
  f.write(quantized_tflite_model)
with open(float_file, 'wb') as f:
  f.write(float_tflite_model)
print("Float model in Mb:", os.path.getsize(float_file) / float(2**20))
print("Quantized model in Mb:", os.path.getsize(quant_file) / float(2**20))
Explanation: See 4x smaller model from quantization
You create a float TFLite model and then see that the quantized TFLite model
is 4x smaller.
End of explanation
<END_TASK>