<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Post Training Integer Quantization
<table class="tfo-notebook-buttons" align="left">
  <td>
    <a target="_blank" href="https
Step1:  Train and export the model
Step2:  For the example, we only trained the model for a single epoch, so it only trains to ~96% accuracy.
Convert to a TensorFlow Lite model
The savedmodel directory is named with a timestamp. Select the most recent one
Step3:  Using the Python TFLiteConverter, the saved model can be converted into a TensorFlow Lite model.
First load the model using the TFLiteConverter
Step4:  Write it out to a .tflite file
Step5:  To instead quantize the model on export, first set the optimizations flag to optimize for size
Step6:  Now, construct and provide a representative dataset, this is used to get the dynamic range of activations.
Step7:  Finally, convert the model like usual. Note, by default the converted model will still use float input and outputs for invocation convenience.
Step8:  Note how the resulting file is approximately 1/4 the size.
Step9:  Run the TensorFlow Lite models
We can run the TensorFlow Lite model using the Python TensorFlow Lite
Interpreter. 
Load the test data
First, let's load the MNIST test data to feed to the model
Step10:  Load the model into the interpreters
Step11:  Test the models on one image
Step12:  Evaluate the models
Step13:  We can repeat the evaluation on the fully quantized model to obtain
<ASSISTANT_TASK:>
Python Code:
! pip uninstall -y tensorflow
! pip install -U tf-nightly
import tensorflow as tf
tf.enable_eager_execution()
! git clone --depth 1 https://github.com/tensorflow/models
import sys
import os
if sys.version_info.major >= 3:
    import pathlib
else:
    import pathlib2 as pathlib
# Add `models` to the python path.
models_path = os.path.join(os.getcwd(), "models")
sys.path.append(models_path)
Explanation: Post Training Integer Quantization
<table class="tfo-notebook-buttons" align="left">
  <td>
    <a target="_blank" href="https://colab.research.google.com/github/tensorflow/tensorflow/blob/master/tensorflow/lite/tutorials/post_training_quant.ipynb"><img src="https://www.tensorflow.org/images/colab_logo_32px.png" />Run in Google Colab</a>
  </td>
  <td>
    <a target="_blank" href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/tutorials/post_training_quant.ipynb"><img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png" />View source on GitHub</a>
  </td>
</table>
Overview
TensorFlow Lite now supports
converting an entire model (weights and activations) to 8-bit during model conversion from TensorFlow to TensorFlow Lite's flat buffer format. This results in a 4x reduction in model size and a 3 to 4x performance improvement on CPU performance. In addition, this fully quantized model can be consumed by integer-only hardware accelerators.
In contrast to post-training "on-the-fly" quantization
, which only stores weights as 8-bit ints, in this technique all weights and activations are quantized statically during model conversion.
In this tutorial, we train an MNIST model from scratch, check its accuracy in TensorFlow, and then convert the saved model into a Tensorflow Lite flatbuffer
with full quantization. We finally check the
accuracy of the converted model and compare it to the original saved model. We
run the training script mnist.py from
Tensorflow official MNIST tutorial.
Building an MNIST model
Setup
End of explanation
saved_models_root = "/tmp/mnist_saved_model"
# The above path addition is not visible to subprocesses, add the path for the subprocess as well.
# Note: channels_last is required here or the conversion may fail. 
!PYTHONPATH={models_path} python models/official/mnist/mnist.py --train_epochs=1 --export_dir {saved_models_root} --data_format=channels_last
Explanation: Train and export the model
End of explanation
saved_model_dir = str(sorted(pathlib.Path(saved_models_root).glob("*"))[-1])
saved_model_dir
Explanation: For the example, we only trained the model for a single epoch, so it only trains to ~96% accuracy.
Convert to a TensorFlow Lite model
The savedmodel directory is named with a timestamp. Select the most recent one:
End of explanation
import tensorflow as tf
tf.enable_eager_execution()
tf.logging.set_verbosity(tf.logging.DEBUG)
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
tflite_model = converter.convert()
Explanation: Using the Python TFLiteConverter, the saved model can be converted into a TensorFlow Lite model.
First load the model using the TFLiteConverter:
End of explanation
tflite_models_dir = pathlib.Path("/tmp/mnist_tflite_models/")
tflite_models_dir.mkdir(exist_ok=True, parents=True)
tflite_model_file = tflite_models_dir/"mnist_model.tflite"
tflite_model_file.write_bytes(tflite_model)
Explanation: Write it out to a .tflite file:
End of explanation
tf.logging.set_verbosity(tf.logging.INFO)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
Explanation: To instead quantize the model on export, first set the optimizations flag to optimize for size:
End of explanation
mnist_train, _ = tf.keras.datasets.mnist.load_data()
images = tf.cast(mnist_train[0], tf.float32)/255.0
mnist_ds = tf.data.Dataset.from_tensor_slices((images)).batch(1)
def representative_data_gen():
  for input_value in mnist_ds.take(100):
    yield [input_value]
converter.representative_dataset = representative_data_gen
Explanation: Now, construct and provide a representative dataset, this is used to get the dynamic range of activations.
End of explanation
tflite_quant_model = converter.convert()
tflite_model_quant_file = tflite_models_dir/"mnist_model_quant.tflite"
tflite_model_quant_file.write_bytes(tflite_quant_model)
Explanation: Finally, convert the model like usual. Note, by default the converted model will still use float input and outputs for invocation convenience.
End of explanation
!ls -lh {tflite_models_dir}
Explanation: Note how the resulting file is approximately 1/4 the size.
End of explanation
import numpy as np
_, mnist_test = tf.keras.datasets.mnist.load_data()
images, labels = tf.cast(mnist_test[0], tf.float32)/255.0, mnist_test[1]
mnist_ds = tf.data.Dataset.from_tensor_slices((images, labels)).batch(1)
Explanation: Run the TensorFlow Lite models
We can run the TensorFlow Lite model using the Python TensorFlow Lite
Interpreter. 
Load the test data
First, let's load the MNIST test data to feed to the model:
End of explanation
interpreter = tf.lite.Interpreter(model_path=str(tflite_model_file))
interpreter.allocate_tensors()
interpreter_quant = tf.lite.Interpreter(model_path=str(tflite_model_quant_file))
interpreter_quant.allocate_tensors()
Explanation: Load the model into the interpreters
End of explanation
for img, label in mnist_ds:
  break
interpreter.set_tensor(interpreter.get_input_details()[0]["index"], img)
interpreter.invoke()
predictions = interpreter.get_tensor(
    interpreter.get_output_details()[0]["index"])
import matplotlib.pylab as plt
plt.imshow(img[0])
template = "True:{true}, predicted:{predict}"
_ = plt.title(template.format(true= str(label[0].numpy()),
                              predict=str(predictions[0])))
plt.grid(False)
interpreter_quant.set_tensor(
    interpreter_quant.get_input_details()[0]["index"], img)
interpreter_quant.invoke()
predictions = interpreter_quant.get_tensor(
    interpreter_quant.get_output_details()[0]["index"])
plt.imshow(img[0])
template = "True:{true}, predicted:{predict}"
_ = plt.title(template.format(true= str(label[0].numpy()),
                              predict=str(predictions[0])))
plt.grid(False)
Explanation: Test the models on one image
End of explanation
def eval_model(interpreter, mnist_ds):
  total_seen = 0
  num_correct = 0
  input_index = interpreter.get_input_details()[0]["index"]
  output_index = interpreter.get_output_details()[0]["index"]
  for img, label in mnist_ds:
    total_seen += 1
    interpreter.set_tensor(input_index, img)
    interpreter.invoke()
    predictions = interpreter.get_tensor(output_index)
    if predictions == label.numpy():
      num_correct += 1
    if total_seen % 500 == 0:
      print("Accuracy after %i images: %f" %
            (total_seen, float(num_correct) / float(total_seen)))
  return float(num_correct) / float(total_seen)
print(eval_model(interpreter, mnist_ds))
Explanation: Evaluate the models
End of explanation
# NOTE: Colab runs on server CPUs. At the time of writing this, TensorFlow Lite
# doesn't have super optimized server CPU kernels. For this reason this may be
# slower than the above float interpreter. But for mobile CPUs, considerable
# speedup can be observed.
print(eval_model(interpreter_quant, mnist_ds))
Explanation: We can repeat the evaluation on the fully quantized model to obtain:
End of explanation
<END_TASK>