<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Learning to Resize in Computer Vision
Author
Step1:  Define hyperparameters
In order to facilitate mini-batch learning, we need to have a fixed shape for the images
inside a given batch. This is why an initial resizing is required. We first resize all
the images to (300 x 300) shape and then learn their optimal representation for the
(150 x 150) resolution.
Step2:  In this example, we will use the bilinear interpolation but the learnable image resizer
module is not dependent on any specific interpolation method. We can also use others,
such as bicubic.
Load and prepare the dataset
For this example, we will only use 40% of the total training dataset.
Step3:  Define the learnable resizer utilities
The figure below (courtesy
Step4:  Visualize the outputs of the learnable resizing module
Here, we visualize how the resized images would look like after being passed through the
random weights of the resizer.
Step5:  Model building utility
Step6:  The structure of the learnable image resizer module allows for flexible integrations with
different vision models.
Compile and train our model with learnable resizer
Step7:  Visualize the outputs of the trained visualizer
<ASSISTANT_TASK:>
Python Code:
from tensorflow.keras import layers
from tensorflow import keras
import tensorflow as tf
import tensorflow_datasets as tfds
tfds.disable_progress_bar()
import matplotlib.pyplot as plt
import numpy as np
Explanation: Learning to Resize in Computer Vision
Author: Sayak Paul<br>
Date created: 2021/04/30<br>
Last modified: 2021/05/13<br>
Description: How to optimally learn representations of images for a given resolution.
It is a common belief that if we constrain vision models to perceive things as humans do,
their performance can be improved. For example, in this work,
Geirhos et al. showed that the vision models pre-trained on the ImageNet-1k dataset are
biased toward texture whereas human beings mostly use the shape descriptor to develop a
common perception. But does this belief always apply especially when it comes to improving
the performance of vision models?
It turns out it may not always be the case. When training vision models, it is common to
resize images to a lower dimension ((224 x 224), (299 x 299), etc.) to allow mini-batch
learning and also to keep up the compute limitations.  We generally make use of image
resizing methods like bilinear interpolation for this step and the resized images do
not lose much of their perceptual character to the human eyes. In
Learning to Resize Images for Computer Vision Tasks, Talebi et al. show
that if we try to optimize the perceptual quality of the images for the vision models
rather than the human eyes, their performance can further be improved. They investigate
the following question:
For a given image resolution and a model, how to best resize the given images?
As shown in the paper, this idea helps to consistently improve the performance of the
common vision models (pre-trained on ImageNet-1k) like DenseNet-121, ResNet-50,
MobileNetV2, and EfficientNets. In this example, we will implement the learnable image
resizing module as proposed in the paper and demonstrate that on the
Cats and Dogs dataset
using the DenseNet-121 architecture.
This example requires TensorFlow 2.4 or higher.
Setup
End of explanation
INP_SIZE = (300, 300)
TARGET_SIZE = (150, 150)
INTERPOLATION = "bilinear"
AUTO = tf.data.AUTOTUNE
BATCH_SIZE = 64
EPOCHS = 5
Explanation: Define hyperparameters
In order to facilitate mini-batch learning, we need to have a fixed shape for the images
inside a given batch. This is why an initial resizing is required. We first resize all
the images to (300 x 300) shape and then learn their optimal representation for the
(150 x 150) resolution.
End of explanation
train_ds, validation_ds = tfds.load(
    "cats_vs_dogs",
    # Reserve 10% for validation
    split=["train[:40%]", "train[40%:50%]"],
    as_supervised=True,
)
def preprocess_dataset(image, label):
    image = tf.image.resize(image, (INP_SIZE[0], INP_SIZE[1]))
    label = tf.one_hot(label, depth=2)
    return (image, label)
train_ds = (
    train_ds.shuffle(BATCH_SIZE * 100)
    .map(preprocess_dataset, num_parallel_calls=AUTO)
    .batch(BATCH_SIZE)
    .prefetch(AUTO)
)
validation_ds = (
    validation_ds.map(preprocess_dataset, num_parallel_calls=AUTO)
    .batch(BATCH_SIZE)
    .prefetch(AUTO)
)
Explanation: In this example, we will use the bilinear interpolation but the learnable image resizer
module is not dependent on any specific interpolation method. We can also use others,
such as bicubic.
Load and prepare the dataset
For this example, we will only use 40% of the total training dataset.
End of explanation
def conv_block(x, filters, kernel_size, strides, activation=layers.LeakyReLU(0.2)):
    x = layers.Conv2D(filters, kernel_size, strides, padding="same", use_bias=False)(x)
    x = layers.BatchNormalization()(x)
    if activation:
        x = activation(x)
    return x
def res_block(x):
    inputs = x
    x = conv_block(x, 16, 3, 1)
    x = conv_block(x, 16, 3, 1, activation=None)
    return layers.Add()([inputs, x])
def get_learnable_resizer(filters=16, num_res_blocks=1, interpolation=INTERPOLATION):
    inputs = layers.Input(shape=[None, None, 3])
    # First, perform naive resizing.
    naive_resize = layers.Resizing(
        *TARGET_SIZE, interpolation=interpolation
    )(inputs)
    # First convolution block without batch normalization.
    x = layers.Conv2D(filters=filters, kernel_size=7, strides=1, padding="same")(inputs)
    x = layers.LeakyReLU(0.2)(x)
    # Second convolution block with batch normalization.
    x = layers.Conv2D(filters=filters, kernel_size=1, strides=1, padding="same")(x)
    x = layers.LeakyReLU(0.2)(x)
    x = layers.BatchNormalization()(x)
    # Intermediate resizing as a bottleneck.
    bottleneck = layers.Resizing(
        *TARGET_SIZE, interpolation=interpolation
    )(x)
    # Residual passes.
    for _ in range(num_res_blocks):
        x = res_block(bottleneck)
    # Projection.
    x = layers.Conv2D(
        filters=filters, kernel_size=3, strides=1, padding="same", use_bias=False
    )(x)
    x = layers.BatchNormalization()(x)
    # Skip connection.
    x = layers.Add()([bottleneck, x])
    # Final resized image.
    x = layers.Conv2D(filters=3, kernel_size=7, strides=1, padding="same")(x)
    final_resize = layers.Add()([naive_resize, x])
    return tf.keras.Model(inputs, final_resize, name="learnable_resizer")
learnable_resizer = get_learnable_resizer()
Explanation: Define the learnable resizer utilities
The figure below (courtesy: Learning to Resize Images for Computer Vision Tasks)
presents the structure of the learnable resizing module:
End of explanation
sample_images, _ = next(iter(train_ds))
plt.figure(figsize=(16, 10))
for i, image in enumerate(sample_images[:6]):
    image = image / 255
    ax = plt.subplot(3, 4, 2 * i + 1)
    plt.title("Input Image")
    plt.imshow(image.numpy().squeeze())
    plt.axis("off")
    ax = plt.subplot(3, 4, 2 * i + 2)
    resized_image = learnable_resizer(image[None, ...])
    plt.title("Resized Image")
    plt.imshow(resized_image.numpy().squeeze())
    plt.axis("off")
Explanation: Visualize the outputs of the learnable resizing module
Here, we visualize how the resized images would look like after being passed through the
random weights of the resizer.
End of explanation
def get_model():
    backbone = tf.keras.applications.DenseNet121(
        weights=None,
        include_top=True,
        classes=2,
        input_shape=((TARGET_SIZE[0], TARGET_SIZE[1], 3)),
    )
    backbone.trainable = True
    inputs = layers.Input((INP_SIZE[0], INP_SIZE[1], 3))
    x = layers.Rescaling(scale=1.0 / 255)(inputs)
    x = learnable_resizer(x)
    outputs = backbone(x)
    return tf.keras.Model(inputs, outputs)
Explanation: Model building utility
End of explanation
model = get_model()
model.compile(
    loss=keras.losses.CategoricalCrossentropy(label_smoothing=0.1),
    optimizer="sgd",
    metrics=["accuracy"],
)
model.fit(train_ds, validation_data=validation_ds, epochs=EPOCHS)
Explanation: The structure of the learnable image resizer module allows for flexible integrations with
different vision models.
Compile and train our model with learnable resizer
End of explanation
plt.figure(figsize=(16, 10))
for i, image in enumerate(sample_images[:6]):
    image = image / 255
    ax = plt.subplot(3, 4, 2 * i + 1)
    plt.title("Input Image")
    plt.imshow(image.numpy().squeeze())
    plt.axis("off")
    ax = plt.subplot(3, 4, 2 * i + 2)
    resized_image = learnable_resizer(image[None, ...])
    plt.title("Resized Image")
    plt.imshow(resized_image.numpy().squeeze() / 10)
    plt.axis("off")
Explanation: Visualize the outputs of the trained visualizer
End of explanation
<END_TASK>