<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 迁移学习与微调，Transfer learning & fine-tuning
Author
Step1:  介绍，Introduction
Transfer learning consists of taking features learned on one problem, and
leveraging them on a new, similar problem. For instance, features from a model that has
learned to identify racoons may be useful to kick-start a model meant to identify
 tanukis.
迁移学习 就是把从一个问题中学习到的特征使用新的、类似的问题。例如，一个识别浣熊的模型中学到的特征，可能用来初始化识别狸猫的模型中是有用的。
Transfer learning is usually done for tasks where your dataset has too little data to
 train a full-scale model from scratch.
迁移学习通常在当你只有较少的数据难以从头开始训练一个模型的时候使用。
The most common incarnation of transfer learning in the context of deep learning is the
 following workflow
Step2:  通常，所有的权重都是可以训练的权重。keras自带的layer中只有BatchNormalization有不可训练的权重。BatchNormalization使用不可训练的权重来跟踪训练过程中输入的mean和variance。
学习在自定义layers，如何使用不可训练权重，请看
guide to writing new layers from scratch.
In general, all weights are trainable weights. The only built-in layer that has
non-trainable weights is the BatchNormalization layer. It uses non-trainable weights
 to keep track of the mean and variance of its inputs during training.
To learn how to use non-trainable weights in your own custom layers, see the
guide to writing new layers from scratch.
Example
Step3:  Layers & models also feature a boolean attribute trainable. Its value can be changed.
Setting layer.trainable to False moves all the layer's weights from trainable to
non-trainable.  This is called "freezing" the layer
Step4:  When a trainable weight becomes non-trainable, its value is no longer updated during
 training.
当可训练的权重被设置成不可训练non-trainable，其值在训练过程中不再可以更新。
Step5:  不要混淆layer.trainable属性和layer.__call__()中参数training ，后者是控制该层在推理或者训练模式下是否运行前向过程。
Do not confuse the layer.trainable attribute with the argument training in
layer.__call__() (which controls whether the layer should run its forward pass in
 inference mode or training mode). For more information, see the
Keras FAQ.
Recursive setting of the trainable attribute 递归地设置 trainable
If you set trainable = False on a model or on any layer that has sublayers,
all children layers become non-trainable as well.
如果我们再一个模型或者任何一个layer上设置trainable = False，则其所有子layers 也会变成non-trainable
Example
Step6:  典型的迁移学习流程， The typical transfer-learning workflow
This leads us to how a typical transfer learning workflow can be implemented in Keras
Step7:  These are the first 9 images in the training dataset -- as you can see, they're all
 different sizes.
以下展示训练集中前9张图片，都不一样。
Step8:  We can also see that label 1 is "dog" and label 0 is "cat".
标签为1的是 "dog"，0是"cat".
Standardizing the data
数据标准化
Our raw images have a variety of sizes. In addition, each pixel consists of 3 integer
values between 0 and 255 (RGB level values). This isn't a great fit for feeding a
 neural network. We need to do 2 things
Step9:  Besides, let's batch the data and use caching & prefetching to optimize loading speed.
此外，把数据准备成batch小批，并使用缓冲和预读取来优化读取速度。
Step10:  Using random data augmentation
使用随机数据增强
When you don't have a large image dataset, it's a good practice to artificially
 introduce sample diversity by applying random yet realistic transformations to
the training images, such as random horizontal flipping or small random rotations. This
helps expose the model to different aspects of the training data while slowing down
 overfitting.
当没有大的数据集时，人工地通过使用随机但现实的变换原始训练数据，从而引入样本的多样性是一个好的办法。
例如随机水平翻转或小的随机旋转。这有助于把训练数据的不同方面暴露给模型，从而减慢过学习。
Step11:  Let's visualize what the first image of the first batch looks like after various random
 transformations
Step12:  Build a model
构建模型
Now let's built a model that follows the blueprint we've explained earlier.
现在开始构建模型，如前面所述。
Note that
Step13:  Train the top layer
训练最上面的层
Step14:  Do a round of fine-tuning of the entire model
微调整个模型一轮
Finally, let's unfreeze the base model and train the entire model end-to-end with a low
 learning rate.
最后，解冻基础模型并且使用较小的学习率端到端训练整个模型
Importantly, although the base model becomes trainable, it is still running in
inference mode since we passed training=False when calling it when we built the
model. This means that the batch normalization layers inside won't update their batch
statistics. If they did, they would wreck havoc on the representations learned by the
 model so far.
重要的是，尽管基础模型变得可以训练了，它任然是运行在推理模式，因为构建模型时传入了 training=False 。
内部的batch normalization层将不会更新batch统计量。如果更新了，将会破坏模型到目前为止所学习到的表示。
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import tensorflow as tf
from tensorflow import keras
Explanation: 迁移学习与微调，Transfer learning & fine-tuning
Author: fchollet<br>
Date created: 2020/04/15<br>
Last modified: 2020/05/12<br>
Description: Complete guide to transfer learning & fine-tuning in Keras. <br>
翻译： 叶正
设置，Setup
End of explanation
layer = keras.layers.Dense(3)
layer.build((None, 4))  # Create the weights
print("weights:", len(layer.weights), layer.weights)
print("trainable_weights:", len(layer.trainable_weights),layer.trainable_weights)
print("non_trainable_weights:", len(layer.non_trainable_weights))
Explanation: 介绍，Introduction
Transfer learning consists of taking features learned on one problem, and
leveraging them on a new, similar problem. For instance, features from a model that has
learned to identify racoons may be useful to kick-start a model meant to identify
 tanukis.
迁移学习 就是把从一个问题中学习到的特征使用新的、类似的问题。例如，一个识别浣熊的模型中学到的特征，可能用来初始化识别狸猫的模型中是有用的。
Transfer learning is usually done for tasks where your dataset has too little data to
 train a full-scale model from scratch.
迁移学习通常在当你只有较少的数据难以从头开始训练一个模型的时候使用。
The most common incarnation of transfer learning in the context of deep learning is the
 following workflow:
深度学习中最典型的迁移学习工作流如下：
从已经训练好的模型中取出部分层
固定这些层的参数，在后面的训练中不破坏之前学到的信息
在这些固定的层上面，添加一些新的、可以训练的层。这些层将学习如何把之前学到的老的特征变成在新的数据集上的预测。
在新的数据集上，开始训练新的层。
Take layers from a previously trained model.
Freeze them, so as to avoid destroying any of the information they contain during
 future training rounds.
Add some new, trainable layers on top of the frozen layers. They will learn to turn
 the old features into predictions on a  new dataset.
Train the new layers on your dataset.
A last, optional step, is fine-tuning, which consists of unfreezing the entire
model you obtained above (or part of it), and re-training it on the new data with a
very low learning rate. This can potentially achieve meaningful improvements, by
 incrementally adapting the pretrained features to the new data.
最后，也是可选的步骤，就是所谓的微调fine-tuning。这里包括unfreezing整个模型（或者部分模型），并且使用非常小的学习率重新训练。通过逐步地在新的数据上调节预训练好的特征，有潜力获得可观的提高。
First, we will go over the Keras trainable API in detail, which underlies most
 transfer learning & fine-tuning workflows.
首先，我们将复习keras trainable API，这套api是大多数迁移学习和微调工作量的基础。
Then, we'll demonstrate the typical workflow by taking a model pretrained on the
ImageNet dataset, and retraining it on the Kaggle "cats vs dogs" classification
 dataset.
然后，展示典型的工作量：选取一个从ImageNet 数据集上训练好的模型，然后在Kaggle的"cats vs dogs"分类数据集上重新训练
该实例从下列例子中修改的来。
Deep Learning with Python
 and the 2016 blog post
"building powerful image classification models using very little
 data".
Freezing layers: understanding the trainable attribute
冻住一些层：理解trainable属性
Layers & models have three weight attributes:
层和模型有三个权重属性：
weights ：层所有权重变量的列表 list
trainable_weights : 训练过程中，以上中可以更新（通过梯度下降）来最小化损失函数。
non_trainable_weights： 以上中不能被训练的固定参数。
通常，这些变量在前向传播过程中被更新。
weights is the list of all weights variables of the layer.
trainable_weights is the list of those that are meant to be updated (via gradient
 descent) to minimize the loss during training.
non_trainable_weights is the list of those that aren't meant to be trained.
 Typically they are updated by the model during the forward pass.
Example: the Dense layer has 2 trainable weights (kernel & bias)
例子：Dense层有2个可训练的权重(kernel & bias)
End of explanation
layer = keras.layers.BatchNormalization()
layer.build((None, 4))  # Create the weights
print("weights:", len(layer.weights))
print("trainable_weights:", len(layer.trainable_weights))
print("non_trainable_weights:", len(layer.non_trainable_weights))
Explanation: 通常，所有的权重都是可以训练的权重。keras自带的layer中只有BatchNormalization有不可训练的权重。BatchNormalization使用不可训练的权重来跟踪训练过程中输入的mean和variance。
学习在自定义layers，如何使用不可训练权重，请看
guide to writing new layers from scratch.
In general, all weights are trainable weights. The only built-in layer that has
non-trainable weights is the BatchNormalization layer. It uses non-trainable weights
 to keep track of the mean and variance of its inputs during training.
To learn how to use non-trainable weights in your own custom layers, see the
guide to writing new layers from scratch.
Example: the BatchNormalization layer has 2 trainable weights and 2 non-trainable
 weights
例子：BatchNormalization层有3个可训练权重和2个不可训练权重
End of explanation
layer = keras.layers.Dense(3)
layer.build((None, 4))  # Create the weights
layer.trainable = False  # Freeze the layer
print("weights:", len(layer.weights))
print("trainable_weights:", len(layer.trainable_weights))
print("non_trainable_weights:", len(layer.non_trainable_weights))
Explanation: Layers & models also feature a boolean attribute trainable. Its value can be changed.
Setting layer.trainable to False moves all the layer's weights from trainable to
non-trainable.  This is called "freezing" the layer: the state of a frozen layer won't
be updated during training (either when training with fit() or when training with
 any custom loop that relies on trainable_weights to apply gradient updates).
Example: setting trainable to False
层和模型都有一个布尔型的属性trainable，其值可以改变。把layer.trainable的值设置为False就可以把该层所有权重都变成不可训练。该过程我们称为冷冻"freezing"该层：该层的状态在训练过程中不会改变。
例子：设置trainable to `False
End of explanation
# Make a model with 2 layers
layer1 = keras.layers.Dense(3, activation="relu")
layer2 = keras.layers.Dense(3, activation="sigmoid")
model = keras.Sequential([keras.Input(shape=(3,)), layer1, layer2])
# Freeze the first layer
layer1.trainable = False
# Keep a copy of the weights of layer1 for later reference
initial_layer1_weights_values = layer1.get_weights()
# Train the model
model.compile(optimizer="adam", loss="mse")
model.fit(np.random.random((2, 3)), np.random.random((2, 3)))
# Check that the weights of layer1 have not changed during training
final_layer1_weights_values = layer1.get_weights()
np.testing.assert_allclose(
    initial_layer1_weights_values[0], final_layer1_weights_values[0]
)
np.testing.assert_allclose(
    initial_layer1_weights_values[1], final_layer1_weights_values[1]
)
Explanation: When a trainable weight becomes non-trainable, its value is no longer updated during
 training.
当可训练的权重被设置成不可训练non-trainable，其值在训练过程中不再可以更新。
End of explanation
inner_model = keras.Sequential(
    [
        keras.Input(shape=(3,)),
        keras.layers.Dense(3, activation="relu"),
        keras.layers.Dense(3, activation="relu"),
    ]
)
model = keras.Sequential(
    [keras.Input(shape=(3,)), inner_model, keras.layers.Dense(3, activation="sigmoid"),]
)
model.trainable = False  # Freeze the outer model
assert inner_model.trainable == False  # All layers in `model` are now frozen
assert inner_model.layers[0].trainable == False  # `trainable` is propagated recursively
Explanation: 不要混淆layer.trainable属性和layer.__call__()中参数training ，后者是控制该层在推理或者训练模式下是否运行前向过程。
Do not confuse the layer.trainable attribute with the argument training in
layer.__call__() (which controls whether the layer should run its forward pass in
 inference mode or training mode). For more information, see the
Keras FAQ.
Recursive setting of the trainable attribute 递归地设置 trainable
If you set trainable = False on a model or on any layer that has sublayers,
all children layers become non-trainable as well.
如果我们再一个模型或者任何一个layer上设置trainable = False，则其所有子layers 也会变成non-trainable
Example:
End of explanation
import tensorflow_datasets as tfds
tfds.disable_progress_bar()
train_ds, validation_ds, test_ds = tfds.load(
    "cats_vs_dogs",
    # Reserve 10% for validation and 10% for test
    split=["train[:40%]", "train[40%:50%]", "train[50%:60%]"],
    as_supervised=True,  # Include labels
)
print("Number of training samples: %d" % tf.data.experimental.cardinality(train_ds))
print(
    "Number of validation samples: %d" % tf.data.experimental.cardinality(validation_ds)
)
print("Number of test samples: %d" % tf.data.experimental.cardinality(test_ds))
Explanation: 典型的迁移学习流程， The typical transfer-learning workflow
This leads us to how a typical transfer learning workflow can be implemented in Keras:
Instantiate a base model and load pre-trained weights into it.
Freeze all layers in the base model by setting trainable = False.
Create a new model on top of the output of one (or several) layers from the base
 model.
Train your new model on your new dataset.
keras中典型迁移学习流程实现如下：
生成一个基础模型并加载预训练好的权重
冷冻该模型中所有参数：trainable = False
在该基础模型的输出层（或者几个不同的输出层）基础上创建新的模型
在新的数据集上训练新的模型。
Note that an alternative, more lightweight workflow could also be:
Instantiate a base model and load pre-trained weights into it.
Run your new dataset through it and record the output of one (or several) layers
 from the base model. This is called feature extraction.
Use that output as input data for a new, smaller model.
注意一个可替换且更轻量级的流程如下：
生成一个基础模型并加载预训练好的权重
把新数据放入该模型，然后记录该模型的输出（或者几个不同的输出层的输出）。这个过程称作为 特征抽取
使用以上输出为一个新的且更小的模型的输入
A key advantage of that second workflow is that you only run the base model once on
 your data, rather than once per epoch of training. So it's a lot faster & cheaper.
该流程的一个关键好处是：你只需要运行base模型一次，而不需要每个epoch的训练中都运行。所以将会更快和低成本。
An issue with that second workflow, though, is that it doesn't allow you to dynamically
modify the input data of your new model during training, which is required when doing
data augmentation, for instance. Transfer learning is typically used for tasks when
your new dataset has too little data to train a full-scale model from scratch, and in
such scenarios data augmentation is very important. So in what follows, we will focus
 on the first workflow.
但是第二个流程的问题是，在训练过程中我们无法动态的修改模型的输入，也就是说我们无法做数据的增强等操作。
 迁移学习一般用在只有很少数据可用于训练的任务中，因为数据太少我们无法从头开始训练一个好的模型，而且该条件下旺旺数据增强是非常重要的。接下来，我们主要关注第一个流程。
Here's what the first workflow looks like in Keras:
First, instantiate a base model with pre-trained weights.
以下是第一个流程在Keras中使用的样例：
首先，使用预训练好的权重实例化一个基础模型：
python
base_model = keras.applications.Xception(
    weights='imagenet',  # Load weights pre-trained on ImageNet.
    input_shape=(150, 150, 3),
    include_top=False)  # Do not include the ImageNet classifier at the top.
Then, freeze the base model.
然后，冷冻该模型。
python
base_model.trainable = False
Create a new model on top.
在旧模型上，创建一个新的模型：
```python
inputs = keras.Input(shape=(150, 150, 3))
We make sure that the base_model is running in inference mode here,
by passing training=False. This is important for fine-tuning, as you will
learn in a few paragraphs.
x = base_model(inputs, training=False)
Convert features of shape base_model.output_shape[1:] to vectors
x = keras.layers.GlobalAveragePooling2D()(x)
A Dense classifier with a single unit (binary classification)
outputs = keras.layers.Dense(1)(x)
model = keras.Model(inputs, outputs)
```
Train the model on new data.
在新数据上训练该模型：
python
model.compile(optimizer=keras.optimizers.Adam(),
              loss=keras.losses.BinaryCrossentropy(from_logits=True),
              metrics=[keras.metrics.BinaryAccuracy()])
model.fit(new_dataset, epochs=20, callbacks=..., validation_data=...)
微调，Fine-tuning
Once your model has converged on the new data, you can try to unfreeze all or part of
 the base model and retrain the whole model end-to-end with a very low learning rate.
一旦模型在新的数据上收敛了，我们就可以解冻所有或者部分base模型，并使用非常小的学习率端到端地重新训练整个模型。
This is an optional last step that can potentially give you incremental improvements.
 It could also potentially lead to quick overfitting -- keep that in mind.
最后一步是可选的，但有潜力进一步提高模型的性能，也可以导致overfitting过学习。
It is critical to only do this step after the model with frozen layers has been
trained to convergence. If you mix randomly-initialized trainable layers with
trainable layers that hold pre-trained features, the randomly-initialized layers will
cause very large gradient updates during training, which will destroy your pre-trained
 features.
重要的是仅当带冷冻层的模型训练到收敛再进行这一步。
如果把随机初始化的可训练的层与保存预训练特征的可训练层会在一起的话，随机初始化的层会导致非常大的梯度更新，
从而破坏预训练的特征。
It's also critical to use a very low learning rate at this stage, because
you are training a much larger model than in the first round of training, on a dataset
 that is typically very small.
As a result, you are at risk of overfitting very quickly if you apply large weight
 updates. Here, you only want to readapt the pretrained weights in an incremental way.
另外一个关键点是，微调阶段使用非常小的学习率，因为该阶段要训练的模型比第一阶段大不少而且数据集较小。
如果使用较大的权重更新，过学习的风险很大。这里，我们只应该逐步的改造预训练好的权重。
This is how to implement fine-tuning of the whole base model:
以下是如何实现对整个基础模型的微调
```python
Unfreeze the base model，解冻基础模型
base_model.trainable = True
It's important to recompile your model after you make any changes
to the trainable attribute of any inner layer, so that your changes
are take into account
重新编译模型，使得更改生效
model.compile(optimizer=keras.optimizers.Adam(1e-5),  # Very low learning rate
              loss=keras.losses.BinaryCrossentropy(from_logits=True),
              metrics=[keras.metrics.BinaryAccuracy()])
Train end-to-end. Be careful to stop before you overfit!
端到端训练，overfit前停止训练
model.fit(new_dataset, epochs=10, callbacks=..., validation_data=...)
```
Important note about compile() and trainable
重要注意：compile() 和 trainable
Calling compile() on a model is meant to "freeze" the behavior of that model. This
 implies that the trainable
attribute values at the time the model is compiled should be preserved throughout the
 lifetime of that model,
until compile is called again. Hence, if you change any trainable value, make sure
 to call compile() again on your
model for your changes to be taken into account.
调用compile()意味着其对应的模型的行为就固定了。这意味着trainable属性在模型编译后就不能改了，除非compile被重新调用。因此，修改trainable 后必须重新编译。
Important notes about BatchNormalization layer
BatchNormalization层的重要注意事项
Many image models contain BatchNormalization layers. That layer is a special case on
 every imaginable count. Here are a few things to keep in mind.
很多图像模型包含BatchNormalization层。
 以下几点需要注意。
BatchNormalization 包含2个在训练过程中需要更新得non-trainable的权重。这些事跟踪输入的均值和方差。
当bn_layer.trainable = False时， BatchNormalization层将运行推理模式，并且不会更新mean & variance。
 这跟其它层基本都不一样，因为权重可训练性与推理/训练模型是两个正交的概念。
weight trainability & inference/training modes are two orthogonal concepts.
但是这两个概念在BatchNormalization中纠缠在一起
当我们解冻一个包含BatchNormalization层的模型来做微调时，需要记住在推理模式时设置training=False。
否则学习到的模型整个都会被破坏。
在该指南的的最后面，将在一个端到端的例子中实践展示这种模式。
BatchNormalization contains 2 non-trainable weights that get updated during
training. These are the variables tracking the mean and variance of the inputs.
When you set bn_layer.trainable = False, the BatchNormalization layer will
run in inference mode, and will not update its mean & variance statistics. This is not
the case for other layers in general, as
weight trainability & inference/training modes are two orthogonal concepts.
But the two are tied in the case of the BatchNormalization layer.
When you unfreeze a model that contains BatchNormalization layers in order to do
fine-tuning, you should keep the BatchNormalization layers in inference mode by
 passing training=False when calling the base model.
Otherwise the updates applied to the non-trainable weights will suddenly destroy
what the model has learned.
You'll see this pattern in action in the end-to-end example at the end of this guide.
Transfer learning & fine-tuning with a custom training loop
迁移学习&微调中自定义训练loop
If instead of fit(), you are using your own low-level training loop, the workflow
stays essentially the same. You should be careful to only take into account the list
 model.trainable_weights when applying gradient updates:
如果不使用fit()，需要使用自定义的底层的训练循环，该流程本质上还是一样。
你需要非常小心，在应用梯度更新时，仅对 model.trainable_weights列表中的更新。
```python
Create base model
base_model = keras.applications.Xception(
    weights='imagenet',
    input_shape=(150, 150, 3),
    include_top=False)
Freeze base model
base_model.trainable = False
Create new model on top.
inputs = keras.Input(shape=(150, 150, 3))
x = base_model(inputs, training=False)
x = keras.layers.GlobalAveragePooling2D()(x)
outputs = keras.layers.Dense(1)(x)
model = keras.Model(inputs, outputs)
loss_fn = keras.losses.BinaryCrossentropy(from_logits=True)
optimizer = keras.optimizers.Adam()
Iterate over the batches of a dataset.
for inputs, targets in new_dataset:
    # Open a GradientTape.
    with tf.GradientTape() as tape:
        # Forward pass.
        predictions = model(inputs)
        # Compute the loss value for this batch.
        loss_value = loss_fn(targets, predictions)
# Get gradients of loss wrt the *trainable* weights.
gradients = tape.gradient(loss_value, model.trainable_weights)
# Update the weights of the model.
optimizer.apply_gradients(zip(gradients, model.trainable_weights))
```
Likewise for fine-tuning.
微调也类似
An end-to-end example: fine-tuning an image classification model on a cats vs. dogs
端到端例子：在cats vs. dogs数据集上微调图像分类模型
dataset 数据集
To solidify these concepts, let's walk you through a concrete end-to-end transfer
learning & fine-tuning example. We will load the Xception model, pre-trained on
 ImageNet, and use it on the Kaggle "cats vs. dogs" classification dataset.
为了巩固这些概念，我们一起过一个具体的端到端的迁移学习和微调的例子。本例子中使用Xception模型，在ImageNet上训练的，并且使用它来解决kaggle的"cats vs. dogs"的数据集分类问题。
Getting the data
获取数据
First, let's fetch the cats vs. dogs dataset using TFDS. If you have your own dataset,
you'll probably want to use the utility
tf.keras.preprocessing.image_dataset_from_directory to generate similar labeled
 dataset objects from a set of images on disk filed into class-specific folders.
首先，使用TFDS获取该数据集。如果你已经有了该数据集，可以使用tf.keras.preprocessing.image_dataset_from_directory来从硬盘上图像集中生产类似的带标签的数据集对象。
Transfer learning is most useful when working with very small datasets. To keep our
dataset small, we will use 40% of the original training data (25,000 images) for
 training, 10% for validation, and 10% for testing.
迁移学习在数据集非常小的时候是最有用的。为了保持我们的数据集较小，将使用40%的原始数据 (25,000 images)来训练，10% 为验证集，10%为测试集。
End of explanation
import matplotlib.pyplot as plt
plt.figure(figsize=(10, 10))
for i, (image, label) in enumerate(train_ds.take(9)):
    ax = plt.subplot(3, 3, i + 1)
    plt.imshow(image)
    plt.title(int(label))
    plt.axis("off")
Explanation: These are the first 9 images in the training dataset -- as you can see, they're all
 different sizes.
以下展示训练集中前9张图片，都不一样。
End of explanation
size = (150, 150)
train_ds = train_ds.map(lambda x, y: (tf.image.resize(x, size), y))
validation_ds = validation_ds.map(lambda x, y: (tf.image.resize(x, size), y))
test_ds = test_ds.map(lambda x, y: (tf.image.resize(x, size), y))
Explanation: We can also see that label 1 is "dog" and label 0 is "cat".
标签为1的是 "dog"，0是"cat".
Standardizing the data
数据标准化
Our raw images have a variety of sizes. In addition, each pixel consists of 3 integer
values between 0 and 255 (RGB level values). This isn't a great fit for feeding a
 neural network. We need to do 2 things:
Standardize to a fixed image size. We pick 150x150.
Normalize pixel values between -1 and 1. We'll do this using a Normalization layer as
 part of the model itself.
原始图片有着不同的大小。此外，每个像素点包含3个0-255的整型数值（RGB）。不便于直接放入神经网络训练，我们需要做一下两件事：
标准化每张图片到固定大小。本例子选择150*150
规范化像素值到-1到1之间，我们使用Normalization层来实现这个，该层也作为模型的一层。
In general, it's a good practice to develop models that take raw data as input, as
opposed to models that take already-preprocessed data. The reason being that, if your
model expects preprocessed data, any time you export your model to use it elsewhere
(in a web browser, in a mobile app), you'll need to reimplement the exact same
preprocessing pipeline. This gets very tricky very quickly. So we should do the least
 possible amount of preprocessing before hitting the model.
通常来说，相对使用已经处理好的数据作为输入，使用原始数据作为输入input来开发模型是很好的做法。
原因是在不同应用环境（app，浏览器），我们不必重新实现预处理过程。
Here, we'll do image resizing in the data pipeline (because a deep neural network can
only process contiguous batches of data), and we'll do the input value scaling as part
 of the model, when we create it.
这里，我们将在数据pipeline中实现图像resizing大小调整（因为深度神经网络只能处理了连续小批的数据），
并且将把输入数据缩放作为模型的一部分。
Let's resize images to 150x150:
调整图像大小为150x150:
End of explanation
batch_size = 32
train_ds = train_ds.cache().batch(batch_size).prefetch(buffer_size=10)
validation_ds = validation_ds.cache().batch(batch_size).prefetch(buffer_size=10)
test_ds = test_ds.cache().batch(batch_size).prefetch(buffer_size=10)
Explanation: Besides, let's batch the data and use caching & prefetching to optimize loading speed.
此外，把数据准备成batch小批，并使用缓冲和预读取来优化读取速度。
End of explanation
from tensorflow import keras
from tensorflow.keras import layers
data_augmentation = keras.Sequential(
    [
        layers.experimental.preprocessing.RandomFlip("horizontal"),
        layers.experimental.preprocessing.RandomRotation(0.1),
    ]
)
Explanation: Using random data augmentation
使用随机数据增强
When you don't have a large image dataset, it's a good practice to artificially
 introduce sample diversity by applying random yet realistic transformations to
the training images, such as random horizontal flipping or small random rotations. This
helps expose the model to different aspects of the training data while slowing down
 overfitting.
当没有大的数据集时，人工地通过使用随机但现实的变换原始训练数据，从而引入样本的多样性是一个好的办法。
例如随机水平翻转或小的随机旋转。这有助于把训练数据的不同方面暴露给模型，从而减慢过学习。
End of explanation
import numpy as np
for images, labels in train_ds.take(1):
    plt.figure(figsize=(10, 10))
    first_image = images[0]
    for i in range(9):
        ax = plt.subplot(3, 3, i + 1)
        augmented_image = data_augmentation(
            tf.expand_dims(first_image, 0), training=True
        )
        plt.imshow(augmented_image[0].numpy().astype("int32"))
        plt.title(int(labels[i]))
        plt.axis("off")
Explanation: Let's visualize what the first image of the first batch looks like after various random
 transformations:
下面可视化第一个batch中的第一张图片随机变换后的样子：
End of explanation
base_model = keras.applications.Xception(
    weights="imagenet",  # Load weights pre-trained on ImageNet.
    input_shape=(150, 150, 3),
    include_top=False,
)  # Do not include the ImageNet classifier at the top.
# Freeze the base_model
base_model.trainable = False
# Create new model on top
inputs = keras.Input(shape=(150, 150, 3))
x = data_augmentation(inputs)  # Apply random data augmentation
# Pre-trained Xception weights requires that input be normalized
# from (0, 255) to a range (-1., +1.), the normalization layer
# does the following, outputs = (inputs - mean) / sqrt(var)
norm_layer = keras.layers.experimental.preprocessing.Normalization()
mean = np.array([127.5] * 3)
var = mean ** 2
# Scale inputs to [-1, +1]
x = norm_layer(x)
norm_layer.set_weights([mean, var])
# The base model contains batchnorm layers. We want to keep them in inference mode
# when we unfreeze the base model for fine-tuning, so we make sure that the
# base_model is running in inference mode here.
x = base_model(x, training=False)
x = keras.layers.GlobalAveragePooling2D()(x)
x = keras.layers.Dropout(0.2)(x)  # Regularize with dropout
outputs = keras.layers.Dense(1)(x)
model = keras.Model(inputs, outputs)
model.summary()
Explanation: Build a model
构建模型
Now let's built a model that follows the blueprint we've explained earlier.
现在开始构建模型，如前面所述。
Note that:
We add a Normalization layer to scale input values (initially in the [0, 255]
 range) to the [-1, 1] range.
We add a Dropout layer before the classification layer, for regularization.
We make sure to pass training=False when calling the base model, so that
it runs in inference mode, so that batchnorm statistics don't get updated
even after we unfreeze the base model for fine-tuning.
注意：
我们需要增加一个Normalization层把输入数值缩放到[-1, 1]
在分类层中增加Dropout层进行规范化
调用基础模型时确保传入training=False，因为此刻是推理模式批量。这样规范化统计量不会被更新，即使当我们解冻基础模型进行微调时也一样。
End of explanation
model.compile(
    optimizer=keras.optimizers.Adam(),
    loss=keras.losses.BinaryCrossentropy(from_logits=True),
    metrics=[keras.metrics.BinaryAccuracy()],
)
epochs = 20
model.fit(train_ds, epochs=epochs, validation_data=validation_ds)
Explanation: Train the top layer
训练最上面的层
End of explanation
# Unfreeze the base_model. Note that it keeps running in inference mode
# since we passed `training=False` when calling it. This means that
# the batchnorm layers will not update their batch statistics.
# This prevents the batchnorm layers from undoing all the training
# we've done so far.
base_model.trainable = True
model.summary()
model.compile(
    optimizer=keras.optimizers.Adam(1e-5),  # Low learning rate
    loss=keras.losses.BinaryCrossentropy(from_logits=True),
    metrics=[keras.metrics.BinaryAccuracy()],
)
epochs = 10
model.fit(train_ds, epochs=epochs, validation_data=validation_ds)
Explanation: Do a round of fine-tuning of the entire model
微调整个模型一轮
Finally, let's unfreeze the base model and train the entire model end-to-end with a low
 learning rate.
最后，解冻基础模型并且使用较小的学习率端到端训练整个模型
Importantly, although the base model becomes trainable, it is still running in
inference mode since we passed training=False when calling it when we built the
model. This means that the batch normalization layers inside won't update their batch
statistics. If they did, they would wreck havoc on the representations learned by the
 model so far.
重要的是，尽管基础模型变得可以训练了，它任然是运行在推理模式，因为构建模型时传入了 training=False 。
内部的batch normalization层将不会更新batch统计量。如果更新了，将会破坏模型到目前为止所学习到的表示。
End of explanation
<END_TASK>