<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Kaggle Dogs and Cats Image Identification Problem
Achieved 98.9% accuracy - average of two test sets.  Data taken from the 25k images of the Kaggle cats vs. dogs problem.  16k images were used for training. 3k images for validation, 3k each for two test sets.  Each set was balanced, 50% dogs, 50% cats.  In the future I may further divide the test sets so that a mean and standard deviation of test set accuracy could be calculated.
TODO
Step1:  Get train, validation and 2 test data sets - data had previously been split by a Python script.
Validation set has variable images so that it can be doubled to produce a larger validation set. This can work since each replicated image is randomized in rotation, flip, skew, shift and so is in a sense a 'different' image.
Having two test data sets allows gor some glimpse of the repeatability of the model on new data - in the future I may split these further so a standard deviation of accuracy on the test sets can be determined.
Step2:  Set up base model - had success for this problem with the Xception model.  It will not be retrained for the first training phase which will output the training for the added dense layers only.
Step3:  Build the model.
Step4:  Pre-train the added dense layers.  Set workers to a reasonable number for the CPU. I have an 8 core, 16 thread, Ryzen 7.  We could go higher on workers but this seemed enough.  Note that this is set up to run Keras / TensorFlow with a GPU.
Step5:  Set the base model to have the last few layers be trainable.  Preserve most of the layers from the pre-trained model.
Step6:  Train the model.  Now training both the dense layers and last few of the base Xception model.
Step7:  Score the model on two previously unseen data sets.
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
import os
import numpy as np
import matplotlib.pyplot as plt
from keras.applications import Xception
from keras.preprocessing.image import ImageDataGenerator
from keras import models
from keras import layers
from keras import optimizers
import tensorflow as tf
Explanation: Kaggle Dogs and Cats Image Identification Problem
Achieved 98.9% accuracy - average of two test sets.  Data taken from the 25k images of the Kaggle cats vs. dogs problem.  16k images were used for training. 3k images for validation, 3k each for two test sets.  Each set was balanced, 50% dogs, 50% cats.  In the future I may further divide the test sets so that a mean and standard deviation of test set accuracy could be calculated.
TODO: Plot history to look for overfitting, but with class and work this will need to wait.
Note: Image sizes are a smaller than the default for the Xception base model.  This is because my GPU memory could not handle a full-size Xception model.
Set up imports
End of explanation
base_dir = r'C:\Users\Vette\Desktop\Regis\#MSDS686 Deep Learning\cats_dogs'
train_dir = os.path.join(base_dir, 'train')
validation_dir = os.path.join(base_dir, 'validation')
test_dir = os.path.join(base_dir, 'test')
test2_dir = os.path.join(base_dir, 'test2')
batch_size = 20
seed = 321
train_datagen = ImageDataGenerator(rescale=1./255,
                                      rotation_range=30,
                                      width_shift_range=0.2,
                                      height_shift_range=0.2,
                                      shear_range=0.2,
                                      zoom_range=0.2,
                                      horizontal_flip=True,
                                      fill_mode='nearest')
validation_datagen = ImageDataGenerator(rescale=1./255,
                                      rotation_range=30,
                                      width_shift_range=0.2,
                                      height_shift_range=0.2,
                                      shear_range=0.2,
                                      zoom_range=0.2,
                                      horizontal_flip=True,
                                      fill_mode='nearest')
test_datagen = ImageDataGenerator(rescale=1./255)
test2_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(train_dir,
                                                        target_size=(240, 240),
                                                        batch_size=50,
                                                        class_mode='binary')
validation_generator = validation_datagen.flow_from_directory(validation_dir,
                                                                target_size=(240, 240),
                                                                batch_size=50,
                                                                class_mode='binary')
test_generator = test_datagen.flow_from_directory(test_dir,
                                                        target_size=(240, 240),
                                                        batch_size=50,
                                                        class_mode='binary')
test2_generator = test2_datagen.flow_from_directory(test2_dir,
                                                        target_size=(240, 240),
                                                        batch_size=50,
                                                        class_mode='binary')
Explanation: Get train, validation and 2 test data sets - data had previously been split by a Python script.
Validation set has variable images so that it can be doubled to produce a larger validation set. This can work since each replicated image is randomized in rotation, flip, skew, shift and so is in a sense a 'different' image.
Having two test data sets allows gor some glimpse of the repeatability of the model on new data - in the future I may split these further so a standard deviation of accuracy on the test sets can be determined.
End of explanation
conv_base = Xception(weights='imagenet',
                              include_top=False,
                              input_shape=(240, 240, 3))
conv_base.summary()
conv_base.trainable = False
Explanation: Set up base model - had success for this problem with the Xception model.  It will not be retrained for the first training phase which will output the training for the added dense layers only.
End of explanation
def build_model():
    model = models.Sequential()
    model.add(conv_base)
    model.add(layers.Flatten())
    model.add(layers.Dense(256, activation='relu'))
    model.add(layers.Dropout(0.2))
    model.add(layers.Dense(32, activation='relu'))
    model.add(layers.Dense(1, activation='sigmoid')) 
    model.compile(loss='binary_crossentropy',
                      optimizer=optimizers.RMSprop(lr=2e-5),
                      metrics=['acc'])
    return model
Explanation: Build the model.
End of explanation
with tf.device('/gpu:0'):
    np.random.seed(seed)
    model = build_model()
    print('Pre-train dense layers')
    history = model.fit_generator(train_generator,
                                  steps_per_epoch=160,
                                  epochs=8,
                                  validation_data=validation_generator,
                                  validation_steps=30,
                                  verbose=1,
                                  workers=10)
Explanation: Pre-train the added dense layers.  Set workers to a reasonable number for the CPU. I have an 8 core, 16 thread, Ryzen 7.  We could go higher on workers but this seemed enough.  Note that this is set up to run Keras / TensorFlow with a GPU.
End of explanation
conv_base.trainable = True
set_trainable = False
for layer in conv_base.layers:
    if 'block13' in layer.name:
        set_trainable = True
    if set_trainable:
        layer.trainable = True
    else:
        layer.trainable = False
    
Explanation: Set the base model to have the last few layers be trainable.  Preserve most of the layers from the pre-trained model.
End of explanation
with tf.device('/gpu:0'):
    print('Train Model')
    np.random.seed(seed)
    model = build_model()
    history = model.fit_generator(train_generator,
                                  steps_per_epoch=320,
                                  epochs=20,
                                  validation_data=validation_generator,
                                  validation_steps=60,
                                  verbose=1,
                                  initial_epoch=8,
                                  workers=10)
Explanation: Train the model.  Now training both the dense layers and last few of the base Xception model.
End of explanation
with tf.device('/gpu:0'):
    scores = model.evaluate_generator(test_generator, workers=8)
    print('#1 Loss, Accuracy: ', scores)
    scores = model.evaluate_generator(test2_generator, workers=8)
    print('#2 Loss, Accuracy: ', scores)
Explanation: Score the model on two previously unseen data sets.
End of explanation
<END_TASK>