<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Convolutional Neural Networks With BatchFlow
Now it's time to talk about convolutional neural networks and in this notebook you will find out how to do
Step1:  You don't need to implement a MNIST dataset. It is already done for you.
Step2:  We can use deep learning frameworks such as TensorFlow or PyTorch to make a neural network. These frameworks have a lot of differences under the hood. Batchflow allows us not to dive deep into each of them and use the same model configuration, thereby allowing us to build framework-agnostic models. 
But before,  we should define model class 'model' and channels positions 'channels' (for TensorFlow models - 'last', for PyTorch models - 'first') in config. 
There are also predefined models of both frameworks. You can use them without additional configuration.
Model configuration
Step3:  As we already learned from the previous tutorials, first of all you have to define model configuration and create train and test pipelines. 
A little bit about the structure of batchflow model
Step5:  Train pipeline
We define our custom function for data augmentation.
Step6:  When config is defined, next step is to create a pipeline. Note that rotate and scale are methods of the ImagesBatch class. You can see all avalible augmentations in images tutorial.
In contrast to them apply_transform is a function from Batch class. It is worth mentioning because it runs our function custom_filter in parallel. About parallel method read docs.
Step7:  Validation pipeline
Testing on the augmented data
Step8:  Training process
We introduce an early stopping to terminate the model training when an average accuracy for a few last epochs will exceed 90 percent.
Step9:  Take a look at the loss history during training.
Step10:  Results
Our network is ready for inference. Now we don't use data augmentations. Let's take a look at the predictions.
Step11:  It's always interesting to look at the images, so let's draw them.
<ASSISTANT_TASK:>
Python Code:
import sys
import warnings
warnings.filterwarnings("ignore")
import numpy as np
import PIL
from matplotlib import pyplot as plt
from tqdm import tqdm
%matplotlib inline
# the following line is not required if BatchFlow is installed as a python package.
sys.path.append('../..')
from batchflow import D, B, V, C, R, P
from batchflow.utils import plot_images
from batchflow.opensets import MNIST
from batchflow.models.tf import TFModel
from batchflow.models.torch import TorchModel
from batchflow.models.metrics import ClassificationMetrics
plt.style.use('ggplot')
Explanation: Convolutional Neural Networks With BatchFlow
Now it's time to talk about convolutional neural networks and in this notebook you will find out how to do:
* data augmentation;
* early stopping.
End of explanation
mnist = MNIST()
Explanation: You don't need to implement a MNIST dataset. It is already done for you.
End of explanation
config = {
    'model': TorchModel,
    'channels': 'first'}
# or for TensorFlow model
# config = {
#     'model': TFModel,
#     'channels': 'last'}
Explanation: We can use deep learning frameworks such as TensorFlow or PyTorch to make a neural network. These frameworks have a lot of differences under the hood. Batchflow allows us not to dive deep into each of them and use the same model configuration, thereby allowing us to build framework-agnostic models. 
But before,  we should define model class 'model' and channels positions 'channels' (for TensorFlow models - 'last', for PyTorch models - 'first') in config. 
There are also predefined models of both frameworks. You can use them without additional configuration.
Model configuration
End of explanation
model_config = {
    'inputs/images/shape': B.image_shape,
    'inputs/labels/classes': D.num_classes,
    'initial_block/inputs': 'images',
    'body': {'layout': 'cna cna cna', 
             'filters': [16, 32, 64], 
             'kernel_size': [7, 5, 3], 
             'strides': 2},
    'head': {'layout': 'Pf',
             'units': 10},
    'loss': 'ce',
    'optimizer': 'Adam',
    'output': dict(predicted=['proba', 'labels'])
}
Explanation: As we already learned from the previous tutorials, first of all you have to define model configuration and create train and test pipelines. 
A little bit about the structure of batchflow model:
* initial_block - block containing the input layers;
* body - the main part of the model;
* head - outputs layers, like global average pooling or dense layers.
Let's create a dict with configuration for our model — model_config. This dict is used when model is initialized. You can override default parameters or add new parameters by typing in a model_config key like 'body/layout' and params to this key. Similar way use it in the key 'initial_block/inputs' or 'head/units'. 
The main parameter of each architecture is 'layout'. It is a sequence of letters, each letter meaning operation. For example, operations in our model: 
 * c - convolution layer, 
 * b - batch normalization,
 * a - activation,
 * P - global pooling,
 * f - dense layer (fully connected).
In our configuration  'body/filters', 'body/kernel_size' are lists with a length equal to the number of convolutions, store individual parameters for each convolution. And 'body/strides' is an integer — therefore, the same value is used for all convolutional layers.
In docs you can read more.
End of explanation
def custom_filter(image, kernel_weights=None):
     Apply filter with custom kernel to image
    Parameters
    ----------
    kernel_weights: np.array
        Weights of kernel. 
    Returns
    -------
    filtered image 
    
    if kernel_weights is None:
        kernel_weights = np.ones((3,3))
        kernel_weights[1][1] = 10
        
    kernel = PIL.ImageFilter.Kernel(kernel_weights.shape, kernel_weights.ravel())
    
    return image.filter(kernel)
Explanation: Train pipeline
We define our custom function for data augmentation.
End of explanation
train_pipeline = (
    mnist.train.p
    .init_variable('loss_history', default=[])
    .init_model('dynamic', C('model'), 'conv', config=model_config)
    .apply_transform(custom_filter, src='images', p=0.8)
    .shift(offset=P(R('randint', 8, size=2)), p=0.8)
    .rotate(angle=P(R('uniform', -10, 10)), p=0.8)
    .scale(factor=P(R('uniform', 0.8, 1.2, size=R([1, 2]))), preserve_shape=True, p=0.8)
    .to_array(channels=C('channels'), dtype=np.float32)
    .multiply(multiplier=1/255)
    .train_model('conv', fetches='loss', images=B('images'), targets=B('labels'), 
                 save_to=V('loss_history', mode='a'))
) << config
Explanation: When config is defined, next step is to create a pipeline. Note that rotate and scale are methods of the ImagesBatch class. You can see all avalible augmentations in images tutorial.
In contrast to them apply_transform is a function from Batch class. It is worth mentioning because it runs our function custom_filter in parallel. About parallel method read docs.
End of explanation
validation_pipeline = (
    mnist.test.p
    .init_variable('predictions')
    .init_variable('metrics', default=None)
    .import_model('conv', train_pipeline)
    .apply_transform(custom_filter, src='images', p=0.8)
    .shift(offset=P(R('randint', 8, size=2)), p=0.8)
    .rotate(angle=P(R('uniform', -10, 10)), p=0.8)
    .scale(factor=P(R('uniform', 0.8, 1.2, size=R([1, 2]))), preserve_shape=True, p=0.8)
    .to_array(channels=C('channels'), dtype=np.float32)
    .multiply(multiplier=1/255)
    .predict_model('conv', images=B('images'),
                   fetches='predictions', save_to=V('predictions'))
    .gather_metrics(ClassificationMetrics, targets=B('labels'), predictions=V('predictions'),
                    fmt='logits', axis=-1, save_to=V('metrics', mode='a'))
) << config
Explanation: Validation pipeline
Testing on the augmented data
End of explanation
MAX_ITER = 500
FREQUENCY = N_LAST = 20
batch_size = 128
for curr_iter in tqdm(range(1, MAX_ITER + 1)):
    train_pipeline.next_batch(batch_size)
    validation_pipeline.next_batch(batch_size)
    if curr_iter % FREQUENCY == 0:
        metrics = validation_pipeline.v('metrics')
        accuracy = metrics[-N_LAST:].evaluate('accuracy')
        #Early stopping   
        if accuracy > 0.9:
            print('Early stop on {} iteration. Accuracy: {}'.format(curr_iter, accuracy))
            break
Explanation: Training process
We introduce an early stopping to terminate the model training when an average accuracy for a few last epochs will exceed 90 percent.
End of explanation
plt.figure(figsize=(15, 5))
plt.plot(train_pipeline.v('loss_history'))
plt.xlabel("Iterations"), plt.ylabel("Loss")
plt.show()
Explanation: Take a look at the loss history during training.
End of explanation
inference_pipeline = (mnist.test.p
    .init_variables('proba', 'labels')
    .import_model('conv', train_pipeline) 
    .to_array(channels=C('channels'), dtype=np.float32)
    .multiply(multiplier=1/255)
    .predict_model('conv', images=B('images'),
        fetches=['predicted_proba', 'predicted_labels'],
        save_to=[V('proba'), V('labels')])) << config
Explanation: Results
Our network is ready for inference. Now we don't use data augmentations. Let's take a look at the predictions.
End of explanation
batch = inference_pipeline.next_batch(12, shuffle=True)
plot_images(np.squeeze(batch.images), batch.labels,
            batch.pipeline.v('proba'), ncols=4, figsize=(30, 35))
Explanation: It's always interesting to look at the images, so let's draw them.
End of explanation
<END_TASK>