<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <a href="https
Step1:  load data
Step2:  pre-process data into chunks
Step3:  Recurrent Neural Networks
RNNs work on sequences of input data and can learn which part of the history is relevant. Thus, they can learn which mouse events are relevant.
Simple RNN
Inexpensive in terms of model size and compute, but are bad at remembering events far in the past.
LSTM
Can learn which events in the past are important and can thus much better match a certain mouse path. However, 4x more expensive than the simple RNN.
GRU
Like LSTMs, but only 3x more expensive. Often as good as LSTMs.
Training
Step4:  Convert Model into tfjs format
<ASSISTANT_TASK:>
Python Code:
# for colab
!pip install -q tf-nightly-gpu-2.0-preview
import tensorflow as tf
print(tf.__version__)
# a small sanity check, does tf seem to work ok?
hello = tf.constant('Hello TF!')
print("This works: {}".format(hello))
# this should return True even on Colab
tf.test.is_gpu_available()
tf.test.is_built_with_cuda()
tf.executing_eagerly()
Explanation: <a href="https://colab.research.google.com/github/DJCordhose/deep-learning-crash-course-notebooks/blob/master/tf-v2/ux-rnn.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
Training on Mouse-Movements
Our challenge is to predict which button a user is going to click based on previous mouse movements. One application might be to highlight the button before hover and give a clearer guidance to the user: 
Even though the model will vary between different machines and different users, we have a default data set you can use to train the model.
Read the complete article here https://dev.to/djcordhose/improving-user-experience-with-tensorflow-js-4693 and have a look at the browser application http://djcordhose.github.io/ux-by-tfjs/dist that can load our model to make online predictions.
End of explanation
import pandas as pd
print(pd.__version__)
# local
# URL = '../data/sample4.json'
# remote
URL = 'https://raw.githubusercontent.com/DJCordhose/ux-by-tfjs/master//data/sample4.json'
df = pd.read_json(URL, typ='series')
len(df)
df.head()
X = [item['x'] for item in df]
X[0]
y = [item['y'] - 1 for item in df]
y[0]
Explanation: load data
End of explanation
from math import floor
def make_chunks(list_to_chunk, chunk_size):
    length = len(list_to_chunk)
    assert length / chunk_size == floor(length / chunk_size), "length of data must be multiple of segment length"
    for chunk_start in range(0, length, chunk_size):
        yield list_to_chunk[chunk_start : chunk_start + chunk_size]
import numpy as np
CHUNK_SIZE = 25
# only use the final segments 
SEGMENTS = 2
X_expanded = []
y_expanded = []
for x_el, y_el in zip(X, y):
    chunks = list(make_chunks(x_el, CHUNK_SIZE))
    chunks = chunks[len(chunks) - SEGMENTS:]
    labels = [y_el] * SEGMENTS
    for seq, label in zip(chunks, labels):
        X_expanded.append(seq)
        y_expanded.append(label)
X_expanded = np.array(X_expanded)
y_expanded = np.array(y_expanded)
X_expanded.shape
X_expanded[100]
X_expanded[100][0]
y_expanded[100]
np.unique(y_expanded)
assert np.array_equal(np.unique(y_expanded), [0, 1, 2])
Explanation: pre-process data into chunks
End of explanation
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Dense, LSTM, GRU, SimpleRNN, BatchNormalization
from tensorflow.keras.models import Sequential, Model
# experiment with
# - type of RNN: SimpleRNN, LSTM, GRU
# - number of units
# - dropout
# - BatchNormalization: yes/no
n_steps = len(X_expanded[0])
n_features = len(X_expanded[0][0])
n_buttons = 3
model = Sequential()
model.add(SimpleRNN(units=50, activation='tanh', input_shape=(n_steps, n_features), name="RNN_Input", 
# model.add(GRU(units=50, activation='tanh', input_shape=(n_steps, n_features), name="RNN_Input", 
# recurrent_dropout makes things slow              
#               dropout=0.1, recurrent_dropout=0.1))
                dropout=0.1))
# model.add(GRU(units=50, activation='tanh', input_shape=(n_steps, n_features), name="RNN_Input"))
model.add(BatchNormalization())
model.add(Dense(units=n_buttons, name='softmax', activation='softmax'))
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
%%time
EPOCHS = 200
BATCH_SIZE = 200
history = model.fit(X_expanded, y_expanded, 
                    batch_size=BATCH_SIZE,
                    epochs=EPOCHS, verbose=0, validation_split=0.2)
loss, accuracy = model.evaluate([X_expanded], y_expanded, batch_size=BATCH_SIZE)
accuracy
%matplotlib inline
import matplotlib.pyplot as plt
# plt.yscale('log')
plt.ylabel('loss')
plt.xlabel('epochs')
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.legend(['loss', 'val_loss'])
plt.ylabel('accuracy')
plt.xlabel('epochs')
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.legend(['accuracy', 'val_accuracy'])
model.predict([[X_expanded[0]]])
model.predict([[X_expanded[0]]]).argmax()
y_expanded[0]
y_pred = model.predict([X_expanded]).argmax(axis=1)
cm = tf.math.confusion_matrix(labels=y_expanded, predictions=y_pred)
cm
import seaborn as sns
classes = ["Left Button", "Middle Button", "Right Button"]
sns.heatmap(cm, annot=True, fmt="d", xticklabels=classes, yticklabels=classes)
Explanation: Recurrent Neural Networks
RNNs work on sequences of input data and can learn which part of the history is relevant. Thus, they can learn which mouse events are relevant.
Simple RNN
Inexpensive in terms of model size and compute, but are bad at remembering events far in the past.
LSTM
Can learn which events in the past are important and can thus much better match a certain mouse path. However, 4x more expensive than the simple RNN.
GRU
Like LSTMs, but only 3x more expensive. Often as good as LSTMs.
Training
End of explanation
model.save('ux.hd5')
!ls -l
!pip install -q tensorflowjs
!tensorflowjs_converter --input_format keras ux.hd5 tfjs
!ls -l tfjs
Explanation: Convert Model into tfjs format
End of explanation
<END_TASK>