<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 This notebook will use a simple model to predict ratings for a user, anime pair
Step1:  Let's plot the objective and see how it decreases.
Step2:  So by 50 iterations, the model hits a bend and from there we see incremental improvement. We can see quite clearly that this model does not overfit. It's a very simple model (that likely underfits the data) and we use L2 regularization, so to that end it's not surprising at all. But it's good to confirm anyway.
Let's test the model on our test data now.
<ASSISTANT_TASK:>
Python Code:
import matplotlib.pyplot as plt
import matplotlib
%matplotlib inline
matplotlib.style.use('seaborn')
from animerec.data import get_data
users, anime = get_data()
from sklearn.model_selection import train_test_split
train, test = train_test_split(users, test_size = 0.1) #let's split up the dataset into a train and test set.
train, valid = train_test_split(train, test_size = 0.2) #let's split up the dataset into a train and valid set.
from animerec.data import remove_users
train = remove_users(train, 10)
#define validation set
valid_users = valid['user_id']
valid_anime = valid['anime_id']
valid_ratings = valid['rating']
#initialize some local variables
nUsers = len(train.user_id.unique())
nAnime = len(train.anime_id.unique())
# we'll need some data structures in order to vectorize computations
from collections import defaultdict
user_ids = train.user_id
item_ids = train.anime_id
user_index = defaultdict(lambda: -1) # maps a user_id to the index in the bias term.
item_index = defaultdict(lambda: -1) # maps an anime_id to the index in the bias term.
counter = 0
for user in user_ids:
    if user_index[user] == -1:
        user_index[user] = counter
        counter += 1 
counter = 0
for item in item_ids:
    if item_index[item] == -1:
        item_index[item] = counter
        counter += 1 
import tensorflow as tf
y = tf.cast(tf.constant(train['rating'].as_matrix(), shape=[len(train),1]), tf.float32)
def objective(alpha, Bi, Bu, y, lam):
    #construct the full items and user matrix.
    Bi_full = tf.gather(Bi, train.anime_id.map(lambda _id: item_index[_id]).as_matrix())
    Bu_full = tf.gather(Bu, train.user_id.map(lambda _id: user_index[_id]).as_matrix())
    alpha_full = tf.tile(alpha, (len(train), 1))
    
    return tf.reduce_sum(abs(alpha_full+Bi_full+Bu_full-y)) + lam * (tf.reduce_sum(Bi**2) + tf.reduce_sum(Bu**2))
#initialize alpha, Bi, Bu
alpha = tf.Variable(tf.constant([6.9], shape=[1, 1]))
Bi = tf.Variable(tf.constant([0.0]*nAnime, shape=[nAnime, 1]))
Bu = tf.Variable(tf.constant([0.0]*nUsers, shape=[nUsers, 1]))
optimizer = tf.train.AdamOptimizer(0.01)
obj = objective(alpha, Bi, Bu, y, 1)
trainer = optimizer.minimize(obj)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
tLoss = []
vLoss = []
prev = 10e10
for iteration in range(500):
    cvalues = sess.run([trainer, obj])
    print("objective = " + str(cvalues[1]))
    tLoss.append(cvalues[1])
    
    if not iteration % 5:
        cAlpha, cBi, cBu, cLoss = sess.run([alpha, Bi, Bu, obj])
        indices = valid_users.map(lambda x: user_index[x])
        bu = indices.map(lambda x: 0.0 if x == -1 else float(cBu[x]))
        indices = valid_anime.map(lambda x: item_index[x])
        bi = indices.map(lambda x: 0.0 if x == -1 else float(cBi[x]))
        preds = bu + bi + float(cAlpha)
        MAE = 1.0/len(valid) * sum(abs(valid_ratings-preds))
        vLoss.append(MAE)
        if MAE > prev: break
        else: prev = MAE
    
cAlpha, cBi, cBu, cLoss = sess.run([alpha, Bi, Bu, obj])
print("\nFinal train loss is ", cLoss)
Explanation: This notebook will use a simple model to predict ratings for a user, anime pair:
rating(user, item) = offset + user_bias + item_bias
Put simply, the predicted rating is the sum of three terms. The user_bias refers to whether a user is predisposed to rating higher or lower. For example, a user who gives all perfect scores will have a highly positive user_bias term. The item_bias is similar, except it refers to an anime. So a highly rated anime will have a highly positive anime_bias term. Finally, there's an offset to adjust for the general anime score. This will be initialized to 6.9 (the mean rating in the exploratory analysis) but since this is being run for only users with 10 or more ratings, that number is only a rough guess. Note that this entire program is optimized for users with 10 or more ratings, but the validation and test sets may include users with fewer ratings. 
The objective function will be mean absolute error (MAE) with L2 regularization. This model is implemented in tensorflow, just for the sake of practice. 
Broadly speaking, this code is organized into two parts.
In the first part, I do some basic data prep:
1) Import some packages for plotting
2) Use my animerec package to get data
3) split the data into train, test, and validation sets
4) Define some data structures that'll be used to vectorize computations. 
In the second part, I use tensorflow to implement this simple model, using a validation set for early stopping. I then see the performance on the test set.
End of explanation
fig, ax1 = plt.subplots()
plt.title('Linear model performance vs. iterations')
ax1.plot(tLoss, 'b-')
ax1.set_xlabel('Training Iterations')
ax1.set_ylabel('Train Loss')
ax2 = ax1.twinx()
ax2.plot(range(0, len(vLoss)*5, 5), vLoss, 'r.')
ax2.set_ylabel('Validation Classification MAE')
fig.tight_layout()
Explanation: Let's plot the objective and see how it decreases.
End of explanation
test_users = test['user_id']
test_anime = test['anime_id']
test_ratings = test['rating']
indices = test_users.map(lambda x: user_index[x])
bu = indices.map(lambda x: 0.0 if x == -1 else float(cBu[x]))
indices = test_anime.map(lambda x: item_index[x])
bi = indices.map(lambda x: 0.0 if x == -1 else float(cBi[x]))
preds = bu + bi + float(cAlpha)
MAE = 1.0/len(test) * sum(abs(test_ratings-preds))
print ('MAE on test set is: ', float(MAE))
sess.close()
Explanation: So by 50 iterations, the model hits a bend and from there we see incremental improvement. We can see quite clearly that this model does not overfit. It's a very simple model (that likely underfits the data) and we use L2 regularization, so to that end it's not surprising at all. But it's good to confirm anyway.
Let's test the model on our test data now.
End of explanation
<END_TASK>