<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 model05
Model
Linear models
Step1:  Make training set
For training model, we might need to make feature and lable pair. In this case, we will use only uid, qid, and position for feature.
Step2:  It means that user 0 tried to solve question number 1 which has 77 tokens for question and he or she answered at 61st token.
Train model and make predictions
Let's train model and make predictions.
Step3:  Here is 4749 predictions.
Writing submission.
OK, let's writing submission into guess.csv file. In the given submission form, we realized that we need to put header. So, we will insert header at the first of predictions, and then make it as a file.
<ASSISTANT_TASK:>
Python Code:
import gzip
import cPickle as pickle
with gzip.open("../data/train.pklz", "rb") as train_file:
    train_set = pickle.load(train_file)
with gzip.open("../data/test.pklz", "rb") as test_file:
    test_set = pickle.load(test_file)
with gzip.open("../data/questions.pklz", "rb") as questions_file:
    questions = pickle.load(questions_file)
Explanation: model05
Model
Linear models: LinearRegression, Ridge, Lasso, ElasticNet
Features
uid
qid
q_length
category
answer
avg_per_uid: average response time per user
avg_per_qid: average response time per question
End of explanation
print train_set[1]
print questions[1].keys()
X = []
Y = []
avg_time_per_user = {}
avg_time_per_que = {}
for key in train_set:
    # We only care about positive case at this time
    #if train_set[key]['position'] < 0:
    #    continue
    uid = train_set[key]['uid']
    qid = train_set[key]['qid']
    pos = train_set[key]['position']
    q_length = max(questions[qid]['pos_token'].keys())
    category = questions[qid]['category'].lower()
    answer = questions[qid]['answer'].lower()
    
    # Calculate average response time per user
    temp = 0; num = 0
    if uid not in avg_time_per_user.keys():
        for keysubset in train_set:
            if train_set[keysubset]['uid'] == uid:
                temp += train_set[keysubset]['position']
                num += 1
        avg_time_per_user[uid] = temp/num
        temp=0; num = 0
    # Calculate average response time per question
    temp=0; num = 0
    if qid not in avg_time_per_que.keys():
        for keysubset in train_set:
            if train_set[keysubset]['qid'] == qid:
                temp += train_set[keysubset]['position']
                num += 1
        avg_time_per_que[qid] = temp/num
        temp=0; num = 0
    
    feat = {"uid": str(uid), "qid": str(qid), "q_length": q_length, "category": category, "answer": answer, "avg_per_uid": avg_time_per_user[uid], "avg_per_qid":avg_time_per_que[qid]}
    X.append(feat)
    Y.append([pos])
print len(X)
print len(Y)
print X[0], Y[0]
Explanation: Make training set
For training model, we might need to make feature and lable pair. In this case, we will use only uid, qid, and position for feature.
End of explanation
from sklearn.feature_extraction import DictVectorizer
vec = DictVectorizer()
X = vec.fit_transform(X)
print X[0]
from sklearn.linear_model import LinearRegression, Ridge, Lasso, ElasticNet
from sklearn.cross_validation import train_test_split, cross_val_score
import math
X_train, X_test, Y_train, Y_test = train_test_split (X, Y)
regressor = LinearRegression()
scores = cross_val_score(regressor, X, Y, cv=10, scoring= 'mean_squared_error')
# Flip the sign of MSE and take sqrt of that values.
for ii in xrange(len(scores)):
    scores[ii] = math.sqrt(-1*scores[ii])
print 'Linear Cross validation RMSE scores:', scores.mean()
print scores
regressor = Ridge()
scores = cross_val_score(regressor, X, Y, cv=10, scoring= 'mean_squared_error')
# Flip the sign of MSE and take sqrt of that values.
for ii in xrange(len(scores)):
    scores[ii] = math.sqrt(-1*scores[ii])
print 'Ridge Cross validation RMSE scores:', scores.mean()
print scores
regressor = Lasso()
scores = cross_val_score(regressor, X, Y, cv=10, scoring= 'mean_squared_error')
# Flip the sign of MSE and take sqrt of that values.
for ii in xrange(len(scores)):
    scores[ii] = math.sqrt(-1*scores[ii])
print 'Lasso Cross validation RMSE scores:', scores.mean()
print scores
regressor = ElasticNet()
scores = cross_val_score(regressor, X, Y, cv=10, scoring= 'mean_squared_error')
# Flip the sign of MSE and take sqrt of that values.
for ii in xrange(len(scores)):
    scores[ii] = math.sqrt(-1*scores[ii])
print 'ElasticNet Cross validation RMSE scores:', scores.mean()
print scores
a = [{1: 2}, {2: 3}]
b = [{3: 2}, {4: 3}]
c = a + b
print c[:len(a)]
print c[len(a):]
X_train = []
Y_train = []
for key in train_set:
    # We only care about positive case at this time
    #if train_set[key]['position'] < 0:
    #    continue
    uid = train_set[key]['uid']
    qid = train_set[key]['qid']
    pos = train_set[key]['position']
    q_length = max(questions[qid]['pos_token'].keys())
    category = questions[qid]['category'].lower()
    answer = questions[qid]['answer'].lower()
    feat = {"uid": str(uid), "qid": str(qid), "q_length": q_length, "category": category, "answer": answer}
    X_train.append(feat)
    Y_train.append(pos)
X_test = []
Y_test = []
for key in test_set:
    uid = test_set[key]['uid']
    qid = test_set[key]['qid']
    q_length = max(questions[qid]['pos_token'].keys())
    category = questions[qid]['category'].lower()
    answer = questions[qid]['answer'].lower()
    feat = {"uid": str(uid), "qid": str(qid), "q_length": q_length, "category": category, "answer": answer}
    X_test.append(feat)
    Y_test.append(key)
print "Before transform: ", len(X_test)
X_train_length = len(X_train)
X = vec.fit_transform(X_train + X_test)
X_train = X[:X_train_length]
X_test = X[X_train_length:]
regressor = Ridge()
regressor.fit(X_train, Y_train)
predictions = regressor.predict(X_test)
predictions = sorted([[id, predictions[index]] for index, id in enumerate(Y_test)])
print len(predictions)
predictions[:5]
Explanation: It means that user 0 tried to solve question number 1 which has 77 tokens for question and he or she answered at 61st token.
Train model and make predictions
Let's train model and make predictions.
End of explanation
import csv
predictions.insert(0,["id", "position"])
with open('guess.csv', 'wb') as fp:
    writer = csv.writer(fp, delimiter=',')
    writer.writerows(predictions)
Explanation: Here is 4749 predictions.
Writing submission.
OK, let's writing submission into guess.csv file. In the given submission form, we realized that we need to put header. So, we will insert header at the first of predictions, and then make it as a file.
End of explanation
<END_TASK>