<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 First model
Here is the first model by our own. In this note, you can explorer how read data, train model, make prediction, and save it into submission form.
Load train, test, questions data from pklz
First of all, we need to read those three data set.
Step1:  Let's take a look the loaded data set.
Step2:  Make training set
For training model, we might need to make feature and lable pair. In this case, we will use only uid, qid, and position for feature.
Step3:  It means that user 0 tried to solve question number 1 which has 77 tokens for question and he or she answered at 61st token.
Train model and make predictions
Let's train model and make predictions. We will use simple Linear Regression at this moment.
Step4:  Let's make test set for testing.
Step5:  Here is 4749 predictions.
Writing submission.
OK, let's writing submission into guess.csv file. In the given submission form, we realized that we need to put header. So, we will insert header at the first of predictions, and then make it as a file.
<ASSISTANT_TASK:>
Python Code:
import gzip
import cPickle as pickle
with gzip.open("../data/train.pklz", "rb") as train_file:
    train_set = pickle.load(train_file)
with gzip.open("../data/test.pklz", "rb") as test_file:
    test_set = pickle.load(test_file)
with gzip.open("../data/questions.pklz", "rb") as questions_file:
    questions = pickle.load(questions_file)
Explanation: First model
Here is the first model by our own. In this note, you can explorer how read data, train model, make prediction, and save it into submission form.
Load train, test, questions data from pklz
First of all, we need to read those three data set.
End of explanation
print "train_set: ", len(train_set)
print "test_set: ", len(test_set)
print "questions: ", len(questions)
print sorted(train_set.keys())[:10]
print train_set[1]
print train_set[1].keys()
print sorted(test_set.keys())[:10]
print test_set[7]
print test_set[7].keys()
print sorted(questions.keys())[:10]
print questions[1]
print questions[1].keys()
Explanation: Let's take a look the loaded data set.
End of explanation
X_train = []
Y_train = []
for key in train_set:
    # We only care about positive case at this time
    if train_set[key]['position'] < 0:
        continue
    uid = train_set[key]['uid']
    qid = train_set[key]['qid']
    pos = train_set[key]['position']
    q_length = max(questions[qid]['pos_token'].keys())
    feat = [uid, qid, q_length]
    X_train.append(feat)
    Y_train.append([pos])
print len(X_train)
print len(Y_train)
print X_train[0], Y_train[0]
Explanation: Make training set
For training model, we might need to make feature and lable pair. In this case, we will use only uid, qid, and position for feature.
End of explanation
from sklearn.linear_model import LinearRegression
model = LinearRegression()
model.fit(X_train, Y_train)
Explanation: It means that user 0 tried to solve question number 1 which has 77 tokens for question and he or she answered at 61st token.
Train model and make predictions
Let's train model and make predictions. We will use simple Linear Regression at this moment.
End of explanation
X_test = []
test_id = []
for key in test_set:
    test_id.append(key)
    uid = test_set[key]['uid']
    qid = test_set[key]['qid']
    q_length = max(questions[qid]['pos_token'].keys())
    feat = [uid, qid, q_length]
    X_test.append(feat)
predictions = model.predict(X_test)
predictions = sorted([[id, predictions[index][0]] for index, id in enumerate(test_id)])
print len(predictions)
predictions[:5]
Explanation: Let's make test set for testing.
End of explanation
import csv
predictions.insert(0,["id", "position"])
with open('guess.csv', 'wb') as fp:
    writer = csv.writer(fp, delimiter=',')
    writer.writerows(predictions)
Explanation: Here is 4749 predictions.
Writing submission.
OK, let's writing submission into guess.csv file. In the given submission form, we realized that we need to put header. So, we will insert header at the first of predictions, and then make it as a file.
End of explanation
<END_TASK>