<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Preparing data
get test users data
Step1:  This data is not a part of the Movielens-1M dataset, however, it contains ratings for the same movies. You are expected to use this dataset to generate recommendations with your recommendation model.
get movielens data
Movielens-1M dataset to train your model.
Step2:  As previously, you need to convert it into appropriate format
Step3:  Important
Step4:  You also have to remove gaps in user and movie indices. In standard usage scenarios this is done by the method _prepare() (and even doesn't require your intervention). However, this is not applicable in the custom setup (you want to prevent some of the actions performed by the method).
Anyway, this step is still relatively easy with _reindex_data() method. It will not only build new index, but also will save index mapping in a special attribute index.itemid
Step5:  The last step is to "emulate" the splitting of the test data into the observed part and holdout
Step6:  Building your model
Step7:  Submitting your solution
Before submitting you have to "reverse" movies index back to original values. It can be done in one line
Step8:  Save you model and submit results. Note, that both upload address and the leaderbord itself have new location
<ASSISTANT_TASK:>
Python Code:
test_data = pd.read_csv("https://github.com/Evfro/RecSys_ISP2017/raw/master/test_data_new.gz", compression='gzip')
test_data.head()
Explanation: Preparing data
get test users data
End of explanation
ml_data = get_movielens_data()
Explanation: This data is not a part of the Movielens-1M dataset, however, it contains ratings for the same movies. You are expected to use this dataset to generate recommendations with your recommendation model.
get movielens data
Movielens-1M dataset to train your model.
End of explanation
data_model = RecommenderData(ml_data, 'userid', 'movieid', 'rating')
Explanation: As previously, you need to convert it into appropriate format:
End of explanation
data_model._training = data_model._data #set training data to full movielens dataset
data_model._test = test_data.copy() # setting custom test data
Explanation: Important:
As you'll use custom test data,  the extra step should be taken in order to prepare your data model. You only have to do it once!
End of explanation
data_model._reindex_data() # build new index of users and movies with no gaps and stores it in index.itemid attribute 
data_model._test['movieid'] = data_model._test['movieid'].map(data_model.index.itemid.set_index('old').new)
Explanation: You also have to remove gaps in user and movie indices. In standard usage scenarios this is done by the method _prepare() (and even doesn't require your intervention). However, this is not applicable in the custom setup (you want to prevent some of the actions performed by the method).
Anyway, this step is still relatively easy with _reindex_data() method. It will not only build new index, but also will save index mapping in a special attribute index.itemid:
End of explanation
data_model._test = namedtuple('TestData', 'testset evalset')._make([data_model._test, None])
Explanation: The last step is to "emulate" the splitting of the test data into the observed part and holdout:
End of explanation
svd = SVDModel(data_model)
svd.build()
recs = svd.get_recommendations()
recs.shape
Explanation: Building your model
End of explanation
recs = pd.Series(recs.ravel()).map(data_model.index.itemid.set_index('new').old).values.reshape(recs.shape)
Explanation: Submitting your solution
Before submitting you have to "reverse" movies index back to original values. It can be done in one line:
End of explanation
np.savez('your-team-name', recs=recs)
import requests
files = {'upload': open('your-team-name.npz','rb')}
url = "http://isp2017.azurewebsites.net/team/upload"
r = requests.post(url, files=files)
print r.status_code, r.reason
Explanation: Save you model and submit results. Note, that both upload address and the leaderbord itself have new location:
End of explanation
<END_TASK>