<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Train / Test
We'll start by creating some data set that we want to build a model for (in this case a polynomial regression)
Step1:  Now we'll split the data in two - 80% of it will be used for "training" our model, and the other 20% for testing it. This way we can avoid overfitting.
Step2:  Here's our training dataset
Step3:  And our test dataset
Step4:  Now we'll try to fit an 8th-degree polynomial to this data (which is almost certainly overfitting, given what we know about how it was generated!)
Step5:  Let's plot our polynomial against the training data
Step6:  And against our test data
Step7:  Doesn't look that bad when you just eyeball it, but the r-squared score on the test data is kind of horrible! This tells us that our model isn't all that great...
Step8:  ...even though it fits the training data better
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
import numpy as np
from pylab import *
np.random.seed(2)
pageSpeeds = np.random.normal(3.0, 1.0, 100)
purchaseAmount = np.random.normal(50.0, 30.0, 100) / pageSpeeds
scatter(pageSpeeds, purchaseAmount)
Explanation: Train / Test
We'll start by creating some data set that we want to build a model for (in this case a polynomial regression):
End of explanation
trainX = pageSpeeds[:80]
testX = pageSpeeds[80:]
trainY = purchaseAmount[:80]
testY = purchaseAmount[80:]
Explanation: Now we'll split the data in two - 80% of it will be used for "training" our model, and the other 20% for testing it. This way we can avoid overfitting.
End of explanation
scatter(trainX, trainY)
Explanation: Here's our training dataset:
End of explanation
scatter(testX, testY)
Explanation: And our test dataset:
End of explanation
x = np.array(trainX)
y = np.array(trainY)
p4 = np.poly1d(np.polyfit(x, y, 8))
Explanation: Now we'll try to fit an 8th-degree polynomial to this data (which is almost certainly overfitting, given what we know about how it was generated!)
End of explanation
import matplotlib.pyplot as plt
xp = np.linspace(0, 7, 100)
axes = plt.axes()
axes.set_xlim([0,7])
axes.set_ylim([0, 200])
plt.scatter(x, y)
plt.plot(xp, p4(xp), c='r')
plt.show()
Explanation: Let's plot our polynomial against the training data:
End of explanation
testx = np.array(testX)
testy = np.array(testY)
axes = plt.axes()
axes.set_xlim([0,7])
axes.set_ylim([0, 200])
plt.scatter(testx, testy)
plt.plot(xp, p4(xp), c='r')
plt.show()
Explanation: And against our test data:
End of explanation
from sklearn.metrics import r2_score
r2 = r2_score(testy, p4(testx))
print(r2)
Explanation: Doesn't look that bad when you just eyeball it, but the r-squared score on the test data is kind of horrible! This tells us that our model isn't all that great...
End of explanation
from sklearn.metrics import r2_score
r2 = r2_score(np.array(trainY), p4(np.array(trainX)))
print(r2)
Explanation: ...even though it fits the training data better:
End of explanation
<END_TASK>