<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Simple linear regression
Predict price of pizza based on size
The sklearn.linear_model.LinearRegression class is an estimator. Estimators predict a value based on the observed data. In scikit-learn, all estimators implement the fit() and predict() methods. The former method is used to learn the parameters of a model, and the latter method is used to predict the value of a response variable
for an explanatory variable using the learned parameters. It is easy to experiment
with different models using scikit-learn because all estimators implement the fit and predict methods.
Step1:  Evaluate fitness of model using cost (lost) function
Cost function for simple linear regression is the sum total of residuals, called RSS
Step2:  Solving OLS for simple regression
Goal is to calculate vector of coefficients beta that minimizes cost function.
Step3:  Evaluate fit using r square
Step4:  Multiple linear regression
Add topings to our model of pizza price prediction
Step5:  Polynomial regression
Use PolynomialFeatures to transform the data
Step6:  Apply linear regression on Wine dataset from UCI
Step7:  Fit and evaluate the model
Step8:  Cross validation
Step9:  Fitting using gradient descent
<ASSISTANT_TASK:>
Python Code:
import matplotlib.pyplot as plt
%matplotlib inline
X = [[6], [8], [10], [14],   [18]]
Y = [[7], [9], [13], [17.5], [18]]
plt.figure()
plt.title("Pizza price plotted against diameter")
plt.xlabel("Diameter in inches")
plt.ylabel("Price in dollars")
plt.plot(X,Y,"k.")
plt.axis([0,25,0,25])
plt.grid(True)
plt.show()
#Make simple linear regression
from sklearn.linear_model import LinearRegression
#Create and fit the model
model = LinearRegression()
model.fit(X,Y)
#Make preidiction
print('A 12" pizza should cost: %0.2f' %(model.predict([12])[0]))
Explanation: Simple linear regression
Predict price of pizza based on size
The sklearn.linear_model.LinearRegression class is an estimator. Estimators predict a value based on the observed data. In scikit-learn, all estimators implement the fit() and predict() methods. The former method is used to learn the parameters of a model, and the latter method is used to predict the value of a response variable
for an explanatory variable using the learned parameters. It is easy to experiment
with different models using scikit-learn because all estimators implement the fit and predict methods.
End of explanation
import numpy as np
print("Residual sum of squares: %.2f" %np.mean((model.predict(X)-Y)**2))
Explanation: Evaluate fitness of model using cost (lost) function
Cost function for simple linear regression is the sum total of residuals, called RSS
End of explanation
from __future__ import division
xbar = np.mean(X)
ybar = np.mean(Y)
print("Mean of X is:", xbar)
print("Mean of Y is:", ybar)
#Make own function for variance and covariance to better understand how it works
def variance(X):
    return np.sum((X - np.mean(X))**2 / (len(X)-1))
def covariance(X,Y):
    return np.sum((X - np.mean(X)) * (Y - np.mean(Y)) / (len(X)-1))
print("Variance of X: ", variance(X))
print("Covariance of X, Y is: ", covariance(X,Y))
#For simple linear regression, beta is cov/var. 
#Following calculation of beta, I can also get alpha a = y - bx
beta = covariance(X,Y) / variance(X)
beta
Explanation: Solving OLS for simple regression
Goal is to calculate vector of coefficients beta that minimizes cost function.
End of explanation
#Load another set
X_test = np.array([8,9,11,16,12])
Y_test = np.array([11,8.5,15,18,11])
model.fit(X_test.reshape(-1,1),Y_test)
model.predict(X_test.reshape(-1,1))
def total_sum_squares(Y):
    return np.sum((Y_test - np.mean(Y_test))**2)
#Residual sum of squares
def residual_sum_squares(Y):
    return np.sum( (Y_test - model.predict(X_test.reshape(-1,1)))**2)
#Get R square
1 - residual_sum_squares(Y_test)/total_sum_squares(Y_test)
#From sklearn
model.score(X_test.reshape(-1,1),Y_test)
Explanation: Evaluate fit using r square
End of explanation
from numpy.linalg import inv
from numpy import dot, transpose
X = [[1, 6, 2], [1, 8, 1], [1, 10, 0], [1, 14, 2], [1, 18, 0]]
X
y = [[7], [9], [13], [17.5], [18]]
#Solve using linear algebra
dot(inv(dot(transpose(X),X)), dot(transpose(X),y))
#Solve using numpy least squares procedure
from numpy.linalg import lstsq
lstsq(X,y)[0]
#Compare simple vs multinomial
X = [[6, 2], [8, 1], [10, 0], [14, 2], [18, 0]]
Y = [[7],    [9],    [13],    [17.5],  [18]]
model = LinearRegression()
model.fit(X,Y)
X_test = [[8, 2], [9, 0], [11, 2], [16, 2], [12, 0]]
Y_test = [[11],   [8.5],  [15],    [18],    [11]]
predictions = model.predict(X_test)
for i, prediction in enumerate(predictions):
    print("Predicted: %s, Target: %s" %(prediction, Y_test[i]))
print("R square:", model.score(X_test, Y_test))
Explanation: Multiple linear regression
Add topings to our model of pizza price prediction
End of explanation
from sklearn.preprocessing import PolynomialFeatures
X_train = [[6], [8], [10], [14],   [18]]
y_train = [[7], [9], [13], [17.5], [18]]
X_test = [[6],  [8],   [11], [16]]
y_test = [[8], [12], [15], [18]]
regressor = LinearRegression()
regressor.fit(X_train, y_train)
xx = np.linspace(0, 26, 100)
xx
yy = regressor.predict(xx.reshape(xx.shape[0],1))
plt.plot(xx,yy)
quadratic_featureziser = PolynomialFeatures(degree=2)
X_train_quadratic = quadratic_featureziser.fit_transform(X_train)
X_train_quadratic
X_test_quadratic = quadratic_featureziser.transform(X_test)
regressor_quadratic = LinearRegression()
regressor_quadratic.fit(X_train_quadratic,y_train)
xx_quadratic = quadratic_featureziser.transform(xx.reshape(xx.shape[0],1))
plt.plot(xx, regressor_quadratic.predict(xx_quadratic), c='r',linestyle = '--')
plt.title("Pizza price regressed on diameter")
plt.xlabel("Diameter in inches")
plt.ylabel("Price in dollars")
plt.axis([0,25,0,25])
plt.grid(True)
plt.scatter(X_train, y_train)
plt.show()
Explanation: Polynomial regression
Use PolynomialFeatures to transform the data
End of explanation
import pandas as pd
target_url = ("http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv")
df = pd.read_csv(target_url,header=0, sep=";")
df.head()
df.describe()
plt.scatter(df['alcohol'], df['quality'])
plt.xlabel("Alcohol")
plt.ylabel("Quality")
plt.title("Alcohol against Quality")
plt.show()
Explanation: Apply linear regression on Wine dataset from UCI
End of explanation
from sklearn.cross_validation import train_test_split
#Split into feature and target, train and test
X = df[list(df.columns)[:-1]]
y = df['quality']
X.head()
y.tail()
X_train, X_test, y_train, y_test = train_test_split(X, y)
regressor = LinearRegression()
regressor.fit(X_train, y_train)
y_predictions = regressor.predict(X_test)
#Check R squared
print("R squared is: ", regressor.score(X_test, y_test))
Explanation: Fit and evaluate the model
End of explanation
#Make cross validation
from sklearn.cross_validation import cross_val_score
scores = cross_val_score(regressor, X, y, cv = 5)
print(scores.mean(), scores)
Explanation: Cross validation
End of explanation
from sklearn.datasets import load_boston
from sklearn.linear_model import SGDRegressor
from sklearn.preprocessing import StandardScaler
data = load_boston()
X_train, X_test, y_train, y_test = train_test_split(data.data, data.target)
X_scaler = StandardScaler()
y_scaler = StandardScaler()
X_train = X_scaler.fit_transform(X_train)
y_train = y_scaler.fit_transform(y_train)
X_test = X_scaler.transform(X_test)
y_test = y_scaler.transform(y_test)
regressor = SGDRegressor(loss='squared_loss')
scores = cross_val_score(regressor, X_train, y_train, cv=5)
print('Cross validation r-squared scores:', scores)
print('Average cross validation r-squared score:', np.mean(scores))
regressor.fit_transform(X_train, y_train)
print('Test set r-squared score', regressor.score(X_test, y_test))
Explanation: Fitting using gradient descent
End of explanation
<END_TASK>