<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 http
Step1:  p
Step2:  Train - Test
<ASSISTANT_TASK:>
Python Code:
from __future__ import division
import pandas as pd
from pandas import read_csv
from pandas import datetime
from matplotlib import pyplot as plt
from pandas.plotting import autocorrelation_plot
from statsmodels.tsa.arima_model import ARIMA
from pandas import DataFrame
from sklearn.metrics import mean_squared_error
from statsmodels.graphics.tsaplots import plot_pacf
%matplotlib inline
def date_parser(x):
    return datetime.strptime('190'+x, '%Y-%m')
series = read_csv('shampoo_sales.csv',
                  header=0, 
                  index_col=0, #time
                  squeeze=True, #this returns a series
                  parse_dates=[0],  #allows for automatic parsing of dates!
                  date_parser = date_parser
                 )
series.head()
series.plot()
plt.show()
autocorrelation_plot(series)
plt.show()
plot_pacf(series)
plt.show()
Explanation: http://machinelearningmastery.com/arima-for-time-series-forecasting-with-python/
The idea with ARIMA models is that the final residual should look like white noise otherwise, there is juice or information available in the data to extract
End of explanation
#fit entire model
model = ARIMA(series, order=(5,1,0))
model_fit = model.fit(disp=0)
model_fit.summary()
residuals = DataFrame(model_fit.resid)
residuals.plot()
plt.show()
residuals.plot(kind='kde')
plt.show()
residuals.describe()
Explanation: p: The number of lag observations included in the model, also called the lag order.
d: The number of times that the raw observations are differenced, also called the degree of differencing.
q: The size of the moving average window, also called the order of moving average.
End of explanation
XX = series.values
XX.shape
#splitting
train_size = int(len(XX)*2/3)
train_size
train_set = XX[:train_size]
test_set = XX[train_size:]
train_set.shape, test_set.shape
train_set[0]
history = list(train_set)
history[0] = 3333
history = list(train_set)
len(history)
predictions = []
test_len = len(test_set)
test_len
%%time
for tt in range(len(test_set)):
    output = ARIMA(history, order=(5,1,0)).fit(method='css', disp=0).forecast()
    y_hat = output[0] #0th is the index of the prediction
    predictions.append(y_hat)
    
    observation = test_set[tt]
    #history.append(observation)
    history.append(y_hat)
    
    print "predicted: {}, expected: {}".format(y_hat, observation)
error = mean_squared_error(predictions, test_set)
error
plt.figure(figsize=(15,6))
plt.plot(predictions, label='preds')
plt.plot(test_set, label='test set')
plt.legend()
plt.show()
fitted = ARIMA(list(train_set), order=(5,1,0)).fit(disp=0)
fitted
autocorrelation_plot(fitted.resid)
plot_pacf(fitted.resid)
plt.show()
Explanation: Train - Test
End of explanation
<END_TASK>