<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 In the last notebooks the best parameters for the datasets (base period, training period) were found, for some models, and for the prediction of 1, 7, 14, 28 and 56 days ahead. Two models were finally considered
Step1:  Let's first organize the results from the previous notebooks
Step2:  Which is the best model before hyperparameter tuning?
Step3:  Before hyperparameter tuning, it seems like the linear regression is doing better in all the predictions. Clearly, as the days ahead are more, the r2 value drops, and the mre goes up.
Let's search for better hyperparameters for the Random Forest models
Step4:  Build the hyperparameters DataFrame
Step5:  Ahead days = 1
Step6:  Ahead days = 7
Step7:  Ahead days = 14
Step8:  Ahead days = 28
Step9:  Ahead days = 56
<ASSISTANT_TASK:>
Python Code:
# Basic imports
import os
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import datetime as dt
import scipy.optimize as spo
import sys
from time import time
from sklearn.metrics import r2_score, median_absolute_error
%matplotlib inline
%pylab inline
pylab.rcParams['figure.figsize'] = (20.0, 10.0)
%load_ext autoreload
%autoreload 2
sys.path.append('../../')
import utils.misc as misc
Explanation: In the last notebooks the best parameters for the datasets (base period, training period) were found, for some models, and for the prediction of 1, 7, 14, 28 and 56 days ahead. Two models were finally considered: Linear Regression and Random Forest. In this notebook the hyperparameters will be tuned for the Random Forest, and the best model will be chosen for each "ahead day" (1, 7, 14, 28, 56).
End of explanation
res_df = pd.read_pickle('../../data/results_ahead1_linear_df.pkl')
res_df.head()
RELEVANT_COLUMNS = ['base_days', 
                    'train_days', 
                    'r2',
                    'mre',
                    'ahead_days',
                    'train_val_time',
                    'step_days',
                    'GOOD_DATA_RATIO',
                    'SAMPLES_GOOD_DATA_RATIO',
                    'x_filename',
                    'y_filename']
best_params_df = res_df[RELEVANT_COLUMNS].loc[np.argmin(res_df['mre']),:]
best_params_df['model'] = 'linear'
best_params_df
test_df = pd.DataFrame()
test_df.append(best_params_df, ignore_index=True)
RELEVANT_COLUMNS = ['base_days', 
                    'train_days', 
                    'r2',
                    'mre',
                    'ahead_days',
                    'train_val_time',
                    'step_days',
                    'GOOD_DATA_RATIO',
                    'SAMPLES_GOOD_DATA_RATIO',
                    'x_filename',
                    'y_filename']
ahead_days_list = [1, 7, 14, 28, 56]
models_list = ['linear', 'random_forest']
results_df = pd.DataFrame()
for ahead_days in ahead_days_list:
    for model in models_list:
        res_df = pd.read_pickle('../../data/results_ahead{}_{}_df.pkl'.format(ahead_days, model))
        best_params_df = res_df[RELEVANT_COLUMNS].loc[np.argmax(res_df['r2']),:]
        best_params_df['ahead_days'] = ahead_days
        best_params_df['model'] = model
        results_df = results_df.append(best_params_df, ignore_index=True)
results_df
results_df.to_pickle('../../data/best_dataset_params_raw_df.pkl')
Explanation: Let's first organize the results from the previous notebooks
End of explanation
def keep_max_r2(record):
    return record.loc[np.argmax(record['r2']),:]
best_r2_df = results_df.groupby('ahead_days').apply(keep_max_r2)
best_r2_df
best_r2_df[['mre', 'r2']].plot()
Explanation: Which is the best model before hyperparameter tuning?
End of explanation
initial_performance_df = results_df[results_df['model']=='random_forest']
initial_performance_df.set_index('ahead_days', inplace=True)
initial_performance_df
initial_performance_df.loc[14, 'base_days']
Explanation: Before hyperparameter tuning, it seems like the linear regression is doing better in all the predictions. Clearly, as the days ahead are more, the r2 value drops, and the mre goes up.
Let's search for better hyperparameters for the Random Forest models
End of explanation
n_estimators = [10, 50, 100, 200]
max_depth = [None, 5, 10, 15]
hyper_df = pd.DataFrame([(x, y) for x in n_estimators for y in max_depth], columns=['n_estimators', 'max_depth'])
hyper_df['n_jobs'] = -1
hyper_df
params_df = initial_performance_df.loc[1]
params_df
Explanation: Build the hyperparameters DataFrame
End of explanation
AHEAD_DAYS = 1
# Get the normal parameters set
params_df = initial_performance_df.loc[AHEAD_DAYS].copy()
params_df['ahead_days'] = AHEAD_DAYS
tic = time()
from predictor.random_forest_predictor import RandomForestPredictor
PREDICTOR_NAME = 'random_forest'
# Global variables
eval_predictor_class = RandomForestPredictor
step_eval_days = 60  # The step to move between training/validation pairs
# Build the params list
params = {'params_df': params_df,
          'step_eval_days': step_eval_days,
          'eval_predictor_class': eval_predictor_class}
results_df = misc.parallelize_dataframe(hyper_df, misc.search_mean_score_eval, params)
# Some postprocessing... -----------------------------------------------------------
results_df['r2'] = results_df.apply(lambda x: x['scores'][0], axis=1)
results_df['mre'] = results_df.apply(lambda x: x['scores'][1], axis=1)
# Pickle that!
results_df.to_pickle('../../data/hyper_ahead{}_{}_df.pkl'.format(AHEAD_DAYS, PREDICTOR_NAME))
results_df['r2'].plot()
print('Minimum MRE param set: \n {}'.format(results_df.iloc[np.argmin(results_df['mre'])]))
print('Maximum R^2 param set: \n {}'.format(results_df.iloc[np.argmax(results_df['r2'])]))
# -----------------------------------------------------------------------------------
toc = time()
print('Elapsed time: {} seconds.'.format((toc-tic)))
Explanation: Ahead days = 1
End of explanation
AHEAD_DAYS = 7
# Get the normal parameters set
params_df = initial_performance_df.loc[AHEAD_DAYS].copy()
params_df['ahead_days'] = AHEAD_DAYS
tic = time()
from predictor.random_forest_predictor import RandomForestPredictor
PREDICTOR_NAME = 'random_forest'
# Global variables
eval_predictor_class = RandomForestPredictor
step_eval_days = 60  # The step to move between training/validation pairs
# Build the params list
params = {'params_df': params_df,
          'step_eval_days': step_eval_days,
          'eval_predictor_class': eval_predictor_class}
results_df = misc.parallelize_dataframe(hyper_df, misc.search_mean_score_eval, params)
# Some postprocessing... -----------------------------------------------------------
results_df['r2'] = results_df.apply(lambda x: x['scores'][0], axis=1)
results_df['mre'] = results_df.apply(lambda x: x['scores'][1], axis=1)
# Pickle that!
results_df.to_pickle('../../data/hyper_ahead{}_{}_df.pkl'.format(AHEAD_DAYS, PREDICTOR_NAME))
results_df['r2'].plot()
print('Minimum MRE param set: \n {}'.format(results_df.iloc[np.argmin(results_df['mre'])]))
print('Maximum R^2 param set: \n {}'.format(results_df.iloc[np.argmax(results_df['r2'])]))
# -----------------------------------------------------------------------------------
toc = time()
print('Elapsed time: {} seconds.'.format((toc-tic)))
Explanation: Ahead days = 7
End of explanation
AHEAD_DAYS = 14
# Get the normal parameters set
params_df = initial_performance_df.loc[AHEAD_DAYS].copy()
params_df['ahead_days'] = AHEAD_DAYS
tic = time()
from predictor.random_forest_predictor import RandomForestPredictor
PREDICTOR_NAME = 'random_forest'
# Global variables
eval_predictor_class = RandomForestPredictor
step_eval_days = 60  # The step to move between training/validation pairs
# Build the params list
params = {'params_df': params_df,
          'step_eval_days': step_eval_days,
          'eval_predictor_class': eval_predictor_class}
results_df = misc.parallelize_dataframe(hyper_df, misc.search_mean_score_eval, params)
# Some postprocessing... -----------------------------------------------------------
results_df['r2'] = results_df.apply(lambda x: x['scores'][0], axis=1)
results_df['mre'] = results_df.apply(lambda x: x['scores'][1], axis=1)
# Pickle that!
results_df.to_pickle('../../data/hyper_ahead{}_{}_df.pkl'.format(AHEAD_DAYS, PREDICTOR_NAME))
results_df['r2'].plot()
print('Minimum MRE param set: \n {}'.format(results_df.iloc[np.argmin(results_df['mre'])]))
print('Maximum R^2 param set: \n {}'.format(results_df.iloc[np.argmax(results_df['r2'])]))
# -----------------------------------------------------------------------------------
toc = time()
print('Elapsed time: {} seconds.'.format((toc-tic)))
Explanation: Ahead days = 14
End of explanation
AHEAD_DAYS = 28
# Get the normal parameters set
params_df = initial_performance_df.loc[AHEAD_DAYS].copy()
params_df['ahead_days'] = AHEAD_DAYS
tic = time()
from predictor.random_forest_predictor import RandomForestPredictor
PREDICTOR_NAME = 'random_forest'
# Global variables
eval_predictor_class = RandomForestPredictor
step_eval_days = 60  # The step to move between training/validation pairs
# Build the params list
params = {'params_df': params_df,
          'step_eval_days': step_eval_days,
          'eval_predictor_class': eval_predictor_class}
results_df = misc.parallelize_dataframe(hyper_df, misc.search_mean_score_eval, params)
# Some postprocessing... -----------------------------------------------------------
results_df['r2'] = results_df.apply(lambda x: x['scores'][0], axis=1)
results_df['mre'] = results_df.apply(lambda x: x['scores'][1], axis=1)
# Pickle that!
results_df.to_pickle('../../data/hyper_ahead{}_{}_df.pkl'.format(AHEAD_DAYS, PREDICTOR_NAME))
results_df['r2'].plot()
print('Minimum MRE param set: \n {}'.format(results_df.iloc[np.argmin(results_df['mre'])]))
print('Maximum R^2 param set: \n {}'.format(results_df.iloc[np.argmax(results_df['r2'])]))
# -----------------------------------------------------------------------------------
toc = time()
print('Elapsed time: {} seconds.'.format((toc-tic)))
Explanation: Ahead days = 28
End of explanation
AHEAD_DAYS = 56
# Get the normal parameters set
params_df = initial_performance_df.loc[AHEAD_DAYS].copy()
params_df['ahead_days'] = AHEAD_DAYS
tic = time()
from predictor.random_forest_predictor import RandomForestPredictor
PREDICTOR_NAME = 'random_forest'
# Global variables
eval_predictor_class = RandomForestPredictor
step_eval_days = 60  # The step to move between training/validation pairs
# Build the params list
params = {'params_df': params_df,
          'step_eval_days': step_eval_days,
          'eval_predictor_class': eval_predictor_class}
results_df = misc.parallelize_dataframe(hyper_df, misc.search_mean_score_eval, params)
# Some postprocessing... -----------------------------------------------------------
results_df['r2'] = results_df.apply(lambda x: x['scores'][0], axis=1)
results_df['mre'] = results_df.apply(lambda x: x['scores'][1], axis=1)
# Pickle that!
results_df.to_pickle('../../data/hyper_ahead{}_{}_df.pkl'.format(AHEAD_DAYS, PREDICTOR_NAME))
results_df['r2'].plot()
print('Minimum MRE param set: \n {}'.format(results_df.iloc[np.argmin(results_df['mre'])]))
print('Maximum R^2 param set: \n {}'.format(results_df.iloc[np.argmax(results_df['r2'])]))
# -----------------------------------------------------------------------------------
toc = time()
print('Elapsed time: {} seconds.'.format((toc-tic)))
Explanation: Ahead days = 56
End of explanation
<END_TASK>