<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Disaggregation
Step1:  Dividing data into train and test set
Step2:  Let us use building 1 for demo purposes
Step3:  Let's split data at April 30th
Step4:  Visualizing the data
Step5:  REDD data set has got appliance level data sampled every 3 or 4 seconds and mains data sampled every 1 second. Let us verify the same.
Step6:  Since, both of these are sampled at different frequencies, we will downsample both to 1 minute resolution. We will also select the top-5 appliances in terms of energy consumption and use them for training our FHMM and CO models.
Selecting top-5 appliances
Step7:  Training and disaggregation
A function to disaggregate the mains data to constituent appliances and return the predictions
Step8:  Train using 2 benchmarking algorithms -  Combinatorial Optimisation (CO) and Factorial Hidden Markov Model (FHMM)
Step9:  Using prettier labels!
Step10:  Taking a look at the ground truth of top 5 appliance power consumption
Step11:  Plotting the predictions against the actual usage
Step12:  Comparing NILM algorithms (CO vs FHMM)
nilmtk.utils.compute_rmse is an extended of the following, handling both missing values and labels better
<ASSISTANT_TASK:>
Python Code:
from __future__ import print_function, division
import time
from matplotlib import rcParams
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
from six import iteritems
from nilmtk import DataSet, TimeFrame, MeterGroup, HDFDataStore
from nilmtk.legacy.disaggregate import CombinatorialOptimisation, FHMM
import nilmtk.utils
%matplotlib inline
rcParams['figure.figsize'] = (13, 6)
Explanation: Disaggregation
End of explanation
train = DataSet('/data/redd.h5')
test = DataSet('/data/redd.h5')
Explanation: Dividing data into train and test set
End of explanation
building = 1
Explanation: Let us use building 1 for demo purposes
End of explanation
train.set_window(end="2011-04-30")
test.set_window(start="2011-04-30")
train_elec = train.buildings[1].elec
test_elec = test.buildings[1].elec
Explanation: Let's split data at April 30th
End of explanation
train_elec.plot()
test_elec.mains().plot()
Explanation: Visualizing the data
End of explanation
fridge_meter = train_elec['fridge']
fridge_df = next(fridge_meter.load())
fridge_df.head()
mains = train_elec.mains()
mains_df = next(mains.load())
mains_df.head()
Explanation: REDD data set has got appliance level data sampled every 3 or 4 seconds and mains data sampled every 1 second. Let us verify the same.
End of explanation
top_5_train_elec = train_elec.submeters().select_top_k(k=5)
top_5_train_elec
Explanation: Since, both of these are sampled at different frequencies, we will downsample both to 1 minute resolution. We will also select the top-5 appliances in terms of energy consumption and use them for training our FHMM and CO models.
Selecting top-5 appliances
End of explanation
def predict(clf, test_elec, sample_period, timezone):
    pred = {}
    gt= {}
    
    # "ac_type" varies according to the dataset used. 
    # Make sure to use the correct ac_type before using the default parameters in this code.    
    for i, chunk in enumerate(test_elec.mains().load(physical_quantity = 'power', ac_type = 'apparent', sample_period=sample_period)):
        chunk_drop_na = chunk.dropna()
        pred[i] = clf.disaggregate_chunk(chunk_drop_na)
        gt[i]={}
        for meter in test_elec.submeters().meters:
            # Only use the meters that we trained on (this saves time!)    
            gt[i][meter] = next(meter.load(physical_quantity = 'power', ac_type = 'active', sample_period=sample_period))
        gt[i] = pd.DataFrame({k:v.squeeze() for k,v in iteritems(gt[i]) if len(v)}, index=next(iter(gt[i].values())).index).dropna()
        
    # If everything can fit in memory
    gt_overall = pd.concat(gt)
    gt_overall.index = gt_overall.index.droplevel()
    pred_overall = pd.concat(pred)
    pred_overall.index = pred_overall.index.droplevel()
    # Having the same order of columns
    gt_overall = gt_overall[pred_overall.columns]
    
    #Intersection of index
    gt_index_utc = gt_overall.index.tz_convert("UTC")
    pred_index_utc = pred_overall.index.tz_convert("UTC")
    common_index_utc = gt_index_utc.intersection(pred_index_utc)
    
    common_index_local = common_index_utc.tz_convert(timezone)
    gt_overall = gt_overall.loc[common_index_local]
    pred_overall = pred_overall.loc[common_index_local]
    appliance_labels = [m for m in gt_overall.columns.values]
    gt_overall.columns = appliance_labels
    pred_overall.columns = appliance_labels
    return gt_overall, pred_overall
Explanation: Training and disaggregation
A function to disaggregate the mains data to constituent appliances and return the predictions
End of explanation
classifiers = {'CO':CombinatorialOptimisation(), 'FHMM':FHMM()}
predictions = {}
sample_period = 120
for clf_name, clf in classifiers.items():
    print("*"*20)
    print(clf_name)
    print("*" *20)
    start = time.time()
    # Note that we have given the sample period to downsample the data to 1 minute. 
    # If instead of top_5 we wanted to train on all appliance, we would write 
    # fhmm.train(train_elec, sample_period=60)
    clf.train(top_5_train_elec, sample_period=sample_period)
    end = time.time()
    print("Runtime =", end-start, "seconds.")
    gt, predictions[clf_name] = predict(clf, test_elec, sample_period, train.metadata['timezone'])
   
Explanation: Train using 2 benchmarking algorithms -  Combinatorial Optimisation (CO) and Factorial Hidden Markov Model (FHMM)
End of explanation
appliance_labels = [m.label() for m in gt.columns.values]
gt.columns = appliance_labels
predictions['CO'].columns = appliance_labels
predictions['FHMM'].columns = appliance_labels
Explanation: Using prettier labels!
End of explanation
gt.head()
predictions['CO'].head()
predictions['FHMM'].head()
Explanation: Taking a look at the ground truth of top 5 appliance power consumption
End of explanation
predictions['CO']['Fridge'].head(300).plot(label="Pred")
gt['Fridge'].head(300).plot(label="GT")
plt.legend()
predictions['FHMM']['Fridge'].head(300).plot(label="Pred")
gt['Fridge'].head(300).plot(label="GT")
plt.legend()
Explanation: Plotting the predictions against the actual usage
End of explanation
? nilmtk.utils.compute_rmse
rmse = {}
for clf_name in classifiers.keys():
    rmse[clf_name] = nilmtk.utils.compute_rmse(gt, predictions[clf_name])
rmse = pd.DataFrame(rmse)
rmse
Explanation: Comparing NILM algorithms (CO vs FHMM)
nilmtk.utils.compute_rmse is an extended of the following, handling both missing values and labels better:
python
def compute_rmse(gt, pred):
    from sklearn.metrics import mean_squared_error
    rms_error = {}
    for appliance in gt.columns:
        rms_error[appliance] = np.sqrt(mean_squared_error(gt[appliance], pred[appliance]))
    return pd.Series(rms_error)
End of explanation
<END_TASK>