<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 MeterGroup, ElecMeter, selection and basic statistics
All NILM datasets consists of various groupings of electricity meters.  We can group the meters by house.  Or by the type of appliance they are directly connected to.  Or by sample rate.  Or by whether the meter is a whole-house "site meter" or an appliance-level submeter, or a circuit-level submeter.
In NILMTK, one of the key classes is MeterGroup which stores a list of meters and allows us to select a subset of meters, aggregate power from all meters and many other functions.
When we first open a DataSet, NILMTK creates several MeterGroup objects.  There's nilmtk.global_meter_group which holds every meter currently loaded (including from multiple datasets if you have opened more than one dataset).  There is also one MeterGroup per building (which live in the Building.elec attribute).  We also nested MeterGroups for aggregating together split-phase mains, 3-phase mains and dual-supply (240 volt) appliances in North American and Canadan datasets.  For example, here is the MeterGroup for building 1 in REDD
Step1:  Note that there are two nested MeterGroups
Step2:  Putting these meters into a MeterGroup allows us to easily sum together the power demand recorded by both meters to get the total power demand for the entire appliance (but it's also very easy to see the individual meter power demand too).
We can easily get a MeterGroup of either the submeters or the mains
Step3:  We can easily get the power data for both mains meters summed together
Step4:  Stats for MeterGroups
Proportion of energy submetered
Let's work out the proportion of energy submetered in REDD building 1
Step5:  Note that NILMTK has raised a warning that Mains uses a different type of power measurement than all the submeters, so it's not an entirely accurate comparison.  Which raises the question
Step6:  Total Energy
Step7:  Energy per submeter
Step8:  column headings are the ElecMeter instance numbers.
The function fraction_per_meter does the same thing as energy_per_submeter but returns the fraction of energy per meter.
Select meters on the basis of their energy consumption
Let's make a new MeterGroup which only contains the ElecMeters which used more than 20 kWh
Step9:  Draw wiring diagram
We can get the wiring diagram for the MeterGroup
Step10:  It's not very pretty but it shows that meters (1,2) (the site meters) are upstream of all other meters.
Buildings in REDD have only two levels in their meter hierarchy (mains and submeters).  If there were more than two levels then it might be useful to get only the meters immediately downstream of mains
Step11:  Stats and info for individual meters
The ElecMeter class represents a single electricity meter.  Each ElecMeter has a list of associated Appliance objects.  ElecMeter has many of the same stats methods as MeterGroup such as total_energy and available_power_ac_types and power_series and power_series_all_data.  We will now explore some more stats functions (many of which are also available on MeterGroup)...
Step12:  Get upstream meter
Step13:  Metadata about the class of meter
Step14:  Dominant appliance
If the metadata specifies that a meter has multiple meters connected to it then one of those can be specified as the 'dominant' appliance, and this appliance can be retrieved with this method
Step15:  Total energy
Step16:  Get good sections
If we plot the raw power data then we see there is one large gap where, supposedly, the metering system was not working.  (if we were to zoom in then we'd see lots of smaller gaps too)
Step17:  We can automatically identify the 'good sections' (i.e. the sections where every pair of consecutive samples is less than max_sample_period specified in the dataset metadata)
Step18:  The blue chunks show where the data is good.  The white gap is the large gap seen in the raw power data.  There are lots of smaller gaps that we cannot see at this zoom level.
We can also see the exact sections identified
Step19:  Dropout rate
As well as large gaps appearing because the entire system is down, we also get frequent small gaps from wireless sensors dropping data.  This is sometimes called 'dropout'.  The dropout rate is a number between 0 and 1 which specifies the proportion of missing samples.  A dropout rate of 0 means no samples are missing.  A value of 1 would mean all samples are missing
Step20:  Note that the dropout rate has gone down (which is good!) now that we are ignoring the gaps.  This value is probably more representative of the performance of the wireless system.
Select subgroups of meters
We use ElecMeter.select_using_appliances() to select a new MeterGroup using an metadata field.  For example, to get all the washer dryers in the whole of the REDD dataset
Step21:  Or all appliances in the 'heating' category
Step22:  Or all appliances in building 1 with a single-phase induction motor(!)
Step23:  (NILMTK imports the 'common metadata' from the NILM Metadata project, which includes a wide range of different category taxonomies)
Step24:  Select a group of meters from properties of the meters (not the appliances)
Step25:  Select a single meter from a MeterGroup
We use [] to retrive a single ElecMeter from a MeterGroup.
Search for a meter using appliances connected to each meter
Step26:  Appliances are uniquely identified within a building by a type (fridge, kettle, television, etc.) and an instance number.  If we do not specify an instance number then ElecMeter retrieves instance 1 (instance numbering starts from 1).  If you want a different instance then just do this
Step27:  To uniquely identify an appliance in nilmtk.global_meter_group then we must specify the dataset name, building instance number, appliance type and appliance instance in a dict
Step28:  Search for a meter using details of the ElecMeter
get ElecMeter with instance = 1
Step29:  Instance numbering
ElecMeter and Appliance instance numbers uniquely identify the meter or appliance type within the building, not globally.  To uniquely identify a meter globally, we need three keys
Step30:  Select nested MeterGroup
We can also select a single, existing nested MeterGroup.  There are two ways to specify a nested MeterGroup
Step31:  We can also specify the mains by asking for meter instance 0
Step32:  which is equivalent to elec.mains()
<ASSISTANT_TASK:>
Python Code:
from matplotlib import rcParams
import matplotlib.pyplot as plt
%matplotlib inline
rcParams['figure.figsize'] = (13, 6)
plt.style.use('ggplot')
from nilmtk import DataSet
redd = DataSet('/data/REDD/redd.h5')
elec = redd.buildings[1].elec
elec
Explanation: MeterGroup, ElecMeter, selection and basic statistics
All NILM datasets consists of various groupings of electricity meters.  We can group the meters by house.  Or by the type of appliance they are directly connected to.  Or by sample rate.  Or by whether the meter is a whole-house "site meter" or an appliance-level submeter, or a circuit-level submeter.
In NILMTK, one of the key classes is MeterGroup which stores a list of meters and allows us to select a subset of meters, aggregate power from all meters and many other functions.
When we first open a DataSet, NILMTK creates several MeterGroup objects.  There's nilmtk.global_meter_group which holds every meter currently loaded (including from multiple datasets if you have opened more than one dataset).  There is also one MeterGroup per building (which live in the Building.elec attribute).  We also nested MeterGroups for aggregating together split-phase mains, 3-phase mains and dual-supply (240 volt) appliances in North American and Canadan datasets.  For example, here is the MeterGroup for building 1 in REDD:
End of explanation
elec.nested_metergroups()
Explanation: Note that there are two nested MeterGroups: one for the electric oven, and one for the washer dryer (both of which are 240 volt appliances and have two meters per appliance):
End of explanation
elec.mains()
Explanation: Putting these meters into a MeterGroup allows us to easily sum together the power demand recorded by both meters to get the total power demand for the entire appliance (but it's also very easy to see the individual meter power demand too).
We can easily get a MeterGroup of either the submeters or the mains:
End of explanation
elec.mains().power_series_all_data().head()
elec.submeters()
Explanation: We can easily get the power data for both mains meters summed together:
End of explanation
elec.proportion_of_energy_submetered()
Explanation: Stats for MeterGroups
Proportion of energy submetered
Let's work out the proportion of energy submetered in REDD building 1:
End of explanation
mains = elec.mains()
mains.available_power_ac_types()
elec.submeters().available_power_ac_types()
elec.load()
Explanation: Note that NILMTK has raised a warning that Mains uses a different type of power measurement than all the submeters, so it's not an entirely accurate comparison.  Which raises the question: which type of power measurements are used for the mains and submeters?  Let's find out...
Active, apparent and reactive power
End of explanation
elec.mains().total_energy() # returns kWh
Explanation: Total Energy
End of explanation
energy_per_meter = elec.submeters().energy_per_meter() # kWh, again
energy_per_meter
Explanation: Energy per submeter
End of explanation
# energy_per_meter is a DataFrame where each row is a 
# power type ('active', 'reactive' or 'apparent').
# All appliance meters in REDD are record 'active' so just select
# the 'active' row:
energy_per_meter = energy_per_meter.loc['active']
more_than_20 = energy_per_meter[energy_per_meter > 20]
more_than_20
instances = more_than_20.index
instances
Explanation: column headings are the ElecMeter instance numbers.
The function fraction_per_meter does the same thing as energy_per_submeter but returns the fraction of energy per meter.
Select meters on the basis of their energy consumption
Let's make a new MeterGroup which only contains the ElecMeters which used more than 20 kWh:
End of explanation
elec.draw_wiring_graph()
Explanation: Draw wiring diagram
We can get the wiring diagram for the MeterGroup:
End of explanation
elec.meters_directly_downstream_of_mains()
Explanation: It's not very pretty but it shows that meters (1,2) (the site meters) are upstream of all other meters.
Buildings in REDD have only two levels in their meter hierarchy (mains and submeters).  If there were more than two levels then it might be useful to get only the meters immediately downstream of mains:
End of explanation
fridge_meter = elec['fridge']
Explanation: Stats and info for individual meters
The ElecMeter class represents a single electricity meter.  Each ElecMeter has a list of associated Appliance objects.  ElecMeter has many of the same stats methods as MeterGroup such as total_energy and available_power_ac_types and power_series and power_series_all_data.  We will now explore some more stats functions (many of which are also available on MeterGroup)...
End of explanation
fridge_meter.upstream_meter() # happens to be the mains meter group!
Explanation: Get upstream meter
End of explanation
fridge_meter.device
Explanation: Metadata about the class of meter
End of explanation
fridge_meter.dominant_appliance()
Explanation: Dominant appliance
If the metadata specifies that a meter has multiple meters connected to it then one of those can be specified as the 'dominant' appliance, and this appliance can be retrieved with this method:
End of explanation
fridge_meter.total_energy() # kWh
Explanation: Total energy
End of explanation
fridge_meter.plot()
Explanation: Get good sections
If we plot the raw power data then we see there is one large gap where, supposedly, the metering system was not working.  (if we were to zoom in then we'd see lots of smaller gaps too):
End of explanation
good_sections = fridge_meter.good_sections(full_results=True)
# specifying full_results=False would give us a simple list of 
# TimeFrames.  But we want the full GoodSectionsResults object so we can
# plot the good sections...
good_sections.plot()
Explanation: We can automatically identify the 'good sections' (i.e. the sections where every pair of consecutive samples is less than max_sample_period specified in the dataset metadata):
End of explanation
good_sections.combined()
Explanation: The blue chunks show where the data is good.  The white gap is the large gap seen in the raw power data.  There are lots of smaller gaps that we cannot see at this zoom level.
We can also see the exact sections identified:
End of explanation
fridge_meter.dropout_rate()
Explanation: Dropout rate
As well as large gaps appearing because the entire system is down, we also get frequent small gaps from wireless sensors dropping data.  This is sometimes called 'dropout'.  The dropout rate is a number between 0 and 1 which specifies the proportion of missing samples.  A dropout rate of 0 means no samples are missing.  A value of 1 would mean all samples are missing:
End of explanation
import nilmtk
nilmtk.global_meter_group.select_using_appliances(type='washer dryer')
Explanation: Note that the dropout rate has gone down (which is good!) now that we are ignoring the gaps.  This value is probably more representative of the performance of the wireless system.
Select subgroups of meters
We use ElecMeter.select_using_appliances() to select a new MeterGroup using an metadata field.  For example, to get all the washer dryers in the whole of the REDD dataset:
End of explanation
nilmtk.global_meter_group.select_using_appliances(category='heating')
Explanation: Or all appliances in the 'heating' category:
End of explanation
nilmtk.global_meter_group.select_using_appliances(building=1, category='single-phase induction motor')
Explanation: Or all appliances in building 1 with a single-phase induction motor(!):
End of explanation
nilmtk.global_meter_group.select_using_appliances(building=2, category='laundry appliances')
Explanation: (NILMTK imports the 'common metadata' from the NILM Metadata project, which includes a wide range of different category taxonomies)
End of explanation
elec.select(device_model='REDD_whole_house')
elec.select(sample_period=3)
Explanation: Select a group of meters from properties of the meters (not the appliances)
End of explanation
elec['fridge']
Explanation: Select a single meter from a MeterGroup
We use [] to retrive a single ElecMeter from a MeterGroup.
Search for a meter using appliances connected to each meter
End of explanation
elec['light', 2]
Explanation: Appliances are uniquely identified within a building by a type (fridge, kettle, television, etc.) and an instance number.  If we do not specify an instance number then ElecMeter retrieves instance 1 (instance numbering starts from 1).  If you want a different instance then just do this:
End of explanation
import nilmtk
nilmtk.global_meter_group[{'dataset': 'REDD', 'building': 1, 'type': 'fridge', 'instance': 1}]
Explanation: To uniquely identify an appliance in nilmtk.global_meter_group then we must specify the dataset name, building instance number, appliance type and appliance instance in a dict:
End of explanation
elec[1]
Explanation: Search for a meter using details of the ElecMeter
get ElecMeter with instance = 1:
End of explanation
from nilmtk.elecmeter import ElecMeterID 
# ElecMeterID is a namedtuple for uniquely identifying each ElecMeter
nilmtk.global_meter_group[ElecMeterID(instance=8, building=1, dataset='REDD')]
Explanation: Instance numbering
ElecMeter and Appliance instance numbers uniquely identify the meter or appliance type within the building, not globally.  To uniquely identify a meter globally, we need three keys:
End of explanation
elec[[ElecMeterID(instance=3, building=1, dataset='REDD'), 
      ElecMeterID(instance=4, building=1, dataset='REDD')]]
elec[ElecMeterID(instance=(3,4), building=1, dataset='REDD')]
Explanation: Select nested MeterGroup
We can also select a single, existing nested MeterGroup.  There are two ways to specify a nested MeterGroup:
End of explanation
elec[ElecMeterID(instance=0, building=1, dataset='REDD')]
Explanation: We can also specify the mains by asking for meter instance 0:
End of explanation
elec.mains() == elec[ElecMeterID(instance=0, building=1, dataset='REDD')]
Explanation: which is equivalent to elec.mains():
End of explanation
<END_TASK>