<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Datastore basic usage
The datastore is a tool for using the eemeter which automates
and helps to scales some of the most frequent tasks accomplished by the
eemeter. These tasks include data loading and storage, meter
running, and result storage and inspection. It puts a REST API
in front of the eemeter.
Note
Step1:  If you followed the datastore development setup instructions, you will
already have run the command to create a superuser and access credentials.
python manage.py dev_seed
If you haven't already done so, do so now. The dev_seed command
creates a demo admin user and a sample project.
username
Step2:  Let's test the API by requesting a list of projects in the datastore. Since the dev_seed command creates a sample project, this will return a response showing that project.
Step3:  Although we'll delete this one in a moment, we can first explore a
bit to get a feel for the API. Then we'll create a project of our own.
Energy trace data will be associated with this project by foreign key.
It is organized into time series by trace_id, and the following request
will show all traces associated with a particular project. Note the
difference between the 'id' field and the 'project_id' field.
The 'project_id' field is the unique label that was associated with
it by an external source; the 'id' field is the database table primary
key.
There are two tables used to store energy data
Step4:  We can also query for consumption records by metadata primary key.
Step5:  Now we'll delete the project that was created by the dev_seed command and make one of our own.
Step6:  If you try to post another project with the same project_id, you'll get an error message.
Step7:  However, there is another endpoint you can hit to sync the project - update it if it exists, create it if it doesn't. This endpoint works almost the same way, but expects a list of data in a slightly different format
Step8:  Now we can give this project some consumption data. Ene
Step9:  Then we'll the sync endpoint for consumption metadata, which will create a new record or update an existing record. We have one trace here
Step10:  Let's turn that CSV into records.
Step11:  We can verify that these records were created by querying by consumption metadata id.
Step12:  We now have a simple project with a single trace of data. Now we will move to running a meter on that project
Step13:  This creates a task to run the meter on the indicated project.
These results can be viewed by requesting the project run by primary key - as it completes, its status will change to SUCCESS or FAILED. If FAILED, it will indicate a traceback of the error that occured. While it runs, its status will be RUNNING; before it has started running, its status will be PENDING.
Step14:  If this project run succeeded, we can inspect its results.
Inspecting results
Results all fall under the ProjectResult API
<ASSISTANT_TASK:>
Python Code:
# library imports
import pandas as pd
import requests
import pytz
Explanation: Datastore basic usage
The datastore is a tool for using the eemeter which automates
and helps to scales some of the most frequent tasks accomplished by the
eemeter. These tasks include data loading and storage, meter
running, and result storage and inspection. It puts a REST API
in front of the eemeter.
Note:
For small and large datasets, the ETL toolkit exists to ease and
speed up this process. That toolkit relies upon the API described
in this tutorial and in the datastore API documentation. For the
purpose of this tutorial, we will not be using the ETL toolkit.
For more information on the ETL toolkit, see its API documentation.
Loading data
For this tutorial, we will use the python requests package to make
requests to the datastore. We will use the same dataset used in the
eemeter tutorial, available for download here:
project data CSV
energy data CSV
This tutorial is also available as a jupyter notebook.
End of explanation
base_url = "http://0.0.0.0:8000"
headers = {"Authorization": "Bearer tokstr"}
Explanation: If you followed the datastore development setup instructions, you will
already have run the command to create a superuser and access credentials.
python manage.py dev_seed
If you haven't already done so, do so now. The dev_seed command
creates a demo admin user and a sample project.
username: demo,
password: demo-password,
API access token: tokstr.
project owner: 1
Ensure that your development server is running locally on port 8000 before continuing.
python manage.py runserver
Each request will include an Authorization header
Authorization: Bearer tokstr
End of explanation
url = base_url + "/api/v1/projects/"
projects = requests.get(url, headers=headers).json()
projects
Explanation: Let's test the API by requesting a list of projects in the datastore. Since the dev_seed command creates a sample project, this will return a response showing that project.
End of explanation
url = base_url + "/api/v1/consumption_metadatas/?summary=True&projects={}".format(projects[0]['id'])
consumption_metadatas = requests.get(url, headers=headers).json()
consumption_metadatas[0]
Explanation: Although we'll delete this one in a moment, we can first explore a
bit to get a feel for the API. Then we'll create a project of our own.
Energy trace data will be associated with this project by foreign key.
It is organized into time series by trace_id, and the following request
will show all traces associated with a particular project. Note the
difference between the 'id' field and the 'project_id' field.
The 'project_id' field is the unique label that was associated with
it by an external source; the 'id' field is the database table primary
key.
There are two tables used to store energy data:
Consumption metadata:
project_id: foreign key of the project this belongs to
trace_id: unique id of the trace
interpretation: the fuel (electricity/natural gas) and the direction and type of flow (net/total consumption, supply, generation)
unit: physical units of the values provided in records
Consumption records:
metadata_id: foreign key to consumption metadata
start: the start date and time of record
value: the energy reading as reported
estimated: boolean indicating whether or not the reading was estimated
For consumption records, the end is implicit in the start of the next temporal record. The last record should be null (if it's not, it will be treated as such).
Let's inspect the traces associated with this project. We can do so
using the project primary key 'id' as a filter (we use the summary flag so that we
don't pull every record):
End of explanation
url = base_url + "/api/v1/consumption_records/?metadata={}".format(consumption_metadatas[0]['id'])
consumption_records = requests.get(url, headers=headers).json()
consumption_records[:3]
Explanation: We can also query for consumption records by metadata primary key.
End of explanation
url = base_url + "/api/v1/projects/{}/".format(projects[0]['id'])
requests.delete(url, headers=headers)
project_data = pd.read_csv('sample-project-data.csv',
                           parse_dates=['retrofit_start_date', 'retrofit_end_date']).iloc[0]
project_data
data = {
    "project_id": project_data.project_id,
    "zipcode": str(project_data.zipcode),
    "baseline_period_end": pytz.UTC.localize(project_data.retrofit_start_date).isoformat(),
    "reporting_period_start": pytz.UTC.localize(project_data.retrofit_end_date).isoformat(),
    "project_owner": 1,
}
print(data)
url = base_url + "/api/v1/projects/"
new_project = requests.post(url, json=data, headers=headers).json()
new_project
Explanation: Now we'll delete the project that was created by the dev_seed command and make one of our own.
End of explanation
url = base_url + "/api/v1/projects/"
requests.post(url, json=data, headers=headers).json()
Explanation: If you try to post another project with the same project_id, you'll get an error message.
End of explanation
data = [
    {
        "project_id": project_data.project_id,
        "zipcode": str(project_data.zipcode),
        "baseline_period_end": pytz.UTC.localize(project_data.retrofit_start_date).isoformat(),
        "reporting_period_start": pytz.UTC.localize(project_data.retrofit_end_date).isoformat(),
        "project_owner_id": 1,
    }
]
print(data)
url = base_url + "/api/v1/projects/sync/"
requests.post(url, json=data, headers=headers).json()
Explanation: However, there is another endpoint you can hit to sync the project - update it if it exists, create it if it doesn't. This endpoint works almost the same way, but expects a list of data in a slightly different format:
End of explanation
energy_data = pd.read_csv('sample-energy-data_project-ABC_zipcode-50321.csv',
                          parse_dates=['date'], dtype={'zipcode': str})
energy_data.head()
Explanation: Now we can give this project some consumption data. Ene
End of explanation
interpretation_mapping = {"electricity": "E_C_S"}
data = [
    {
        "project_project_id": energy_data.iloc[0]["project_id"],
        "interpretation": interpretation_mapping[energy_data.iloc[0]["fuel"]],
        "unit": energy_data.iloc[0]["unit"].upper(),
        "label": energy_data.iloc[0]["trace_id"].upper()
    }
]
data
url = base_url + "/api/v1/consumption_metadatas/sync/"
consumption_metadatas = requests.post(url, json=data, headers=headers).json()
consumption_metadatas
Explanation: Then we'll the sync endpoint for consumption metadata, which will create a new record or update an existing record. We have one trace here:
End of explanation
data = [{
    "metadata_id": consumption_metadatas[0]['id'],
    "start": pytz.UTC.localize(row.date.to_datetime()).isoformat(),
    "value": row.value,
    "estimated": row.estimated,
} for _, row in energy_data.iterrows()]
data[:3]
url = base_url + "/api/v1/consumption_records/sync2/"
consumption_records = requests.post(url, json=data, headers=headers)
consumption_records.text
Explanation: Let's turn that CSV into records.
End of explanation
url = base_url + "/api/v1/consumption_records/?metadata={}".format(consumption_metadatas[0]['id'])
consumption_records = requests.get(url, json=data, headers=headers).json()
consumption_records[:3]
Explanation: We can verify that these records were created by querying by consumption metadata id.
End of explanation
data = {
    "project": new_project['id'],
    "meter_class": "EnergyEfficiencyMeter",
    "meter_settings": {}
}
data
url = base_url + "/api/v1/project_runs/"
project_run = requests.post(url, json=data, headers=headers).json()
project_run
Explanation: We now have a simple project with a single trace of data. Now we will move to running a meter on that project:
Running meters
To run a meter, make a request to create a "project run". This request will start a job that runs a meter and saves its results.
There are a few components to this request.
"project": the primary key of the project.
"meter_class": the name of the class of the eemeter meter to run.
"meter_settings": any special settings to send to the meter class.
End of explanation
url = base_url + "/api/v1/project_runs/{}/".format(project_run['id'])
project_runs = requests.get(url, headers=headers).json()
project_runs
Explanation: This creates a task to run the meter on the indicated project.
These results can be viewed by requesting the project run by primary key - as it completes, its status will change to SUCCESS or FAILED. If FAILED, it will indicate a traceback of the error that occured. While it runs, its status will be RUNNING; before it has started running, its status will be PENDING.
End of explanation
url = base_url + "/api/v1/project_results/"
project_results = requests.get(url, headers=headers).json()
project_results
Explanation: If this project run succeeded, we can inspect its results.
Inspecting results
Results all fall under the ProjectResult API
End of explanation
<END_TASK>