<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <img style="float
Step1:  Starting a Hyperstream instance
First of all, we will create a HyperStream instance. This instance will connect to the MongoDB server that is specified in the configuration file hyperstream_config.json.
Step2:  Selecting a tool
HyperStream comes with a set of predefined tools in hyperstream.tools. These tools can be used to define the nodes of a factor graph that will produce values or compute certain functions given the specified input nodes. For this tutorial, we will focus on the clock tool. This tool produces time ticks given an initial time and a stride in seconds for each following tick. In this case we will generate one tick every hour.
Step3:  Storing the result in a Stream
We need to specify where we want to store the data that will be generated. It is possible to store the Stream in memory or in a MongoDB database. In this tutorial we use the memory channel by creating an instance of memory and then creating the stream in this channel.
Step4:  Definition of a time interval
Now we can specify a time interval in which we want to execute the tool. In this example we will ask for an interval of 10 hours from yesterday.
Step5:  Executing the tool
Now that we defined the tool to use, where we want to store the results and the time interval, it is possible to execute the tool. Then, we will have all the computed results in the Stream specified as a sink.
Step6:  Printing the results
The resulting stream is stored in the ticker. We can get now a list of tuples containing the timestamps and their corresponding clock values.
Step7:  Query the Stream
We can query the stream with any particular time interval. For example, we can ask for the first 5 hours of the precomputed starting time.
Step8:  Executing a new interval
It is possible to execute the tool again specifying a different interval. Because the interval has never been computed in the specified Stream, the tool will compute the new data. If the interval was already computed the tool would not do any computation. As an example, we will execute the tool for the past 5 hours.
Step9:  Now, we can see that we got two different intervals of time in the full Stream
Step10:  Visualization
For this and the following tutorials we will use the JavaScript library highcharts. We have created a Python function called plot_high_chart for one time-serie or line and plot_multiple_stock for multiple time-series.
Then, we can visualize the two intervals where the tool has been executed.
Be aware that there is a missing time interval in the graph and the highchart jumps from one point to the next one ignoring a linear time.
<ASSISTANT_TASK:>
Python Code:
%load_ext watermark
import sys
sys.path.append("../") # Add parent dir in the Path
from hyperstream import HyperStream
from hyperstream import StreamId
from hyperstream import TimeInterval
from pytz import UTC
from datetime import datetime, timedelta
from utils import plot_high_chart
from utils import unix_time_miliseconds
%watermark -v -m -p hyperstream -g
Explanation: <img style="float: right;" src="images/hyperstream.svg">
HyperStream Tutorial 1: Introduction
Requirements
In order to run this and the following tutorials, it is necessary to have access to a MongoDB server running in the localhost port 27017. It is possible to change the host and port of the MongoDB server by modifying the configuration file hyperstream_config.json located in the folder of this notebook.
We also require all the dependencies listed in the HyperStream requirements, the installation instructions can be found in https://github.com/IRC-SPHERE/HyperStream
It is possible to start a MongoDB server, a Jupyter Notebook and a MQTT server by running the script start_tutorial.sh
End of explanation
hs = HyperStream(loglevel=0)
print hs
Explanation: Starting a Hyperstream instance
First of all, we will create a HyperStream instance. This instance will connect to the MongoDB server that is specified in the configuration file hyperstream_config.json.
End of explanation
clock_tool = hs.tools.clock(stride=1.0*60*60)
Explanation: Selecting a tool
HyperStream comes with a set of predefined tools in hyperstream.tools. These tools can be used to define the nodes of a factor graph that will produce values or compute certain functions given the specified input nodes. For this tutorial, we will focus on the clock tool. This tool produces time ticks given an initial time and a stride in seconds for each following tick. In this case we will generate one tick every hour.
End of explanation
M = hs.channel_manager.memory
# M = hs.channel_manager.mongo # To store the results in a MongoDB database
clock_stream = M.get_or_create_stream(stream_id=StreamId(name="clock_stream"))
Explanation: Storing the result in a Stream
We need to specify where we want to store the data that will be generated. It is possible to store the Stream in memory or in a MongoDB database. In this tutorial we use the memory channel by creating an instance of memory and then creating the stream in this channel.
End of explanation
yesterday_start = (datetime.utcnow() - timedelta(days=1)).replace(tzinfo=UTC)
yesterday_end = (yesterday_start + timedelta(hours=10)).replace(tzinfo=UTC)
ti = TimeInterval(yesterday_start, yesterday_end)
Explanation: Definition of a time interval
Now we can specify a time interval in which we want to execute the tool. In this example we will ask for an interval of 10 hours from yesterday.
End of explanation
clock_tool.execute(sources=[], sink=clock_stream, interval=ti, alignment_stream=None)
print clock_stream.calculated_intervals
Explanation: Executing the tool
Now that we defined the tool to use, where we want to store the results and the time interval, it is possible to execute the tool. Then, we will have all the computed results in the Stream specified as a sink.
End of explanation
for timestamp, value in clock_stream.window().items():
    print '[%s]: %s' % (timestamp, value)
Explanation: Printing the results
The resulting stream is stored in the ticker. We can get now a list of tuples containing the timestamps and their corresponding clock values.
End of explanation
ti = TimeInterval(yesterday_start, (yesterday_start + timedelta(hours=5)).replace(tzinfo=UTC))
for timestamp, value in clock_stream.window(ti).items():
    print '[%s]: %s' % (timestamp, value)
Explanation: Query the Stream
We can query the stream with any particular time interval. For example, we can ask for the first 5 hours of the precomputed starting time.
End of explanation
today_end = datetime.utcnow().replace(tzinfo=UTC)
today_start = (today_end - timedelta(hours=5)).replace(tzinfo=UTC)
ti = TimeInterval(today_start, today_end)
clock_tool.execute(sources=[], sink=clock_stream, interval=ti, alignment_stream=None)
print clock_stream.calculated_intervals
Explanation: Executing a new interval
It is possible to execute the tool again specifying a different interval. Because the interval has never been computed in the specified Stream, the tool will compute the new data. If the interval was already computed the tool would not do any computation. As an example, we will execute the tool for the past 5 hours.
End of explanation
ti = TimeInterval(yesterday_start, today_end)
for timestamp, value in clock_stream.window(ti).items():
    print '[%s]: %s' % (timestamp, value)
Explanation: Now, we can see that we got two different intervals of time in the full Stream
End of explanation
my_time, my_data = zip(*[(key.__str__(), unix_time_miliseconds(value)) for key, value in clock_stream.window(ti).items()])
plot_high_chart(my_time, my_data, yax='miliseconds', title='Time from epoch', type='high_stock')
Explanation: Visualization
For this and the following tutorials we will use the JavaScript library highcharts. We have created a Python function called plot_high_chart for one time-serie or line and plot_multiple_stock for multiple time-series.
Then, we can visualize the two intervals where the tool has been executed.
Be aware that there is a missing time interval in the graph and the highchart jumps from one point to the next one ignoring a linear time.
End of explanation
<END_TASK>