<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Shaolin GraphCalculator tutorial
<a id='index'></a>
Index
1 Introduction
2 Loading the sample data
2.1 Calculating the graph matrices
2.2 Calculating some metrics for the nodes
3 GraphCalculator componenets
3.1 Graph creation parameters
3.2 Matrix transformations
3.3 Graph layout
4 GraphCalculator data
4.1 Node graph metrics
4.2 Edge metrics
4.3 External node metrics
4.4 Networkx Graph
<a id='introduction'></a>
1 Introduction
In this tutorial we will analyse the structure of the graph resulting from the correlation matrix of the prices of different currency exchanges. We just need time series data and a few lines of code to map matrices into graphs and calculate its metrics.
<a id='sample_data'></a>
2 Loading the sample data
Back to top
We are using a week of hourly data data from different to conduct our analysis. Each time series has also its normalized price, its returns and its logarithmic returns so you don't have to calculate it if you want to use it. In this example we will start just with the normalized price values.
Step1:  <a id='sample_matrices'></a>
2.1 Calculating the graph matrices
Back to top
In this case we will start calculating the correlation and covariance matrices of the series.
Step3:  <a id='sample_node_metrics'></a>
2.2 Calculating some metrics for the nodes
Back to top
This data will be used to add additional information to every node of the graph. This information will be included in the GraphCalculator.
Step4:  <a id='components'></a>
3 GraphCalculator components
Back to top
The GraphCalculator lets us have complete control over the process of turning a matrix and arbitrary data into a graph. Although every component can be used as a standalone widget it is recommended to use the widget attribute for controlling the GraphCalculator.
Every component can be hidden with the toggle buttons situated at the bottom, and every change you make in the widget will take effect after you click the Calculate button.
Step5:  <a id='gc_parameters'></a>
3.1 Graph creation parameters
Back to top
This widget is used to select which kind of graph will be calculated from the supplied matrix.
<img src='gcalculator_data/creation.png' style="height
Step6:  <a id='edge_metrics'></a>
4.2 Edge metrics
Back to top
The edge attribute is a pandas Panel with all the information regarding the edges of the graph. This means that it will contain two kinds of metrics
Step7:  <a id='external_node'></a>
4.3 External node metrics
Back to top
The node metrics that were passed as a parameter are stored in the node_metrics attribute. They are not currently used by the GraphCalculator. This information is currently used only when displaying the node tooltips on a plot.
Step8:  <a id='networkx_graph'></a>
4.4 Networkx Graph
Back to top
All the data is also stored in a networkx graph containing all the DataFrames of the GraphCalculator as node and edge attributes. It is possible to acces the Graph using the G attribute.
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import pandas as pd
from scipy import stats
from IPython.display import Image #this is for displaying the widgets in the github repo
from shaolin.dashboards.graph import GraphCalculator
forex_data = pd.read_hdf('gcalculator_data/forex_sample.h5')
forex_data.items,forex_data.minor_axis,forex_data.major_axis
fund = forex_data['fund']
fund.head()
Explanation: Shaolin GraphCalculator tutorial
<a id='index'></a>
Index
1 Introduction
2 Loading the sample data
2.1 Calculating the graph matrices
2.2 Calculating some metrics for the nodes
3 GraphCalculator componenets
3.1 Graph creation parameters
3.2 Matrix transformations
3.3 Graph layout
4 GraphCalculator data
4.1 Node graph metrics
4.2 Edge metrics
4.3 External node metrics
4.4 Networkx Graph
<a id='introduction'></a>
1 Introduction
In this tutorial we will analyse the structure of the graph resulting from the correlation matrix of the prices of different currency exchanges. We just need time series data and a few lines of code to map matrices into graphs and calculate its metrics.
<a id='sample_data'></a>
2 Loading the sample data
Back to top
We are using a week of hourly data data from different to conduct our analysis. Each time series has also its normalized price, its returns and its logarithmic returns so you don't have to calculate it if you want to use it. In this example we will start just with the normalized price values.
End of explanation
matrices = {}
matrices['corr'] = fund.corr()
matrices['cov'] = fund.cov()
matrices['exchange'] = fund.corr()#alpha version hack, needs to be a matrix called exchange. you dont
                                  #need to use it
matrix_panel = pd.Panel(matrices)
matrix_panel
Explanation: <a id='sample_matrices'></a>
2.1 Calculating the graph matrices
Back to top
In this case we will start calculating the correlation and covariance matrices of the series.
End of explanation
def calculate_pdf(x,n=1000):
        Fits a Gaussian Kernel, resamples n values and returns (X,p(x=X))
        
        try:
            kernel = stats.gaussian_kde(x,bw_method='scott')
            X= kernel.resample(n)
            p = kernel(X)
        except:
            X= x
            p = np.ones(len(x))*1.0/len(x)
        return X.flatten(),p.flatten()
def shannon_entropy(x,n=1000):
    X,p = calculate_pdf(x,n)
    ent = p*np.log2(p)
    return -ent.sum()/(10*n)# this way we dont get values too big. 
                            #After all, entropy is a purely arbitrary measure ;)
def mean_rets(x):
     return x.pct_change().mean()*10e4
def std_rets(x):
     return x.pct_change().mean()*10e4
    
def total_rets(x):
    return x.pct_change().sum()
node_metrics = pd.DataFrame(columns=fund.columns)
funcs = [total_rets, np.mean, np.std, shannon_entropy,mean_rets,std_rets]
for fun in funcs:
    node_metrics.ix[fun.__name__] = fund.apply(fun)
node_metrics
Explanation: <a id='sample_node_metrics'></a>
2.2 Calculating some metrics for the nodes
Back to top
This data will be used to add additional information to every node of the graph. This information will be included in the GraphCalculator.
End of explanation
gc = GraphCalculator(node_metrics=node_metrics,matrix_panel=matrix_panel)
gc.widget
Image(filename='gcalculator_data/img_1.png') 
Explanation: <a id='components'></a>
3 GraphCalculator components
Back to top
The GraphCalculator lets us have complete control over the process of turning a matrix and arbitrary data into a graph. Although every component can be used as a standalone widget it is recommended to use the widget attribute for controlling the GraphCalculator.
Every component can be hidden with the toggle buttons situated at the bottom, and every change you make in the widget will take effect after you click the Calculate button.
End of explanation
gc.node = gc.node.dropna(axis=1)
gc.node.describe()
gc.node.head()
Explanation: <a id='gc_parameters'></a>
3.1 Graph creation parameters
Back to top
This widget is used to select which kind of graph will be calculated from the supplied matrix.
<img src='gcalculator_data/creation.png' style="height: 450px;"></img>
Graph type
This is how the matrix will be converted into a graph. There are currently three possible options to build a non directed graph:
Full matrix: The matrix will be interpreted as the adjacency matrix of a full connected graph.
MST: The target graph will be a minimum spanning tree constructed and the data matrix.
PMFG: Maximum planary filtered graph, calculated from the data matrix.
Absolute value threshold
Minimum absolute value forn an element of the matrix needed to be taken into account as a valid edge.
Invert distance
Use 1/value instead of the original value of the data matrix.
Metrics
Node metrics that will be calculated when the graph is constructed
Target attribute
This widget allows us to select which matrix of our matrix panel will be used as an adjacency matrix for the graph. In other words, this will select which data matrix we will convert into a graph.
<a id='matrix_transforms'></a>
3.2 Matrix transformations
Back to top
This widget handles the transformations we will apply to the target data matrix before mapping it into a graph. One, or a succession of the following transformations has to be applied to the matrix.
Raw: Do not apply any transformation.<br>
Scale: Rescale the values of the matrix to match the interval [Rescale min, Rescale max].<br>
Clip: All the values lower than Clip min will be changed for Clip min and all the values greater than Clip          max will be changed for Clip max.<br>
Zscore: The matrix will be transformed so its elements have mean 0 and a standard deviation of 1.
Distance: The matrix will be transformed according to the following distance function: 
<img src='gcalculator_data/matrix.png' style="height: 450px;"></img>
<a id='graph_layout'></a>
3.3 Graph Layout
Back to top
<img src='gcalculator_data/layout.png' style="height: 300px;"></img>
The Layout manager is the widget in charge of managing the layout that we will use to draw our graph. It can calculate the layouts of the selected type. Any layout available in the networkx package is included in the LayoutCalculator and any of its parametrs can be tweaked via a widget.
Alpha version disclaimer: 
- Can only calculate both the 3D and the 2D and not only one of them separatedly.
<a id='data'></a>
4 GraphCalculator Data
Back to top
The results of running the GraphCalculator will be sored internally in different object attributes as explained below.
<a id='node_metrics'></a>
4.1 Node graph metrics
Back to top
The metrics that are graph related are sored as a dataframe in the node attribute as a DataFrame.
End of explanation
gc.edge
gc.edge.items
gc.edge['cov']
gc.edge['edge_betweenness']
Explanation: <a id='edge_metrics'></a>
4.2 Edge metrics
Back to top
The edge attribute is a pandas Panel with all the information regarding the edges of the graph. This means that it will contain two kinds of metrics:
Edge: All the metrics calculated on the edges of the graph in the form of an adjacency matrix.
Matrix: All the matrices that were not used to build the graph are stored in the form of an adjacency matrix.
End of explanation
gc.node_metrics
Explanation: <a id='external_node'></a>
4.3 External node metrics
Back to top
The node metrics that were passed as a parameter are stored in the node_metrics attribute. They are not currently used by the GraphCalculator. This information is currently used only when displaying the node tooltips on a plot.
End of explanation
gc.G.node
gc.G.edge
Explanation: <a id='networkx_graph'></a>
4.4 Networkx Graph
Back to top
All the data is also stored in a networkx graph containing all the DataFrames of the GraphCalculator as node and edge attributes. It is possible to acces the Graph using the G attribute.
End of explanation
<END_TASK>