<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <img src="CA_logo.png" alt="Continuum Analytics" width="20%" align="right" border="4"><br><br><br><br>
Interactive Financial Analytics  with Python & IPython
Tutorial with Examples based on the VSTOXX Volatility Index
Dr. Yves J. Hilpisch
Continuum Analytics Europe GmbH
<a href="http
Step1:  The convenience function DataReader makes it easy to read historical stock price data from Yahoo! Finance (http
Step2:  pandas strength is the handling of indexed/labeled/structured data, like times series data.
Step3:  pandas makes it easy to implement vectorized operations, like calculating log-returns over whole time series.
Step4:  In addition, pandas makes plotting quite simple and compact.
Step5:  We now want to check how annual volatility changes over time.
Step6:  Obviously, the annual volatility changes significantly over time.
Step7:  Exercise
Trend-based investment strategy with the EURO STOXX 50 index
Step8:  The EURO STOXX 50 data is not yet in the right format. Some house cleaning is necessary (I).
Step9:  The EURO STOXX 50 data is not yet in the right format. Some house cleaning is necessary (II).
Step10:  The EURO STOXX 50 data is not yet in the right format. Some house cleaning is necessary (III).
Step11:  The EURO STOXX 50 data is not yet in the right format. Some house cleaning is necessary (IV).
Step12:  Now, the data can be safely read into a DataFrame object.
Step13:  The VSTOXX data can be read without touching the raw data.
Step14:  We now merge the data for further analysis.
Step15:  Let's inspect the two time series.
Step16:  A picture can tell almost the complete story.
Step17:  We now generate log returns for both time series.
Step18:  To this new data set, also stored in a DataFrame object, we apply OLS.
Step19:  Again, we want to see how our results look graphically.
Step20:  Let us see if we can identify systematics over time. And indeed, during the crisis 2007/2008 (yellow dots) volatility has been more pronounced than more recently (red dots).
Step21:  Exercise
We want to test whether the EURO STOXX 50 and/or the VSTOXX returns are normally distributed or not (e.g. if they might have fat tails). We want to do a
graphical illustration (using qqplot of statsmodels.api) and a
statistical test (using normaltest of scipy.stats)
Add on
Step22:  First, the initial invest.
Step23:  This can already be considered an static investment strategy.
Step24:  Second, the dynamic strategy with daily adjustments to keep the value ratio constant.
Step25:  Third, the total wealth position.
Step26:  A brief check if the ratios are indeed constant.
Step27:  Let us inspect the performance of the strategy.
Step28:  Exercise
Write a Python function which allows for an arbitrary but constant ratio to be invested in the VSTOXX index and which returns net performance values (in percent) for the constant proportion VSTOXX strategy.
Add on
Step29:  The intraday evolution of the Apple stock price.
Step30:  A resampling of the data is easily accomplished with pandas.
Step31:  Let's have a graphical look at the new data set.
Step32:  With pandas you can easily apply custom functions to time series data.
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import pandas as pd
import pandas.io.data as pdd
from urllib import urlretrieve
%matplotlib inline
Explanation: <img src="CA_logo.png" alt="Continuum Analytics" width="20%" align="right" border="4"><br><br><br><br>
Interactive Financial Analytics  with Python & IPython
Tutorial with Examples based on the VSTOXX Volatility Index
Dr. Yves J. Hilpisch
Continuum Analytics Europe GmbH
<a href="http://www.continuum.io" target="_blank">www.continuum.io</a>
<a href="mailto:yves@continuum.io">yves@continuum.io</a>
<a href="http://twitter.com/dyjh" target="_blank">@dyjh</a>
For Python Quants &ndash; 14. March 2014
You find the presentation and the IPython Notebook here:
<a href="http://www.hilpisch.com/YH_FPQ_Volatility_Tutorial.html" target="_blank">http://www.hilpisch.com/YH_FPQ_Volatility_Tutorial.html</a>
<a href="http://www.hilpisch.com/YH_FPQ_Volatility_Tutorial.ipynb" target="_blank">http://www.hilpisch.com/YH_FPQ_Volatility_Tutorial.ipynb</a>
About Me
A brief bio:
Managing Director Europe of Continuum Analytics Inc.
Founder of Visixion GmbH – The Python Quants
Lecturer Mathematical Finance at Saarland University
Focus on Financial Industry and Financial Analytics
Book "Derivatives Analytics with Python" (2013)
Book "Python for Finance" O'Reilly (2014)
Dr.rer.pol in Mathematical Finance
Graduate in Business Administration
Martial Arts Practitioner and Fan
See <a href="http://www.hilpisch.com" target="_blank">www.hilpisch.com</a>.
<img src="python_for_finance.png" alt="Python for Finance" style="width: 45%; border: 2px solid black;">
<a href="http://shop.oreilly.com/product/0636920032441.do" target="_blank">Python for Finance (O'Reilly Shop)</a>
Python for Analytics
This tutorial focuses on
Python as a general purpose financial analytics environment
interactive analytics examples
prototyping-like Python usage
It does not address such important issues like
architectural issues regarding hardware and software
development processes, testing, documentation and production
real world problem modeling
A fundamental Python stack for interactive data analytics and visualization should at least contain the following libraries tools:
Python &ndash; the Python interpreter itself
NumPy &ndash; high performance, flexible array structures and operations
SciPy &ndash; collection of scientific modules and functions (e.g. for regression, optimization, integration)
pandas &ndash; time series and panel data analysis and I/O 
PyTables &ndash; hierarchical, high performance database (e.g. for out-of-memory analytics)
matplotlib &ndash; 2d and 3d visualization
IPython &ndash; interactive data analytics, visualization, publishing
It is best to use e.g. a Python distribution like Anaconda to ensure consistency of libraries.
First Financial Analytics Example
We need to make a couple of imports for what is to come.
End of explanation
try:    
    index = pdd.DataReader('^GDAXI', data_source='yahoo', start='2007/3/30')
      # e.g. the EURO STOXX 50 ticker symbol -- ^SX5E
except:
    index = pd.read_csv('dax.txt', index_col=0, parse_dates=True)
index.info()
Explanation: The convenience function DataReader makes it easy to read historical stock price data from Yahoo! Finance (http://finance.yahoo.com).
End of explanation
index.tail()
Explanation: pandas strength is the handling of indexed/labeled/structured data, like times series data.
End of explanation
index['Returns'] = np.log(index['Close'] / index['Close'].shift(1))
Explanation: pandas makes it easy to implement vectorized operations, like calculating log-returns over whole time series.
End of explanation
index[['Close', 'Returns']].plot(subplots=True, style='b', figsize=(8, 5))
Explanation: In addition, pandas makes plotting quite simple and compact.
End of explanation
index['Mov_Vol'] = pd.rolling_std(index['Returns'], window=252) * np.sqrt(252)
Explanation: We now want to check how annual volatility changes over time.
End of explanation
index[['Close', 'Returns', 'Mov_Vol']].plot(subplots=True, style='b', figsize=(8, 5))
Explanation: Obviously, the annual volatility changes significantly over time.
End of explanation
import pandas as pd
import datetime as dt
from urllib import urlretrieve
try:
    es_url = 'http://www.stoxx.com/download/historical_values/hbrbcpe.txt'
    vs_url = 'http://www.stoxx.com/download/historical_values/h_vstoxx.txt'
    urlretrieve(es_url, 'es.txt')
    urlretrieve(vs_url, 'vs.txt')
except:
    pass
Explanation: Exercise
Trend-based investment strategy with the EURO STOXX 50 index:
2 trends 42d & 252d 
long, short, cash positions
no transaction costs
Signal generation:
invest (go long) when the 42d trend is more than 100 points above the 252d trend
sell (go short) when the 42d trend is more than 20 points below the 252d trend
invest in cash (no interest) when neither of both is true
Historical Correlation between EURO STOXX 50 and VSTOXX
It is a stylized fact that stock indexes and related volatility indexes are highly negatively correlated. The following example analyzes this stylized fact based on the EURO STOXX 50 stock index and the VSTOXX volatility index using Ordinary Least-Squares regession (OLS).
First, we collect historical data for both the EURO STOXX 50 stock and the VSTOXX volatility index.
End of explanation
lines = open('es.txt').readlines()  # reads the whole file line-by-line
lines[:5]  # header not well formatted
Explanation: The EURO STOXX 50 data is not yet in the right format. Some house cleaning is necessary (I).
End of explanation
lines[3883:3890]  # from 27.12.2001 additional semi-colon
Explanation: The EURO STOXX 50 data is not yet in the right format. Some house cleaning is necessary (II).
End of explanation
lines = open('es.txt').readlines()  # reads the whole file line-by-line
new_file = open('es50.txt', 'w')  # opens a new file
new_file.writelines('date' + lines[3][:-1].replace(' ', '') + ';DEL' + lines[3][-1])
    # writes the corrected third line (additional column name)
    # of the orginal file as first line of new file
new_file.writelines(lines[4:])  # writes the remaining lines of the orginal file
Explanation: The EURO STOXX 50 data is not yet in the right format. Some house cleaning is necessary (III).
End of explanation
list(open('es50.txt'))[:5]  # opens the new file for inspection
Explanation: The EURO STOXX 50 data is not yet in the right format. Some house cleaning is necessary (IV).
End of explanation
es = pd.read_csv('es50.txt', index_col=0, parse_dates=True, sep=';', dayfirst=True)
del es['DEL']  # delete the helper column
es.info()
Explanation: Now, the data can be safely read into a DataFrame object.
End of explanation
vs = pd.read_csv('vs.txt', index_col=0, header=2, parse_dates=True, sep=',', dayfirst=True)
# you can alternatively read from the Web source directly
# without saving the csv file to disk:
# vs = pd.read_csv(vs_url, index_col=0, header=2,
#                  parse_dates=True, sep=',', dayfirst=True)
Explanation: The VSTOXX data can be read without touching the raw data.
End of explanation
import datetime as dt
data = pd.DataFrame({'EUROSTOXX' :
            es['SX5E'][es.index > dt.datetime(1999, 12, 31)]})
data = data.join(pd.DataFrame({'VSTOXX' :
            vs['V2TX'][vs.index > dt.datetime(1999, 12, 31)]}))
data.info()
Explanation: We now merge the data for further analysis.
End of explanation
data.head()
Explanation: Let's inspect the two time series.
End of explanation
data.plot(subplots=True, grid=True, style='b', figsize=(10, 5))
Explanation: A picture can tell almost the complete story.
End of explanation
rets = np.log(data / data.shift(1)) 
rets.head()
Explanation: We now generate log returns for both time series.
End of explanation
xdat = rets['EUROSTOXX']
ydat = rets['VSTOXX']
model = pd.ols(y=ydat, x=xdat)
model
Explanation: To this new data set, also stored in a DataFrame object, we apply OLS.
End of explanation
import matplotlib.pyplot as plt
plt.plot(xdat, ydat, 'r.')
ax = plt.axis()  # grab axis values
x = np.linspace(ax[0], ax[1] + 0.01)
plt.plot(x, model.beta[1] + model.beta[0] * x, 'b', lw=2)
plt.grid(True)
plt.axis('tight')
Explanation: Again, we want to see how our results look graphically.
End of explanation
import matplotlib as mpl
mpl_dates = mpl.dates.date2num(rets.index)
plt.figure(figsize=(8, 4))
plt.scatter(rets['EUROSTOXX'], rets['VSTOXX'], c=mpl_dates, marker='o')
plt.grid(True)
plt.xlabel('EUROSTOXX')
plt.ylabel('VSTOXX')
plt.colorbar(ticks=mpl.dates.DayLocator(interval=250),
          format=mpl.dates.DateFormatter('%d %b %y'))
Explanation: Let us see if we can identify systematics over time. And indeed, during the crisis 2007/2008 (yellow dots) volatility has been more pronounced than more recently (red dots).
End of explanation
data = data.dropna()
data = data / data.ix[0] * 100
data.head()
Explanation: Exercise
We want to test whether the EURO STOXX 50 and/or the VSTOXX returns are normally distributed or not (e.g. if they might have fat tails). We want to do a
graphical illustration (using qqplot of statsmodels.api) and a
statistical test (using normaltest of scipy.stats)
Add on: plot a histogram of the log return frequencies and compare that to a normal distribution with same mean and variance (using e.g. norm.pdf from scipy.stats)
Constant Proportion VSTOXX Investment
There has been a number of studies which have illustrated that constant proportion investments in volatility derivatives &ndash; given a diversified equity portfolio &ndash; might improve investment performance considerably. See, for instance, the study
<a href="http://www.eurexgroup.com/group-en/newsroom/60036/" target="_blank">The Benefits of Volatility Derivatives in Equity Portfolio Management</a>
We now want to replicate (in a simplified fashion) what you can flexibly test here on the basis of two backtesting applications for VSTOXX-based investment strategies:
<a href="http://www.eurexchange.com/vstoxx/app1/" target="_blank">Two Assets Backtesting</a>
<a href="http://www.eurexchange.com/vstoxx/app2/" target="_blank">Four Assets Backtesting</a>
The strategy we are going to implement and test is characterized as follows:
An investor has total wealth of say 100,000 EUR
He invests, say, 70% of that into a diversified equity portfolio
The remainder, i.e. 30%, is invested in the VSTOXX index directly
Through (daily) trading the investor keeps the proportions constant
No transaction costs apply, all assets are infinitely divisible
We already have the necessary data available. However, we want to drop 'NaN' values and want to normalize the index values.
End of explanation
invest = 100
cratio = 0.3
data['Equity'] = (1 - cratio) * invest / data['EUROSTOXX'][0]
data['Volatility'] = cratio * invest / data['VSTOXX'][0]
Explanation: First, the initial invest.
End of explanation
data['Static'] = (data['Equity'] * data['EUROSTOXX']
                + data['Volatility'] * data['VSTOXX'])
data[['EUROSTOXX', 'Static']].plot(figsize=(10, 5))
Explanation: This can already be considered an static investment strategy.
End of explanation
for i in range(1, len(data)):
    evalue = data['Equity'][i - 1] * data['EUROSTOXX'][i]
      # value of equity position
    vvalue = data['Volatility'][i - 1] * data['VSTOXX'][i]
      # value of volatility position
    tvalue = evalue + vvalue
      # total wealth 
    data['Equity'][i] = (1 - cratio) * tvalue / data['EUROSTOXX'][i]
      # re-allocation of total wealth to equity ...
    data['Volatility'][i] = cratio * tvalue / data['VSTOXX'][i]
      # ... and volatility position
Explanation: Second, the dynamic strategy with daily adjustments to keep the value ratio constant.
End of explanation
data['Dynamic'] = (data['Equity'] * data['EUROSTOXX']
                + data['Volatility'] * data['VSTOXX'])
data.head()
Explanation: Third, the total wealth position.
End of explanation
(data['Volatility'] * data['VSTOXX'] / data['Dynamic'])[:5]
(data['Equity'] * data['EUROSTOXX'] / data['Dynamic'])[:5]
Explanation: A brief check if the ratios are indeed constant.
End of explanation
data[['EUROSTOXX', 'Dynamic']].plot(figsize=(10, 5))
Explanation: Let us inspect the performance of the strategy.
End of explanation
try:
    url = 'http://hopey.netfonds.no/posdump.php?'
    url += 'date=%s%s%s&paper=AAPL.O&csv_format=csv' % ('2014', '03', '12')
    # you may have to adjust the date since only recent dates are available
    urlretrieve(url, 'aapl.csv')
except:
    pass
AAPL = pd.read_csv('aapl.csv', index_col=0, header=0, parse_dates=True)
AAPL.info()
Explanation: Exercise
Write a Python function which allows for an arbitrary but constant ratio to be invested in the VSTOXX index and which returns net performance values (in percent) for the constant proportion VSTOXX strategy.
Add on: find the ratio to be invested in the VSTOXX that gives the maximum performance.
Analyzing High Frequency Data
Using standard Python functionality and pandas, the code that follows reads intraday, high-frequency data from a Web source, plots it and resamples it.
End of explanation
AAPL['bid'].plot()
AAPL = AAPL[AAPL.index > dt.datetime(2014, 3, 12, 10, 0, 0)]
  # only data later than 10am at that day
Explanation: The intraday evolution of the Apple stock price.
End of explanation
# this resamples the record frequency to 5 minutes, using mean as aggregation rule
AAPL_5min = AAPL.resample(rule='5min', how='mean').fillna(method='ffill')
AAPL_5min.head()
Explanation: A resampling of the data is easily accomplished with pandas.
End of explanation
AAPL_5min['bid'].plot()
Explanation: Let's have a graphical look at the new data set.
End of explanation
AAPL_5min['bid'].apply(lambda x: 2 * 530 - x).plot()
  # this mirrors the stock price development at 
Explanation: With pandas you can easily apply custom functions to time series data.
End of explanation
<END_TASK>