<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Exercises
Step1:  Exercise 1
Step2:  b. Cointegration Test II
Determine whether the following two artificial series $C$ and $D$ are cointegrated using the coint() function and a reasonable confidence level.
Step3:  Exercise 2
Step4:  b. Real Cointegration Test II
Determine whether the following two assets FCAU and HMC were cointegrated during 2015 using the coint() function and a reasonable confidence level.
Step5:  Exercise 3
Step6:  Exercise 4
Step7:  b. Testing the Coefficient
Use your coefficient from part a to plot the weighted spread using prices from the first half of 2016, and check whether the result is still stationary.
Step8:  Extra Credit Exercise
<ASSISTANT_TASK:>
Python Code:
# Useful Functions
def find_cointegrated_pairs(data):
    n = data.shape[1]
    score_matrix = np.zeros((n, n))
    pvalue_matrix = np.ones((n, n))
    keys = data.keys()
    pairs = []
    for i in range(n):
        for j in range(i+1, n):
            S1 = data[keys[i]]
            S2 = data[keys[j]]
            result = coint(S1, S2)
            score = result[0]
            pvalue = result[1]
            score_matrix[i, j] = score
            pvalue_matrix[i, j] = pvalue
            if pvalue < 0.05:
                pairs.append((keys[i], keys[j]))
    return score_matrix, pvalue_matrix, pairs
# Useful Libraries
import numpy as np
import pandas as pd
import statsmodels
import statsmodels.api as sm
from statsmodels.tsa.stattools import coint, adfuller
# just set the seed for the random number generator
np.random.seed(107)
import matplotlib.pyplot as plt
Explanation: Exercises: Introduction to Pairs Trading
By Chris Fenaroli, Delaney Mackenzie, and Maxwell Margenot
Lecture Link :
https://www.quantopian.com/lectures/introduction-to-pairs-trading
IMPORTANT NOTE:
These exercises correspond to the Introduction to Pairs Trading lecture, which is part of the Quantopian Lecture Series. It is meant to cement a few concepts presented in the lecture, specifically cointegration and construction of cointegrated pair spreads.
We expect you to rely heavily on the code presented in the corresponding lecture. Please copy and paste regularly from that lecture when starting to work on the problems, as trying to do them from scratch will likely be too difficult. There is an answer key available for this problem set. Please email us at feedback@quantopian.com for results.
Part of the Quantopian Lecture Series:
www.quantopian.com/lectures
github.com/quantopian/research_public
Key Concepts
End of explanation
A_returns = np.random.normal(0, 1, 100)
A = pd.Series(np.cumsum(A_returns), name='X') + 50
some_noise = np.random.exponential(1, 100)
B = A - 7 + some_noise
#Your code goes here
Explanation: Exercise 1: Testing Artificial Examples
We'll use some artificially generated series first as they are much cleaner and easier to work with. In general when learning or developing a new technique, use simulated data to provide a clean environment. Simulated data also allows you to control the level of noise and difficulty level for your model.
a. Cointegration Test I
Determine whether the following two artificial series $A$ and $B$ are cointegrated using the coint() function and a reasonable confidence level.
End of explanation
C_returns = np.random.normal(1, 1, 100) 
C = pd.Series(np.cumsum(C_returns), name='X') + 100
D_returns = np.random.normal(2, 1, 100)
D = pd.Series(np.cumsum(D_returns), name='X') + 100
#Your code goes here
Explanation: b. Cointegration Test II
Determine whether the following two artificial series $C$ and $D$ are cointegrated using the coint() function and a reasonable confidence level.
End of explanation
ual = get_pricing('UAL', fields=['price'], 
                        start_date='2015-01-01', end_date='2016-01-01')['price']
aal = get_pricing('AAL', fields=['price'], 
                        start_date='2015-01-01', end_date='2016-01-01')['price']
#Your code goes here
Explanation: Exercise 2: Testing Real Examples
a. Real Cointegration Test I
Determine whether the following two assets UAL and AAL were cointegrated during 2015 using the coint() function and a reasonable confidence level.
End of explanation
fcau = get_pricing('FCAU', fields=['price'], 
                        start_date='2015-01-01', end_date='2016-01-01')['price']
hmc = get_pricing('HMC', fields=['price'], 
                        start_date='2015-01-01', end_date='2016-01-01')['price']
#Your code goes here
Explanation: b. Real Cointegration Test II
Determine whether the following two assets FCAU and HMC were cointegrated during 2015 using the coint() function and a reasonable confidence level.
End of explanation
symbol_list = ['MTRN', 'CMP', 'TRQ', 'SCCO', 'HCLP','SPY']
prices_df = get_pricing(symbol_list, fields=['price']
                               , start_date='2015-01-01', end_date='2016-01-01')['price']
prices_df.columns = map(lambda x: x.symbol, prices_df.columns)
#Your code goes here
Explanation: Exercise 3: Searching for Cointegrated Pairs
Use the find_cointegrated_pairs function, defined in the "Helper Functions" section above, to find any cointegrated pairs among a set of metal and mining securities.
Note that not all of these securities in this exercise are within the QTradableStocksUS. As you continue your development, focus on securities within the QTU to be eligible for an allocation.
End of explanation
S1 = prices_df['MTRN']
S2 = prices_df['SCCO']
#Your code goes here
Explanation: Exercise 4: Out of Sample Validation
a. Calculating the Spread
Using pricing data from 2015, construct a linear regression to find a coefficient for the linear combination of MTRN and SCCO that makes their spread stationary.
End of explanation
S1_out = get_pricing('MTRN', fields=['price'], 
                        start_date='2016-01-01', end_date='2016-07-01')['price']
S2_out = get_pricing('SCCO', fields=['price'], 
                        start_date='2016-01-01', end_date='2016-07-01')['price']
#Your code goes here
Explanation: b. Testing the Coefficient
Use your coefficient from part a to plot the weighted spread using prices from the first half of 2016, and check whether the result is still stationary.
End of explanation
# Your code goes here
Explanation: Extra Credit Exercise: Hurst Exponent
This exercise is more difficult and we will not provide initial structure.
The Hurst exponent is a statistic between 0 and 1 that provides information about how much a time series is trending or mean reverting. We want our spread time series to be mean reverting, so we can use the Hurst exponent to monitor whether our pair is going out of cointegration. Effectively as a means of process control to know when our pair is no longer good to trade.
Please find either an existing Python library that computes, or compute yourself, the Hurst exponent. Then plot it over time for the spread on the above pair of stocks.
These links may be helpful:
https://en.wikipedia.org/wiki/Hurst_exponent
https://www.quantopian.com/posts/pair-trade-with-cointegration-and-mean-reversion-tests
End of explanation
<END_TASK>