<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Pipeline
This notebook is from the official Quantopian Guide on Pipelines. Make sure to visit their documentation for many more great resources!
Many trading algorithms have the following structure
Step1:  Data
Step2:  Factors
Remember, Factors take in an asset and a timestamp and return some numerical value.
Step3:  Combining Factors
Step4:  Filters and Screens
Filters take in an asset and a timestamp and return a boolean
Step5:  Screens
Step6:  Reverse a screen
Step7:  Combine Filters
Step8:  Masking
Sometimes we want to ignore certain assets when computing pipeline expresssions. There are two common cases where ignoring assets is useful
Step9:  Classifiers
A classifier is a function from an asset and a moment in time to a categorical output such as a string or integer label.
Step10:  Classifier Methods
eq (equals)
isnull
startswith
Step11:  Pipelines in Quantopian IDE
<ASSISTANT_TASK:>
Python Code:
from quantopian.pipeline import Pipeline
def make_pipeline():
    return Pipeline()
pipe = make_pipeline()
from quantopian.research import run_pipeline
result = run_pipeline(pipe, '2017-01-01', '2017-01-01')
result.head(10)
result.info()
Explanation: Pipeline
This notebook is from the official Quantopian Guide on Pipelines. Make sure to visit their documentation for many more great resources!
Many trading algorithms have the following structure:
For each asset in a known (large) set, compute N scalar values for the asset based on a trailing window of data.
Select a smaller tradeable set of assets based on the values computed in (1).
Calculate desired portfolio weights on the set of assets selected in (2).
Place orders to move the algorithm’s current portfolio allocations to the desired weights computed in (3).
There are several technical challenges with doing this robustly. These include:
efficiently querying large sets of assets
performing computations on large sets of assets
handling adjustments (splits and dividends)
asset delistings
Pipeline exists to solve these challenges by providing a uniform API for expressing computations on a diverse collection of datasets.
Factors
A factor is a function from an asset and a moment in time to a numerical value.
A simple example of a factor is the most recent price of a security. Given a security and a specific point in time, the most recent price is a number. Another example is the 10-day average trading volume of a security. Factors are most commonly used to assign values to securities which can then be used in a number of ways. A factor can be used in each of the following procedures:
* computing target weights
* generating alpha signal
* constructing other, more complex factors
* constructing filters
Filters
A filter is a function from an asset and a moment in time to a boolean.
An example of a filter is a function indicating whether a security's price is below $10. Given a security and a point in time, this evaluates to either True or False. Filters are most commonly used for describing sets of assets to include or exclude for some particular purpose.
Classifiers
A classifier is a function from an asset and a moment in time to a categorical output.
More specifically, a classifier produces a string or an int that doesn't represent a numerical value (e.g. an integer label such as a sector code). Classifiers are most commonly used for grouping assets for complex transformations on Factor outputs. An example of a classifier is the exchange on which an asset is currently being traded.
End of explanation
from quantopian.pipeline.data.builtin import USEquityPricing
Explanation: Data
End of explanation
from quantopian.pipeline.factors import BollingerBands,SimpleMovingAverage,EWMA
SimpleMovingAverage(inputs = [USEquityPricing.close],
                    window_length = 30)
def make_pipeline():
    
    mean_close_30 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 30)
    
    return Pipeline(columns = {
        '30 Day Mean Close':mean_close_30
    })
results = run_pipeline(make_pipeline(),
                       '2017-01-01',
                       '2017-01-01')
results.head(20)
def make_pipeline():
    
    mean_close_30 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 30)
    latest_close = USEquityPricing.close.latest
    
    return Pipeline(columns = {
        '30 Day Mean Close':mean_close_30,
        'Latest Close':latest_close
    })
results = run_pipeline(make_pipeline(),
                       '2017-01-01',
                       '2017-01-01')
results.head(10)
Explanation: Factors
Remember, Factors take in an asset and a timestamp and return some numerical value.
End of explanation
def make_pipeline():
    
    mean_close_10 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 10)
    mean_close_30 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 30)
    latest_close = USEquityPricing.close.latest
    
    percent_difference = (mean_close_10-mean_close_30) / mean_close_30
    
    return Pipeline(columns = {
        'Percent Difference':percent_difference,
        '30 Day Mean Close':mean_close_30,
        'Latest Close':latest_close
    })
results = run_pipeline(make_pipeline(),
                       '2017-01-01',
                       '2017-01-01')
results.head()
Explanation: Combining Factors
End of explanation
last_close_price = USEquityPricing.close.latest
close_price_filter = last_close_price > 20
close_price_filter
def make_pipeline():
    
    mean_close_10 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 10)
    mean_close_30 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 30)
    latest_close = USEquityPricing.close.latest
    
    percent_difference = (mean_close_10-mean_close_30) / mean_close_30
    
    perc_diff_check = percent_difference > 0 
    
    return Pipeline(columns = {
        'Percent Difference':percent_difference,
        '30 Day Mean Close':mean_close_30,
        'Latest Close':latest_close,
        'Positive Percent Diff': perc_diff_check
    })
results = run_pipeline(make_pipeline(),
                       '2017-01-01',
                       '2017-01-01')
results.head()
Explanation: Filters and Screens
Filters take in an asset and a timestamp and return a boolean
End of explanation
def make_pipeline():
    
    mean_close_10 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 10)
    mean_close_30 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 30)
    latest_close = USEquityPricing.close.latest
    
    percent_difference = (mean_close_10-mean_close_30) / mean_close_30
    
    perc_diff_check = percent_difference > 0 
    
    return Pipeline(columns = {
                            'Percent Difference':percent_difference,
                            '30 Day Mean Close':mean_close_30,
                            'Latest Close':latest_close,
                            'Positive Percent Diff': perc_diff_check},
                    screen=perc_diff_check)
results = run_pipeline(make_pipeline(),
                       '2017-01-01',
                       '2017-01-01')
results.head()
Explanation: Screens
End of explanation
def make_pipeline():
    
    mean_close_10 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 10)
    mean_close_30 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 30)
    latest_close = USEquityPricing.close.latest
    
    percent_difference = (mean_close_10-mean_close_30) / mean_close_30
    
    perc_diff_check = percent_difference > 0 
    
    return Pipeline(columns = {
                            'Percent Difference':percent_difference,
                            '30 Day Mean Close':mean_close_30,
                            'Latest Close':latest_close,
                            'Positive Percent Diff': perc_diff_check},
                    screen = ~perc_diff_check)
results = run_pipeline(make_pipeline(),
                       '2017-01-01',
                       '2017-01-01')
results.head()
Explanation: Reverse a screen
End of explanation
def make_pipeline():
    
    mean_close_10 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 10)
    mean_close_30 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 30)
    latest_close = USEquityPricing.close.latest
    
    percent_difference = (mean_close_10-mean_close_30) / mean_close_30
    
    perc_diff_check = percent_difference > 0 
    small_price = latest_close < 5
    
    final_filter = perc_diff_check & small_price
    
    return Pipeline(columns = {
                            'Percent Difference':percent_difference,
                            '30 Day Mean Close':mean_close_30,
                            'Latest Close':latest_close,
                            'Positive Percent Diff': perc_diff_check},
                    screen = final_filter)
results = run_pipeline(make_pipeline(),
                       '2017-01-01',
                       '2017-01-01')
results.head()
Explanation: Combine Filters
End of explanation
def make_pipeline():
    
    # Create Filters for Masks First
    latest_close = USEquityPricing.close.latest
    small_price = latest_close < 5
    
    # Pass in the mask
    mean_close_10 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 10,
                                        mask = small_price)
    mean_close_30 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 30,
                                        mask = small_price)
    
    
    percent_difference = (mean_close_10-mean_close_30) / mean_close_30
    
    perc_diff_check = percent_difference > 0 
    
    
    final_filter = perc_diff_check
    
    return Pipeline(columns = {
                            'Percent Difference':percent_difference,
                            '30 Day Mean Close':mean_close_30,
                            'Latest Close':latest_close,
                            'Positive Percent Diff': perc_diff_check},
                    screen = final_filter)
results = run_pipeline(make_pipeline(),
                       '2017-01-01',
                       '2017-01-01')
results.head()
len(results)
Explanation: Masking
Sometimes we want to ignore certain assets when computing pipeline expresssions. There are two common cases where ignoring assets is useful:
* We want to compute an expression that's computationally expensive, and we know we only care about results for certain assets.
* We want to compute an expression that performs comparisons between assets, but we only want those comparisons to be performed against a subset of all assets.
End of explanation
from quantopian.pipeline.data import morningstar
from quantopian.pipeline.classifiers.morningstar import Sector
morningstar_sector = Sector()
exchange = morningstar.share_class_reference.exchange_id.latest
exchange
Explanation: Classifiers
A classifier is a function from an asset and a moment in time to a categorical output such as a string or integer label.
End of explanation
nyse_filter = exchange.eq('NYS')
def make_pipeline():
    
    # Create Filters for Masks First
    latest_close = USEquityPricing.close.latest
    small_price = latest_close < 5
    
    # Classifier
    nyse_filter = exchange.eq('NYS')
    
    # Pass in the mask
    mean_close_10 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 10,
                                        mask = small_price)
    mean_close_30 = SimpleMovingAverage(inputs = [USEquityPricing.close],
                                        window_length = 30,
                                        mask = small_price)
    
    
    percent_difference = (mean_close_10-mean_close_30) / mean_close_30
    
    perc_diff_check = percent_difference > 0 
    
    
    final_filter = perc_diff_check & nyse_filter
    
    return Pipeline(columns = {
                            'Percent Difference':percent_difference,
                            '30 Day Mean Close':mean_close_30,
                            'Latest Close':latest_close,
                            'Positive Percent Diff': perc_diff_check},
                    screen=final_filter)
results = run_pipeline(make_pipeline(),
                       '2017-01-01',
                       '2017-01-01')
results.head()
len(results)
Explanation: Classifier Methods
eq (equals)
isnull
startswith
End of explanation
from quantopian.pipeline import Pipeline
from quantopian.algorithm import attach_pipeline, pipeline_output
def initialize(context):
    my_pipe = make_pipeline()
    attach_pipeline(my_pipe, 'my_pipeline')
def make_pipeline():
    return Pipeline()
def before_trading_start(context, data):
    # Store our pipeline output DataFrame in context.
    context.output = pipeline_output('my_pipeline')
Explanation: Pipelines in Quantopian IDE
End of explanation
<END_TASK>