{"action":"created","issue":{"url":"https://api.github.com/repos/quantopian/zipline/issues/630","labels_url":"https://api.github.com/repos/quantopian/zipline/issues/630/labels{/name}","comments_url":"https://api.github.com/repos/quantopian/zipline/issues/630/comments","events_url":"https://api.github.com/repos/quantopian/zipline/issues/630/events","html_url":"https://github.com/quantopian/zipline/pull/630","id":92188082,"number":630,"title":"FFC API","user":{"login":"ssanderson","id":1588357,"avatar_url":"https://avatars.githubusercontent.com/u/1588357?v=3","gravatar_id":"","url":"https://api.github.com/users/ssanderson","html_url":"https://github.com/ssanderson","followers_url":"https://api.github.com/users/ssanderson/followers","following_url":"https://api.github.com/users/ssanderson/following{/other_user}","gists_url":"https://api.github.com/users/ssanderson/gists{/gist_id}","starred_url":"https://api.github.com/users/ssanderson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ssanderson/subscriptions","organizations_url":"https://api.github.com/users/ssanderson/orgs","repos_url":"https://api.github.com/users/ssanderson/repos","events_url":"https://api.github.com/users/ssanderson/events{/privacy}","received_events_url":"https://api.github.com/users/ssanderson/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-06-30T20:10:45Z","updated_at":"2015-07-29T17:57:22Z","closed_at":"2015-07-29T17:47:59Z","pull_request":{"url":"https://api.github.com/repos/quantopian/zipline/pulls/630","html_url":"https://github.com/quantopian/zipline/pull/630","diff_url":"https://github.com/quantopian/zipline/pull/630.diff","patch_url":"https://github.com/quantopian/zipline/pull/630.patch"},"body":"**NOTE:** Throughout this description I've added sections marked with **RFC**, meaning \"Request For Comments\", on locations where I'm particularly interested in feedback.\r\n\r\nThis patch lays the groundwork for a compute engine designed to facilitate construction of factor-based universe screening and portfolio allocation.\r\n\r\n## Goals\r\n1. Provide a high-level, declarative API for representing trailing window computations on large datasets.\r\n2. Provide groundwork for a dynamic portfolio construction API using Filters and Factors.\r\n3. Do 1 and 2 in way that's sufficiently abstracted from the underlying computation process that we can change the internals without breaking public APIs.\r\n4. Be fast enough and memory efficient enough to support computation on universes with thousands of assets.\r\n\r\n## Content\r\n### `zipline.modelling`\r\n\r\nContains entities that can be used to express computations as dependency graphs.  Each node in such a graph is an instance of the base `Term` class, defined in `zipline.modelling.term`.  Dependency graphs are executed by instances of `FFCEngine`, defined in `zipline.modelling.engine`.  The only interesting implementation is `SimpleFFCEngine`, but there's also a `NoOpFFCEngine` which is used when no FFC Loaders can be found.\r\n\r\nThere are three important subclasses of `Term`:\r\n* `Factor`, representing the results of numerical-valued computations.  Currently we only support `float64` as a dtype for Factor, but it would be mostly straightforward to add integral types.  Factors can be computed and referenced directly, and factors can be converted into `Filter`s via comparison operators (`>`, `>=`, `==`, etc).  There are also methods for more fine-grained statistical computations like percentile analysis.  In general, these methods work by producing new Term instances.  For example, `some_factor.percentile_between(10, 20)` returns an instance of `PercentileFilter`, which will perform the desired computation when actually evaluated by an engine.  There is experimental support for defining custom factors, and some examples have been added in `zipline.modelling.factors.technical`.\r\n* `Filter`, representing the results of boolean-valued computations.  Filters are generally with method calls on an instance of `Factor`, but there's an experimental `CustomFilter` for direct expression of boolean-valued computations.\r\n\r\nMost of the complexity of the `Term` class is dedicated to a mechanism that makes it so that calling a Term constructor twice with the same arguments will return the same instance.  (**RFC:** Does the implementation of this make sense to others.  Most of the explanation for how and why we're doing this is in `Term.__new__` and `Term.static_identity`.) This is valuable because it gives us common subexpression folding for free, but it adds some boilerplate to the construction of new Term subclasses.  This should only affect implementers of term combinators (i.e. functions that take terms and construct new terms) like `Rank` and `PercentileFilter`.  For users who just want to define new Filters/Factors with rolling computations, there are experimental `CustomFilter` and `CustomFactor` classes, which just require that the user implement a `compute` function that will be called on rolling windows of data.\r\n\r\n### `zipline.data.ffc`\r\n(**RFC**: I don't like this directory structure right now.  Should we just move it to zipline.modelling?  Somewhere else?)\r\n\r\nContains loaders and dataset definitions for inputs to the modelling API.  The abstract API for a modelling dataloader is defined by `FFCLoader` (**RFC**: I think we should remove references to FFC everywhere.  Thoughts on better names for this?)\r\n\r\nNew `TradingAlgorithm` api methods: `add_factor`, and `add_filter`.  These methods can only be called from `initialize`, and are used to inform the algorithm that each day it should compute the given terms.\r\nComputed factor results are made available through a new `factors` property of the `data` object in `before_trading_start` and `handle_data`. Computed filter results control which assets are available in the factor matrix on each day.  An asset will only appear if it passed **all** registered filters.  (**RFC:**: is this the right combinatorial behavior for filters?  Should we even allow multiple filters to be registered?  Should `data.factors` be a method rather than a property so that the user can request values that passed a specific filters instead?  That would add a fairly significant performance burden."},"comment":{"url":"https://api.github.com/repos/quantopian/zipline/issues/comments/126039382","html_url":"https://github.com/quantopian/zipline/pull/630#issuecomment-126039382","issue_url":"https://api.github.com/repos/quantopian/zipline/issues/630","id":126039382,"user":{"login":"twiecki","id":674200,"avatar_url":"https://avatars.githubusercontent.com/u/674200?v=3","gravatar_id":"","url":"https://api.github.com/users/twiecki","html_url":"https://github.com/twiecki","followers_url":"https://api.github.com/users/twiecki/followers","following_url":"https://api.github.com/users/twiecki/following{/other_user}","gists_url":"https://api.github.com/users/twiecki/gists{/gist_id}","starred_url":"https://api.github.com/users/twiecki/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twiecki/subscriptions","organizations_url":"https://api.github.com/users/twiecki/orgs","repos_url":"https://api.github.com/users/twiecki/repos","events_url":"https://api.github.com/users/twiecki/events{/privacy}","received_events_url":"https://api.github.com/users/twiecki/received_events","type":"User","site_admin":false},"created_at":"2015-07-29T17:57:22Z","updated_at":"2015-07-29T17:57:22Z","body":":+1: "}}