{"action":"created","comment":{"url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/comments/33624100","id":33624100,"diff_hunk":"@@ -0,0 +1,1022 @@\n+:author: Mike Pacer\n+:email: mpacer@berkeley.edu\n+:institution: University of California at Berkeley\n+:bibliography: myBibliography\n+\n+:video: youtube_link_to_come\n+\n+..  latex::\n+    :usepackage: booktabs\n+\n+..  latex::\n+\n+    %:usepackage: mathpalette, graphicx\n+    \\newcommand{\\bigCI}{\\mathrel{\\text{\\scalebox{1.07}{$\\perp\\mkern-10mu\\perp$}}}}\n+    \\newcommand\\independent{\\protect\\mathpalette{\\protect\\independenT}{\\perp}}\n+    \\def\\independenT#1#2{\\mathrel{\\rlap{$#1#2$}\\mkern2mu{#1#2}}}\n+\n+\n+.. raw::latex\n+\n+    \\newcommand{\\DUrolesc}{\\textsc}\n+    \\newcommand{\\DUroleindep}{\\mathrel{\\text{\\scalebox{1}{$\\perp\\mkern-9mu\\perp$}}}}\n+\n+\n+.. role:: indep\n+\n+.. role:: sc\n+\n+\n+\n+------------------------\n+Causal Bayesian NetworkX\n+------------------------\n+\n+..  class:: abstract\n+\n+    Humans are existence proofs for the solubility of computational causal inference.\n+\n+    Computational problems are sometimes thought to be the exclusive domain of computer science, though the solutions found prove vital for many other sciences. But computational cognitive science can also contribute to the solution of computational problems, particularly inductive problems. Causal inference (and inductive problems more generally) have proven resilient to traditional analysis, and the recent progress on these problems observed in machine learning (e.g., neural networks and their extensions) originate in models formulated by cognitive scientists.\n+\n+    As a computational cognitive scientist, I use a technique called rational analysis, which roughly consists of developing formal models of how *any* cognitive agent might optimally solve a computational problem and comparing that to how people actually solve analogous problems. In the course of doing so we find that people turn out to make much more sense than popular psychology might lead you to believe. At the same time, we create formal frameworks that represent entities and relations in the world as well as reasoning processes over those representations. \n+\n+    One of the frameworks successfully used in this way are causal Bayesian networks. Bayesian networks fall within the more general class of probabilistic graphical models, specifically, directed acyclic graphs with associated conditional probability distributions. Directed arrows encode direct dependency relationships going from parents to their children, whose conditional probability distribution is defined in terms of the parents' values. *Causal* Bayesian networks are endowed with an intervention operation that allows \"graph surgery\" in which one cuts variables off from their parents (usually setting it to a particular value). \n+\n+    I have developed tools on top of the :code:`NetworkX` package that allow me to implement some aspects of these models. By treating graph definition as one of enumeration and filtering rather than investigating invidual graphs, we are able to more conveniently state constraints on the graph structures under question. Indeed this gives an alternative view of intervention not as the modification of an single graph, but as a constraint on the total set of graphs. This allows us to treat the graphical aspects of the problem separately from the probabilistic semantics that define particular models on those graphs. I call this set of tools `Causal Bayesian NetworkX`.\n+\n+\n+..  class:: keywords\n+\n+    probabilistic graphical models, causality, intervention\n+\n+Introduction and Aims\n+---------------------\n+\n+My first goal in this paper is to provide enough of an introduction to some formal/mathematical tools such that those familiar with :code:`python` and programming more generally will be able to appreciate both why and how one might implement causal Bayesian networks. Especially to exhibit *how*, I have developed parts of a toolkit that allows the creation of these models on top of the :code:`NetworkX` python package. Given the coincidence of the names, it seemed most apt to refer to this toolkit as :code:`Causal Bayesian NetworkX` (the current implementation of which can be found at `Causal Bayesian NetworkX`_).\n+\n+In order to understand the toolset requires the basics of probabilsitic graphical models, which requires understanding some graph theory and some probability theory. The first few pages are devoted to providing necessary background and illustrative cases for conveying that understanding. \n+\n+Notably, contrary to how Bayesian networks are commonly introduced, I say relatively little about inference from observed data. This is intentional, as is this discussion of it. Many of the most trenchant problems with Bayesian networks are found in critiques of their use to infer these networks from observed data. But, many of the aspects of Bayesian networks (especially causal Bayesian networks) that are most useful for thinking about problems of structure and probabilistic relations do not rely on inference from observed data. In fact, I think the immediate focus on inference has greatly hampered widespread understanding of the power and representative capacity of this class of models. Equally – if not more – importantly, I aim to discuss generalizations of Bayesian networks such as those that appear in :cite:`griffithst09`, and inference in these cases requires a much longer treatement (if a comprehensive treatment can be provided at all). If you are dissatisfied with this approach and wish to read a more conventional introduction to (causal) Bayesian networks I suggest consulting :cite:`pearl2000`.\n+\n+The Causal Bayesian NetworkX toolkit can be seen as consisting of two main parts: graph enumeration/filtering and the storage and use of probabilistic graphical models in a NetworkX compatible format. Because this topic will more be the focus of my talk which can be viewed at the youtube link above and the source code of the most basic implementation is available at `Causal Bayesian NetworkX`_, in this paper I focus more on the other aspects of the problem. Nonetheless, appreciating these other aspects is made easier by also appreciating the problems of implementation/representation and the early solutions that I propose.\n+\n+I focus first on establishing a means of building iterators over sets of directed graphs. I then apply operations to those sets. Beginning with the complete directed graph, we enumerate over the subgraphs of that complete graph and enforce graph theoretic conditions such as acyclicity over the entire graph, guarantees on paths between nodes that are known to be able to communicate with one another, or orphan-hood for individual nodes known to have no parents. We accomplish this by using closures that take graphs as their input along with any explicitly defined arguments needed to define the exact desired conditions. \n+\n+I then shift focus to a case where there is a specific known directed acyclic graph that is imbued with a simple probabilistic semantics over its nodes and edges, also known as a Bayesian network. I demonstrate how to sample independent trials from these variables in a way consistent with these semantics. I discuss briefly some of the challenges of encoding these semantics in dictionaries as afforded by NetworkX without resorting to :code:`eval` statements and discuss compatibility issues I have found with JSON storage formats. \n+\n+I conclude with a discussion of some of the problems that have been addressed in Cognitive Science through the use of graphical models like those described. In particular, I will discuss a framework called **theory based causal induction** :cite:`griffithst09`, or my preferred term: **causal theories**, which allows for defining problems of causal induction. It is out of this framework the perspective expressed in this paper, the associated talk, and the Causal Bayesian NetworkX toolkit developed. \n+\n+.. _Causal Bayesian NetworkX: https://github.com/michaelpacer/Causal-Bayesian-NetworkX\n+\n+Graphical Models\n+----------------\n+\n+Graphs are formal models defined by a set of nodes (:math:`X, |X| = N`) and edges between those nodes (:math:`e \\in E \\equiv e \\in (X \\times X)`, where *E* is the set of edges). \n+\n+Notes on notation\n+=================\n+\n+Nodes\n+^^^^^\n+\n+In the examples in `Causal Bayesian NetworkX`_, nodes are given explicit labels individuating them such as :math:`\\{A,B,C,\\ldots\\}` or {'rain','sprinkler','grass_wet'}. Oftentimes, for the purposes of mathematical notation, it will be helpful to index nodes by the integers over a common variable label, e.g., using  :math:`\\{X_1,X_2,X_3,\\ldots\\}`. [#]_ \n+\n+.. [#] Despite pythonic counting beginning with 0, I chose not to begin this series with 0 because when dealing with variables that might be used in statistical regressions, the 0 subscript will have a specific meaning that separates it from the rest of the notation. For example when expressing multivariate regession as :math:`Y = \\beta X + \\epsilon, \\epsilon \\sim \\mathcal{N}(0,\\Sigma)`, :math:`\\beta_0` refers to the parameter associated with a constant variable :math:`x_0 = 1` and :math:`X` is normally defined as :math:`x_1, x_2, x_3, \\ldots`. This allows a simple additive constant to be estimated, which usually(but not always) is not of interest to statistical tests, acting as a scaling constant more than anything else. This also makes for simpler notation than saying :math:`Y = \\beta_0 + \\beta X + \\epsilon`, since that is equivalent to the previous notation (:math:`Y = \\beta X + \\epsilon`) if :math:`x_0 = 1`. In other cases :cite:`griffithst05,pacerg12`, the 0 index will be used to indicate background sources for events in a system.\n+\n+Edges\n+^^^^^\n+\n+Defined in this way, edges are all *directed* in the sense that an edge from :math:`X_1 \\textrm{ to } X_2` is not the same as the edge from :math:`X_2 \\textrm{ to } X_1`, or :math:`(X_1,X_2) \\neq (X_2,X_1)`. An edge :math:`(X_1,X_2)` will sometimes be written as :math:`X_1 \\rightarrow X_2`, and the relation may be described using language like \":math:`X_1` is the parent of :math:`X_2`\" or \":math:`X_2` is the child of :math:`X_1`\".\n+\n+Directed paths\n+^^^^^^^^^^^^^^\n+\n+Paths are a useful way to understand sequences of edges and the structure of a graph. Informally, to say there is a path between :math:`X_i` and :math:`X_j` is to say that one can start at :math:`X_i` and by traveling from parent to child along the edges leading out from the node that you are currently at, you can eventually reach :math:`X_j`.\n+\n+To define it recursively and more precisely, if the edge :math:`(X_i,X_j)` is in the edge set or if the edges :math:`(X_i,X_k)` and :math:`(X_k,X_j)` are in the edge set there is a path from :math:`X_i` to :math:`X_j`. Otherwise, a graph has a path from node :math:`X_i` to :math:`X_j` if there is a subset of its set of edges such that the set contains edges :math:`(X_i,X_k)` and :math:`(X_l,X_j)` and there is a path from :math:`X_k` to :math:`X_l`. \n+\n+\n+Adjacency Matrix Perspective\n+============================\n+\n+For a fixed set of nodes :math:`X` of size :math:`N`, each graph is uniquely defined by its edge set, which can be seen as a binary :math:`N \\times N` matrix, where each index :math:`(i,j)` in the matrix is :math:`1` if the graph contains an edge from :math:`X_i \\rightarrow X_j`, and :math:`0` if it does not contain such an edge. We will refer to this matrix as :math:`A(G)`.\n+\n+This means that any values of :math:`1` found on the diagonal of the adjacency matrix (i.e., where :math:`X_i \\rightarrow X_j, i=j`) indicate a self-loop on the respective node.\n+\n+.. Finding paths using adjacency matrices\n+.. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+.. It is straightforward to interpret questions of the existence of paths between :math:`X_i` and :math:`X_j` using the adjacency matrix perspective and matrix multiplication. The key step is to recognize that you can think of multiplying the adjacency matrix from the right by a binary vector as taking a step in the graph from the nodes whose values in the vector were 1 to the set of children of those nodes. To continue to have a binary vector then requires resetting values in the vector 0 and 1 by taking (for every element of the resulting vector) the minimum of the value of the vector and 1 (which addresses the case where more than one edge leads into the same node). \n+\n+.. To use this technique to test whether a matrix has an edge between, if you have a value of 1 at index *i*, and 0's elsewhere, if you multiply this vector from the left by the adjacency matrix, then if there is a path between \n+\n+Undirected Graphs\n+=================\n+\n+We can still have a coherent view of *undirected* graphs, despite the fact that our primitive notion of an edge is that of a *directed* edge. If a graph is undirected, then if it has an edge from :math:`X_i \\rightarrow X_j` then it has an edge from :math:`X_j \\rightarrow X_i`. Equivalently, this means that the adjacency matrix of the graph is symmetric, or :math:`A(G)=A(G)^\\top`.\n+\n+\n+Directed Graphs\n+===============\n+\n+From the adjacency matrix perspective we've been considering, all graphs are technically directed, and undirected graphs are a special case where both edges are symmetric.\n+\n+The number of directed graphs that can be obtained from a set of nodes of size :math:`n` can be defined explicitly using the fact that they can be encoded as a unique :math:`n \\times n` matrix:\n+\n+.. math::\n+\n+    R_n = 2^{n^2}\n+\n+\n+Directed Acyclic Graphs\n+^^^^^^^^^^^^^^^^^^^^^^^\n+\n+A cycle in a directed graph can be understood as the existence of a path from a node to itself. This can be as simple as a self-loop (i.e., if there is an edge :math:`(X_i,X_i)` for any node :math:`X_i`). \n+\n+Directed acyclic graphs(:sc:`dag`\\s) are directed graphs that contain no cycles.\n+\n+The number of :sc:`dag`\\s that can be obtained from a set of nodes of size :math:`n` can be defined recursively as follows :cite:`mckay2003acyclic` :\n+\n+.. math::\n+\n+    R_n = \\sum_{k=1}^{n} (-1)^{k+1} {\\binom{n}{k}} 2^{k(n-k)} R_{n-k}\n+\n+Note, because :sc:`dag`\\s do not allow any cycles, this means that there can be no self-loops. As a result, every value on the diagonal of a  :sc:`dag`\\'s adjacency matrix will be 0. \n+\n+.. Topological ordering in :sc:`dag`\\s\n+.. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+.. It is possible to reorder \n+\n+\n+Probability Distributions: Conditional, Joint and Marginal\n+----------------------------------------------------------\n+\n+A random variable defined by a conditional probability distribution [#]_ has a distribution indexed by the realization of some other variable (which itself is often a random variable, especially in the context of Bayesian networks). \n+\n+.. [#] Rather than choose a particular interpretation of probability over event sets (e.g., Bayesian or  frequentist), I will attempt to remain neutral, as those concerns are not central to the issues of graphs and simple sampling.\n+\n+The probability mass function (pmf) of a discrete random variable(:math:`X`) taking on value :math:`x` will be designated with :math:`P(X=x)`. Oftentimes, when one is discussing the full set of potential values (and not just a single value), one leaves out the :math:`=x` and just indicates :math:`P(X)`. [#]_ \n+.. This interpretation works most easily when considering mutually exclusive values, and if one is instead considering the possibility of a more complex event such as a variable taking on one of a set of values, the notation will often need adjusting. \n+\n+.. [#] If one is dealing with continuous quantities rather than discrete quantities one will have to use a probability density function (pdf) which does not have as straightforward an interpretation as a probability mass function. This difficult stems from the fact that (under most cases) the probability of any particular event occuring is \"measure zero\", or \"almost surely\" impossible. Without getting into measure theory and the foundation of calculus and continuity we can simply note that it is not that any individual event has non-zero probability, but that sets of events have non-zero probability.As a result, continuous random variables are more easily understood in terms a cummulative density function (cdf), which states not how likely any individual event is, but how likely it is that the event in question is less than a value :math:`x`. The notation usually given for a cdf of this sort is :math:`F(X\\leq x) = \\int_{-\\infty}^{x}f(u)du`, where :math:`f(u)` is the associated probability density function.\n+\n+The conditional probability of a variable :math:`X` taking on value :math:`x` once it is known that another variable :math:`Y` takes on value :math:`y` is :math:`P(X=x|Y=y)`. Much like above, if we want to consider the probability of each possible event without specifying one, sometimes this will be written as :math:`P(X|Y=y)`. If we are considering conditioning on any of the possible values of the known variable, we might use the notation :math:`P(X|Y)`, but that is a slight abuse of the notation. \n+\n+You *can* view :math:`P(X|Y)` as a function over the space defined by :math:`X\\times Y`. However, if you do so, do not interpret this as a probability function (of any kind). Rather, this defines a probability function for :math:`X` relative to each value of :math:`Y`. Without conditioning on :math:`Y` we have many potential functions of X. Thus, you can think of that as denoting a *family* of probability functions indexed by the various values :math:`Y=y`.\n+\n+The *joint probability* of :math:`X` and :math:`Y` is the probability that both :math:`X` and  :math:`Y` occur in the event set in question. This is noted as :math:`P(X,Y)` or :math:`P(X \\cap Y)` (using the set theoretic intersection operation). Similar to :math:`P(X|Y)`, you *can* view :math:`P(X,Y)` as a function over the space defined by :math:`X\\times Y`. However, :math:`P(X,Y)` is a probability function in the sense that the sum of :math:`P(X=x,Y=y)` over all the possible events in the space defined by :math:`(x,y)\\in X\\times Y` equals 1.\n+\n+The *marginal probability* of :math:`X` is the same :math:`P(X)` that we have seen before. However, the term refers to the notion of summing over values of :math:`Y` in the joint probability, and these summed probabilities were recorded in the *margins* of a probability table. Formally, this can be stated as :math:`P(X) = \\sum_{y\\in Y}P(X,Y)`.\n+\n+Relating conditional and joint probabilities\n+============================================\n+\n+Conditional probabilities are related to joint probabilities using the following form:\n+\n+.. math::\n+\n+    P(X|Y=y) = \\frac{P(X,Y=y)}{P(Y=y)} = \\frac{P(X,Y=y)}{\\sum_{x \\in X}P(X=x,Y=y)}\n+\n+Equivalently:\n+\n+.. math::\n+\n+    P(X,Y=y) = P(X|Y=y)P(X)\n+\n+\n+Bayes' Theorem\n+==============\n+\n+Bayes' Theorem can be seen as a result of how to relate conditional and joint probabilities. Or more importantly, how to compute the probability of a variable once you know something about some other variable.\n+\n+Namely, if we want to know :math:`P(X|Y)` we can transform it into :math:`\\frac{P(X,Y)}{\\sum_{x \\in X}P(X=x,Y)}`, but then can also transform joint probabilities (:math:`P(X,Y)`) into statements about conditional and marginal probabilities (:math:`P(X|Y)P(X)`).\n+\n+This leaves us with\n+\n+..  math::\n+\n+    P(X|Y) = \\frac{P(X|Y)P(X)}{\\sum_{x \\in X}P(X=x|Y)P(X=x)}\n+\n+Probabilistic Independence\n+==========================\n+\n+To say that two variables are independent of each other means that knowing/conditioning on the realization of one variable is irrelevant to the distribution of the other variable. This is equivalent to saying that the joint probability is equal to the multiplication of the probabilities of the two events. \n+\n+If two variables are conditionally independent, that means that conditional on some set of variables, condition\n+\n+\n+\n+Example: Marginal Independence :math:`\\neq` Conditional Independence\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+Consider the following example:\n+\n+..  math::\n+    :type: eqnarray\n+\n+    X &\\sim& \\textrm{Bernoulli}_{\\{0,1\\}}(.5), Y \\sim \\textrm{Bernoulli}_{\\{0,1\\}}(.5)\\\\\n+    Z &=& X \\oplus Y, \\oplus \\equiv \\textsc{xor}\\\\\n+\n+Note that, :math:`X \\independent Y` but :math:`X \\not\\independent Y|Z`.\n+\n+Sampling from Conditional Probability distributions\n+---------------------------------------------------\n+\n+Example - Coins and dice\n+========================\n+\n+Imagine the following game: \n+\n+You have a coin [#]_ (*C*, :sc:`Heads, Tails`), a 6-sided die (:math:`D_6, \\{1,2,\\ldots,6\\}`), and a 20-sided die (:math:`D_{20}, \\{1,2,\\ldots,20\\}`). If for simplicity, you prefer to think of these as fair dice and a fair coin, you are welcome to do so, but my notation will not require that.\n+\n+.. [#] A coin is effectively 2-sided die, but for clarity of exposition I chose to treat the conditioned-on variable as a different kind of object than the variables relying on that conditioning.\n+\n+The rules of the game are as follows: flip the coin, and if it lands on :sc:`Heads`, then you roll the 6-sided die to find your score for the round. If instead your coin lands on :sc:`Tails` your score comes from a roll of the 20-sided die. Your score for one round of the game is the value of the die that you roll, and you will only roll one die in each round. \n+\n+Suppose we wanted to know your expected score on a single round, but we do not know whether the coin will land on :sc:`Heads` or :sc:`Tails`. We cannot directly compute the probabilities for each die without first considering the probability that the coin will land on :sc:`Heads` or :sc:`Tails`. This is the \n+\n+But this discussion hides an important complexity by having the event set of the :math:`D_6` embedded within the event set of the :math:`D_{20}`. Moreover, we assumed that we could treat each event in these sets as belonging to the integers and as a result, that with little interpretation, they can be easily summed.\n+\n+Coins and dice with hierarchically labeled entities, Example\n+============================================================\n+\n+Imagine the following game: \n+\n+You have a coin (*C*, :sc:`Heads, Tails`), a *new* 6-sided die (:math:`D_6, \\{X_1,X_2,\\ldots,X_6\\}`), and a 20-sided die (:math:`D_{20}, \\{X_1,X_2,\\ldots,X_{20}\\}`). \n+\n+The rules are the same as before: your score for one round of the game is the value of the die that you roll, and you will only roll one die in each round. You flip the coin, and if it lands on :sc:`Heads`, then you roll the 6-sided die to find your score for the round. If instead your coin lands on :sc:`Tails` your score comes from a roll of the 20-sided die.\n+\n+But note that now we cannot sum over these in the same way that we did before. Without additional information about how to map these different labels onto values, there's no way to describe the \"score\". Rather, the best we can do is to determine the probability with which each individual case occurs, so that once we know more about the utility curve we can efficiently use the probability distribution regardless of the particular value that is assigned.\n+\n+Thus we can establish the following statements\n+\n+..  latex::\n+\n+    \\begin{center}\n+    \\begin{tabular}{lll}\n+        \\toprule\n+        & \\multicolumn{2}{c}{Parent values} \\\\\n+        \\cmidrule(r){2-3}\n+        Probs & $P(\\cdot|D_6,\\textsc{h})$ & $P(\\cdot|D_{20},\\textsc{t})$\\\\\n+        \\midrule\n+        $P(X_1|\\cdot)$ &$P(X_1|D_6)*P(\\textsc{h})$ & $P(X_1|D_{20})*P(\\textsc{t})$ \\\\\n+        \\vdots     &    \\vdots     & \\vdots       \\\\\n+        $P(X_6|\\cdot)$       &  $P(X_6|D_6)*P(\\textsc{h})$     & $P(X_6|D_{20})*P(\\textsc{t})$      \\\\\n+        \\vdots       & \\vdots     & \\vdots      \\\\\n+        $P(X_{20}|\\cdot)$ & 0      & $P(X_{20}|D_{20})*P(\\textsc{t})$   \\\\\n+        \\bottomrule\n+    \\end{tabular}\n+    \\end{center}\n+\n+.. Coins and dice with disjoint sets of labeled entities, Example\n+.. ==============================================================\n+\n+.. Imagine the following game: \n+\n+.. You have a coin (*C*, :sc:`Heads, Tails`), a *new* 6-sided die (:math:`D_6, \\{\\clubsuit,\\diamondsuit,\\heartsuit,\\spadesuit,\\odot,\\dagger\\}`), and a 20-sided die (:math:`D_{20}, \\{X_1,X_2,\\ldots,X_{20}\\}`). \n+\n+.. The rules are the same as before: your score for one round of the game is the value of the die that you roll, and you will only roll one die in each round. You flip the coin, and if it lands on :sc:`Heads`, then you roll the 6-sided die to find your score for the round. If instead your coin lands on :sc:`Tails` your score comes from a roll of the 20-sided die.\n+\n+.. But note that now we cannot sum over these in the same way that we did before. Indeed, our event sets for the two dice are mutually disjoint, making the event set for the scores that one can receive on a single round :math:`\\{\\clubsuit,\\diamondsuit,\\heartsuit,\\spadesuit,\\odot,\\dagger,X_1,X_2,\\ldots,X_{20}\\}`. Without additional information about how to map these different labels onto values, there's no way to describe the \"score\". Rather, the best we can do is to determine the probability with which each individual case occurs.\n+\n+Bayesian Networks\n+-----------------\n+\n+Bayesian networks are a class of graphical models that have particular probabilistic semantics attached to their nodes and edges. This makes them probabilsitic graphical models. \n+\n+The most important property of Bayesian networks is that a variable when conditioned on the total set of its parents and children, is conditionally independent of any other variables in the graph. This is known as the \"Markov blanket\" of that node. [#]_\n+\n+.. [#] The word \"Markov\" refers to Andrei Markov and appears as a prefix to many other terms. It most often indicates that some kind of independence property holds. For example, a Markov chain is a sequence (chain) of variables in which each variable depends only dependent on the value of the immediate preceding (and by implication) postceding variables in the chain. \n+\n+Common assumptions in Bayesian networks\n+=======================================\n+\n+While there are extensions to these models [#]_ , a number of assumptions commonly hold. \n+\n+.. [#] An important class of extensions to Bayesian networks that I will not have time to discuss at length includes those that consider temporal dependencies: Dynamic Bayesian Networks (:sc:`dbn`\\s) :cite:`deank1989time,ghahramani1998learning`, continuous-time dependencies with Continuous Time Bayesian Networks (:sc:`ctbn`\\s) :cite:`nodelman02`, Poisson Cascades :cite:`simma10`, Continuous Time Causal Theories (:sc:`ct`:math:`^2`) :cite:`pacerg12, pacerg15`, Reciprocal Hawkes Processes :cite:`blundell2012modelling` and the Network Hawkes Model :cite:`lindermana2014`.\n+\n+Fixed node set\n+^^^^^^^^^^^^^^\n+\n+The network is considered to be comprehensive in the sense that there is a fixed set of known nodes with finite cardinality :math:`N`. This rules out the possibility of hidden/latent variables as being part of the network. From this perspective inducing hidden nodes requires postulating a new graph that is potentially unrelated to the previous graph. \n+\n+Trial-based events, complete activation and :sc:`dag`\\-hood\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+Within a trial, all events are presumed to occur simultaneously. This means two things. First, there is no notion of temporal asynchrony, where one node/variable takes on a value before its children take on a value (even if in reality – i.e., outside the model – that variable is known to occur before its child). Secondly, the probabilistic semantics will be defined over the entirety of the graph meaning that one cannot sample a proper subset of the nodes of a graph unless they have no effects or are marginalized out with their effects being incorporated into their children.\n+\n+This property also explains why Bayesian networks need to be acyclic. Most of the time when we consider causal cycles in the world the cycle relies on a temporal delay between the causes and their effects to take place. If the cause and its effect is simultaneous, it becomes difficult (if not nonsensical) to determine which is the cause and which is the effect — they seem instead to be mutually definitional. But, as noted above, when sampling in Bayesian networks simultenaity is presumed for *all* of the nodes.\n+\n+Independence in Bayes Nets\n+==========================\n+\n+One of the standard ways of describing the relation between the semantics (probability values) and syntax (graphical structure) of Bayesian networks is in terms of the graph encoding particular conditional independence assumptions between the nodes of the graph. Indeed, in some cases Bayesian networks are *defined as* a convenient representation for the conditional and marginal independence relationships between different variables. \n+\n+It is the perspective of the graphs as *merely* representing the independence relationships and the focus on inference that leads to the focus on equivalence classes of Bayes nets. The set of graphs :math:`\\{A \\rightarrow B \\rightarrow C,~ A \\leftarrow B \\rightarrow C, \\textrm{ and } A \\leftarrow B \\leftarrow C\\}` represent the same conditional independence relationships, and thus cannot be distinguished on the basis of observational evidence alone. This also leads to the emphasis on finding *v-structures* or common-cause structures where (at least) two arrows are directed into the same child with no direct link between those parents(e.g., :math:`A \\rightarrow B \\leftarrow C`). V-structures are observationally distinguishable because any reversing the direction of any of the arrows will alter the conditional independence relations that are guaranteed by the graphical structure. [#]_\n+\n+.. [#] A more thorough analysis of this relation between graph structures and implied conditional independence relations invokes the discussion of *d-separation*. However, d-separation (despite claims that \"[t]he intuition behind [it] is simple\") is a more subtle concept than it at first appears as it involves both which nodes are obeserved and the underlying structure.\n+\n+While this is accurate, it eschews some important aspects of the semantics that distinguish arrows with different directions when you consider the particular kinds of values that the variables take on.\n+\n+.. Issues surrounding independence in Bayesian networks\n+.. ====================================================\n+\n+.. Misplaced Emphasis on Independence in :sc:`dag`\\s\n+.. =================================================\n+\n+.. I do not agree with the interpretation of Bayes nets as merely representing independence properties, though, not because it is incorrect. Rather, I think it has two unfortunate results. First, it encourages poor statistical practices when it comes to inferring independence from observed data using null hypothesis testing. Second, it deëmphasizes an important assymetry that appears in the semantics of how nodes in Bayes nets relate to one another when they are not exclusively discrete nodes.\n+\n+.. Null hypothesis testing and inference\n+.. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+.. The assumptions embedded in Bayesian networks are assumptions about the independence of different nodes. But most of the measn \n+\n+Directional semantics between different types of nodes\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+The conditional distributions of child nodes are usually defined with parameter functions that take as arguments their parents' realizations for that trial. Bayes nets often are used to exclusively represent discrete (usually, binary) nodes the distribution is usually defined as an arbitrary probability distribution associated with the label of it's parent's realization. \n+\n+If we allow (for example) positive continuous valued nodes to exist in relation to discrete nodes the kind of distributions available to describe relations between these nodes changes depending upon the direction of the arrow. A continuous node taking on positive real values mapping to an arbitrarily labeled binary node taking on values :math:`\\{a,b\\}` will require a function that maps from :math:`\\mathbb{R} \\rightarrow [0,1]`, where it maps to the probability that the child node takes on (for instance) the value :math:`a` . [#]_However, if the relationship goes the other direction, one would need to have a function that maps from :math:`\\{a,b\\} \\rightarrow \\mathbb{R}`. For example, this might be a gaussian distributions for *a* and *b* (:math:`(\\mu_a,\\sigma_a),(\\mu_b,\\sigma_b)`). Regardless of the particular distributions, the key is that the functional form of the distributions are radically different \n+\n+.. [#] If the function maps directly to one of the labeled binary values this can be represented as having probabilty 1 of mapping to either :math:`a` or :math:`b`.\n+\n+\n+Generating samples from Bayes Nets\n+==================================\n+\n+This procedure for sampling a trial from Bayesian networks relies heavily on using what I call the *active sample set*. This is the set of nodes for which we have well-defined distributions at the time of sampling.\n+\n+There will always be at least one node in a Bayesian network that has no parents (for a given trial). We will call these nodes **orphans**. To sample a trial from the Bayesian network we begin with the orphans. \n+\n+Because orphans have no parents – in order for the Bayes net to be well-defined – each orphan will have a well-defined marginal probability distribution that we can directly sample from. Thus we start with the set of orphans as the *active sample set*. \n+\n+After sampling from all of the orphans, we will take the union of the sets of children of the orphans, and at least one of these nodes will have values sampled for all of its parents. We take the set of orphans whose entire parent-set has sampled values, and sample from the conditional distributions defined relative to their parents' sampled values and make this the *active sample set*.\n+\n+After each set of samples from the *active sample set* we will either have new variables whose distributions are well-defined or will have sampled all of the variables in the graph for that trial. [#]_ If we have multiple trials, we repeat this procedure for each trial. \n+\n+.. [#] One potential worry is the case of disconnected graphs (i.e., graphs that can be divided into at least 2 disjoint sets of nodes where there will be no edges between nodes of different sets). However, because disconnected subgraphs of a :sc:`dag` will also be :sc:`dag`\\s, we can count on at least one orphan existing for each of those graphs, and thus we will be able to sample from all disconnected subgraph by following the same algorithm above (they will just be sampled in parallel).\n+\n+Causal Bayesian Networks\n+------------------------\n+\n+Causal Bayesian networks are Bayesian networks that are given an interventional operation that allows for \"graph surgery\" by cutting nodes off from their parents. [#]_ The central idea is that interventions are cases where some external causal force is able to \"reach in\" and set the values of individual nodes, rendering intervened on independent of their parent nodes. \n+\n+.. [#] This is technically a more general definition than that given in :cite:`pearl2000` as in that case there is a specific semantic flavor given to interventions as they affect the probabilistic semantics of the variables within the network. Because here we are considering a version of intervention that affects the *structure* of a set of graphs rather than an intervention's results on a specific parameterized graph, this greater specificity is unnecessary.\n+\n+NetworkX\n+--------\n+\n+This is a framework for graphs that stores graphs as \"a dict of dicts of dicts\".\n+\n+Basic NetworkX operations\n+=========================\n+\n+NetworkX is usually imported using the :code:`nx` abbreviation\n+\n+..  code-block:: python\n+    \n+    import networkx as nx  \n+\n+    G = nx.DiGraph() # initialize a directed graph\n+\n+    edge_list = G.edges() # returns a list of edges\n+    edge_data_list = G.edges(data=True) \n+    # returns list of edges as tuples with data dictionary \n+\n+    node_list = G.nodes() # returns a list of nodes\n+    node_data_list = G.nodes(data=True) \n+    # returns list of nodes as tuples with data dictionary\n+\n+\n+\n+Causal Bayesian NetworkX: Graphs\n+--------------------------------\n+\n+Here we will look at some of the basic operations described in the `ipython notebook` :cite:`perezG2007` found at `Causal Bayesian NetworkX`_.\n+\n+Other packages\n+==============\n+\n+In addition to networkX, we need to import numpy and itertools.\n+\n+..  code-block::python\n+\n+    import numpy as np\n+    from itertools import chain, combinations, tee\n+\n+\n+Beginning with a max-graph\n+==========================\n+\n+Starting with the max graph for a set of nodes (i.e., the graph with :math:`N^2` edges), we build an iterator that returns graphs by successively removing subsets of edges. Because we start with the max graph, this procedure will visit all possible subgraphs. One challenge that arises when visiting *all* possible subgraphs is the sheer magnitude of that search space (:math:`2^{N^2}`).\n+\n+..  code-block:: python\n+\n+    def completeDiGraph(nodes):\n+        \"\"\"\n+        Building a max-graph from a set of n nodes.\n+        This graph has :math:`n^2` edges.\n+        Variables:\n+        nodes are a list of strings comprising node names\n+        \"\"\"\n+\n+        G = nx.DiGraph() # Creates new graph\n+        G.add_nodes_from(nodes) # adds nodes to graph\n+        edgelist = list(combinations(nodes,2)) \n+        # list of directed edges\n+        edgelist.extend([(y,x) for x,y in edgelist)\n+        #add symmetric edges\n+        edgelist.extend([(x,x) for x in nodes]) \n+        # add self-loops\n+        G.add_edges_from(edgelist) # add edges to graph\n+        return G\n+\n+Preëmptive Filters\n+==================\n+\n+In order to reduce the set of edges that we need to iterate over, rather than working over the max-graph for *any* of nodes, it helps to determine which individual edges are known to always be present and which ones are known to never be present. In this way we can reduce the size of the edgeset over which we will be iterating. \n+\n+Interestingly, this allows us to include more variables/nodes without the explosion of edges that would be the consequence of adding additional nodes were we not to include preëmptive filters.\n+\n+One of the most powerful uses I have found for this is the ability to modify a graph set to include interventional nodes without seeing a corresponding explosion in the number of graphs. On the assumption that interventions apply only to a single node () example nodes representing interventions, as nodes without on the preëxisting variables that.\n+\n+..  code-block:: python\n+\n+    def filter_Graph(G,filter_set):\n+        \"\"\"\n+        This allows us to apply a set of filters encoded \n+        as closures that take a graph as input\n+        and return a graph as output.\n+        \"\"\"\n+        graph = G.copy()\n+        for f in filter_set:\n+            graph = f(graph)\n+        return graph\n+\n+Example filter: remove self-loops\n+=================================\n+\n+By default the graph completed by :code:`completeDiGraph()` will have self-loops, often we will not want this (e.g., :sc:`dag`\\s cannot contain self-loops).\n+\n+.. code-block:: python\n+\n+    def extract_remove_self_loops_filter():\n+        def remove_self_loops_filter(G):\n+            graph = G.copy()\n+            graph.remove_edges_from(graph.selfloop_edges())\n+            return graph\n+        return remove_self_loops_filter\n+\n+.. Example filter use-case: add intervening nodes to a existing graph\n+.. ==================================================================\n+\n+.. By default the graph completed by :code:`completeDiGraph()` will have self-loops, often we will not want this (e.g., :sc:`dag`\\s cannot contain self-loops).\n+\n+.. .. code-block:: python\n+\n+..     def extract_remove_self_loops_filter():\n+..         def remove_self_loops_filter(G):\n+..             graph = G.copy()\n+..             graph.remove_edges_from(graph.selfloop_edges())\n+..             return graph\n+..         return remove_self_loops_filter\n+\n+\n+\n+Conditions\n+==========\n+\n+The enumeration portion of this approach is defined in this :code:`conditionalSubgraphs` function.[#]_ This allows you to pass in a graph from which you will want to sample subgraphs that meet the conditions that you also pass in. \n+\n+.. [#] Note that powerset will need to be built (see `Causal Bayesian NetworkX`_ for details).\n+\n+..  code-block:: python\n+\n+    def conditionalSubgraphs(G,condition_list):\n+        \"\"\"\n+        Returns a graph iterator of subgraphs of G \n+        meeting conditions in condition_list.\n+\n+        Variables: \n+        G: a graph from which subgraphs will be taken.\n+        condition_list: a list of condition functions.\n+        \n+        Functions in condition_list have i/o defined as\n+        input: graph, generated as a subgraph of G\n+        output: Bool, whether graph passes condition\n+        \"\"\"\n+\n+        for edges in powerset(G.edges()):\n+            G_test = G.copy()\n+            G_test.remove_edges_from(edges)\n+            if all([c(G_test) for c in condition_list]):\n+                \n+                yield G_test\n+\n+\n+Example condition: detecting :sc:`dag`\\s\n+========================================\n+\n+If we wanted to have examples of all dags that are subgraphs of a passed in graph, we can use a convenient networkX utility.\n+\n+..  code-block:: python\n+\n+    def create_is_dag_condition(node_list):\n+        \"\"\" Returns a function that returns true \n+        if graph is a dag.\"\"\"\n+        def is_dag_condition(G):\n+            return nx.is_directed_acyclic_graph(G)\n+        return is_dag_condition\n+\n+Non-destructive conditional subgraph generators\n+===============================================\n+\n+Because the :code:`conditionalSubgraph` generator produces an iterable, if we want to apply a conditional after that intiial set is generated, we need to split it into two copies of the iterable. This involves the :code:`tee` function from the :code:`itertools` core package.\n+\n+.. code-block:: python\n+\n+    def new_conditional_graph_set(graph_set,cond_list):\n+        \"\"\"\n+        Returns graph_set & a new iterator which has \n+        conditions in cond_list applied to it.\n+        \n+        Warning: This function will devour the iterator \n+        you include as the `graph_set` input, \n+        you need to redeclare the variable as \n+        one of the return values of the function.\n+        \n+        Thus a correct use would be:    \n+        a,b = new_conditional_graph_set(a,c)\n+        \n+        The following would not be a correct use:\n+        x,y = new_conditional_graph_set(a,c)\n+        \n+        Variables: \n+        graph_set: graph-set iterator generator\n+        cond_list: list conditions\n+            input: a graph.\n+            output: boolean value\n+        \"\"\"\n+        \n+        graph_set_newer, graph_set_test = tee(graph_set,2)\n+        def gen():\n+            for G in graph_set_test:\n+                G_test = G.copy()\n+                if all([c(G_test) for c in condition_list]):\n+                    yield G_test\n+        return graph_set_newer, gen()\n+\n+Filters versus Conditions: which to use\n+=======================================\n+\n+The most obvious structural differences between filters and conditions give insight to how they are to be used. \n+\n+Filters are intended to apply to the max graph to reduce the edge set. They take (at least) a graph as an argument and return a graph. This is meant to be a transformation of the graph, or a way to change the value of a graph in place. It does not have any notion of producing both copies of the graph (though that could be done as well).\n+\n+Conditions are intended to be applied to a series of graphs generated by an iterator taking subgraphs of some other graph\n+\n+Naming conventions for filters and conditions\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+The convention I have been following for distinguishing filter and condition functions is that the higher-order function in the case of filters beginning with the word :code:`extract_`, and then both the returned function and the higher-order function ending with the word :code:`filter`. Similarly, conditions have begun with :code:`create_` and finished with :code:`condition`.\n+\n+..  code-block:: python\n+\n+    def extract_name_filter(node_list):\n+        \"\"\"\n+        \"\"\"\n+        def name_filter(G):\n+            graph = G.copy()\n+            # operations removing edges\n+            return graph\n+        return name_filter\n+\n+\n+..  code-block:: python\n+\n+    def create_name_condition(node_list):\n+        \"\"\"\n+        \"\"\"\n+        def name_condition(G):\n+            # operations checking for whether conditions hold\n+            return # truth value\n+        return name_condition\n+\n+.. Complex example: adding interventional nodes\n+.. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+.. ..  code-block:: python\n+\n+..     def add_interventions(G):\n+..         node_list = G.nodes()\n+..         edge_list = G.edges()\n+\n+..         int_node_list = [str(x)+\"_int\" for x in node_list]\n+\n+..     def completeDiGraph(nodes):\n+..         \"\"\"\n+..         Building a max-graph from a set of nodes. This graph has\n+..         :math:`n^2` edges in terms of len(nodes).\n+..         Variables:\n+..         nodes are a list of strings that specify the node names\n+..         \"\"\"\n+\n+..         G = nx.DiGraph() # Creates new graph\n+..         G.add_nodes_from(nodes) # adds nodes to graph\n+..         edgelist = list(combinations(nodes,2)) \n+..         # list of directed edges\n+..         edgelist.extend([(y,x) for x,y in list(combinations(nodes,2))]) \n+..         #add symmetric edges\n+..         edgelist.extend([(x,x) for x in nodes]) # add self-loops\n+..         G.add_edges_from(edgelist) # add edges to graph\n+..         return G\n+\n+..     def extract_remove_self_loops_filter():\n+..         def remove_self_loops_filter(G):\n+..             graph = G.copy()\n+..             graph.remove_edges_from(graph.selfloop_edges())\n+..             return graph\n+..         return remove_self_loops_filter\n+\n+\n+.. Gates: Context-sensitive causal Bayesian networks\n+.. -------------------------------------------------\n+\n+Causal Bayesian NetworkX: Sampling\n+----------------------------------\n+\n+It is possible to identify first those nodes with no parents, and then sample them. Then (and this is the part that is iterated), sample those in the children set whose parent set has a full sample, removing children from the children set when they are sampled from and adding them to the parent set. This is continued until there are no children in the children set. \n+\n+This is the algorithm that sampling follows as can be observed in `Causal Bayesian NetworkX`_. This approach only works for :sc:`dag`\\s and is formally equivalent to identifying a *topological ordering* for the nodes and then sampling accordingly. A graph having a topological ordering is biconditionally equivalent to being a :sc:`dag`. This critrion can roughly be seen as assigning each node an integer such that every child will always have an integer greater than any of its parent nodes (and by recursion any of its ancestor nodes). This provides an order in which to visit the nodes for sampling purposes that will ensure that any nodes in a child's parent set will always be visited first. This also results in choosing those nodes with an empty set as a parent set (i.e., orphans) to have the lowest integers, and therefore to be sampled first.\n+\n+Most of the difficult parts of encoding a sampling procedure though have nothing to do with the algorithm(it is standard for :sc:`dag`\\s). Rather, they arise from attempting to store the relevant information within the NetworkX data nodes, so that a self-contained graphical object can be imported and exported. There is a general problem of a lack of standard storage format for Bayesian networks (and probabilistic graphical models in general). This is just one flavor of that problem. \n+\n+Lack of JSON compatibility\n+==========================\n+\n+When submitting the abstract for this paper and talk, I believed I had created a JSON compatible format for storing the underlying data. I discovered that my method of storing groups of parent-variable realizations as tuple-keys with distribution arguments broke the JSON compatiblity that I was able to maintain in other circumstances. This was useful for being able to call the distributions of variables who have more than one parent nodes. I have not yet fixed this problem. \n+\n+Causal Theories and Computational Cognitive Science\n+---------------------------------------------------\n+\n+**Theory based causal induction** is a formal framework that arose out of the tradition in computational cognitive science to approach problems of human cogntiion with rational, comptuational-level analyses :cite:`griffithst09`. In particular, causal theories form generative models for defining classes of parameterized probabilistic graphical models. They rely on defining a set of classes of entities (ontology), potential relationships between those classes of entities and particular entities (plausible relations), and particular parameterizations of how those relations manifest in observable data (or in how other relations eventually ground out into observable data). This allows Griffiths and Tenenbaum to subsume the prediction of a wide array of human causal inductive, learning and reasoning behavior using this framework for generating graphical models and doing inference over the structures they generate.\n+\n+Rational analysis\n+=================\n+\n+A technique used that allows us to model not cognition per se, but the situation into which cognitive capacities are to be placed. If we assume that we know the inputs, the outputs and the goal state of the arbtirary cognitive agent, we can iteratively predict the agent's behavior[#]_.\n+\n+This is often coupled with comptuational-level analysis inspired by Marr's :cite:`marr82` level's of analysis.  ","path":"papers/mike_pacer/scipy_first_draft.rst","position":670,"original_position":670,"commit_id":"ffb7f35705522a628fb0b02fea785ebf6c2dbb83","original_commit_id":"ffb7f35705522a628fb0b02fea785ebf6c2dbb83","user":{"login":"katyhuff","id":393899,"avatar_url":"https://avatars.githubusercontent.com/u/393899?v=3","gravatar_id":"","url":"https://api.github.com/users/katyhuff","html_url":"https://github.com/katyhuff","followers_url":"https://api.github.com/users/katyhuff/followers","following_url":"https://api.github.com/users/katyhuff/following{/other_user}","gists_url":"https://api.github.com/users/katyhuff/gists{/gist_id}","starred_url":"https://api.github.com/users/katyhuff/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/katyhuff/subscriptions","organizations_url":"https://api.github.com/users/katyhuff/orgs","repos_url":"https://api.github.com/users/katyhuff/repos","events_url":"https://api.github.com/users/katyhuff/events{/privacy}","received_events_url":"https://api.github.com/users/katyhuff/received_events","type":"User","site_admin":false},"body":"level's -> levels","created_at":"2015-06-30T21:10:06Z","updated_at":"2015-06-30T21:10:06Z","html_url":"https://github.com/scipy-conference/scipy_proceedings/pull/147#discussion_r33624100","pull_request_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/147","_links":{"self":{"href":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/comments/33624100"},"html":{"href":"https://github.com/scipy-conference/scipy_proceedings/pull/147#discussion_r33624100"},"pull_request":{"href":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/147"}}},"pull_request":{"url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/147","id":37754091,"html_url":"https://github.com/scipy-conference/scipy_proceedings/pull/147","diff_url":"https://github.com/scipy-conference/scipy_proceedings/pull/147.diff","patch_url":"https://github.com/scipy-conference/scipy_proceedings/pull/147.patch","issue_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/issues/147","number":147,"state":"open","locked":false,"title":"Paper: Causal Bayesian NetworkX (Pacer)","user":{"login":"michaelpacer","id":2482408,"avatar_url":"https://avatars.githubusercontent.com/u/2482408?v=3","gravatar_id":"","url":"https://api.github.com/users/michaelpacer","html_url":"https://github.com/michaelpacer","followers_url":"https://api.github.com/users/michaelpacer/followers","following_url":"https://api.github.com/users/michaelpacer/following{/other_user}","gists_url":"https://api.github.com/users/michaelpacer/gists{/gist_id}","starred_url":"https://api.github.com/users/michaelpacer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/michaelpacer/subscriptions","organizations_url":"https://api.github.com/users/michaelpacer/orgs","repos_url":"https://api.github.com/users/michaelpacer/repos","events_url":"https://api.github.com/users/michaelpacer/events{/privacy}","received_events_url":"https://api.github.com/users/michaelpacer/received_events","type":"User","site_admin":false},"body":"Note: The paper as it stands is incomplete. Apologies —  I will be continuing to work on it in the coming days. \r\n\r\nNote2: Part of the time was spent trying to alter `paper_build` and and `writer.__init__` to be able to allow a local bibtex file to be used with a simple declaration at the head of the file after the `:name:` `:email:` and `:institution:` class declarations. I worked on this in conversation with @stefanv's, so apologies if my file doesn't build quite right as is (since the changes I've made are not likely to have been ported over to the master branch until others can look through them more closely).\r\n\r\nFor others who want to use an external bib file, just include `:bibliography: fileName` for a `fileName.bib` and it should work. If you don't have any declaration then none of the bibtex specific code should run. The file name cannot contain spaces or any special characters that need to be escaped. \r\n\r\nCitations can be called then using :cite:`citekey`. \r\n\r\nCheers,\r\nMike\r\n\r\n[Edit by jaberg] http://nipy.bic.berkeley.edu:5000/download/12","created_at":"2015-06-16T02:23:27Z","updated_at":"2015-06-30T21:10:06Z","closed_at":null,"merged_at":null,"merge_commit_sha":"95a4ac046fab53c61045e8eb7ca4bb2485e8ebc5","assignee":{"login":"jaberg","id":171276,"avatar_url":"https://avatars.githubusercontent.com/u/171276?v=3","gravatar_id":"","url":"https://api.github.com/users/jaberg","html_url":"https://github.com/jaberg","followers_url":"https://api.github.com/users/jaberg/followers","following_url":"https://api.github.com/users/jaberg/following{/other_user}","gists_url":"https://api.github.com/users/jaberg/gists{/gist_id}","starred_url":"https://api.github.com/users/jaberg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jaberg/subscriptions","organizations_url":"https://api.github.com/users/jaberg/orgs","repos_url":"https://api.github.com/users/jaberg/repos","events_url":"https://api.github.com/users/jaberg/events{/privacy}","received_events_url":"https://api.github.com/users/jaberg/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/147/commits","review_comments_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/147/comments","review_comment_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/comments{/number}","comments_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/issues/147/comments","statuses_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/statuses/ffb7f35705522a628fb0b02fea785ebf6c2dbb83","head":{"label":"michaelpacer:2015","ref":"2015","sha":"ffb7f35705522a628fb0b02fea785ebf6c2dbb83","user":{"login":"michaelpacer","id":2482408,"avatar_url":"https://avatars.githubusercontent.com/u/2482408?v=3","gravatar_id":"","url":"https://api.github.com/users/michaelpacer","html_url":"https://github.com/michaelpacer","followers_url":"https://api.github.com/users/michaelpacer/followers","following_url":"https://api.github.com/users/michaelpacer/following{/other_user}","gists_url":"https://api.github.com/users/michaelpacer/gists{/gist_id}","starred_url":"https://api.github.com/users/michaelpacer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/michaelpacer/subscriptions","organizations_url":"https://api.github.com/users/michaelpacer/orgs","repos_url":"https://api.github.com/users/michaelpacer/repos","events_url":"https://api.github.com/users/michaelpacer/events{/privacy}","received_events_url":"https://api.github.com/users/michaelpacer/received_events","type":"User","site_admin":false},"repo":{"id":37351679,"name":"scipy_proceedings","full_name":"michaelpacer/scipy_proceedings","owner":{"login":"michaelpacer","id":2482408,"avatar_url":"https://avatars.githubusercontent.com/u/2482408?v=3","gravatar_id":"","url":"https://api.github.com/users/michaelpacer","html_url":"https://github.com/michaelpacer","followers_url":"https://api.github.com/users/michaelpacer/followers","following_url":"https://api.github.com/users/michaelpacer/following{/other_user}","gists_url":"https://api.github.com/users/michaelpacer/gists{/gist_id}","starred_url":"https://api.github.com/users/michaelpacer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/michaelpacer/subscriptions","organizations_url":"https://api.github.com/users/michaelpacer/orgs","repos_url":"https://api.github.com/users/michaelpacer/repos","events_url":"https://api.github.com/users/michaelpacer/events{/privacy}","received_events_url":"https://api.github.com/users/michaelpacer/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/michaelpacer/scipy_proceedings","description":"Tools used to generate the SciPy conference proceedings","fork":true,"url":"https://api.github.com/repos/michaelpacer/scipy_proceedings","forks_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/forks","keys_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/keys{/key_id}","collaborators_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/teams","hooks_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/hooks","issue_events_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/issues/events{/number}","events_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/events","assignees_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/assignees{/user}","branches_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/branches{/branch}","tags_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/tags","blobs_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/git/refs{/sha}","trees_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/git/trees{/sha}","statuses_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/statuses/{sha}","languages_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/languages","stargazers_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/stargazers","contributors_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/contributors","subscribers_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/subscribers","subscription_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/subscription","commits_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/commits{/sha}","git_commits_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/git/commits{/sha}","comments_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/comments{/number}","issue_comment_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/issues/comments{/number}","contents_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/contents/{+path}","compare_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/compare/{base}...{head}","merges_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/merges","archive_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/downloads","issues_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/issues{/number}","pulls_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/pulls{/number}","milestones_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/milestones{/number}","notifications_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/labels{/name}","releases_url":"https://api.github.com/repos/michaelpacer/scipy_proceedings/releases{/id}","created_at":"2015-06-13T00:49:36Z","updated_at":"2015-06-16T04:12:13Z","pushed_at":"2015-06-30T02:49:54Z","git_url":"git://github.com/michaelpacer/scipy_proceedings.git","ssh_url":"git@github.com:michaelpacer/scipy_proceedings.git","clone_url":"https://github.com/michaelpacer/scipy_proceedings.git","svn_url":"https://github.com/michaelpacer/scipy_proceedings","homepage":"","size":1346,"stargazers_count":0,"watchers_count":0,"language":"TeX","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"scipy-conference:2015","ref":"2015","sha":"4e7f0ee5967290cb3ad92468133949c518186a11","user":{"login":"scipy-conference","id":4884274,"avatar_url":"https://avatars.githubusercontent.com/u/4884274?v=3","gravatar_id":"","url":"https://api.github.com/users/scipy-conference","html_url":"https://github.com/scipy-conference","followers_url":"https://api.github.com/users/scipy-conference/followers","following_url":"https://api.github.com/users/scipy-conference/following{/other_user}","gists_url":"https://api.github.com/users/scipy-conference/gists{/gist_id}","starred_url":"https://api.github.com/users/scipy-conference/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scipy-conference/subscriptions","organizations_url":"https://api.github.com/users/scipy-conference/orgs","repos_url":"https://api.github.com/users/scipy-conference/repos","events_url":"https://api.github.com/users/scipy-conference/events{/privacy}","received_events_url":"https://api.github.com/users/scipy-conference/received_events","type":"Organization","site_admin":false},"repo":{"id":1825629,"name":"scipy_proceedings","full_name":"scipy-conference/scipy_proceedings","owner":{"login":"scipy-conference","id":4884274,"avatar_url":"https://avatars.githubusercontent.com/u/4884274?v=3","gravatar_id":"","url":"https://api.github.com/users/scipy-conference","html_url":"https://github.com/scipy-conference","followers_url":"https://api.github.com/users/scipy-conference/followers","following_url":"https://api.github.com/users/scipy-conference/following{/other_user}","gists_url":"https://api.github.com/users/scipy-conference/gists{/gist_id}","starred_url":"https://api.github.com/users/scipy-conference/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scipy-conference/subscriptions","organizations_url":"https://api.github.com/users/scipy-conference/orgs","repos_url":"https://api.github.com/users/scipy-conference/repos","events_url":"https://api.github.com/users/scipy-conference/events{/privacy}","received_events_url":"https://api.github.com/users/scipy-conference/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/scipy-conference/scipy_proceedings","description":"Tools used to generate the SciPy conference proceedings","fork":false,"url":"https://api.github.com/repos/scipy-conference/scipy_proceedings","forks_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/forks","keys_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/keys{/key_id}","collaborators_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/teams","hooks_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/hooks","issue_events_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/issues/events{/number}","events_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/events","assignees_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/assignees{/user}","branches_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/branches{/branch}","tags_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/tags","blobs_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/git/refs{/sha}","trees_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/git/trees{/sha}","statuses_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/statuses/{sha}","languages_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/languages","stargazers_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/stargazers","contributors_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/contributors","subscribers_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/subscribers","subscription_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/subscription","commits_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/commits{/sha}","git_commits_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/git/commits{/sha}","comments_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/comments{/number}","issue_comment_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/issues/comments{/number}","contents_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/contents/{+path}","compare_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/compare/{base}...{head}","merges_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/merges","archive_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/downloads","issues_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/issues{/number}","pulls_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls{/number}","milestones_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/milestones{/number}","notifications_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/labels{/name}","releases_url":"https://api.github.com/repos/scipy-conference/scipy_proceedings/releases{/id}","created_at":"2011-05-31T10:26:48Z","updated_at":"2015-06-22T07:22:47Z","pushed_at":"2015-06-30T19:29:23Z","git_url":"git://github.com/scipy-conference/scipy_proceedings.git","ssh_url":"git@github.com:scipy-conference/scipy_proceedings.git","clone_url":"https://github.com/scipy-conference/scipy_proceedings.git","svn_url":"https://github.com/scipy-conference/scipy_proceedings","homepage":"","size":155374,"stargazers_count":76,"watchers_count":76,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":142,"mirror_url":null,"open_issues_count":38,"forks":142,"open_issues":38,"watchers":76,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/147"},"html":{"href":"https://github.com/scipy-conference/scipy_proceedings/pull/147"},"issue":{"href":"https://api.github.com/repos/scipy-conference/scipy_proceedings/issues/147"},"comments":{"href":"https://api.github.com/repos/scipy-conference/scipy_proceedings/issues/147/comments"},"review_comments":{"href":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/147/comments"},"review_comment":{"href":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/scipy-conference/scipy_proceedings/pulls/147/commits"},"statuses":{"href":"https://api.github.com/repos/scipy-conference/scipy_proceedings/statuses/ffb7f35705522a628fb0b02fea785ebf6c2dbb83"}}}}