Bayesian networks in neuroscience: a survey
---
paper_title: Probabilistic Graphical Models: Principles and Techniques

paper_content:

Most tasks require a person or an automated system to reason -- to reach conclusions based on available information. The framework of probabilistic graphical models, presented in this book, provides a general approach for this task. The approach is model-based, allowing interpretable models to be constructed and then manipulated by reasoning algorithms. These models can also be learned automatically from data, allowing the approach to be used in cases where manually constructing a model is difficult or even impossible. Because uncertainty is an inescapable aspect of most real-world applications, the book focuses on probabilistic models, which make the uncertainty explicit and provide models that are more faithful to reality. Probabilistic Graphical Models discusses a variety of models, spanning Bayesian networks, undirected Markov networks, discrete and continuous models, and extensions to deal with dynamical systems and relational data. For each class of models, the text describes the three fundamental cornerstones: representation, inference, and learning, presenting both basic concepts and advanced techniques. Finally, the book considers the use of the proposed framework for causal reasoning and decision making under uncertainty. The main text in each chapter provides the detailed technical development of the key ideas. Most chapters also include boxes with additional material: skill boxes, which describe techniques; case study boxes, which discuss empirical cases related to the approach described in the text, including applications in computer vision, robotics, natural language understanding, and computational biology; and concept boxes, which present significant concepts drawn from the material in the chapter. Instructors (and readers) can group chapters in various combinations, from core topics to more technically advanced material, to suit their particular needs.
---
paper_title: Unsupervised training of Bayesian networks for data clustering

paper_content:

This paper presents a new approach to the unsupervised training of Bayesian network classifiers. Three models have been analysed: the Chow and Liu (CL) multinets; the tree-augmented naive Bayes; and a new model called the simple Bayesian network classifier, which is more robust in its structure learning. To perform the unsupervised training of these models, the classification maximum likelihood criterion is used. The maximization of this criterion is derived for each model under the classification expectation–maximization (EM) algorithm framework. To test the proposed unsupervised training approach, 10 well-known benchmark datasets have been used to measure their clustering performance. Also, for comparison, the results for the k -means and the EM algorithm, as well as those obtained when the three Bayesian network classifiers are trained in a supervised way, are analysed. A real-world image processing application is also presented, dealing with clustering of wood board images described by 165 attributes. Results show that the proposed learning method, in general, outperforms traditional clustering algorithms and, in the wood board image application, the CL multinets obtained a 12 per cent increase, on average, in clustering accuracy when compared with the k -means method and a 7 per cent increase, on average, when compared with the EM algorithm.
---
paper_title: Learning Bayesian networks: approaches and issues

paper_content:

Bayesian networks have become a widely used method in the modelling of uncertain knowledge. Owing to the difficulty domain experts have in specifying them, techniques that learn Bayesian networks from data have become indispensable. Recently, however, there have been many important new developments in this field. This work takes a broad look at the literature on learning Bayesian networks-in particular their structure-from data. Specific topics are not focused on in detail, but it is hoped that all the major fields in the area are covered. This article is not intended to be a tutorial-for this, there are many books on the topic, which will be presented. However, an effort has been made to locate all the relevant publications, so that this paper can be used as a ready reference to find the works on particular sub-topics.
---
paper_title: Probabilistic Graphical Models: Principles and Techniques

paper_content:

Most tasks require a person or an automated system to reason -- to reach conclusions based on available information. The framework of probabilistic graphical models, presented in this book, provides a general approach for this task. The approach is model-based, allowing interpretable models to be constructed and then manipulated by reasoning algorithms. These models can also be learned automatically from data, allowing the approach to be used in cases where manually constructing a model is difficult or even impossible. Because uncertainty is an inescapable aspect of most real-world applications, the book focuses on probabilistic models, which make the uncertainty explicit and provide models that are more faithful to reality. Probabilistic Graphical Models discusses a variety of models, spanning Bayesian networks, undirected Markov networks, discrete and continuous models, and extensions to deal with dynamical systems and relational data. For each class of models, the text describes the three fundamental cornerstones: representation, inference, and learning, presenting both basic concepts and advanced techniques. Finally, the book considers the use of the proposed framework for causal reasoning and decision making under uncertainty. The main text in each chapter provides the detailed technical development of the key ideas. Most chapters also include boxes with additional material: skill boxes, which describe techniques; case study boxes, which discuss empirical cases related to the approach described in the text, including applications in computer vision, robotics, natural language understanding, and computational biology; and concept boxes, which present significant concepts drawn from the material in the chapter. Instructors (and readers) can group chapters in various combinations, from core topics to more technically advanced material, to suit their particular needs.
---
paper_title: Discrete dynamic Bayesian network analysis of fMRI data

paper_content:

We examine the efficacy of using discrete Dynamic Bayesian Networks (dDBNs), a data-driven modeling technique employed in machine learning, to identify functional correlations among neuroanatomical regions of interest. Unlike many neuroimaging analysis techniques, this method is not limited by linear and/or Gaussian noise assumptions. It achieves this by modeling the time series of neuroanatomical regions as discrete, as opposed to continuous, random variables with multinomial distributions. We demonstrated this method using an fMRI dataset collected from healthy and demented elderly subjects and identify correlates based on a diagnosis of dementia. The results are validated in three ways. First, the elicited correlates are shown to be robust over leave-one-out cross-validation and, via a Fourier bootstrapping method, that they were not likely due to random chance. Second, the dDBNs identified correlates that would be expected given the experimental paradigm. Third, the dDBN's ability to predict dementia is competitive with two commonly employed machine-learning classifiers: the support vector machine and the Gaussian naive Bayesian network. We also verify that the dDBN selects correlates based on non-linear criteria. Finally, we provide a brief analysis of the correlates elicited from Buckner et al.'s data that suggests that demented elderly subjects have reduced involvement of entorhinal and occipital cortex and greater involvement of the parietal lobe and amygdala in brain activity compared with healthy elderly (as measured via functional correlations among BOLD measurements). Limitations and extensions to the dDBN method are discussed.
---
paper_title: Causality: Models, Reasoning, and Inference

paper_content:

1. Introduction to probabilities, graphs, and causal models 2. A theory of inferred causation 3. Causal diagrams and the identification of causal effects 4. Actions, plans, and direct effects 5. Causality and structural models in the social sciences 6. Simpson's paradox, confounding, and collapsibility 7. Structural and counterfactual models 8. Imperfect experiments: bounds and counterfactuals 9. Probability of causation: interpretation and identification Epilogue: the art and science of cause and effect.
---
paper_title: Learning Gaussian Networks

paper_content:

We describe scoring metrics for learning Bayesian networks from a combination of user knowledge and statistical data. Previous work has concentrated on metrics for domains containing only discrete variables, under the assumption that data represents a multinomial sample. In this paper, we extend this work, developing scoring metrics for domains containing only continuous variables under the assumption that continuous data is sampled from a multivariate normal distribution. Our work extends traditional statistical approaches for identifying vanishing regression coefficients in that we identify two important assumptions, called event equivalence and parameter modularity, that when combined allow the construction of prior distributions for multivariate normal parameters from a single prior Bayesian network specified by a user.
---
paper_title: Mixtures of Truncated Exponentials in Hybrid Bayesian Networks

paper_content:

In this paper we propose the use of mixtures of truncated exponential (MTE) distributions in hybrid Bayesian networks. We study the properties of the MTE distribution and show how exact probability propagation can be carried out by means of a local computation algorithm. One feature of this model is that no restriction is made about the order among the variables either discrete or continuous. Computations are performed over a representation of probabilistic potentials based on probability trees, expanded to allow discrete and continuous variables simultaneously. Finally, a Markov chain Monte Carlo algorithm is described with the aim of dealing with complex networks.
---
paper_title: Gaussian Influence Diagrams

paper_content:

An influence diagram is a network representation of probabilistic inference and decision analysis models. The nodes correspond to variables that can be either constants, uncertain quantities, decisions, or objectives. The arcs reveal probabilistic dependence of the uncertain quantities and information available at the time of the decisions. The influence diagram focuses attention on relationships among the variables. As a result, it is increasingly popular for eliciting and communicating the structure of a decision or probabilistic model. ::: ::: This paper develops the framework for assessment and analysis of linear-quadratic-Gaussian models within the influence diagram representation. The "Gaussian influence diagram" exploits conditional independence in a model to simplify elicitation of parameters for the multivariate normal distribution. It is straightforward to assess and maintain a positive semi-definite covariance matrix. Problems of inference and decision making can be analyzed using simple transformations to the assessed model, and these procedures have attractive numerical properties. Algorithms are also provided to translate between the Gaussian influence diagram and covariance matrix representations for the normal distribution.
---
paper_title: ✦ Mixtures of Truncated Basis Functions

paper_content:

In this paper we propose a framework, called mixtures of truncated basis functions (MoTBFs), for representing general hybrid Bayesian networks. The proposed framework generalizes both the mixture of truncated exponentials (MTEs) framework and the Mixture of Polynomials (MoPs) framework. Similar to MTEs and MoPs, MoTBFs are defined so that the potentials are closed under combination and marginalization, which ensures that inference in MoTBF networks can be performed efficiently using the Shafer-Shenoy architecture. Based on a generalized Fourier series approximation, we devise a method for efficiently approximating an arbitrary density function using the MoTBF framework. The translation method is more flexible than existing MTE or MoP-based methods, and it supports an online/anytime tradeoff between the accuracy and the complexity of the approximation. Experimental results show that the approximations obtained are either comparable or significantly better than the approximations obtained using existing methods.
---
paper_title: Inference in hybrid Bayesian networks using mixtures of polynomials

paper_content:

The main goal of this paper is to describe inference in hybrid Bayesian networks (BNs) using mixture of polynomials (MOP) approximations of probability density functions (PDFs). Hybrid BNs contain a mix of discrete, continuous, and conditionally deterministic random variables. The conditionals for continuous variables are typically described by conditional PDFs. A major hurdle in making inference in hybrid BNs is marginalization of continuous variables, which involves integrating combinations of conditional PDFs. In this paper, we suggest the use of MOP approximations of PDFs, which are similar in spirit to using mixtures of truncated exponentials (MTEs) approximations. MOP functions can be easily integrated, and are closed under combination and marginalization. This enables us to propagate MOP potentials in the extended Shenoy-Shafer architecture for inference in hybrid BNs that can include deterministic variables. MOP approximations have several advantages over MTE approximations of PDFs. They are easier to find, even for multi-dimensional conditional PDFs, and are applicable for a larger class of deterministic functions in hybrid BNs.
---
paper_title: A Model for Reasoning About Persistence and Causation

paper_content:

In this paper, we describe a model of causal reasoning that accounts for knowledge concerning cause-and-effect relationships and knowledge concerning the tendency for propositions to persist or not as a function of time passing. Our model has a natural encoding in the form of a network representation for probabilistic models. We explore the computational properties of our model by considering recent advances in computing the consequences of models encoded in this network representation.
---
paper_title: The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks

paper_content:

Abstract Bayesian belief networks provide a natural, efficient method for representing probabilistic dependencies among a set of variables. For these reasons, numerous researchers are exploring the use of belief networks as a knowledge representation in artificial intelligence. Algorithms have been developed previously for efficient probabilistic inference using special classes of belief networks. More general classes of belief networks, however, have eluded efforts to develop efficient inference algorithms. We show that probabilistic inference using belief networks is NP-hard. Therefore, it seems unlikely that an exact algorithm can be developed to perform probabilistic inference efficiently over all classes of belief networks. This result suggests that research should be directed away from the search for a general, efficient probabilistic inference algorithm, and toward the design of efficient special-case, average-case, and approximation algorithms.
---
paper_title: Approximate probability propagation with mixtures of truncated exponentials

paper_content:

Mixtures of truncated exponentials (MTEs) are a powerful alternative to discretisation when working with hybrid Bayesian networks. One of the features of the MTE model is that standard propagation algorithms can be used. However, the complexity of the process is too high and therefore approximate methods, which tradeoff complexity for accuracy, become necessary. In this paper we propose an approximate propagation algorithm for MTE networks which is based on the Penniless propagation method already known for discrete variables. We also consider how to use Markov Chain Monte Carlo to carry out the probability propagation. The performance of the proposed methods is analysed in a series of experiments with random networks.
---
paper_title: Local Propagation in Conditional Gaussian Bayesian Networks

paper_content:

This paper describes a scheme for local computation in conditional Gaussian Bayesian networks that combines the approach of Lauritzen and Jensen (2001) with some elements of Shachter and Kenley (1989). Message passing takes place on an elimination tree structure rather than the more compact (and usual) junction tree of cliques. This yields a local computation scheme in which all calculations involving the continuous variables are performed by manipulating univariate regressions, and hence matrix operations are avoided.
---
paper_title: Propagating Uncertainty in Bayesian Networks by Probabilistic Logic Sampling

paper_content:

Bayesian belief networks and influence diagrams are attractive approaches for representing uncertain expert knowledge in coherent probabilistic form. But current algorithms for propagating updates are either restricted to singly connected networks (Chow trees), as the scheme of Pearl and Kim, or they are liable to exponential complexity when dealing with multiply connected networks. Probabilistic logic sampling is a new scheme employing stochastic simulation which can make probabilistic inferences in large, multiply connected networks, with an arbitrary degree of precision controlled by the sample size. A prototype implementation, named Pulse, is illustrated, which provides efficient methods to estimate conditional probabilities, perform systematic sensitivity analysis, and compute evidence weights to explain inferences.
---
paper_title: The EM algorithm for graphical association models with missing data

paper_content:

Abstract It is shown how the computational scheme of Lauritzen and Spiegelhalter (1988) can be exploited to perform the E-step of the EM algorithm when applied to finding maximum likelihood estimates or penalized maximum likelihood estimates in hierarchical log-linear models and recursive models for contingency tables with missing data. The generalization to mixed association models introduced in Lauritzen and Wermuth (1989) and Edwards (1990) is indicated.
---
paper_title: Local computations with probabilities on graphical structures and their application to expert systems

paper_content:

The invention comprises pigments of the general Formula I in which each X is hydrogen, chlorine or bromine, A is (a) an aliphatic hydrocarbon radical, (b) the phenylene group or (c) the naphthylene group and R is identical with the 1: 2-phthaloylcarbazole-aminocarbonyl radical shown in the above formula when A has the meaning defined under (a), and R is alkoxy, alkylamino, alkoylamino, trifluoroalkyl, alkylsulphonyl or fluoro when A has the meaning defined under (b) or (c), and in which general formula the group R-A-CO-NH- is in the 3- to 9-position. The invention further comprises a process for making pigments of the above Formula I in which each X is hydrogen, chloride, bromine or alkyl and (i) A is an aliphatic hydrocarbon radical and R is the 1: 2-phthaloylcarbazole-aminocarbonyl radical shown in the formula or (ii) A is an aromatic hydrocarbon radical and R is hydrogen, alkyl, alkoxy, alkylamino, alkoylamino, trifluoralkyl, alkylsulphonyl, halogen or the 1: 2-phthaloylcarbazoleaminocarbonyl radical, R-A-CO-NH- being linked in the 3- or 9-position, wherein an amino-1: 2-phthaloylcarbazole of the above Formula I in which the group R-A-CO- is replaced by hydrogen, is acylated with an appropriate aliphatic or aromatic dicarboxylic acid or an appropriate substituted or unsubstituted aromatic mono-carboxylic acid. The products are pigments for lacquers, printing pastes and plastics such as polyvinyl chloride.
---
paper_title: Approximating Probabilistic Inference in Bayesian Belief Networks is NP-Hard

paper_content:

Abstract It is known that exact computation of conditional probabilities in belief networks is NP -hard. Many investigators in the AI community have tacitly assumed that algorithms for performing approximate inference with belief networks are of polynomial complexity. Indeed, special cases of approximate inference can be performed in time polynomial in the input size. However, we have discovered that the general problem of approximating conditional probabilities with belief networks, like exact inference, resides in the NP -hard complexity class. We develop a complexity analysis to elucidate the difficulty of approximate probabilistic inference. More specifically, we show that the existence of a polynomial-time relative approximation algorithm for major classes of problem instances implies that NP ⊆ P. We present our proof and explore the implications of the result.
---
paper_title: Inference in hybrid Bayesian networks with mixtures of truncated exponentials

paper_content:

Mixtures of truncated exponentials (MTE) potentials are an alternative to discretization for solving hybrid Bayesian networks. Any probability density function (PDF) can be approximated with an MTE potential, which can always be marginalized in closed form. This allows propagation to be done exactly using the Shenoy-Shafer architecture for computing marginals, with no restrictions on the construction of a join tree. This paper presents MTE potentials that approximate an arbitrary normal PDF with any mean and a positive variance. The properties of these MTE potentials are presented, along with examples that demonstrate their use in solving hybrid Bayesian networks. Assuming that the joint density exists, MTE potentials can be used for inference in hybrid Bayesian networks that do not fit the restrictive assumptions of the conditional linear Gaussian (CLG) model, such as networks containing discrete nodes with continuous parents.
---
paper_title: Inference in hybrid Bayesian networks using mixtures of polynomials

paper_content:

The main goal of this paper is to describe inference in hybrid Bayesian networks (BNs) using mixture of polynomials (MOP) approximations of probability density functions (PDFs). Hybrid BNs contain a mix of discrete, continuous, and conditionally deterministic random variables. The conditionals for continuous variables are typically described by conditional PDFs. A major hurdle in making inference in hybrid BNs is marginalization of continuous variables, which involves integrating combinations of conditional PDFs. In this paper, we suggest the use of MOP approximations of PDFs, which are similar in spirit to using mixtures of truncated exponentials (MTEs) approximations. MOP functions can be easily integrated, and are closed under combination and marginalization. This enables us to propagate MOP potentials in the extended Shenoy-Shafer architecture for inference in hybrid BNs that can include deterministic variables. MOP approximations have several advantages over MTE approximations of PDFs. They are easier to find, even for multi-dimensional conditional PDFs, and are applicable for a larger class of deterministic functions in hybrid BNs.
---
paper_title: The EM algorithm for graphical association models with missing data

paper_content:

Abstract It is shown how the computational scheme of Lauritzen and Spiegelhalter (1988) can be exploited to perform the E-step of the EM algorithm when applied to finding maximum likelihood estimates or penalized maximum likelihood estimates in hierarchical log-linear models and recursive models for contingency tables with missing data. The generalization to mixed association models introduced in Lauritzen and Wermuth (1989) and Edwards (1990) is indicated.
---
paper_title: Sequential updating of conditional probabilities on directed graphical structures

paper_content:

A directed acyclic graph or influence diagram is frequently used as a representation for qualitative knowledge in some domains in which expert system techniques have been applied, and conditional probability tables on appropriate sets of variables form the quantitative part of the accumulated experience. It is shown how one can introduce imprecision into such probabilities as a data base of cases accumulates. By exploiting the graphical structure, the updating can be performed locally, either approximately or exactly, and the setup makes it possible to take advantage of a range of well-established statistical techniques. As examples we discuss discrete models, models based on Dirichlet distributions and models of the logistic regression type.
---
paper_title: The Recovery of Causal Poly-Trees from Statistical Data

paper_content:

Poly-trees are singly connected causal networks in which variables may arise from multiple causes. This paper develops a method of recovering ply-trees from empirically measured probability distributions of pairs of variables. The method guarantees that, if the measured distributions are generated by a causal process structured as a ply-tree then the topological structure of such tree can be recovered precisely and, in addition, the causal directionality of the branches can be determined up to the maximum extent possible. The method also pinpoints the minimum (if any) external semantics required to determine the causal relationships among the variables considered.
---
paper_title: Bayesian Network Induction via Local Neighborhoods

paper_content:

In recent years, Bayesian networks have become highly successful tool for diagnosis, analysis, and decision making in real-world domains. We present an efficient algorithm for learning Bayes networks from data. Our approach constructs Bayesian networks by first identifying each node's Markov blankets, then connecting nodes in a maximally consistent way. In contrast to the majority of work, which typically uses hill-climbing approaches that may produce dense and causally incorrect nets, our approach yields much more compact causal networks by heeding independencies in the data. Compact causal networks facilitate fast inference and are also easier to understand. We prove that under mild assumptions, our approach requires time polynomial in the size of the data and the number of nodes. A randomized variant, also presented here, yields comparable results at much higher speeds.
---
paper_title: An Algorithm for Fast Recovery of Sparse Causal Graphs

paper_content:

Previous asymptotically correct algorithms for recovering causal structure from sample probabilities have been limited even in sparse causal graphs to a few variables. We describe an asymptotically...
---
paper_title: Controlling the False Discovery Rate of the Association/Causality Structure Learned with the PC Algorithm

paper_content:

In real world applications, graphical statistical models are not only a tool for operations such as classification or prediction, but usually the network structures of the models themselves are also of great interest (e.g., in modeling brain connectivity). The false discovery rate (FDR), the expected ratio of falsely claimed connections to all those claimed, is often a reasonable error-rate criterion in these applications. However, current learning algorithms for graphical models have not been adequately adapted to the concerns of the FDR. The traditional practice of controlling the type I error rate and the type II error rate under a conventional level does not necessarily keep the FDR low, especially in the case of sparse networks. In this paper, we propose embedding an FDR-control procedure into the PC algorithm to curb the FDR of the skeleton of the learned graph. We prove that the proposed method can control the FDR under a user-specified level at the limit of large sample sizes. In the cases of moderate sample size (about several hundred), empirical experiments show that the method is still able to control the FDR under the user-specified level, and a heuristic modification of the method is able to control the FDR more accurately around the user-specified level. The proposed method is applicable to any models for which statistical tests of conditional independence are available, such as discrete models and Gaussian models.
---
paper_title: Estimating high-dimensional directed acyclic graphs with the PC-algorithm

paper_content:

We consider the PC-algorithm Spirtes et. al. (2000) for estimating the skeleton of a very high-dimensional acyclic directed graph (DAG) with corresponding Gaussian distribution. The PC-algorithm is computationally feasible for sparse problems with many nodes, i.e. variables, and it has the attractive property to automatically achieve high computational efficiency as a function of sparseness of the true underlying DAG. We prove consistency of the algorithm for very high-dimensional, sparse DAGs where the number of nodes is allowed to quickly grow with sample size n, as fast as O(n^a) for any 0<a<infinity. The sparseness assumption is rather minimal requiring only that the neighborhoods in the DAG are of lower order than sample size n. We empirically demonstrate the PC-algorithm for simulated data and argue that the algorithm is rather insensitive to the choice of its single tuning parameter.
---
paper_title: A new look at the statistical model identification

paper_content:

The history of the development of statistical hypothesis testing in time series analysis is reviewed briefly and it is pointed out that the hypothesis testing procedure is not adequately defined as the procedure for statistical model identification. The classical maximum likelihood estimation procedure is reviewed and a new estimate minimum information theoretical criterion (AIC) estimate (MAICE) which is designed for the purpose of statistical identification is introduced. When there are several competing models the MAICE is defined by the model and the maximum likelihood estimates of the parameters which give the minimum of AIC defined by AIC = (-2)log-(maximum likelihood) + 2(number of independently adjusted parameters within the model). MAICE provides a versatile procedure for statistical model identification which is free from the ambiguities inherent in the application of conventional hypothesis testing procedure. The practical utility of MAICE in time series analysis is demonstrated with some numerical examples.
---
paper_title: Learning Bayesian Networks is NP-Complete

paper_content:

Algorithms for learning Bayesian networks from data have two components: a scoring metric and a search procedure. The scoring metric computes a score reflecting the goodness-of-fit of the structure to the data. The search procedure tries to identify network structures with high scores. Heckerman et al. (1995) introduce a Bayesian metric, called the BDe metric, that computes the relative posterior probability of a network structure given data. In this paper, we show that the search problem of identifying a Bayesian network—among those where each node has at most K parents—that has a relative posterior probability greater than a given constant is NP-complete, when the BDe metric is used.
---
paper_title: Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks

paper_content:

In many multivariate domains, we are interested in analyzing the dependency structure of the underlying distribution, e.g., whether two variables are in direct interaction. We can represent dependency structures using Bayesian network models. To analyze a given data set, Bayesian model selection attempts to find the most likely (MAP) model, and uses its structure to answer these questions. However, when the amount of available data is modest, there might be many models that have non-negligible posterior. Thus, we want compute the Bayesian posterior of a feature, i.e., the total posterior probability of all models that contain it. In this paper, we propose a new approach for this task. We first show how to efficiently compute a sum over the exponential number of networks that are consistent with a fixed order over network variables. This allows us to compute, for a given order, both the marginal probability of the data and the posterior of a feature. We then use this result as the basis for an algorithm that approximates the Bayesian posterior of a feature. Our approach uses a Markov Chain Monte Carlo (MCMC) method, but over orders rather than over network structures. The space of orders is smaller and more regular than the space of structures, and has much a smoother posterior “landscape”. We present empirical results on synthetic and real-life datasets that compare our approach to full model averaging (when possible), to MCMC over network structures, and to a non-Bayesian bootstrap approach.
---
paper_title: Theory Refinement on Bayesian Networks

paper_content:

Theory refinement is the task of updating a domain theory in the light of new cases, to be done automatically or with some expert assistance. The problem of theory refinement under uncertainty is reviewed here in the context of Bayesian statistics, a theory of belief revision. The problem is reduced to an incremental learning task as follows: the learning system is initially primed with a partial theory supplied by a domain expert, and thereafter maintains its own internal representation of alternative theories which is able to be interrogated by the domain expert and able to be incrementally refined from data. Algorithms for refinement of Bayesian networks are presented to illustrate what is meant by "partial theory", "alternative theory representation", etc. The algorithms are an incremental variant of batch learning algorithms from the literature so can work well in batch and incremental mode.
---
paper_title: Counting unlabeled acyclic digraphs

paper_content:

The previously known ways to count acyclic digraphs, both labeled and unlabeled, are reviewed. Then a new method of enumerating unlabeled acyclic digraphs is developed. It involves computing the sum of the cyclic indices of the automorphism groups of the acyclic digraphs, achieving a considerable gain in efficiency through an application of the inclusion-exclusion principle. Numerical results are reported on, and a table of the numbers of unlabeled acyclic digraphs on up to 18 points is included.
---
paper_title: On Local Optima in Learning Bayesian Networks

paper_content:

This paper proposes and evaluates the k-greedy equivalence search algorithm (KES) for learning Bayesian networks (BNs) from complete data. The main characteristic of KES is that it allows a trade-off between greediness and randomness, thus exploring different good local optima when run repeatedly. When greediness is set at maximum, KES corresponds to the greedy equivalence search algorithm (GES). When greediness is kept at minimum, we prove that under mild conditions KES asymptotically returns any inclusion optimal BN with nonzero probability. Experimental results for both synthetic and real data are reported showing that KES finds a better local optimum than GES considerably often. Additionally, these results illustrate that the number of different local optima is usually huge.
---
paper_title: Learning Gaussian Networks

paper_content:

We describe scoring metrics for learning Bayesian networks from a combination of user knowledge and statistical data. Previous work has concentrated on metrics for domains containing only discrete variables, under the assumption that data represents a multinomial sample. In this paper, we extend this work, developing scoring metrics for domains containing only continuous variables under the assumption that continuous data is sampled from a multivariate normal distribution. Our work extends traditional statistical approaches for identifying vanishing regression coefficients in that we identify two important assumptions, called event equivalence and parameter modularity, that when combined allow the construction of prior distributions for multivariate normal parameters from a single prior Bayesian network specified by a user.
---
paper_title: Optimizing Causal Orderings for Generating DAGs from Data

paper_content:

An algorithm for generating the structure of a directed acyclic graph from data using the notion of causal input lists is presented. The algorithm manipulates the ordering of the variables with operations which very much resemble arc reversal. Operations are only applied if the DAG after the operation represents at least the independencies represented by the DAG before the operation until no more arcs can be removed from the DAG. The resulting DAG is a minimal I-map.
---
paper_title: Learning an L1-Regularized Gaussian Bayesian Network in the Equivalence Class Space

paper_content:

Learning the structure of a graphical model from data is a common task in a wide range of practical applications. In this paper, we focus on Gaussian Bayesian networks, i.e., on continuous data and directed acyclic graphs with a joint probability density of all variables given by a Gaussian. We propose to work in an equivalence class search space, specifically using the k-greedy equivalence search algorithm. This, combined with regularization techniques to guide the structure search, can learn sparse networks close to the one that generated the data. We provide results on some synthetic networks and on modeling the gene network of the two biological pathways regulating the biosynthesis of isoprenoids for the Arabidopsis thaliana plant.
---
paper_title: Learning Bayesian networks in the space of structures by estimation of distribution algorithms

paper_content:

The induction of the optimal Bayesian network structure is NP-hard, justifying the use of search heuristics. Two novel population-based stochastic search approaches, univariate marginal distribution algorithm (UMDA) and population-based incremental learning (PBIL), are used to learn a Bayesian network structure from a database of cases in a score + search framework. A comparison with a genetic algorithm (GA) approach is performed using three different scores: penalized maximum likelihood, marginal likelihood, and information-theory-based entropy. Experimental results show the interesting capabilities of both novel approaches with respect to the score value and the number of generations needed to converge.
---
paper_title: A Bayesian Method for the Induction of Probabilistic Networks from Data

paper_content:

This paper presents a Bayesian method for constructing probabilistic networks from databases. In particular, we focus on constructing Bayesian belief networks. Potential applications include computer-assisted hypothesis testing, automated scientific discovery, and automated construction of probabilistic expert systems. We extend the basic method to handle missing data and hidden (latent) variables. We show how to perform probabilistic inference by averaging over the inferences of multiple belief networks. Results are presented of a preliminary evaluation of an algorithm for constructing a belief network from a database of cases. Finally, we relate the methods in this paper to previous work, and we discuss open problems.
---
paper_title: Learning Equivalence Classes of Bayesian Networks Structures

paper_content:

Approaches to learning Bayesian networks from data typically combine a scoring function with a heuristic search procedure. Given a Bayesian network structure, many of the scoring functions derived in the literature return a score for the entire equivalence class to which the structure belongs. When using such a scoring function, it is appropriate for the heuristic search algorithm to search over equivalence classes of Bayesian networks as opposed to individual structures. We present the general formulation of a search space for which the states of the search correspond to equivalence classes of structures. Using this space, any one of a number of heuristic search algorithms can easily be applied. We compare greedy search performance in the proposed search space to greedy search performance in a search space for which the states correspond to individual Bayesian network structures.
---
paper_title: An Algorithm for the Construction of Bayesian Network Structures from Data

paper_content:

Previous algorithms for the construction of Bayesian belief network structures from data have been either highly dependent on conditional independence (CI) tests, or have required an ordering on the nodes to be supplied by the user. We present an algorithm that integrates these two approaches - CI tests are used to generate an ordering on the nodes from the database which is then used to recover the underlying Bayesian network structure using a non CI based method. Results of preliminary evaluation of the algorithm on two networks (ALARM and LED) are presented. We also discuss some algorithm performance issues and open problems.
---
paper_title: The Size Distribution for Markov Equivalence Classes of Acyclic Digraph Models

paper_content:

Bayesian networks, equivalently graphical Markov models determined by acyclic digraphs or ADGs (also called directed acyclic graphs or dags), have proved to be both effective and efficient for representing complex multivariate dependence structures in terms of local relations. However, model search and selection is potentially complicated by the many-to-one correspondence between ADGs and the statistical models that they represent. If the ADGs/models ratio is large, search procedures based on unique graphical representations of equivalence classes of ADGs could provide substantial computational efficiency. Hitherto, the value of the ADGs/models ratio has been calculated only for graphs with n = 5 or fewer vertices. In the present study, a computer program was written to enumerate the equivalence classes of ADG models and study the distributions of class sizes and number of edges for graphs up to n = 10 vertices. The ratio of ADGs to numbers of classes appears to approach an asymptote of about 3.7. Distributions of the classes according to number of edges and class size were produced which also appear to be approaching asymptotic limits. Imposing a bound on the maximum number of parents to any vertex causes little change if the bound is sufficiently large, with four being a possible minimum. The program also includes a new variation of orderly algorithm for generating undirected graphs.
---
paper_title: LEARNING BAYESIAN NETWORKS IN THE SPACE OF ORDERINGS WITH ESTIMATION OF DISTRIBUTION ALGORITHMS

paper_content:

The search for the optimal ordering of a set of variables in order to solve a computational problem is a dicult y that can appear in several circumstances. One of these situations is the automatic learning of a network structure, for example, a Bayesian Network structure (BN) starting from a dataset. Searching in the space of structures is often unmanageable, especially if the number of variables is high. Popular heuristic approaches, like Cooper and Herskovits’s K2 algorithm, depend on a given ordering of variables. Estimation of Distribution Algorithms (EDAs) are a new paradigm for Evolutionary Computation that have been used as a search engine in the BN structure learning problem. In this paper, we will use two dieren t EDAs to obtain not the best structure, but the optimal ordering of variables for the K2 algorithm: UMDA and MIMIC, both of them in discrete and continuous domains. We will also check whether the individual representation and its relation to the corresponding ordering play important roles, and whether MIMIC outperforms the results of UMDA.
---
paper_title: Learning Bayesian networks: the combination of knowledge and statistical data

paper_content:

We describe scoring metrics for learning Bayesian networks from a combination of user knowledge and statistical data. We identify two important properties of metrics, which we call event equivalence and parameter modularity. These properties have been mostly ignored, but when combined, greatly simplify the encoding of a user's prior knowledge. In particular, a user can express his knowledge--for the most part--as a single prior Bayesian network for the domain.
---
paper_title: Learning Bayesian Network Structures by Searching For the Best Ordering With Genetic Algorithms

paper_content:

Presents a new methodology for inducing Bayesian network structures from a database of cases. The methodology is based on searching for the best ordering of the system variables by means of genetic algorithms. Since this problem of finding an optimal ordering of variables resembles the traveling salesman problem, the authors use genetic operators that were developed for the latter problem. The quality of a variable ordering is evaluated with the structure-learning algorithm K2. The authors present empirical results that were obtained with a simulation of the ALARM network.
---
paper_title: Machine Learning: A Probabilistic Perspective

paper_content:

Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.
---
paper_title: Evaluating Learning Algorithms: A Classification Perspective

paper_content:

The field of machine learning has matured to the point where many sophisticated learning approaches can be applied to practical applications. Thus it is of critical importance that researchers have the proper tools to evaluate learning approaches and understand the underlying issues. This book examines various aspects of the evaluation process with an emphasis on classification algorithms. The authors describe several techniques for classifier performance assessment, error estimation and resampling, obtaining statistical significance as well as selecting appropriate domains for evaluation. They also present a unified evaluation framework and highlight how different components of evaluation are both significantly interrelated and interdependent. The techniques presented in the book are illustrated using R and WEKA facilitating better practical insight as well as implementation.Aimed at researchers in the theory and applications of machine learning, this book offers a solid basis for conducting performance evaluations of algorithms in practical settings.
---
paper_title: Knowledge representation and inference in similarity networks and Bayesian multinets

paper_content:

We examine two representation schemes for uncertain knowledge: the similarity network (Heckerman, 1991) and the Bayesian multinet. These schemes are extensions of the Bayesian network model in that they represent asymmetric independence assertions. We explicate the notion of relevance upon which similarity networks are based and present an efficient inference algorithm that works under the assumption that every event has a nonzero probability. Another inference algorithm is developed that works under no restriction albeit less efficiently. We show that similarity networks are not inferentially complete-namely-not every query can be answered. Nonetheless, we show that a similarity network can always answer any query of the form: “What is the posterior probability of an hypothesis given evidence?” We call this property diagnostic completeIZESS. Finally, we describe a generalization of similarity networks that can encode more types of asymmetric conditional independence assertions than can ordinary similarity networks.
---
paper_title: Toward Optimal Feature Selection

paper_content:

In this paper, we examine a method for feature subset selection based on Information Theory. Initially, a framework for defining the theoretically optimal, but computationally intractable, method for feature subset selection is presented. We show that our goal should be to eliminate a feature if it gives us little or no additional information beyond that subsumed by the remaining features. In particular, this will be the case for both irrelevant and redundant features. We then give an efficient algorithm for feature selection which computes an approximation to the optimal feature selection criterion. The conditions under which the approximate algorithm is successful are examined. Empirical results are given on a number of data sets, showing that the algorithm effectively handles datasets with a very large number of features.
---
paper_title: Constructing Bayesian networks to predict uncollectible telecommunications accounts

paper_content:

The complexities of building models that can predict whether a customer account or transaction is collectible are greater than most current learning systems can handle. The authors describe software that builds Bayesian network models for such predictions. They also examine how varying model parameters and hence model structure can affect predictive accuracy.
---
paper_title: Model Averaging for Prediction with Discrete Bayesian Networks

paper_content:

In this paper we consider the problem of performing Bayesian model-averaging over a class of discrete Bayesian network structures consistent with a partial ordering and with bounded in-degree k. We show that for N nodes this class contains in the worst-case at least distinct network structures, and yet model averaging over these structures can be performed using operations. Furthermore we show that there exists a single Bayesian network that defines a joint distribution over the variables that is equivalent to model averaging over these structures. Although constructing this network is computationally prohibitive, we show that it can be approximated by a tractable network, allowing approximate model-averaged probability calculations to be performed in O(N) time. Our result also leads to an exact and linear-time solution to the problem of averaging over the 2N possible feature sets in a naive Bayes model, providing an exact Bayesian solution to the troublesome feature-selection problem for naive Bayes classifiers. We demonstrate the utility of these techniques in the context of supervised classification, showing empirically that model averaging consistently beats other generative Bayesian-network-based models, even when the generating model is not guaranteed to be a member of the class being averaged over. We characterize the performance over several parameters on simulated and real-world data.
---
paper_title: On the shortest spanning subtree of a graph and the traveling salesman problem

paper_content:

7. A. Kurosh, Ringtheoretische Probleme die mit dem Burnsideschen Problem uber periodische Gruppen in Zussammenhang stehen, Bull. Acad. Sei. URSS, Ser. Math. vol. 5 (1941) pp. 233-240. 8. J. Levitzki, On the radical of a general ring, Bull. Amer. Math. Soc. vol. 49 (1943) pp. 462^66. 9. -, On three problems concerning nil rings, Bull. Amer. Math. Soc. vol. 49 (1943) pp. 913-919. 10. -, On the structure of algebraic algebras and related rings, Trans. Amer. Math. Soc. vol. 74 (1953) pp. 384-409.
---
paper_title: Learning Limited Dependence Bayesian Classifiers

paper_content:

We present a framework for characterizing Bayesian classification methods. This framework can be thought of as a spectrum of allowable dependence in a given probabilistic model with the Naive Bayes algorithm at the most restrictive end and the learning of full Bayesian networks at the most general extreme. While much work has been carried out along the two ends of this spectrum, there has been surprising little done along the middle. We analyze the assumptions made as one moves along this spectrum and show the tradeoffs between model accuracy and learning speed which become critical to consider in a variety of data mining domains. We then present a general induction algorithm that allows for traversal of this spectrum depending on the available computational power for carrying out induction and show its application in a number of domains with different properties.
---
paper_title: Induction of Selective Bayesian Classifiers

paper_content:

In this paper, we examine previous work on the naive Bayesian classifier and review its limitations, which include a sensitivity to correlated features. We respond to this problem by embedding the naive Bayesian induction scheme within an algorithm that carries out a greedy search through the space of features. We hypothesize that this approach will improve asymptotic accuracy in domains that involve correlated features without reducing the rate of learning in ones that do not. We report experimental results on six natural domains, including comparisons with decision-tree induction, that support these hypotheses. In closing, we discuss other approaches to extending naive Bayesian classifiers and outline some directions for future research.
---
paper_title: On Relevance, Probabilistic Indexing and Information Retrieval

paper_content:

This paper reports on a novel technique for literature indexing and searching in a mechanized library system. The notion of relevance is taken as the key concept in the theory of information retrieval and a comparative concept of relevance is explicated in terms of the theory of probability. The resulting technique called “Probabilistic Indexing,” allows a computing machine, given a request for information, to make a statistical inference and derive a number (called the “relevance number”) for each document, which is a measure of the probability that the document will satisfy the given request. The result of a search is an ordered list of those documents which satisfy the request ranked according to their probable relevance. The paper goes on to show that whereas in a conventional library system the cross-referencing (“see” and “see also”) is based solely on the “semantical closeness” between index terms, statistical measures of closeness between index terms can be defined and computed. Thus, given an arbitrary request consisting of one (or many) index term(s), a machine can elaborate on it to increase the probability of selecting relevant documents that would not otherwise have been selected. Finally, the paper suggests an interpretation of the whole library problem as one where the request is considered as a clue on the basis of which the library system makes a concatenated statistical inference in order to provide as an output an ordered list of those documents which most probably satisfy the information needs of the user.
---
paper_title: Learning the Structure of Augmented Bayesian Classifiers

paper_content:

The naive Bayes classifier is built on the assumption of conditional independence between the attributes given the class. The algorithm has been shown to be surprisingly robust to obvious violations of this condition, but is is natural to ask if it is possible to further improve the accuracy by relaxing this assumption. We examine an approach where naive Bayes is augmented by the addition of correlation arcs between attributes. We explore two methods for finding the set of augmenting arcs, a greedy hill-climbing search, and a novel, more computationally efficient algorithm that we call SuperParent. We compare these methods to TAN; a state-of the-art distribution-based approach to finding the augmenting arcs.
---
paper_title: Constructive Induction of Cartesian Product Attributes

paper_content:

Constructive induction is the process of changing the representation of examples by creating new attributes from existing attributes. In classification, the goal of constructive induction is to find a representation that facilitates learning a concept description by a particular learning system. Typically, the new attributes are Boolean or arithmetic combinations of existing attributes and the learning algorithms used are decision trees or rule learners. We describe the construction of new attributes that are the Cartesian product of existing attributes. We consider the effects of this operator on three learning algorithms and compare two different methods for determining when to construct new attributes with this operator
---
paper_title: Bayesian classifiers based on kernel density estimation: Flexible classifiers

paper_content:

When learning Bayesian network based classifiers continuous variables are usually handled by discretization, or assumed that they follow a Gaussian distribution. This work introduces the kernel based Bayesian network paradigm for supervised classification. This paradigm is a Bayesian network which estimates the true density of the continuous variables using kernels. Besides, tree-augmented naive Bayes, k-dependence Bayesian classifier and complete graph classifier are adapted to the novel kernel based Bayesian network paradigm. Moreover, the strong consistency properties of the presented classifiers are proved and an estimator of the mutual information based on kernels is presented. The classifiers presented in this work can be seen as the natural extension of the flexible naive Bayes classifier proposed by John and Langley [G.H. John, P. Langley, Estimating continuous distributions in Bayesian classifiers, in: Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, 1995, pp. 338-345], breaking with its strong independence assumption. Flexible tree-augmented naive Bayes seems to have superior behavior for supervised classification among the flexible classifiers. Besides, flexible classifiers presented have obtained competitive errors compared with the state-of-the-art classifiers.
---
paper_title: Estimating Continuous Distributions in Bayesian Classifiers

paper_content:

When modeling a probability distribution with a Bayesian network, we are faced with the problem of how to handle continuous variables. Most previous work has either solved the problem by discretizing, or assumed that the data are generated by a single Gaussian. In this paper we abandon the normality assumption and instead use statistical methods for nonparametric density estimation. For a naive Bayesian classifier, we present experimental results on a variety of natural and artificial domains, comparing two methods of density estimation: assuming normality and modeling each conditional distribution with a single Gaussian; and using nonparametric kernel density estimation. We observe large reductions in error on several natural and artificial data sets, which suggests that kernel estimation is a useful tool for learning Bayesian models.
---
paper_title: The Bayesian Structural EM Algorithm

paper_content:

In recent years there has been a flurry of works on learning Bayesian networks from data. One of the hard problems in this area is how to effectively learn the structure of a belief network from incomplete data--that is, in the presence of missing values or hidden variables. In a recent paper, I introduced an algorithm called Structural EM that combines the standard Expectation Maximization (EM) algorithm, which optimizes parameters, with structure search for model selection. That algorithm learns networks based on penalized likelihood scores, which include the BIC/MDL score and various approximations to the Bayesian score. In this paper, I extend Structural EM to deal directly with Bayesian model selection. I prove the convergence of the resulting algorithm and show how to apply it for learning a large class of probabilistic models, including Bayesian networks and some variants thereof.
---
paper_title: Learning Bayesian networks for clustering by means of constructive induction

paper_content:

Abstract The purpose of this paper is to present and evaluate a heuristic algorithm for learning Bayesian networks for clustering. Our approach is based upon improving the Naive-Bayes model by means of constructive induction. A key idea in this approach is to treat expected data as real data. This allows us to complete the database and to take advantage of factorable closed forms for the marginal likelihood. In order to get such an advantage, we search for parameter values using the EM algorithm or another alternative approach that we have developed: a hybridization of the Bound and Collapse method and the EM algorithm, which results in a method that exhibits a faster convergence rate and a more effective behaviour than the EM algorithm. Also, we consider the possibility of interleaving runnings of these two methods after each structural change. We evaluate our approach on synthetic and real-world databases.
---
paper_title: Data clustering: a review

paper_content:

Clustering is the unsupervised classification of patterns (observations, data items, or feature vectors) into groups (clusters). The clustering problem has been addressed in many contexts and by researchers in many disciplines; this reflects its broad appeal and usefulness as one of the steps in exploratory data analysis. However, clustering is a difficult problem combinatorially, and differences in assumptions and contexts in different communities has made the transfer of useful generic concepts and methodologies slow to occur. This paper presents an overviewof pattern clustering methods from a statistical pattern recognition perspective, with a goal of providing useful advice and references to fundamental concepts accessible to the broad community of clustering practitioners. We present a taxonomy of clustering techniques, and identify cross-cutting themes and recent advances. We also describe some important applications of clustering algorithms such as image segmentation, object recognition, and information retrieval.
---
paper_title: Estimating the components of a mixture of normal distributions

paper_content:

SUMMARY The problem of estimating the components of a mixture of two normal distributions, multivariate or otherwise, with common but unknown covariance matrices is examined. The maximum likelihood equations are shown to be not unduly laborious to solve and the sampling properties of the resulting estimates are investigated, mainly by simulation. Moment estimators, minimum x2 and Bayes estimators are discussed but they appear greatly inferior to maximum likelihood except in the univariate case, the inferiority lying either in the sampling properties of the estimates or in the complexity of the computation. The wider problems obtained by allowing the components in the mixture to have different covariance matrices, or by having more than two components in the mixture, are briefly discussed, as is the relevance of this problem to cluster analysis.
---
paper_title: Learning Recursive Bayesian Multinets for Data Clustering by Means of Constructive Induction

paper_content:

This paper introduces and evaluates a new class of knowledge model, the recursive Bayesian multinet (RBMN), which encodes the joint probability distribution of a given database. RBMNs extend Bayesian networks (BNs) as well as partitional clustering systems. Briefly, a RBMN is a decision tree with component BNs at the leaves. A RBMN is learnt using a greedy, heuristic approach akin to that used by many supervised decision tree learners, but where BNs are learnt at leaves using constructive induction. A key idea is to treat expected data as real data. This allows us to complete the database and to take advantage of a closed form for the marginal likelihood of the expected complete data that factorizes into separate marginal likelihoods for each family (a node and its parents). Our approach is evaluated on synthetic and real-world databases.
---
paper_title: Bayesian Model Averaging of Naive Bayes for Clustering

paper_content:

This paper considers a Bayesian model-averaging (MA) approach to learn an unsupervised naive Bayes classification model. By using the expectation model-averaging (EMA) algorithm, which is proposed in this paper, a unique naive Bayes model that approximates an MA over selective naive Bayes structures is obtained. This algorithm allows to obtain the parameters for the approximate MA clustering model in the same time complexity needed to learn the maximum-likelihood model with the expectation-maximization algorithm. On the other hand, the proposed method can also be regarded as an approach to an unsupervised feature subset selection due to the fact that the model obtained by the EMA algorithm incorporates information on how dependent every predictive variable is on the cluster variable
---
paper_title: Models and Simulation of 3D Neuronal Dendritic Trees Using Bayesian Networks

paper_content:

Neuron morphology is crucial for neuronal connectivity and brain information processing. Computational models are important tools for studying dendritic morphology and its role in brain function. We applied a class of probabilistic graphical models called Bayesian networks to generate virtual dendrites from layer III pyramidal neurons from three different regions of the neocortex of the mouse. A set of 41 morphological variables were measured from the 3D reconstructions of real dendrites and their probability distributions used in a machine learning algorithm to induce the model from the data. A simulation algorithm is also proposed to obtain new dendrites by sampling values from Bayesian networks. The main advantage of this approach is that it takes into account and automatically locates the relationships between variables in the data instead of using predefined dependencies. Therefore, the methodology can be applied to any neuronal class while at the same time exploiting class-specific properties. Also, a Bayesian network was defined for each part of the dendrite, allowing the relationships to change in the different sections and to model heterogeneous developmental factors or spatial influences. Several univariate statistical tests and a novel multivariate test based on Kullback–Leibler divergence estimation confirmed that virtual dendrites were similar to real ones. The analyses of the models showed relationships that conform to current neuroanatomical knowledge and support model correctness. At the same time, studying the relationships in the models can help to identify new interactions between variables related to dendritic morphology.
---
paper_title: Comparison Between Supervised and Unsupervised Classifications of Neuronal Cell Types: A Case Study

paper_content:

In the study of neural circuits, it becomes essential to discern the different neuronal cell types that build the circuit. Traditionally, neuronal cell types have been classified using qualitative descriptors. More recently, several attempts have been made to classify neurons quantitatively, using unsupervised clustering methods. While useful, these algorithms do not take advantage of previous information known to the investigator, which could improve the classification task. For neocortical GABAergic interneurons, the problem to discern among different cell types is particularly difficult and better methods are needed to perform objective classifications. Here we explore the use of supervised classification algorithms to classify neurons based on their morphological features, using a database of 128 pyramidal cells and 199 interneurons from mouse neocortex. To evaluate the performance of different algorithms we used, as a “benchmark,” the test to automatically distinguish between pyramidal cells and interneurons, defining “ground truth” by the presence or absence of an apical dendrite. We compared hierarchical clustering with a battery of different supervised classification algorithms, finding that supervised classifications outperformed hierarchical clustering. In addition, the selection of subsets of distinguishing features enhanced the classification accuracy for both sets of algorithms. The analysis of selected variables indicates that dendritic features were most useful to distinguish pyramidal cells from interneurons when compared with somatic and axonal morphological variables. We conclude that supervised classification algorithms are better matched to the general problem of distinguishing neuronal cell types when some information on these cell groups, in our case being pyramidal or interneuron, is known a priori. As a spin-off of this methodological study, we provide several methods to automatically distinguish neocortical pyramidal cells from interneurons, based on their morphologies. © 2010 Wiley Periodicals, Inc. Develop Neurobiol 71: 71–82, 2011
---
paper_title: New insights into the classification and nomenclature of cortical GABAergic interneurons

paper_content:

A systematic classification and accepted nomenclature of neuron types is much needed but is currently lacking. This article describes a possible taxonomical solution for classifying GABAergic interneurons of the cerebral cortex based on a novel, web-based interactive system that allows experts to classify neurons with pre-determined criteria. Using Bayesian analysis and clustering algorithms on the resulting data, we investigated the suitability of several anatomical terms and neuron names for cortical GABAergic interneurons. Moreover, we show that supervised classification models could automatically categorize interneurons in agreement with experts' assignments. These results demonstrate a practical and objective approach to the naming, characterization and classification of neurons based on community consensus.
---
paper_title: Computational Inference of Neural Information Flow Networks

paper_content:

Determining how information flows along anatomical brain pathways is a fundamental requirement for understanding how animals perceive their environments, learn, and behave. Attempts to reveal such neural information flow have been made using linear computational methods, but neural interactions are known to be nonlinear. Here, we demonstrate that a dynamic Bayesian network (DBN) inference algorithm we originally developed to infer nonlinear transcriptional regulatory networks from gene expression data collected with microarrays is also successful at inferring nonlinear neural information flow networks from electrophysiology data collected with microelectrode arrays. The inferred networks we recover from the songbird auditory pathway are correctly restricted to a subset of known anatomical paths, are consistent with timing of the system, and reveal both the importance of reciprocal feedback in auditory processing and greater information flow to higher-order auditory areas when birds hear natural as opposed to synthetic sounds. A linear method applied to the same data incorrectly produces networks with information flow to non-neural tissue and over paths known not to exist. To our knowledge, this study represents the first biologically validated demonstration of an algorithm to successfully infer neural information flow networks.
---
paper_title: Inference of combinatorial neuronal synchrony with Bayesian networks.

paper_content:

Various methods have been used to infer functional synchrony between neuronal channels using electrode signal recordings. Such methods vary from approaches to identify the groups of neuronal channels that show similar signal patterns to approaches to figure out connectivity between neuronal channels. The inference of detailed connectivity between neuronal channels from electrode signal recordings can be computationally more complex than identifying the groups of neuronal channels. For this reason, most of previous approaches to infer connectivity between neuronal channels were based on pairwise measures. In this work, we propose the degree of combinatorial synchrony (DoCS) based on Bayesian networks for more enhanced inference of neuronal synchrony. DoCS is evaluated as the likelihood of edge connections in Bayesian network structures that capture the combinatorial dependency between neuronal channels. From the comparison with a cross-correlation measure using artificial neuronal networks, we validate that the proposed DoCS shows more accurate inference of neuronal synchrony when target neuronal networks include combinatorial synchrony.
---
paper_title: Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons

paper_content:

An important open problem of computational neuroscience is the generic organization of computations in networks of neurons in the brain. We show here through rigorous theoretical analysis that inherent stochastic features of spiking neurons, in combination with simple nonlinear computational operations in specific network motifs and dendritic arbors, enable networks of spiking neurons to carry out probabilistic inference through sampling in general graphical models. In particular, it enables them to carry out probabilistic inference in Bayesian networks with converging arrows (“explaining away”) and with undirected loops, that occur in many real-world tasks. Ubiquitous stochastic features of networks of spiking neurons, such as trial-to-trial variability and spontaneous activity, are necessary ingredients of the underlying computational organization. We demonstrate through computer simulations that this approach can be scaled up to neural emulations of probabilistic inference in fairly large graphical models, yielding some of the most complex computations that have been carried out so far in networks of spiking neurons.
---
paper_title: On the Use of Dynamic Bayesian Networks in Reconstructing Functional Neuronal Networks from Spike Train Ensembles

paper_content:

Coordination among cortical neurons is believed to be a key element in mediating many high-level cortical processes such as perception, attention, learning, and memory formation. Inferring the structure of the neural circuitry underlying this coordination is important to characterize the highly nonlinear, time-varying interactions between cortical neurons in the presence of complex stimuli. In this work, we investigate the applicability of dynamic Bayesian networks (DBNs) in inferring the effective connectivity between spiking cortical neurons from their observed spike trains. We demonstrate that DBNs can infer the underlying nonlinear and time-varying causal interactions between these neurons and can discriminate between mono-and polysynaptic links between them under certain constraints governing their putative connectivity. We analyzed conditionally Poisson spike train data mimicking spiking activity of cortical networks of small and moderately large size. The performance was assessed and compared to other methods under systematic variations of the network structure to mimic a wide range of responses typically observed in the cortex. Results demonstrate the utility of DBN in inferring the effective connectivity in cortical networks.
---
paper_title: The application of naive Bayes model averaging to predict Alzheimer's disease from genome-wide data.

paper_content:

Objective Predicting patient outcomes from genome-wide measurements holds significant promise for improving clinical care. The large number of measurements (eg, single nucleotide polymorphisms (SNPs)), however, makes this task computationally challenging. This paper evaluates the performance of an algorithm that predicts patient outcomes from genome-wide data by efficiently model averaging over an exponential number of naive Bayes (NB) models. ::: ::: Design This model-averaged naive Bayes (MANB) method was applied to predict late onset Alzheimer's disease in 1411 individuals who each had 312 318 SNP measurements available as genome-wide predictive features. Its performance was compared to that of a naive Bayes algorithm without feature selection (NB) and with feature selection (FSNB). ::: ::: Measurement Performance of each algorithm was measured in terms of area under the ROC curve (AUC), calibration, and run time. ::: ::: Results The training time of MANB (16.1 s) was fast like NB (15.6 s), while FSNB (1684.2 s) was considerably slower. Each of the three algorithms required less than 0.1 s to predict the outcome of a test case. MANB had an AUC of 0.72, which is significantly better than the AUC of 0.59 by NB (p<0.00001), but not significantly different from the AUC of 0.71 by FSNB. MANB was better calibrated than NB, and FSNB was even better in calibration. A limitation was that only one dataset and two comparison algorithms were included in this study. ::: ::: Conclusion MANB performed comparatively well in predicting a clinical outcome from a high-dimensional genome-wide dataset. These results provide support for including MANB in the methods used to predict outcomes from large, genome-wide datasets.
---
paper_title: Ensemble transcript interaction networks: A case study on Alzheimer's disease

paper_content:

Systems biology techniques are a topic of recent interest within the neurological field. Computational intelligence (CI) addresses this holistic perspective by means of consensus or ensemble techniques ultimately capable of uncovering new and relevant findings. In this paper, we propose the application of a CI approach based on ensemble Bayesian network classifiers and multivariate feature subset selection to induce probabilistic dependences that could match or unveil biological relationships. The research focuses on the analysis of high-throughput Alzheimer's disease (AD) transcript profiling. The analysis is conducted from two perspectives. First, we compare the expression profiles of hippocampus subregion entorhinal cortex (EC) samples of AD patients and controls. Second, we use the ensemble approach to study four types of samples: EC and dentate gyrus (DG) samples from both patients and controls. Results disclose transcript interaction networks with remarkable structures and genes not directly related to AD by previous studies. The ensemble is able to identify a variety of transcripts that play key roles in other neurological pathologies. Classical statistical assessment by means of non-parametric tests confirms the relevance of the majority of the transcripts. The ensemble approach pinpoints key metabolic mechanisms that could lead to new findings in the pathogenesis and development of AD
---
paper_title: Pro-inflammatory cytokine network in peripheral inflammation response to cerebral ischemia

paper_content:

The key circulating pro-inflammatory cytokines and their interaction in peripheral inflammation after acute cerebral ischemia are poorly understood. CD40L, IFN-γ, IL-1α, IL-1β, IL-6, IL-8, IL-17 and TNF-α were determined using multi-ELISA kit in stroke patients within 72 h of an acute ischemic attack. Leukocyte mRNAs were determined using real-time polymerase chain reactions (PCR). Stroke severity and clinical outcomes were evaluated by National Institutes of Health Stroke Scores (NIHSS) and modified Rankin Scale (mRS). Plasma/mRNA cytokine interactions were analyzed using the Bayesian network learning procedure. Compared to controls, stroke patients had higher IL-6, IL-8 and TNFα protein in plasma and lower IL-6, IL-8, TNFα, IL-1α, and IL-1β mRNA in leukocyte within 72 h after stroke. However, only the elevation of IL-6 correlated with the severity and prognosis of their stroke. This was associated with a decreased IL-6 mRNA in leukocyte. Further study showed that Bayesian network analysis revealed that changes in the other cytokines were subsequent to IL-6 leukocyte cytokine RNA. The change of other cytokines in plasma proteins after ischemic brain injury appeared secondary to IL-6. Pro-inflammatory cytokines up-regulation in plasma and compensatory immunity depression in leukocyte involve in peripheral inflammation response to cerebral ischemia. IL-6 appears to be the key mediator of circulating pro-inflammatory cytokines network.
---
paper_title: Learning genetic epistasis using Bayesian network scoring criteria

paper_content:

BackgroundGene-gene epistatic interactions likely play an important role in the genetic basis of many common diseases. Recently, machine-learning and data mining methods have been developed for learning epistatic relationships from data. A well-known combinatorial method that has been successfully applied for detecting epistasis is Multifactor Dimensionality Reduction (MDR). Jiang et al. created a combinatorial epistasis learning method called BNMBL to learn Bayesian network (BN) epistatic models. They compared BNMBL to MDR using simulated data sets. Each of these data sets was generated from a model that associates two SNPs with a disease and includes 18 unrelated SNPs. For each data set, BNMBL and MDR were used to score all 2-SNP models, and BNMBL learned significantly more correct models. In real data sets, we ordinarily do not know the number of SNPs that influence phenotype. BNMBL may not perform as well if we also scored models containing more than two SNPs. Furthermore, a number of other BN scoring criteria have been developed. They may detect epistatic interactions even better than BNMBL.Although BNs are a promising tool for learning epistatic relationships from data, we cannot confidently use them in this domain until we determine which scoring criteria work best or even well when we try learning the correct model without knowledge of the number of SNPs in that model.ResultsWe evaluated the performance of 22 BN scoring criteria using 28,000 simulated data sets and a real Alzheimer's GWAS data set. Our results were surprising in that the Bayesian scoring criterion with large values of a hyperparameter called α performed best. This score performed better than other BN scoring criteria and MDR at recall using simulated data sets, at detecting the hardest-to-detect models using simulated data sets, and at substantiating previous results using the real Alzheimer's data set.ConclusionsWe conclude that representing epistatic interactions using BN models and scoring them using a BN scoring criterion holds promise for identifying epistatic genetic variants in data. In particular, the Bayesian scoring criterion with large values of a hyperparameter α appears more promising than a number of alternatives.
---
paper_title: Beyond Structural Equation Modeling: model properties and effect size from a Bayesian viewpoint. An example of complex phenotype - genotype associations in depression.

paper_content:

Despite the rapid evolution of measurement technologies in biomedicine and genetics, most of the recent studies aiming to explore the genetic background of multifactorial diseases were only moderately successful. One of the causes of this phenomenon is that the bottleneck of genetic research is no longer the measurement process related to various laboratory technologies, but rather the analysis and interpretation of results. The commonly applied univariate methods are inadequate for exploring complex dependency patterns of multifactorial diseases which includes nearly all common diseases, such as depression, hypertension, and asthma. A comprehensive investigation requires multivariate modeling methods that enable the analysis of interactions between factors, and allow a more detailed interpretation of studies measuring complex phenotype descriptors. In this paper we discuss various aspects of multivariate modeling through a case study analyzing the effect of the single nucleotide polymorphism rs6295 in the HTR1A gene on depression and impulsivity. We overview basic concepts related to multivariate modeling and compare the properties of two investigated modeling techniques: Structural Equation Modeling and Bayesian network based learning algorithms. The resulting models demonstrate the advantages of the Bayesian approach in terms of model properties and effect size as it allows coherent handling of the weakly significant effect of rs6295. Results also confirm the mediating role of impulsivity between the SNP rs6295 of HTR1A and depression.
---
paper_title: Genetic studies of complex human diseases: Characterizing SNP-disease associations using Bayesian networks

paper_content:

BackgroundDetecting epistatic interactions plays a significant role in improving pathogenesis, prevention, diagnosis, and treatment of complex human diseases. Applying machine learning or statistical methods to epistatic interaction detection will encounter some common problems, e.g., very limited number of samples, an extremely high search space, a large number of false positives, and ways to measure the association between disease markers and the phenotype.ResultsTo address the problems of computational methods in epistatic interaction detection, we propose a score-based Bayesian network structure learning method, EpiBN, to detect epistatic interactions. We apply the proposed method to both simulated datasets and three real disease datasets. Experimental results on simulation data show that our method outperforms some other commonly-used methods in terms of power and sample-efficiency, and is especially suitable for detecting epistatic interactions with weak or no marginal effects. Furthermore, our method is scalable to real disease data.ConclusionsWe propose a Bayesian network-based method, EpiBN, to detect epistatic interactions. In EpiBN, we develop a new scoring function, which can reflect higher-order epistatic interactions by estimating the model complexity from data, and apply a fast Branch-and-Bound algorithm to learn the structure of a two-layer Bayesian network containing only one target node. To make our method scalable to real data, we propose the use of a Markov chain Monte Carlo (MCMC) method to perform the screening process. Applications of the proposed method to some real GWAS (genome-wide association studies) datasets may provide helpful insights into understanding the genetic basis of Age-related Macular Degeneration, late-onset Alzheimer's disease, and autism.
---
paper_title: Integrative Modeling Defines the Nova Splicing-Regulatory Network and Its Combinatorial Controls

paper_content:

The control of RNA alternative splicing is critical for generating biological diversity. Despite emerging genome-wide technologies to study RNA complexity, reliable and comprehensive RNA-regulatory networks have not been defined. Here we used Bayesian networks to probabilistically model diverse datasets and predict the target networks of specific regulators. We applied this strategy to identify ~700 alternative splicing events directly regulated by the neuron-specific factor Nova in the mouse brain, integrating RNA-binding data, splicing microarray data, Nova-binding motifs, and evolutionary signatures. The resulting integrative network revealed combinatorial regulation by Nova and the neuronal splicing factor Fox, interplay between phosphorylation and splicing, and potential links to neurologic disease. Thus we have developed a general approach to understanding mammalian RNA regulation at the systems level.
---
paper_title: A Transcriptomic Atlas of Mouse Neocortical Layers

paper_content:

In the mammalian cortex, neurons and glia form a patterned structure across six layers whose complex cytoarchitectonic arrangement is likely to contribute to cognition. We sequenced transcriptomes from layers 1-6b of different areas (primary and secondary) of the adult (postnatal day 56) mouse somatosensory cortex to understand the transcriptional levels and functional repertoires of coding and noncoding loci for cells constituting these layers. A total of 5,835 protein-coding genes and 66 noncoding RNA loci are differentially expressed (“patterned”) across the layers, on the basis of a machine-learning model (naive Bayes) approach. Layers 2-6b are each associated with specific functional and disease annotations that provide insights into their biological roles. This new resource (http://genserv.anat.ox.ac.uk/layers) greatly extends currently available resources, such as the Allen Mouse Brain Atlas and microarray data sets, by providing quantitative expression levels, by being genome-wide, by including novel loci, and by identifying candidate alternatively spliced transcripts that are differentially expressed across layers.
---
paper_title: Investigation of serum proteome alterations in human glioblastoma multiforme

paper_content:

Glioblastoma multiforme (GBM) or grade IV astrocytoma is the most common and lethal adult malignant brain tumor. The present study was conducted to investigate the alterations in the serum proteome in GBM patients compared to healthy controls. Comparative proteomic analysis was performed employing classical 2DE and 2D-DIGE combined with MALDI TOF/TOF MS and results were further validated through Western blotting and immunoturbidimetric assay. Comparison of the serum proteome of GBM and healthy subjects revealed 55 differentially expressed and statistically significant (p <0.05) protein spots. Among the identified proteins, haptoglobin, plasminogen precursor, apolipoprotein A-1 and M, and transthyretin are very significant due to their functional consequences in glioma tumor growth and migration, and could further be studied as glioma biomarkers and grade-specific protein signatures. Analysis of the lipoprotein pattern indicated elevated serum levels of cholesterol, triacylglycerol, and low-density lipoproteins in GBM patients. Functional pathway analysis was performed using multiple software including ingenuity pathway analysis (IPA), protein analysis through evolutionary relationships (PANTHER), database for annotation, visualization and integrated discovery (DAVID), and GeneSpring to investigate the biological context of the identified proteins, which revealed the association of candidate proteins in a few essential physiological pathways such as intrinsic prothrombin activation pathway, plasminogen activating cascade, coagulation system, glioma invasiveness signaling, and PI3K signaling in B lymphocytes. A subset of the differentially expressed proteins was applied to build statistical sample class prediction models for discrimination of GBM patients and healthy controls employing partial least squares discriminant analysis (PLS-DA) and other machine learning methods such as support vector machine (SVM), Decision Tree and Naive Bayes, and excellent discrimination between GBM and control groups was accomplished.
---
paper_title: Network modelling methods for FMRI.

paper_content:

There is great interest in estimating brain “networks” from FMRI data. This is often attempted by identifying a set of functional “nodes” (e.g., spatial ROIs or ICA maps) and then conducting a connectivity analysis between the nodes, based on the FMRI timeseries associated with the nodes. Analysis methods range from very simple measures that consider just two nodes at a time (e.g., correlation between two nodes' timeseries) to sophisticated approaches that consider all nodes simultaneously and estimate one global network model (e.g., Bayes net models). Many different methods are being used in the literature, but almost none has been carefully validated or compared for use on FMRI timeseries data. In this work we generate rich, realistic simulated FMRI data for a wide range of underlying networks, experimental protocols and problematic confounds in the data, in order to compare different connectivity estimation approaches. Our results show that in general correlation-based approaches can be quite successful, methods based on higher-order statistics are less sensitive, and lag-based approaches perform very poorly. More specifically: there are several methods that can give high sensitivity to network connection detection on good quality FMRI data, in particular, partial correlation, regularised inverse covariance estimation and several Bayes net methods; however, accurate estimation of connection directionality is more difficult to achieve, though Patel's τ can be reasonably successful. With respect to the various confounds added to the data, the most striking result was that the use of functionally inaccurate ROIs (when defining the network nodes and extracting their associated timeseries) is extremely damaging to network estimation; hence, results derived from inappropriate ROI definition (such as via structural atlases) should be regarded with great caution.
---
paper_title: Bayesian Network Analysis Reveals Alterations to Default Mode Network Connectivity in Individuals at Risk for Alzheimer's Disease

paper_content:

Alzheimer's disease (AD) is associated with abnormal functioning of the default mode network (DMN). Functional connectivity (FC) changes to the DMN have been found in patients with amnestic mild cognitive impairment (aMCI), which is the prodromal stage of AD. However, whether or not aMCI also alters the effective connectivity (EC) of the DMN remains unknown. We employed a combined group independent component analysis (ICA) and Bayesian network (BN) learning approach to resting-state functional MRI (fMRI) data from 17 aMCI patients and 17 controls, in order to establish the EC pattern of DMN, and to evaluate changes occurring in aMCI. BN analysis demonstrated heterogeneous regional convergence degree across DMN regions, which were organized into two closely interacting subsystems. Compared to controls, the aMCI group showed altered directed connectivity weights between DMN regions in the fronto-parietal, temporo-frontal, and temporo-parietal pathways. The aMCI group also exhibited altered regional convergence degree in the right inferior parietal lobule. Moreover, we found EC changes in DMN regions in aMCI were correlated with regional FC levels, and the connectivity metrics were associated with patients' cognitive performance. This study provides novel sights into our understanding of the functional architecture of the DMN and adds to a growing body of work demonstrating the importance of the DMN as a mechanism of aMCI.
---
paper_title: Inferring functional connectivity in MRI using Bayesian network structure learning with a modified PC algorithm

paper_content:

Abstract Resting state functional connectivity MRI (rs-fcMRI) is a popular technique used to gauge the functional relatedness between regions in the brain for typical and special populations. Most of the work to date determines this relationship by using Pearson's correlation on BOLD fMRI timeseries. However, it has been recognized that there are at least two key limitations to this method. First, it is not possible to resolve the direct and indirect connections/influences. Second, the direction of information flow between the regions cannot be differentiated. In the current paper, we follow-up on recent work by Smith et al. (2011), and apply PC algorithm to both simulated data and empirical data to determine whether these two factors can be discerned with group average, as opposed to single subject, functional connectivity data. When applied on simulated individual subjects, the algorithm performs well determining indirect and direct connection but fails in determining directionality. However, when applied at group level, PC algorithm gives strong results for both indirect and direct connections and the direction of information flow. Applying the algorithm on empirical data, using a diffusion-weighted imaging (DWI) structural connectivity matrix as the baseline, the PC algorithm outperformed the direct correlations. We conclude that, under certain conditions, the PC algorithm leads to an improved estimate of brain network structure compared to the traditional connectivity analysis based on correlations.
---
paper_title: Learning effective brain connectivity with dynamic Bayesian networks

paper_content:

Abstract We propose to use dynamic Bayesian networks (DBN) to learn the structure of effective brain connectivity from functional MRI data in an exploratory manner. In our previous work, we used Bayesian networks (BN) to learn the functional structure of the brain (Zheng, X., Rajapakse, J.C., 2006. Learning functional structure from fMR images. NeuroImage 31 (4), 1601–1613). However, BN provides a single snapshot of effective connectivity of the entire experiment and therefore is unable to accurately capture the temporal characteristics of connectivity. Dynamic Bayesian networks (DBN) use a Markov chain to model fMRI time-series and thereby determine temporal relationships of interactions among brain regions. Experiments on synthetic fMRI data demonstrate that the performance of DBN is comparable to Granger causality mapping (GCM) in determining the structure of linearly connected networks. Dynamic Bayesian networks render more accurate and informative brain connectivity than earlier methods as connectivity is described in complete statistical sense and temporal characteristics of time-series are explicitly taken into account. The functional structures inferred on two real fMRI datasets are consistent with the previous literature and more accurate than those discovered by BN. Furthermore, we study the effects of hemodynamic noise, scanner noise, inter-scan interval, and the variability of hemodynamic parameters on the derived connectivity.
---
paper_title: Hybrid ICA-Bayesian Network approach reveals distinct effective connectivity differences in schizophrenia

paper_content:

We utilized a discrete dynamic Bayesian network (dDBN) approach (Burge, J., Lane, T., Link, H., Qiu, S., Clark, V.P., 2007. Discrete dynamic Bayesian network analysis of fMRI data. Hum Brain Mapp.) to determine differences in brain regions between patients with schizophrenia and healthy controls on a measure of effective connectivity, termed the approximate conditional likelihood score (ACL) (Burge, J., Lane, T., 2005. Learning Class-Discriminative Dynamic Bayesian Networks. Proceedings of the International Conference on Machine Learning, Bonn, Germany, pp. 97-104.). The ACL score represents a class-discriminative measure of effective connectivity by measuring the relative likelihood of the correlation between brain regions in one group versus another. The algorithm is capable of finding non-linear relationships between brain regions because it uses discrete rather than continuous values and attempts to model temporal relationships with a first-order Markov and stationary assumption constraint (Papoulis, A., 1991. Probability, random variables, and stochastic processes. McGraw-Hill, New York.). Since Bayesian networks are overly sensitive to noisy data, we introduced an independent component analysis (ICA) filtering approach that attempted to reduce the noise found in fMRI data by unmixing the raw datasets into a set of independent spatial component maps. Components that represented noise were removed and the remaining components reconstructed into the dimensions of the original fMRI datasets. We applied the dDBN algorithm to a group of 35 patients with schizophrenia and 35 matched healthy controls using an ICA filtered and unfiltered approach. We determined that filtering the data significantly improved the magnitude of the ACL score. Patients showed the greatest ACL scores in several regions, most markedly the cerebellar vermis and hemispheres. Our findings suggest that schizophrenia patients exhibit weaker connectivity than healthy controls in multiple regions, including bilateral temporal, frontal, and cerebellar regions during an auditory paradigm.
---
paper_title: Inferring consistent functional interaction patterns from natural stimulus FMRI data

paper_content:

There has been increasing interest in how the human brain responds to natural stimulus such as video watching in the neuroimaging field. Along this direction, this paper presents our effort in inferring consistent and reproducible functional interaction patterns under natural stimulus of video watching among known functional brain regions identified by task-based fMRI. Then, we applied and compared four statistical approaches, including Bayesian network modeling with searching algorithms: greedy equivalence search (GES), Peter and Clark (PC) analysis, independent multiple greedy equivalence search (IMaGES), and the commonly used Granger causality analysis (GCA), to infer consistent and reproducible functional interaction patterns among these brain regions. It is interesting that a number of reliable and consistent functional interaction patterns were identified by the GES, PC and IMaGES algorithms in different participating subjects when they watched multiple video shots of the same semantic category. These interaction patterns are meaningful given current neuroscience knowledge and are reasonably reproducible across different brains and video shots. In particular, these consistent functional interaction patterns are supported by structural connections derived from diffusion tensor imaging (DTI) data, suggesting the structural underpinnings of consistent functional interactions. Our work demonstrates that specific consistent patterns of functional interactions among relevant brain regions might reflect the brain's fundamental mechanisms of online processing and comprehension of video messages.
---
paper_title: Comparison of pattern recognition methods in classifying high-resolution BOLD signals obtained at high magnetic field in monkeys

paper_content:

Pattern recognition methods have shown that functional magnetic resonance imaging (fMRI) data can reveal significant information about brain activity. For example, in the debate of how object categories are represented in the brain, multivariate analysis has been used to provide evidence of a distributed encoding scheme [Science 293:5539 (2001) 2425–2430]. Many follow-up studies have employed different methods to analyze human fMRI data with varying degrees of success [Nature reviews 7:7 (2006) 523–534]. In this study, we compare four popular pattern recognition methods: correlation analysis, support-vector machines (SVM), linear discriminant analysis (LDA) and Gaussian naive Bayes (GNB), using data collected at high field (7 Tesla) with higher resolution than usual fMRI studies. We investigate prediction performance on single trials and for averages across varying numbers of stimulus presentations. The performance of the various algorithms depends on the nature of the brain activity being categorized: for several tasks, many of the methods work well, whereas for others, no method performs above chance level. An important factor in overall classification performance is careful preprocessing of the data, including dimensionality reduction, voxel selection and outlier elimination.
---
paper_title: Large-scale directional connections among multi resting-state neural networks in human brain: A functional MRI and Bayesian network modeling study

paper_content:

Abstract This study examined the large-scale connectivity among multiple resting-state networks (RSNs) in the human brain. Independent component analysis was first applied to the resting-state functional MRI (fMRI) data acquired from 12 healthy young subjects for the separation of RSNs. Four sensory (lateral and medial visual, auditory, and sensory-motor) RSNs and four cognitive (default-mode, self-referential, dorsal and ventral attention) RSNs were identified. Gaussian Bayesian network (BN) learning approach was then used for the examination of the conditional dependencies among these RSNs and the construction of the network-to-network directional connectivity patterns. The BN based results demonstrated that sensory networks and cognitive networks were hierarchically organized. Specially, we found the sensory networks were highly intra-dependent and the cognitive networks were strongly intra-influenced. In addition, the results depicted dominant bottom-up connectivity from sensory networks to cognitive networks in which the self-referential and the default-mode networks might play respectively important roles in the process of resting-state information transfer and integration. The present study characterized the global connectivity relations among RSNs and delineated more characteristics of spontaneous activity dynamics.
---
paper_title: Modeling Neuronal Interactivity using Dynamic Bayesian Networks

paper_content:

Functional Magnetic Resonance Imaging (fMRI) has enabled scientists to look into the active brain. However, interactivity between functional brain regions, is still little studied. In this paper, we contribute a novel framework for modeling the interactions between multiple active brain regions, using Dynamic Bayesian Networks (DBNs) as generative models for brain activation patterns. This framework is applied to modeling of neuronal circuits associated with reward. The novelty of our framework from a Machine Learning perspective lies in the use of DBNs to reveal the brain connectivity and interactivity. Such interactivity models which are derived from fMRI data are then validated through a group classification task. We employ and compare four different types of DBNs: Parallel Hidden Markov Models, Coupled Hidden Markov Models, Fully-linked Hidden Markov Models and Dynamically Multi-Linked HMMs (DML-HMM). Moreover, we propose and compare two schemes of learning DML-HMMs. Experimental results show that by using DBNs, group classification can be performed even if the DBNs are constructed from as few as 5 brain regions. We also demonstrate that, by using the proposed learning algorithms, different DBN structures characterize drug addicted subjects vs. control subjects. This finding provides an independent test for the effect of psychopathology on brain function. In general, we demonstrate that incorporation of computer science principles into functional neuroimaging clinical studies provides a novel approach for probing human brain function.
---
paper_title: Dynamic Bayesian network modeling of fMRI: A comparison of group-analysis methods

paper_content:

Bayesian network (BN) modeling has recently been introduced as a tool for determining the dependencies between brain regions from functional-magnetic-resonance-imaging (fMRI) data. However, studies to date have yet to explore the optimum way for meaningfully combining individually determined BN models to make group inferences. We contrasted the results from three broad approaches: the “virtual-typical- subject” (VTS) approach which pools or averages group data as if they are sampled from a single, hypothetical virtual typical subject; the “individual-structure” (IS) approach that learns a separate BN for each subject, and then finds commonality across the individual structures, and the “common-structure” (CS) approach that imposes the same network structure on the BN of every subject, but allows the parameters to differ across subjects. To explore the effects of these three approaches, we applied them to an fMRI study exploring the motor effect of L-dopa medication on ten subjects with Parkinson's disease (PD), as the profound clinical effects of this medication suggest that fMRI activation in PD subjects after medication should start approaching that of age-matched controls. We found that none of these approaches is generally superior over the others, according to Bayesian-information-criterion (BIC) scores, and that they led to considerably different group-level results. The IS approach was more sensitive to the normalization effect of the L-dopa medication on brain connectivity. However, for the more homogeneous control population, the VTS approach was superior. Group-analysis approaches should be selected carefully with consideration of both statistical and biomedical evidence.
---
paper_title: Discrete dynamic Bayesian network analysis of fMRI data

paper_content:

We examine the efficacy of using discrete Dynamic Bayesian Networks (dDBNs), a data-driven modeling technique employed in machine learning, to identify functional correlations among neuroanatomical regions of interest. Unlike many neuroimaging analysis techniques, this method is not limited by linear and/or Gaussian noise assumptions. It achieves this by modeling the time series of neuroanatomical regions as discrete, as opposed to continuous, random variables with multinomial distributions. We demonstrated this method using an fMRI dataset collected from healthy and demented elderly subjects and identify correlates based on a diagnosis of dementia. The results are validated in three ways. First, the elicited correlates are shown to be robust over leave-one-out cross-validation and, via a Fourier bootstrapping method, that they were not likely due to random chance. Second, the dDBNs identified correlates that would be expected given the experimental paradigm. Third, the dDBN's ability to predict dementia is competitive with two commonly employed machine-learning classifiers: the support vector machine and the Gaussian naive Bayesian network. We also verify that the dDBN selects correlates based on non-linear criteria. Finally, we provide a brief analysis of the correlates elicited from Buckner et al.'s data that suggests that demented elderly subjects have reduced involvement of entorhinal and occipital cortex and greater involvement of the parietal lobe and amygdala in brain activity compared with healthy elderly (as measured via functional correlations among BOLD measurements). Limitations and extensions to the dDBN method are discussed.
---
paper_title: Learning Equivalence Classes of Bayesian Networks Structures

paper_content:

Approaches to learning Bayesian networks from data typically combine a scoring function with a heuristic search procedure. Given a Bayesian network structure, many of the scoring functions derived in the literature return a score for the entire equivalence class to which the structure belongs. When using such a scoring function, it is appropriate for the heuristic search algorithm to search over equivalence classes of Bayesian networks as opposed to individual structures. We present the general formulation of a search space for which the states of the search correspond to equivalence classes of structures. Using this space, any one of a number of heuristic search algorithms can easily be applied. We compare greedy search performance in the proposed search space to greedy search performance in a search space for which the states correspond to individual Bayesian network structures.
---
paper_title: Bayesian networks for fMRI: A primer

paper_content:

Abstract Bayesian network analysis is an attractive approach for studying the functional integration of brain networks, as it includes both the locations of connections between regions of the brain (functional connectivity) and more importantly the direction of the causal relationship between the regions (directed functional connectivity). Further, these approaches are more attractive than other functional connectivity analyses in that they can often operate on larger sets of nodes and run searches over a wide range of candidate networks. An important study by Smith et al. (2011) illustrated that many Bayesian network approaches did not perform well in identifying the directionality of connections in simulated single-subject data. Since then, new Bayesian network approaches have been developed that have overcome the failures in the Smith work. Additionally, an important discovery was made that shows a preprocessing step used in the Smith data puts some of the Bayesian network methods at a disadvantage. This work provides a review of Bayesian network analyses, focusing on the methods used in the Smith work as well as methods developed since 2011 that have improved estimation performance. Importantly, only approaches that have been specifically designed for fMRI data perform well, as they have been tailored to meet the challenges of fMRI data. Although this work does not suggest a single best model, it describes the class of models that perform best and highlights the features of these models that allow them to perform well on fMRI data. Specifically, methods that rely on non-Gaussianity to direct causal relationships in the network perform well.
---
paper_title: Cerebral modeling and dynamic Bayesian networks

paper_content:

The understanding and the prediction of the clinical outcomes of focal or degenerative cerebral lesions, as well as the assessment of rehabilitation procedures, necessitate knowing the cerebral substratum of cognitive or sensorimotor functions. This is achieved by activation studies, where subjects are asked to perform a specific task while data of their brain functioning are obtained through functional neuroimaging techniques. Such studies, as well as animal experiments, have shown that sensorimotor or cognitive functions are the offspring of the activity of large-scale networks of anatomically connected cerebral regions. However, no one-to-one correspondence between activated networks and functions can be found. Our research aims at understanding how the activation of large-scale networks derives from cerebral information processing mechanisms, which can only explain apparently conflicting activation data. Our work falls at the crossroads of neuroimaging interpretation techniques and computational neuroscience. Since knowledge in cognitive neuroscience is permanently evolving, our research aims more precisely at defining a new modeling formalism and at building a flexible simulator, allowing a quick implementation of the models, for a better interpretation of cerebral functional images. It also aims at providing plausible models, at the level of large-scale networks, of cerebral information processing mechanisms in humans. In this paper, we propose a formalism, based on dynamic Bayesian networks (DBNs), that respects the following constraints: an oriented, networked architecture, whose nodes (the cerebral structures) can all be different, the implementation of causality-the activation of a structure is caused by upstream nodes' activation-the explicit representation of different time scales (from 1ms for the cerebral activity to many seconds for a PET scan image acquisition), the representation of cerebral information at the integrated level of neuronal populations, the imprecision of functional neuroimaging data, the nonlinearity and the uncertainty in cerebral mechanisms, and brain's plasticity (learning, reorganization, modulation). One of the main problems, nonlinearity, has been tackled thanks to new extensions of the Kalman filter. The capabilities of the formalism's current version are illustrated by the modeling of a phoneme categorization process, explaining the different cerebral activations in normal and dyslexic subjects.
---
paper_title: Clinical Diagnosis Based on Bayesian Classification of Functional Magnetic-Resonance Data

paper_content:

We describe a method for classifying subjects based on functional magnetic-resonance (fMR) data, using a method combining a Bayesian-network classifier with inverse-tree structure (BNCIT), and ensemble learning. The central challenge is to generate a classifier from a small sample of high-dimensional data. The principal strengths of our method include the nonparametric multivariate Bayesian-network representation, and joint performance of feature selection and classification. Preliminary results indicate that this method can detect regions characterizing group differences, and can, on the basis of activation levels in these regions, accurately classify new subjects.
---
paper_title: Performance comparison of machine learning algorithms and number of independent components used in fMRI decoding of belief vs . disbelief

paper_content:

Machine learning (ML) has become a popular tool for mining functional neuroimaging data, and there are now hopes of performing such analyses efficiently in real-time. Towards this goal, we compared accuracy of six different ML algorithms applied to neuroimaging data of persons engaged in a bivariate task, asserting their belief or disbelief of a variety of propositional statements. We performed unsupervised dimension reduction and automated feature extraction using independent component (IC) analysis and extracted IC time courses. Optimization of classification hyperparameters across each classifier occurred prior to assessment. Maximum accuracy was achieved at 92% for Random Forest, followed by 91% for AdaBoost, 89% for Naive Bayes, 87% for a J48 decision tree, 86% for K*, and 84% for support vector machine. For real-time decoding applications, finding a parsimonious subset of diagnostic ICs might be useful. We used a forward search technique to sequentially add ranked ICs to the feature subspace. For the current data set, we determined that approximately six ICs represented a meaningful basis set for classification. We then projected these six IC spatial maps forward onto a later scanning session within subject. We then applied the optimized ML algorithms to these new data instances, and found that classification accuracy results were reproducible. Additionally, we compared our classification method to our previously published general linear model results on this same data set. The highest ranked IC spatial maps show similarity to brain regions associated with contrasts for belief > disbelief, and disbelief < belief.
---
paper_title: Analysis of fMRI Data by Blind Separation Into Independent Spatial Components

paper_content:

r r Abstract: Current analytical techniques applied to functional magnetic resonance imaging (fMRI) data require a priori knowledge or specific assumptions about the time courses of processes contributing to the measured signals. Here we describe a new method for analyzing fMRI data based on the independent component analysis (ICA) algorithm of Bell and Sejnowski ((1995): Neural Comput 7:1129-1159). We decomposed eight fMRI data sets from 4 normal subjects performing Stroop color-naming, the Brown and Peterson word/number task, and control tasks into spatially independent components. Each component consisted of voxel values at fixed three-dimensional locations (a component ''map''), and a unique associated time course of activation. Given data from 144 time points collected during a 6-min trial, ICA extracted an equal number of spatially independent components. In all eight trials, ICA derived one and only one component with a time course closely matching the time course of 40-sec alternations between experimental and control tasks. The regions of maximum activity in these consistently task-related components generally overlapped active regions detected by standard correlational analysis, but included frontal regions not detected by correlation. Time courses of other ICA components were transiently task-related, quasiperiodic, or slowly varying. By utilizing higher-order statistics to enforce successively stricter criteria for spatial independence between component maps, both the ICA algorithm and a related fourth-order decomposition technique (Comon (1994): Signal Processing 36:11-20) were superior to principal component analysis (PCA) in determining the spatial and temporal extent of task-related activation. For each subject, the time courses and active regions of the task-related ICA components were consistent across trials and were robust to the addition of simulated noise. Simulated movement artifact and simulated task-related activations added to actual fMRI data were clearly separated by the algorithm. ICA can be used to distinguish between nontask-related signal components, movements, and other artifacts, as well as consistently or transiently task-related fMRI activations, based on only weak
---
paper_title: Investigating directed cortical interactions in time-resolved fMRI data using vector autoregressive modeling and Granger causality mapping.

paper_content:

We present a framework aimed to reveal directed interactions of activated brain areas using time-resolved fMRI and vector autoregressive (VAR) modeling in the context of Granger causality. After describing the underlying mathematical concepts, we present simulations helping to characterize the conditions under which VAR modeling and Granger causality can reveal directed interactions from fluctuations in BOLD-like signal time courses. We apply the proposed approach to a dynamic sensorimotor mapping paradigm. In an event-related fMRI experiment, subjects performed a visuomotor mapping task for which the mapping of two stimuli ("faces" vs "houses") to two responses ("left" or "right") alternated periodically between the two possible mappings. Besides expected activity in sensory and motor areas, a fronto-parietal network was found to be active during presentation of a cue indicating a change in the stimulus-response (S-R) mapping. The observed network includes the superior parietal lobule and premotor areas. These areas might be involved in setting up and maintaining stimulus-response associations. The Granger causality analysis revealed a directed influence exerted by the left lateral prefrontal cortex and premotor areas on the left posterior parietal cortex.
---
paper_title: The counting Stroop: a cognitive interference task

paper_content:

The counting Stroop is a validated Stroop task variant. Initially designed as a functional magnetic resonance imaging (fMRI) task for identifying brain regions subserving cognition and attention (dorsal anterior midcingulate cortex (daMCC) and dorsolateral prefrontal cortex (DLPFC)), it has been used to study cognition in healthy volunteers and to identify functional brain abnormalities in neuropsychiatric disorders, such as attention deficit hyperactivity disorder (ADHD). During the counting Stroop, subjects report by button-press the number of words (one to four) appearing on the screen, regardless of word meaning. Neutral-word control trials contain single semantic category common animals (e.g., 'dog' written three times), while interference trials contain number words that are incongruent with the correct response (e.g., 'two' written four times). The counting Stroop can be completed in approximately 20 min per subject and can be used offline (behavioral performance) or with fMRI, positron emission tomography, event-related potentials, magnetoencephalography or intracranial recordings.
---
paper_title: Learning partially directed functional networks from meta-analysis imaging data

paper_content:

We propose a new exploratory method for the discovery of partially directed functional networks from fMRI meta-analysis data. The method performs structure learning of Bayesian networks in search of directed probabilistic dependencies between brain regions. Learning is based on the co-activation of brain regions observed across several independent imaging experiments. In a series of simulations, we first demonstrate the reliability of the method. We then present the application of our approach in an extensive meta-analysis including several thousand activation coordinates from more than 500 imaging studies. Results show that our method is able to automatically infer Bayesian networks that capture both directed and undirected probabilistic dependencies between a number of brain regions, including regions that are frequently observed in motor-related and cognitive control tasks.
---
paper_title: Six problems for causal inference from fMRI

paper_content:

Abstract Neuroimaging (e.g. fMRI) data are increasingly used to attempt to identify not only brain regions of interest (ROIs) that are especially active during perception, cognition, and action, but also the qualitative causal relations among activity in these regions (known as effective connectivity; Friston, 1994). Previous investigations and anatomical and physiological knowledge may somewhat constrain the possible hypotheses, but there often remains a vast space of possible causal structures. To find actual effective connectivity relations, search methods must accommodate indirect measurements of nonlinear time series dependencies, feedback, multiple subjects possibly varying in identified regions of interest, and unknown possible location-dependent variations in BOLD response delays. We describe combinations of procedures that under these conditions find feed-forward sub-structure characteristic of a group of subjects. The method is illustrated with an empirical data set and confirmed with simulations of time series of non-linear, randomly generated, effective connectivities, with feedback, subject to random differences of BOLD delays, with regions of interest missing at random for some subjects, measured with noise approximating the signal to noise ratio of the empirical data.
---
paper_title: Smoothness without Smoothing: Why Gaussian Naive Bayes Is Not Naive for Multi-Subject Searchlight Studies

paper_content:

Spatial smoothness is helpful when averaging fMRI signals across multiple subjects, as it allows different subjects' corresponding brain areas to be pooled together even if they are slightly misaligned. However, smoothing is usually not applied when performing multivoxel pattern-based analyses (MVPA), as it runs the risk of blurring away the information that fine-grained spatial patterns contain. It would therefore be desirable, if possible, to carry out pattern-based analyses which take unsmoothed data as their input but which produce smooth images as output. We show here that the Gaussian Naive Bayes (GNB) classifier does precisely this, when it is used in "searchlight" pattern-based analyses. We explain why this occurs, and illustrate the effect in real fMRI data. Moreover, we show that analyses using GNBs produce results at the multi-subject level which are statistically robust, neurally plausible, and which replicate across two independent data sets. By contrast, SVM classifiers applied to the same data do not generate a replication, even if the SVM-derived searchlight maps have smoothing applied to them. An additional advantage of GNB classifiers for searchlight analyses is that they are orders of magnitude faster to compute than more complex alternatives such as SVMs. Collectively, these results suggest that Gaussian Naive Bayes classifiers may be a highly non-naive choice for multi-subject pattern-based fMRI studies.
---
paper_title: Learning to Decode Cognitive States from Brain Images

paper_content:

Over the past decade, functional Magnetic Resonance Imaging (fMRI) has emerged as a powerful new instrument to collect vast quantities of data about activity in the human brain. A typical fMRI experiment can produce a three-dimensional image related to the human subject's brain activity every half second, at a spatial resolution of a few millimeters. As in other modern empirical sciences, this new instrumentation has led to a flood of new data, and a corresponding need for new data analysis methods. We describe recent research applying machine learning methods to the problem of classifying the cognitive state of a human subject based on fRMI data observed over a single time interval. In particular, we present case studies in which we have successfully trained classifiers to distinguish cognitive states such as (1) whether the human subject is looking at a picture or a sentence, (2) whether the subject is reading an ambiguous or non-ambiguous sentence, and (3) whether the word the subject is viewing is a word describing food, people, buildings, etc. This learning problem provides an interesting case study of classifier learning from extremely high dimensional (105 features), extremely sparse (tens of training examples), noisy data. This paper summarizes the results obtained in these three case studies, as well as lessons learned about how to successfully apply machine learning methods to train classifiers in such settings.
---
paper_title: Evaluation and calibration of functional network modeling methods based on known anatomical connections

paper_content:

Abstract Recent studies have identified large scale brain networks based on the spatio-temporal structure of spontaneous fluctuations in resting-state fMRI data. It is expected that functional connectivity based on resting-state data is reflective of — but not identical to — the underlying anatomical connectivity. However, which functional connectivity analysis methods reliably predict the network structure remains unclear. Here we tested and compared network connectivity analysis methods by applying them to fMRI resting-state time-series obtained from the human visual cortex. The methods evaluated here are those previously tested against simulated data in Smith et al. (Neuroimage, 2011) . To this end, we defined regions within retinotopic visual areas V1, V2, and V3 according to their eccentricity in the visual field, delineating central, intermediate, and peripheral eccentricity regions of interest (ROIs). These ROIs served as nodes in the models we study. We based our evaluation on the “ground-truth”, thoroughly studied retinotopically-organized anatomical connectivity in the monkey visual cortex. For each evaluated method, we computed the fractional rate of detecting connections known to exist (“c-sensitivity”), while using a threshold of the 95th percentile of the distribution of interaction magnitudes of those connections not expected to exist. Under optimal conditions — including session duration of 68 min, a relatively small network consisting of 9 nodes and artifact-free regression of the global effect — each of the top methods predicted the expected connections with 67–85% c-sensitivity. Correlation methods, including Correlation (Corr; 85%), Regularized Inverse Covariance (ICOV; 84%) and Partial Correlation (PCorr; 81%) performed best, followed by Patel's Kappa (80%), Bayesian Network method PC (BayesNet; 77%), General Synchronization measures (67–77%), and Coherence (CohB; 74%). With decreased session duration, these top methods saw decreases in c-sensitivities, achieving 59–76% for 17 min sessions. With a short resting-state fMRI scan of 8.5 min, none of the methods predicted the real network well, with Corr (65%) performing best. With increased complexity of the network from 9 to 36 nodes, multivariate methods including PCorr and BayesNet saw a decrease in performance. Artifact-free regression of the global effect increased the c-sensitivity of the top-performing methods. In an overall evaluation across all tests we performed, correlation methods (Corr, ICOV, and PCorr), Patel's Kappa, and BayesNet method PC set themselves somewhat above all other methods. We propose that data-based calibration based on known anatomical connections be integrated into future network studies, in order to maximize sensitivity and reduce false positives.
---
paper_title: Dynamic Bayesian network modeling for longitudinal brain morphometry

paper_content:

Identifying interactions among brain regions from structural magnetic-resonance images presents one of the major challenges in computational neuroanatomy. We propose a Bayesian data-mining approach to the detection of longitudinal morphological changes in the human brain. Our method uses a dynamic Bayesian network to represent evolving inter-regional dependencies. The major advantage of dynamic Bayesian network modeling is that it can represent complicated interactions among temporal processes. We validated our approach by analyzing a simulated atrophy study, and found that this approach requires only a small number of samples to detect the ground-truth temporal model. We further applied dynamic Bayesian network modeling to a longitudinal study of normal aging and mild cognitive impairment — the Baltimore Longitudinal Study of Aging. We found that interactions among regional volume-change rates for the mild cognitive impairment group are different from those for the normal-aging group.
---
paper_title: Predicting dementia development in Parkinson's disease using Bayesian network classifiers

paper_content:

Parkinson's disease (PD) has broadly been associated with mild cognitive impairment (PDMCI) and dementia (PDD). Researchers have studied surrogate, neuroanatomic biomarkers provided by magnetic resonance imaging (MRI) that may help in the early diagnosis of this condition. In this article, four classification models (naive Bayes, multivariate filter-based naive Bayes, filter selective naive Bayes and support vector machines, SVM) have been applied to evaluate their capacity to discriminate between cognitively intact patients with Parkinson's disease (PDCI), PDMCI and PDD. For this purpose, the MRI studies of 45 subjects (16 PDCI, 15 PDMCI and 14 PDD) were acquired and post-processed with Freesurfer, obtaining 112 variables (volumes of subcortical structures and thickness of cortical parcels) per subject. A multivariate filter-based naive Bayes model was found to be the best classifier, having the highest cross-validated sensitivity, specificity and accuracy. Additionally, the most relevant variables related to dementia in PD, as predicted by our classifiers, were cerebral white matter, and volumes of the lateral ventricles and hippocampi.
---
paper_title: Identification of Mild Alzheimer's Disease through automated classification of structural MRI features

paper_content:

The significant potential for early and accurate detection of Alzheimer's disease (AD) through neuroimaging data is becoming increasingly attractive in view of the possible advent of drugs which are able to modify or delay disease progression. In this paper, we aimed at developing an effective machine learning scheme which leverages structural magnetic resonance imaging features in order to identify and discriminate individuals affected by mild AD on a single subject basis. Selected features included one- and two-way combinations of subcortical and cortical volumes as well as cortical thickness and curvature of numerous brain regions which are known to be vulnerable to AD. Additionally, several feature combinations were fed into support vector machines (SVMs) as well as Naive Bayes classifiers in order to compare scheme accuracy. The most efficient combination of features and classification scheme, which employed both subcortical and cortical volumes feature vectors and a SVM classifier, was able to distinguish mild AD patients from healthy controls with 86% accuracy (82% sensitivity and 90% specificity). While this investigation is of preliminary nature, and further efforts are currently underway towards automated feature selection, best classifier determination and parameter optimization, our results appear very promising in terms of automated high-accuracy discrimination of disease stages which cannot easily be distinguished though routine clinical investigation.
---
paper_title: Ontology driven decision support for the diagnosis of mild cognitive impairment

paper_content:

In recent years, mild cognitive impairment (MCI) has attracted significant attention as an indicator of high risk for Alzheimer's disease (AD), and the diagnosis of MCI can alert patient to carry out appropriate strategies to prevent AD. To avoid subjectivity in diagnosis, we propose an ontology driven decision support method which is an automated procedure for diagnosing MCI through magnetic resonance imaging (MRI). In this approach, we encode specialized MRI knowledge into an ontology and construct a rule set using machine learning algorithms. Then we apply these two parts in conjunction with reasoning engine to automatically distinguish MCI patients from normal controls (NC). The rule set is trained by MRI data of 187 MCI patients and 177 normal controls selected from Alzheimer's Disease Neuroimaging Initiative (ADNI) using C4.5 algorithm. By using a 10-fold cross validation, we prove that the performance of C4.5 with 80.2% sensitivity is better than other algorithms, such as support vector machine (SVM), Bayesian network (BN) and back propagation (BP) neural networks, and C4.5 is suitable for the construction of reasoning rules. Meanwhile, the evaluation results suggest that our approach would be useful to assist physicians efficiently in real clinical diagnosis for the disease of MCI.
---
paper_title: Bayesian approach for network modeling of brain structural features

paper_content:

Brain connectivity patterns are useful in understanding brain function and organization. Anatomical brain ::: connectivity is largely determined using the physical synaptic connections between neurons. In contrast statistical ::: brain connectivity in a given brain population refers to the interaction and interdependencies of statistics of ::: multitudes of brain features including cortical area, volume, thickness etc. Traditionally, this dependence has ::: been studied by statistical correlations of cortical features. In this paper, we propose the use of Bayesian network ::: modeling for inferring statistical brain connectivity patterns that relate to causal (directed) as well as non-causal ::: (undirected) relationships between cortical surface areas. We argue that for multivariate cortical data, the ::: Bayesian model provides for a more accurate representation by removing the effect of confounding correlations ::: that get introduced due to canonical dependence between the data. Results are presented for a population of 466 ::: brains, where a SEM (structural equation modeling) approach is used to generate a Bayesian network model, as ::: well as a dependency graph for the joint distribution of cortical areas.
---
paper_title: Identification of a strategic brain network underlying processing speed deficits in vascular cognitive impairment

paper_content:

Abstract Patients with vascular cognitive impairment (VCI) commonly exhibit deficits in processing speed. This has been attributed to a disruption of frontal-subcortical neuronal circuits by ischemic lesions, but the exact mechanisms and underlying anatomical structures are poorly understood. We set out to identify a strategic brain network for processing speed by applying graph-based data-mining techniques to MRI lesion maps from patients with small vessel disease. We studied 235 patients with CADASIL, a genetic small vessel disease causing pure VCI. Using a probabilistic atlas in standard space we first determined the regional volumes of white matter hyperintensities (WMH) and lacunar lesions (LL) within major white matter tracts. Conditional dependencies between the regional lesion volumes and processing speed were then examined using Bayesian network analysis. Exploratory analysis identified a network of five imaging variables as the best determinant of processing speed. The network included LL in the left anterior thalamic radiation and the left cingulum as well as WMH in the left forceps minor, the left parahippocampal white matter and the left corticospinal tract. Together these variables explained 34% of the total variance in the processing speed score. Structural equation modeling confirmed the findings obtained from the Bayesian models. In summary, using graph-based models we identified a strategic brain network having the highest predictive value for processing speed in our cohort of patients with pure small vessel disease. Our findings confirm and extend previous results showing a role of frontal–subcortical neuronal circuits, in particular dorsolateral prefrontal and cingulate circuits, in VCI.
---
paper_title: Prediction of Conversion from Mild Cognitive Impairment to Alzheimer Disease Based on Bayesian Data Mining with Ensemble Learning

paper_content:

Prediction of disease progress is of great importance to Alzheimer disease (AD) researchers and clinicians. Previous attempts at constructing predictive models have been hindered by undersampling, and restriction to linear associations among variables, among other problems. To address these problems, we propose a novel Bayesian data-mining method called Bayesian Outcome Prediction with Ensemble Learning (BOPEL). BOPEL uses a Bayesian-network representation with boosting, to allow the detection of nonlinear multivariate associations, and incorporates resampling-based feature selection to prevent over-fitting caused by undersampling. We demonstrate the use of this approach in predicting conversion to AD in individuals with mild cognitive impairment (MCI), based on structural magnetic-resonance and magnetic-resonance- spectroscopy data. This study includes 26 subjects with amnestic MCI: the converter group (n = 8) met MCI criteria at baseline, but converted to AD within five years, whereas the non-converter group (n = 18) met MCI criteria at baseline and at follow-up. We found that BOPEL accurately differentiates MCI converters from non-converters, based on the baseline volumes of the left hippocampus, the banks of the right superior temporal sulcus, the right entorhinal cortex, the left lingual gyrus, and the rostral aspect of the left middle frontal gyrus. Prediction accuracy was 0.81, sensitivity was 0.63 and specificity was 0.89. We validated the generated predictive model with an independent data set constructed from the Alzheimer Disease Neuroimaging Initiative database, and again found high predictive accuracy (0.75).
---
paper_title: Structural Interactions within the Default Mode Network Identified by Bayesian Network Analysis in Alzheimer’s Disease

paper_content:

Alzheimer’s disease (AD) is a well-known neurodegenerative disease that is associated with dramatic morphological abnormalities. The default mode network (DMN) is one of the most frequently studied resting-state networks. However, less is known about specific structural dependency or interactions among brain regions within the DMN in AD. In this study, we performed a Bayesian network (BN) analysis based on regional grey matter volumes to identify differences in structural interactions among core DMN regions in structural MRI data from 80 AD patients and 101 normal controls (NC). Compared to NC, the structural interactions between the medial prefrontal cortex (mPFC) and other brain regions, including the left inferior parietal cortex (IPC), the left inferior temporal cortex (ITC) and the right hippocampus (HP), were significantly reduced in the AD group. In addition, the AD group showed prominent increases in structural interactions from the left ITC to the left HP, the left HP to the right ITC, the right HP to the right ITC, and the right IPC to the posterior cingulate cortex (PCC). The BN models significantly distinguished AD patients from NC with 87.12% specificity and 81.25% sensitivity. We then used the derived BN models to examine the replicability and stability of AD-associated BN models in an independent dataset and the results indicated discriminability with 83.64% specificity and 80.49% sensitivity. The results revealed that the BN analysis was effective for characterising regional structure interactions and the AD-related BN models could be considered as valid and predictive structural brain biomarker models for AD. Therefore, our study can assist in further understanding the pathological mechanism of AD, based on the view of the structural network, and may provide new insights into classification and clinical application in the study of AD in the future.
---
paper_title: Pattern analysis of EEG responses to speech and voice: Influence of feature grouping

paper_content:

Pattern recognition algorithms are becoming increasingly used in functional neuroimaging. These algorithms exploit information contained in temporal, spatial, or spatio-temporal patterns of independent variables (features) to detect subtle but reliable differences between brain responses to external stimuli or internal brain states. When applied to the analysis of electroencephalography (EEG) or magnetoencephalography (MEG) data, a choice needs to be made on how the input features to the algorithm are obtained from the signal amplitudes measured at the various channels. In this article, we consider six types of pattern analyses deriving from the combination of three types of feature selection in the temporal domain (predefined windows, shifting window, whole trial) with two approaches to handle the channel dimension (channel wise, multi-channel). We combined these different types of analyses with a Gaussian Naive Bayes classifier and analyzed a multi-subject EEG data set from a study aimed at understanding the task dependence of the cortical mechanisms for encoding speaker's identity and speech content (vowels) from short speech utterances (Bonte, Valente, & Formisano, 2009). Outcomes of the analyses showed that different grouping of available features helps highlighting complementary (i.e. temporal, topographic) aspects of information content in the data. A shifting window/multi-channel approach proved especially valuable in tracing both the early build up of neural information reflecting speaker or vowel identity and the late and task-dependent maintenance of relevant information reflecting the performance of a working memory task. Because it exploits the high temporal resolution of EEG (and MEG), such a shifting window approach with sequential multi-channel classifications seems the most appropriate choice for tracing the temporal profile of neural information processing.
---
paper_title: Natural Language Processing with Dynamic Classification Improves P300 Speller Accuracy and Bit Rate

paper_content:

The P300 speller is an example of a brain-computer interface that can restore functionality to victims of neuromuscular disorders. Although the most common application of this system has been communicating language, the properties and constraints of the linguistic domain have not to date been exploited when decoding brain signals that pertain to language. We hypothesized that combining the standard stepwise linear discriminant analysis with a Naive Bayes classifier and a trigram language model would increase the speed and accuracy of typing with the P300 speller. With integration of natural language processing, we observed significant improvements in accuracy and 40%–60% increases in bit rate for all six subjects in a pilot study. This study suggests that integrating information about the linguistic domain can significantly improve signal classification.
---
paper_title: Different classification techniques considering brain computer interface applications.

paper_content:

In this work the application of different machine learning techniques for classification of mental tasks from electroencephalograph (EEG) signals is investigated. The main application for this research is the improvement of brain computer interface (BCI) systems. For this purpose, Bayesian graphical network, neural network, Bayesian quadratic, Fisher linear and hidden Markov model classifiers are applied to two known EEG datasets in the BCI field. The Bayesian network classifier is used for the first time in this work for classification of EEG signals. The Bayesian network appeared to have a significant accuracy and more consistent classification compared to the other four methods. In addition to classical correct classification accuracy criteria, the mutual information is also used to compare the classification results with other BCI groups.
---
paper_title: Time-varying dynamic bayesian networks

paper_content:

Directed graphical models such as Bayesian networks are a favored formalism for modeling the dependency structures in complex multivariate systems such as those encountered in biology and neural science. When a system is undergoing dynamic transformation, temporally rewiring networks are needed for capturing the dynamic causal influences between covariates. In this paper, we propose time-varying dynamic Bayesian networks (TV-DBN) for modeling the structurally varying directed dependency structures underlying non-stationary biological/neural time series. This is a challenging problem due the non-stationarity and sample scarcity of time series data. We present a kernel reweighted l1-regularized auto-regressive procedure for this problem which enjoys nice properties such as computational efficiency and provable asymptotic consistency. To our knowledge, this is the first practical and statistically sound method for structure learning of TV-DBNs. We applied TV-DBNs to time series measurements during yeast cell cycle and brain response to visual stimuli. In both cases, TV-DBNs reveal interesting dynamics underlying the respective biological systems.
---
paper_title: Integrating Language Information With a Hidden Markov Model to Improve Communication Rate in the P300 Speller

paper_content:

The P300 speller is a common brain-computer interface (BCI) application designed to communicate language by detecting event related potentials in a subject's electroencephalogram (EEG) signal. Information about the structure of natural language can be valuable for BCI communication systems, but few attempts have been made to incorporate this domain knowledge into the classifier. In this study, we treat BCI communication as a hidden Markov model (HMM) where hidden states are target characters and the EEG signal is the visible output. Using the Viterbi algorithm, language information can be incorporated in classification and errors can be corrected automatically. This method was first evaluated offline on a dataset of 15 healthy subjects who had a significant increase in bit rate from a previously published naive Bayes approach and an average 32% increase from standard classification with dynamic stopping. An online pilot study of five healthy subjects verified these results as the average bit rate achieved using the HMM method was significantly higher than that using the naive Bayes and standard methods. These findings strongly support the integration of domain-specific knowledge into BCI classification to improve system performance and accuracy.
---
paper_title: Subject identification through standard EEG signals during resting states

paper_content:

In the present work, we used the brain electroencephalografic activity as an alternative means to identify individuals. 50 healthy subjects participated to the study and 56 EEG signals were recorded through a high-density cap during one minute of resting state either with eyes open and eyes closed. By computing the power spectrum density (PSD) on segments of 10 seconds, we obtained a feature vector of 40 points, notably the PSD values in the standard frequency range (1–40 Hz), for each EEG channel. By using a naive Bayes classifier and K-fold cross-validations, we observed high correct recognition rates (CRR) at the parieto-occipital electrodes (∼78% during eyes open, ∼89% during eyes closed). Notably, the eyes closed resting state elicited the highest CRRs at the occipital electrodes (92% O2, 91% O1), suggesting these biometric characteristics as the most suitable, among those investigated here, for identifying individuals.
---
paper_title: Application of recurrence quantification analysis for the automated identification of epileptic EEG signals.

paper_content:

Epilepsy is a common neurological disorder that is characterized by the recurrence of seizures. Electroencephalogram (EEG) signals are widely used to diagnose seizures. Because of the non-linear and dynamic nature of the EEG signals, it is difficult to effectively decipher the subtle changes in these signals by visual inspection and by using linear techniques. Therefore, non-linear methods are being researched to analyze the EEG signals. In this work, we use the recorded EEG signals in Recurrence Plots (RP), and extract Recurrence Quantification Analysis (RQA) parameters from the RP in order to classify the EEG signals into normal, ictal, and interictal classes. Recurrence Plot (RP) is a graph that shows all the times at which a state of the dynamical system recurs. Studies have reported significantly different RQA parameters for the three classes. However, more studies are needed to develop classifiers that use these promising features and present good classification accuracy in differentiating the three...
---
paper_title: Combining Wavelet Analysis and Bayesian Networks for the Classification of Auditory Brainstem Response

paper_content:

The auditory brainstem response (ABR) has become a routine clinical tool for hearing and neurological assessment. In order to pick out the ABR from the background EEG activity that obscures it, stimulus-synchronized averaging of many repeated trials is necessary, typically requiring up to 2000 repetitions. This number of repetitions can be very difficult, time consuming and uncomfortable for some subjects. In this study, a method combining wavelet analysis and Bayesian networks is introduced to reduce the required number of repetitions, which could offer a great advantage in the clinical situation. 314 ABRs with 64 repetitions and 155 ABRs with 128 repetitions recorded from eight subjects are used here. A wavelet transform is applied to each of the ABRs, and the important features of the ABRs are extracted by thresholding and matching the wavelet coefficients. The significant wavelet coefficients that represent the extracted features of the ABRs are then used as the variables to build the Bayesian network for classification of the ABRs. In order to estimate the performance of this approach, stratified ten-fold cross-validation is used
---
paper_title: Interconnection between biological abnormalities in borderline personality disorder: Use of the Bayesian networks model

paper_content:

There is agreement in that strengthening the sets of neurobiological data would reinforce the diagnostic objectivity of many psychiatric entities. This article attempts to use this approach in borderline personality disorder (BPD). Assuming that most of the biological findings in BPD reflect common underlying pathophysiological processes we hypothesized that most of the data involved in the findings would be statistically interconnected and interdependent, indicating biological consistency for this diagnosis. Prospectively obtained data on scalp and sleep electroencephalography (EEG), clinical neurologic soft signs, the dexamethasone suppression and thyrotropin-releasing hormone stimulation tests of 20 consecutive BPD patients were used to generate a Bayesian network model, an artificial intelligence paradigm that visually illustrates eventual associations (or inter-dependencies) between otherwise seemingly unrelated variables. The Bayesian network model identified relationships among most of the variables. EEG and TSH were the variables that influence most of the others, especially sleep parameters. Neurological soft signs were linked with EEG, TSH, and sleep parameters. The results suggest the possibility of using objective neurobiological variables to strengthen the validity of future diagnostic criteria and nosological characterization of BPD.
---
paper_title: Automatic detection of interictal spikes using data mining models

paper_content:

Abstract A prospective candidate for epilepsy surgery is studied both the ictal and interictal spikes (IS) to determine the localization of the epileptogenic zone. In this work, data mining (DM) classification techniques were utilized to build an automatic detection model. The selected DM algorithms are: Decision Trees (J4.8), and Statistical Bayesian Classifier (naive model). The main objective was the detection of IS, isolating them from the EEG's base activity. On the other hand, DM has an attractive advantage in such applications, in that the recognition of epileptic discharges does not need a clear definition of spike morphology. Furthermore, previously ‘unseen’ patterns could be recognized by the DM with proper ‘training’. The results obtained showed that the efficacy of the selected DM algorithms is comparable to the current visual analysis used by the experts. Moreover, DM is faster than the time required for the visual analysis of the EEG. So this tool can assist the experts by facilitating the analysis of a patient's information, and reducing the time and effort required in the process.
---
paper_title: Decoding semantic information from human electrocorticographic (ECoG) signals

paper_content:

This study examined the feasibility of decoding semantic information from human cortical activity. Four human subjects undergoing presurgical brain mapping and seizure foci localization participated in this study. Electrocorticographic (ECoG) signals were recorded while the subjects performed simple language tasks involving semantic information processing, such as a picture naming task where subjects named pictures of objects belonging to different semantic categories. Robust high-gamma band (60–120Hz) activation was observed at the left inferior frontal gyrus (LIFG) and the posterior portion of the superior temporal gyrus (pSTG) with a temporal sequence corresponding to speech production and perception. Furthermore, Gaussian Naive Bayes and Support Vector Machine classifiers, two commonly used machine learning algorithms for pattern recognition, were able to predict the semantic category of an object using cortical activity captured by ECoG electrodes covering the frontal, temporal and parietal cortices. These findings have implications for both basic neuroscience research and development of semantic-based brain-computer interface systems (BCI) that can help individuals with severe motor or communication disorders to express their intention and thoughts.
---
paper_title: Learning an L1-Regularized Gaussian Bayesian Network in the Equivalence Class Space

paper_content:

Learning the structure of a graphical model from data is a common task in a wide range of practical applications. In this paper, we focus on Gaussian Bayesian networks, i.e., on continuous data and directed acyclic graphs with a joint probability density of all variables given by a Gaussian. We propose to work in an equivalence class search space, specifically using the k-greedy equivalence search algorithm. This, combined with regularization techniques to guide the structure search, can learn sparse networks close to the one that generated the data. We provide results on some synthetic networks and on modeling the gene network of the two biological pathways regulating the biosynthesis of isoprenoids for the Arabidopsis thaliana plant.
---
paper_title: Brain effective connectivity modeling for alzheimer's disease by sparse gaussian bayesian network

paper_content:

Recent studies have shown that Alzheimer's disease (AD) is related to alteration in brain connectivity networks. One type of connectivity, called effective connectivity, defined as the directional relationship between brain regions, is essential to brain function. However, there have been few studies on modeling the effective connectivity of AD and characterizing its difference from normal controls (NC). In this paper, we investigate the sparse Bayesian Network (BN) for effective connectivity modeling. Specifically, we propose a novel formulation for the structure learning of BNs, which involves one L1-norm penalty term to impose sparsity and another penalty to ensure the learned BN to be a directed acyclic graph - a required property of BNs. We show, through both theoretical analysis and extensive experiments on eleven moderate and large benchmark networks with various sample sizes, that the proposed method has much improved learning accuracy and scalability compared with ten competing algorithms. We apply the proposed method to FDG-PET images of 42 AD and 67 NC subjects, and identify the effective connectivity models for AD and NC, respectively. Our study reveals that the effective connectivity of AD is different from that of NC in many ways, including the global-scale effective connectivity, intra-lobe, inter-lobe, and inter-hemispheric effective connectivity distributions, as well as the effective connectivity associated with specific brain regions. These findings are consistent with known pathology and clinical progression of AD, and will contribute to AD knowledge discovery.
---
paper_title: Classification of Juvenile Myoclonic Epilepsy Data Acquired Through Scanning Electromyography with Machine Learning Algorithms

paper_content:

In this paper, classification of Juvenile Myoclonic Epilepsy (JME) patients and healthy volunteers included into Normal Control (NC) groups was established using Feed-Forward Neural Networks (NN), Support Vector Machines (SVM), Decision Trees (DT), and Naive Bayes (NB) methods by utilizing the data obtained through the scanning EMG method used in a clinical study. An experimental setup was built for this purpose. 105 motor units were measured. 44 of them belonged to JME group consisting of 9 patients and 61 of them belonged to NC group comprising ten healthy volunteers. k-fold cross validation was applied to train and test the models. ROC curves were drawn for k values of 4, 6, 8 and 10. 100% of detection sensitivity was obtained for DT, NN, and NB classification methods. The lowest FP number, which was obtained by NN, was 5.
---
paper_title: Online transcranial Doppler ultrasonographic control of an onscreen keyboard

paper_content:

Brain-computer interface (BCI) systems exploit brain activity for generating a control command and may be used by individuals with severe motor disabilities as an alternative means of communication. An emerging brain monitoring modality for BCI development is transcranial Doppler ultrasonography (TCD), which facilitates the tracking of cerebral blood flow velocities associated with mental tasks. However, TCD-BCI studies to date have exclusively been offline. The feasibility of a TCD-based BCI system hinges on its online performance. In this paper, an online TCD-BCI system was implemented, bilaterally tracking blood flow velocities in the middle cerebral arteries for system-paced control of a scanning keyboard. Target letters or words were selected by repetitively rehearsing the spelling while imagining the writing of the intended word, a left-lateralized task. Undesired letters or words were bypassed by performing visual tracking, a non-lateralized task. The keyboard scanning period was 15s. With 10 able-bodied right-handed young adults, the two mental tasks were differentiated online using a Naive Bayes classification algorithm and a set of time-domain, user-dependent features. The system achieved an average specificity and sensitivity of 81.44 ± 8.35% and 82.30 ± 7.39%, respectively. The level of agreement between the intended and machine-predicted selections was moderate (=0.60). The average information transfer rate was 0.87 bits/min with an average throughput of 0.31 ± 0.12 character/min. These findings suggest that an online TCD-BCI can achieve reasonable accuracies with an intuitive language task, but with modest throughput. Future interface and signal classification enhancements are required to improve communication rate.
---
paper_title: Learning graphical model structure using L1-regularization paths

paper_content:

Sparsity-promoting L1-regularization has recently been succesfully used to learn the structure of undirected graphical models. In this paper, we apply this technique to learn the structure of directed graphical models. Specifically, we make three contributions. First, we show how the decomposability of the MDL score, plus the ability to quickly compute entire regularization paths, allows us to efficiently pick the optimal regularization parameter on a per-node basis. Second, we show how to use L1 variable selection to select the Markov blanket, before a DAG search stage. Finally, we show how L1 variable selection can be used inside of an order search algorithm. The effectiveness of these L1-based approaches are compared to current state of the art methods on 10 datasets.
---
paper_title: Exploitation of 3D stereotactic surface projection for predictive modelling of Alzheimer's disease

paper_content:

Alzheimer's Disease AD is one major cause of dementia. Previous studies have indicated that the use of features derived from Positron Emission Tomography PET scans lead to more accurate and earlier diagnosis of AD, compared to the traditional approaches that use a combination of clinical assessments. In this study, we compare Naive Bayes NB with variations of Support Vector Machines SVMs for the automatic diagnosis of AD. 3D Stereotactic Surface Projection 3D-SSP is utilised to extract features from PET scans. At the most detailed level, the dimensionality of the feature space is very high. Hence we evaluate the benefits of a correlation-based feature selection method to find a small number of highly relevant features; we also provide an analysis of selected features, which is generally supportive of the literature. However, we have also encountered patterns that may be new and relevant to prediction of the progression of AD.
---
paper_title: Robust Automated Detection of Microstructural White Matter Degeneration in Alzheimer’s Disease Using Machine Learning Classification of Multicenter DTI Data

paper_content:

Diffusion tensor imaging (DTI) based assessment of white matter fiber tract integrity can support the diagnosis of Alzheimer’s disease (AD). The use of DTI as a biomarker, however, depends on its applicability in a multicenter setting accounting for effects of different MRI scanners. We applied multivariate machine learning (ML) to a large multicenter sample from the recently created framework of the European DTI study on Dementia (EDSD). We hypothesized that ML approaches may amend effects of multicenter acquisition. We included a sample of 137 patients with clinically probable AD (MMSE 20.6±5.3) and 143 healthy elderly controls, scanned in nine different scanners. For diagnostic classification we used the DTI indices fractional anisotropy (FA) and mean diffusivity (MD) and, for comparison, gray matter and white matter density maps from anatomical MRI. Data were classified using a Support Vector Machine (SVM) and a Naive Bayes (NB) classifier. We used two cross-validation approaches, (i) test and training samples randomly drawn from the entire data set (pooled cross-validation) and (ii) data from each scanner as test set, and the data from the remaining scanners as training set (scanner-specific cross-validation). In the pooled cross-validation, SVM achieved an accuracy of 80% for FA and 83% for MD. Accuracies for NB were significantly lower, ranging between 68% and 75%. Removing variance components arising from scanners using principal component analysis did not significantly change the classification results for both classifiers. For the scanner-specific cross-validation, the classification accuracy was reduced for both SVM and NB. After mean correction, classification accuracy reached a level comparable to the results obtained from the pooled cross-validation. Our findings support the notion that machine learning classification allows robust classification of DTI data sets arising from multiple scanners, even if a new data set comes from a scanner that was not part of the training sample.
---
paper_title: A Bayesian Diagnostic System to Differentiate Glioblastomas from Solitary Brain Metastases

paper_content:

This paper aimed to construct a Bayesian network-based decision support system to differentiate glioblastomas from solitary metastases, based on multimodality MR examination. We enrolled 51 patients with solitary brain tumors (26 with glioblastomas and 25 with solitary brain metastases). These patients underwent contrast-enhanced T1-weighted magnetic resonance (MR) examination, diffusion tensor imaging (DTI), dynamic susceptibility contrast (DSC) MRI, and fluid-attenuated inversion recovery (FLAIR). We generated a set of MR biomarkers, including relative cerebral blood volume in the enhancing region, and fractional anisotropy measured in the immediate peritumoral area. We then generated a Bayesian network model to represent associations among these imaging-derived predictors, and the group membership variable, (glioblastoma or solitary metastasis). This Bayesian network can be used to classify new patients' tumors based on their MR appearance. The Bayesian network model accurately differentiated glioblast...
---
paper_title: Classification methods for the differentiation of atypical meningiomas using diffusion and perfusion techniques at 3-T MRI.

paper_content:

Abstract The purpose was to investigate the contribution of machine learning algorithms using diffusion and perfusion techniques in the differentiation of atypical meningiomas from glioblastomas and metastases. Apparent diffusion coefficient, fractional anisotropy, and relative cerebral blood volume were measured in different tumor regions. Naive Bayes, k-Nearest Neighbor, and Support Vector Machine classifiers were used in the classification procedure. The application of classification methods adds incremental differential diagnostic value. Differentiation is mainly achieved using diffusion metrics, while perfusion measurements may provide significant information for the peritumoral regions.
---
paper_title: MEG and fMRI Fusion for Non-Linear Estimation of Neural and BOLD Signal Changes

paper_content:

The combined analysis of MEG/EEG and functional MRI measurements can lead to improvement in the description of the dynamical and spatial properties of brain activity. In this paper we empirically demonstrate this improvement using simulated and recorded task related MEG and fMRI activity. Neural activity estimates were derived using a dynamic Bayesian network with continuous real valued parameters by means of a sequential Monte Carlo technique. In synthetic data, we show that MEG and fMRI fusion improves estimation of the indirectly observed neural activity and smooths tracking of the BOLD response. In recordings of task related neural activity the combination of MEG and fMRI produces a result with greater SNR, that confirms the expectation arising from the nature of the experiment. The highly nonlinear model of the BOLD response poses a difficult inference problem for neural activity estimation; computational requirements are also high due to the time and space complexity. We show that joint analysis of the data improves the system's behavior by stabilizing the differential equations system and by requiring fewer computational resources.
---
paper_title: Automated differentiation of glioblastomas from intracranial metastases using 3T MR spectroscopic and perfusion data

paper_content:

Purpose Differentiation of glioblastomas from metastases is clinical important, but may be difficult even for expert observers. To investigate the contribution of machine learning algorithms in the differentiation of glioblastomas multiforme (GB) from metastases, we developed and tested a pattern recognition system based on 3T magnetic resonance (MR) data.
---
paper_title: Effective connectivity analysis of fMRI and MEG data collected under identical paradigms

paper_content:

Estimation of effective connectivity, a measure of the influence among brain regions, can potentially reveal valuable information about organization of brain networks. Effective connectivity is usually evaluated from the functional data of a single modality. In this paper we show why that may lead to incorrect conclusions about effective connectivity. In this paper we use Bayesian networks to estimate connectivity on two different modalities. We analyze structures of estimated effective connectivity networks using aggregate statistics from the field of complex networks. Our study is conducted on functional MRI and magnetoencephalography data collected from the same subjects under identical paradigms. Results showed some similarities but also revealed some striking differences in the conclusions one would make on the fMRI data compared with the MEG data and are strongly supportive of the use of multiple modalities in order to gain a more complete picture of how the brain is organized given the limited information one modality is able to provide.
---
paper_title: Automated annotation of functional imaging experiments via multi-label classification

paper_content:

Identifying the experimental methods in human neuroimaging papers is important for grouping meaningfully similar experiments for meta-analyses. Currently, this can only be done by human readers. We present the performance of common machine learning (text mining) methods applied to the problem of automatically classifying or labeling this literature. Labeling terms are from the Cognitive Paradigm Ontology (CogPO), the text corpora are abstracts of published functional neuroimaging papers, and the methods use the performance of a human expert as training data. We aim to replicate the expert's annotation of multiple labels per abstract identifying the experimental stimuli, cognitive paradigms, response types, and other relevant dimensions of the experiments. We use several standard machine learning methods: naive Bayes, k-nearest neighbor, and support vector machines (specifically SMO or sequential minimal optimization). Exact match performance ranged from only 15% in the worst cases to 78% in the best cases. Naive Bayes methods combined with binary relevance transformations performed strongly and were robust to overfitting. This collection of results demonstrates what can be achieved with off-the-shelf software components and little to no pre-processing of raw text.
---
paper_title: Conditional Density Approximations with Mixtures of Polynomials

paper_content:

Mixtures of polynomials MoPs are a nonparametric density estimation technique especially designed for hybrid Bayesian networks with continuous and discrete variables. Algorithms to learn one- and multidimensional marginal MoPs from data have recently been proposed. In this paper, we introduce two methods for learning MoP approximations of conditional densities from data. Both approaches are based on learning MoP approximations of the joint density and the marginal density of the conditioning variables, but they differ as to how the MoP approximation of the quotient of the two densities is found. We illustrate and study the methods using data sampled from known parametric distributions, and demonstrate their applicability by learning models based on real neuroscience data. Finally, we compare the performance of the proposed methods with an approach for learning mixtures of truncated basis functions MoTBFs. The empirical results show that the proposed methods generally yield models that are comparable to or significantly better than those found using the MoTBF-based method.
---

Title: Bayesian Networks in Neuroscience: A Survey

Section 1: INTRODUCTION
Description 1: Write about the importance and applicability of Bayesian Networks (BNs) in representing probability distributions and their evolution in the field of computational neuroscience. Give an overview of the paper's structure.

Section 2: DEFINITION
Description 2: Provide a detailed description of Bayesian Networks, including their components like directed acyclic graphs (DAGs), conditional probability tables (CPTs), and the concept of conditional independence.

Section 3: GAUSSIAN BAYESIAN NETWORKS
Description 3: Explain Gaussian Bayesian Networks, which operate on continuous variables and their assumptions, characteristics, and any associated challenges.

Section 4: DYNAMIC BAYESIAN NETWORKS
Description 4: Describe Dynamic Bayesian Networks (DBNs) and their applications in time-evolving domains, their structure, example models, and their relevance in modeling neural processes over time.

Section 5: WHAT IS INFERENCE?
Description 5: Define inference in BNs, including tasks like prediction, diagnosis, abduction, and the types of reasoning within Bayesian Networks like predictive, diagnostic, and intercausal reasoning.

Section 6: INFERENCE METHODS
Description 6: Outline various methods of performing inference in Bayesian Networks, including exact methods like variable elimination and message-passing, as well as approximate methods like Monte Carlo techniques.

Section 7: LEARNING BAYESIAN NETWORKS FROM DATA
Description 7: Discuss methods for learning the structure and parameters of Bayesian Networks from data, differentiating between structure learning and parametric learning.

Section 8: LEARNING BAYESIAN NETWORK PARAMETERS
Description 8: Elaborate on methods for learning the parameters of Bayesian Networks, such as maximum likelihood estimation and Bayesian estimation, and approaches to handle missing data.

Section 9: LEARNING BAYESIAN NETWORK STRUCTURES. ASSOCIATIONS
Description 9: Review methods for learning the structure of Bayesian Networks based on associations between variables, including constraint-based and score-and-search methods.

Section 10: LEARNING BAYESIAN NETWORK STRUCTURES. SUPERVISED CLASSIFICATION
Description 10: Cover the different types of Bayesian Network classifiers for supervised classification tasks, their advantages, and performance measures.

Section 11: MORPHOLOGICAL DATA
Description 11: Present applications of Bayesian Networks in analyzing morphological data in neuroscience, including distinguishing different types of neurons.

Section 12: ELECTROPHYSIOLOGICAL DATA
Description 12: Explore how Bayesian Networks are used to interpret electrophysiological data, such as inferring neural information flow and understanding synchronization between neuronal channels.

Section 13: GENOMICS, PROTEOMICS, AND TRANSCRIPTOMICS
Description 13: Examine the application of Bayesian Networks in genomics, proteomics, and transcriptomics for discovering gene interactions, SNP associations, and protein markers.

Section 14: NEUROIMAGING DATA
Description 14: Discuss the use of Bayesian Networks in analyzing different types of neuroimaging data, including fMRI and MRI, for connectivity analysis and supervised classification tasks.

Section 15: DISCUSSION
Description 15: Summarize the findings of the survey, discuss the strengths and limitations of using Bayesian Networks in neuroscience, and propose future research directions in both the development of BN methods and their application to neuroscience problems.