A survey of models for inference of gene regulatory networks
---
paper_title: Gene regulatory network inference: Data integration in dynamic models - A review

paper_content:

Systems biology aims to develop mathematical models of biological systems by integrating experimental and theoretical techniques. During the last decade, many systems biological approaches that base on genome-wide data have been developed to unravel the complexity of gene regulation. This review deals with the reconstruction of gene regulatory networks (GRNs) from experimental data through computational methods. Standard GRN inference methods primarily use gene expression data derived from microarrays. However, the incorporation of additional information from heterogeneous data sources, e.g. genome sequence and protein–DNA interaction data, clearly supports the network inference process. This review focuses on promising modelling approaches that use such diverse types of molecular biological information. In particular, approaches are discussed that enable the modelling of the dynamics of gene regulatory systems. The review provides an overview of common modelling schemes and learning algorithms and outlines current challenges in GRN modelling.
---
paper_title: Gene networks: how to put the function in genomics

paper_content:

An increasingly popular model of regulation is to represent networks of genes as if they directly affect each other. Although such gene networks are phenomenological because they do not explicitly represent the proteins and metabolites that mediate cell interactions, they are a logical way of describing phenomena observed with transcription profiling, such as those that occur with popular microarray technology. The ability to create gene networks from experimental data and use them to reason about their dynamics and design principles will increase our understanding of cellular function. We propose that gene networks are also a good way to describe function unequivocally, and that they could be used for genome functional annotation. Here, we review some of the concepts and methods associated with gene networks, with emphasis on their construction based on experimental data.
---
paper_title: The Fuzzy Logic of MicroRNA Regulation: A Key to Control Cell Complexity

paper_content:

Genomic and clinical evidence suggest a major role of microRNAs (miRNAs) in the regulatory mechanisms of gene expression, with a clear impact on development and physiology; miRNAs are a class of endogenous 22-25 nt single-stranded RNA molecules, that negatively regulate gene expression post-transcriptionally, by imperfect base pairing with the 3' UTR of the corresponding mRNA target. Because of this imperfection, each miRNA can bind multiple targets, and multiple miRNAs can bind the same mRNA target; although digital, the miRNAs control mechanism is characterized by an imprecise action, naturally understandable in the theoretical framework of fuzzy logic.A major practical application of fuzzy logic is represented by the design and the realization of efficient and robust control systems, even when the processes to be controlled show chaotic, deterministic as well unpredictable, behaviours. The vagueness of miRNA action, when considered together with the controlled and chaotic gene expression, is a hint of a cellular fuzzy control system. As a demonstration of the possibility and the effectiveness of miRNA based fuzzy mechanism, a fuzzy cognitive map -a mathematical formalism combining neural network and fuzzy logic- has been developed to study the apoptosis/proliferation control performed by the miRNA-17-92 cluster/E2F1/cMYC circuitry.When experimentally demonstrated, the concept of fuzzy control could modify the way we analyse and model gene expression, with a possible impact on the way we imagine and design therapeutic intervention based on miRNA silencing.
---
paper_title: Mechanisms of microRNA-mediated gene regulation

paper_content:

microRNAs (miRNAs) are identified as a class of non-protein regulators and a new source for broad control of gene expression in eukaryotes. The past years have witnessed substantial progress in understanding miRNA functions and mechanisms, although a few controversies remain. Various hypotheses and models have been suggested for the mechanisms of miRNA repression, including translational inhibition at the level of initiation or elongation, rapid degradation of the nascent peptide, mRNA degradation, and mRNA sequestration into P bodies (processing bodies) and SGs (stress granules) for degradation or/and storage. Recently, some noncanonical miRNA regulation, such as miRNA activation and de-repression of miRNA inhibition, have been uncovered. This review discusses some recent advances about how miRNAs regulate their targets and various modes of miRNA function.
---
paper_title: Modeling and analysis of ChIP-chip experiments.

paper_content:

Chromatin immunoprecipitation on microarrays, also known as ChIP-chip, is a popular technique for genome-wide localization of DNA-binding proteins. However, the high density (several million genomic sequences for small eukaryote genomes) and the high noise-to-signal ratio of microarrays make the analysis of ChIP-chip data very challenging. In this chapter, we review some of the issues involved in the analysis of ChIP-chip data and present a few statistical methods that can be used to overcome these issues and improve the detection of DNA-protein binding sites.
---
paper_title: MicroRNA Gene Networks in Oncogenesis

paper_content:

MicroRNAs are small non-coding RNAs that regulate gene expression at the transcriptional or posttranscriptional level. They are involved in cellular development, differentiation, proliferation and apoptosis and play a significant role in cancer. Examination of tumor-specific microRNA expression profiles has revealed widespread deregulation of these molecules in diverse cancers. Several studies have shown that microRNAs function either as tumor suppressor genes or oncogenes, whose loss or overexpression respectively has diagnostic and prognostic significance. It seems that microRNAs act as major regulators of gene expression. In this review, we discuss microRNAs’ role in cancer and how microRNAs exert their functions through regulation of their gene targets. Bioinformatic analysis of putative miRNA binding sites has indicated several novel potential gene targets involved in apoptosis, angiogenesis and metastatic mechanisms. Matching computational prediction analysis together with microarray data seems the best method for microRNA gene target identification. MicroRNAs together with transcription factors generate a complex combinatorial code regulating gene expression. Thus, manipulation of microRNA-transcription factor gene networks may be provides a novel approach for developing cancer therapies.
---
paper_title: Identification of microRNA regulatory modules in Arabidopsis via a probabilistic graphical model

paper_content:

MOTIVATION ::: MicroRNAs miRNAs play important roles in gene regulation and are regarded as key components in gene regulatory pathways. Systematically understanding functional roles of miRNAs is essential to define core transcriptional units regulating key biological processes. Here, we propose a method based on the probabilistic graphical model to identify the regulatory modules of miRNAs and the core regulatory motifs involved in their ability to regulate gene expression. ::: ::: ::: RESULTS ::: We applied our method to datasets of different sources from Arabidopsis consisting of miRNA-target pair information, upstream sequences of miRNAs, transcriptional regulatory motifs and gene expression profiles. The graphical model used in this study can efficiently capture the relationship between miRNAs and diverse conditions such as various developmental processes, thus allowing us to detect functionally correlated miRNA regulatory modules involved in specific biological processes. Furthermore, this approach can reveal core transcriptional elements associated with their miRNAs. The proposed method found clusters of miRNAs, as well as putative regulators controlling the expression of miRNAs, which were highly related to diverse developmental processes of Arabidopsis. Consequently, our method can provide hypothetical miRNA regulatory circuits for functional testing that represent transcriptional events of miRNAs and transcriptional factors involved in gene regulatory pathways.
---
paper_title: Recovering Genetic Regulatory Networks from Chromatin Immunoprecipitation and Steady-State Microarray Data

paper_content:

Recent advances in high-throughput DNA microarrays and chromatin immunoprecipitation (ChIP) assays have enabled the learning of the structure and functionality of genetic regulatory networks. In light of these heterogeneous data sets, this paper proposes a novel approach for reconstruction of genetic regulatory networks based on the posterior probabilities of gene regulations. Built within the framework of Bayesian statistics and computational Monte Carlo techniques, the proposed approach prevents the dichotomy of classifying gene interactions as either being connected or disconnected, thereby it reduces significantly the inference errors. Simulation results corroborate the superior performance of the proposed approach relative to the existing state-of-the-art algorithms. A genetic regulatory network for Saccharomyces cerevisiae is inferred based on the published real data sets, and biological meaningful results are discussed.
---
paper_title: Combined experimental and computational approaches to study the regulatory elements in eukaryotic genes

paper_content:

The recognition of transcription factor binding sites (TFBSs) is the first step on the way to deciphering the DNA regulatory code. There is a large variety of experimental approaches providing information on TFBS location in genomic sequences. Many computational approaches to TFBS recognition based on the experimental data obtained are available, each having its own advantages and shortcomings. This article provides short review of approaches to computational recognition of TFBS in genomic sequences and methods of experimental verification of predicted sites. We also present a case study of the interplay between experimental and theoretical approaches to the successful prediction of Steroidogenic Factor 1 (SF1).
---
paper_title: MicroRNA Interference Technologies

paper_content:

The miRNA pathways are highly responsive to interventions of any kind, being excellent candidates for pharmacological manipulation. Mature miR-NAs belong to a category of non-coding small RNAs. They are very different from traditional messenger, ribosomal, or transfer RNA in their biogenesis, structures and functions. The technologies used to study traditional RNAs in basic research and to manipulate traditional RNAs for therapeutic purpose, for the most part, would be ineffective for miRNAs. New technologies for studying miRNA are therefore in need and this demand is being met by the scientific community at a breakneck pace. In the past few years, from the discovery of miRNAs in humans, numerous new technologies have been developed to clone, profile, visualize, quantify and manipulate miRNAs. This book centers on the technologies for manipulating miRNAs.
---
paper_title: ChIP–seq: advantages and challenges of a maturing technology

paper_content:

Chromatin immunoprecipitation followed by sequencing (ChIP-seq) is a technique for genome-wide profiling of DNA-binding proteins, histone modifications or nucleosomes. Owing to the tremendous progress in next-generation sequencing technology, ChIP-seq offers higher resolution, less noise and greater coverage than its array-based predecessor ChIP-chip. With the decreasing cost of sequencing, ChIP-seq has become an indispensable tool for studying gene regulation and epigenetic mechanisms. In this Review, I describe the benefits and challenges in harnessing this technique with an emphasis on issues related to experimental design and data analysis. ChIP-seq experiments generate large quantities of data, and effective computational analysis will be crucial for uncovering biological mechanisms.
---
paper_title: MicroRNA function in cancer: oncogene or a tumor suppressor?

paper_content:

MicroRNAs (miRNAs) are small noncoding, double-stranded RNA molecules that can mediate the expression of target genes with complementary sequences. About 5,300 human genes have been implicated as targets for miRNAs, making them one of the most abundant classes of regulatory genes in humans. MiRNAs recognize their target mRNAs based on sequence complementarity and act on them to cause the inhibition of protein translation by degradation of mRNA. Besides contributing to development and normal function, microRNAs have functions in various human diseases. Given the importance of miRNAs in regulating cellular differentiation and proliferation, it is not surprising that their misregulation is linked to cancer. In cancer, miRNAs function as regulatory molecules, acting as oncogenes or tumor suppressors. Amplification or overexpression of miRNAs can down-regulate tumor suppressors or other genes involved in cell differentiation, thereby contributing to tumor formation by stimulating proliferation, angiogenesis, and invasion; i.e., they act as oncogenes. Similarly, miRNAs can down-regulate different proteins with oncogenic activity; i.e., they act as tumor suppressors. This review will highlight the recent discoveries regarding miRNAs and their importance in cancer.
---
paper_title: Therapeutic MicroRNA Strategies in Human Cancer

paper_content:

MicroRNAs (miRNAs) are ~22 nucleotide long, noncoding, endogenous RNA molecules which exert their functions by base pairing with messenger RNAs (mRNAs), thereby regulate protein-coding gene expression. In eukaryotic cells, miRNAs play important roles in regulating biological processes such as proliferation, differentiation, apoptosis, and stem cell self-renewal. The human genome may contain as many as 1,000 miRNAs, and more than 700 of them have been identified. miRNAs are predicted to target up to one third of mRNAs. Each miRNA can target hundreds of transcripts directly or indirectly, while more than one miRNA can converge on a single transcript target. Therefore, the potential regulatory circuitry afforded by miRNA is enormous. Recently, mounting evidence implicates miRNAs as a new class of modulator for human tumor initiation and progression. Therefore, it has been proposed that manipulating miRNA activity and miRNA biogenesis may be a novel avenue for developing efficient therapies against cancer.
---
paper_title: Gene regulatory network inference: Data integration in dynamic models - A review

paper_content:

Systems biology aims to develop mathematical models of biological systems by integrating experimental and theoretical techniques. During the last decade, many systems biological approaches that base on genome-wide data have been developed to unravel the complexity of gene regulation. This review deals with the reconstruction of gene regulatory networks (GRNs) from experimental data through computational methods. Standard GRN inference methods primarily use gene expression data derived from microarrays. However, the incorporation of additional information from heterogeneous data sources, e.g. genome sequence and protein–DNA interaction data, clearly supports the network inference process. This review focuses on promising modelling approaches that use such diverse types of molecular biological information. In particular, approaches are discussed that enable the modelling of the dynamics of gene regulatory systems. The review provides an overview of common modelling schemes and learning algorithms and outlines current challenges in GRN modelling.
---
paper_title: Analysis of feedback loops and robustness in network evolution based on Boolean models

paper_content:

BackgroundMany biological networks such as protein-protein interaction networks, signaling networks, and metabolic networks have topological characteristics of a scale-free degree distribution. Preferential attachment has been considered as the most plausible evolutionary growth model to explain this topological property. Although various studies have been undertaken to investigate the structural characteristics of a network obtained using this growth model, its dynamical characteristics have received relatively less attention.ResultsIn this paper, we focus on the robustness of a network that is acquired during its evolutionary process. Through simulations using Boolean network models, we found that preferential attachment increases the number of coupled feedback loops in the course of network evolution. Whereas, if networks evolve to have more coupled feedback loops rather than following preferential attachment, the resulting networks are more robust than those obtained through preferential attachment, although both of them have similar degree distributions.ConclusionThe presented analysis demonstrates that coupled feedback loops may play an important role in network evolution to acquire robustness. The result also provides a hint as to why various biological networks have evolved to contain a number of coupled feedback loops.
---
paper_title: Reveal, A General Reverse Engineering Algorithm For Inference Of Genetic Network Architectures

paper_content:

Given the immanent gene expression mapping covering whole genomes during development, health and disease, we seek computational methods to maximize functional inference from such large data sets. Is it possible, in principle, to completely infer a complex regulatory network architecture from input/output patterns of its variables? We investigated this possibility using binary models of genetic networks. Trajectories, or state transition tables of Boolean nets, resemble time series of gene expression. By systematically analyzing the mutual information between input states and output states, one is able to infer the sets of input elements controlling each element or gene in the network. This process is unequivocal and exact for complete state transition tables. We implemented this REVerse Engineering ALgorithm (REVEAL) in a C program, and found the problem to be tractable within the conditions tested so far. For n = 50 (elements) and k = 3 (inputs per element), the analysis of incomplete state transition tables (100 state transition pairs out of a possible 10(15)) reliably produced the original rule and wiring sets. While this study is limited to synchronous Boolean networks, the algorithm is generalizable to include multi-state models, essentially allowing direct application to realistic biological data sets. The ability to adequately solve the inverse problem may enable in-depth analysis of complex dynamic systems in biology and other fields.
---
paper_title: From Boolean to Probabilistic Boolean Networks as Models of Genetic Regulatory Networks

paper_content:

Mathematical and computational modeling of genetic regulatory networks promises to uncover the fundamental principles governing biological systems in an integrative and holistic manner. It also paves the way toward the development of systematic approaches for effective therapeutic intervention in disease. The central theme in this paper is the Boolean formalism as a building block for modeling complex, large-scale, and dynamical networks of genetic interactions. We discuss the goals of modeling genetic networks as well as the data requirements. The Boolean formalism is justified from several points of view. We then introduce Boolean networks and discuss their relationships to nonlinear digital filters. The role of Boolean networks in understanding cell differentiation and cellular functional states is discussed. The inference of Boolean networks from real gene expression data is considered from the viewpoints of computational learning theory and nonlinear signal processing, touching on computational complexity of learning and robustness. Then, a discussion of the need to handle uncertainty in a probabilistic framework is presented, leading to an introduction of probabilistic Boolean networks and their relationships to Markov chains. Methods for quantifying the influence of genes on other genes are presented. The general question of the potential effect of individual genes on the global dynamical network behavior is considered using stochastic perturbation analysis. This discussion then leads into the problem of target identification for therapeutic intervention via the development of several computational tools based on first-passage times in Markov chains. Examples from biology are presented throughout the paper.
---
paper_title: A clustering-based approach for inferring recurrent neural networks as gene regulatory networks

paper_content:

Constructing genetic regulatory networks is one of the most important issues in system biology research. Yet, building regulatory models manually is a tedious task, especially when the number of genes involved increases with the complexity of regulation. To automate the procedure of network construction, in this work we establish a clustering-based approach to infer recurrent neural networks as regulatory systems. Our approach also deals with the scalability problem by developing a clustering method with several data analysis techniques. To verify the presented approach, experiments have been conducted and the results show that it can be used to infer gene regulatory networks successfully.
---
paper_title: Learning Bayesian Networks with Local Structure

paper_content:

We examine a novel addition to the known methods for learning Bayesian networks from data that improves the quality of the learned networks. Our approach explicitly represents and learns the local structure in the conditional probability distributions (CPDs) that quantify these networks. This increases the space of possible models, enabling the representation of CPDs with a variable number of parameters. The resulting learning procedure induces models that better emulate the interactions present in the data. We describe the theoretical foundations and practical aspects of learning local structures and provide an empirical evaluation of the proposed learning procedure. This evaluation indicates that learning curves characterizing this procedure converge faster, in the number of training instances, than those of the standard procedure, which ignores the local structure of the CPDs. Our results also show that networks learned with local structures tend to be more complex (in terms of arcs), yet require fewer parameters.
---
paper_title: A new Dynamic Bayesian Network ( DBN ) approach for identifying gene regulatory networks from time course microarray data

paper_content:

Motivation: Signaling pathways are dynamic events that take place over a given period of time. In order to identify these pathways, expression data over time are required. Dynamic Bayesian network (DBN) is an important approach for predicting the gene regulatory networks from time course expression data. However, two fundamental problems greatly reduce the effectiveness of current DBN methods. The first problem is the relatively low accuracy of prediction, and the second is the excessive computational time. ::: ::: Results: In this paper, we present a DBN-based approach with increased accuracy and reduced computational time compared with existing DBN methods. Unlike previous methods, our approach limits potential regulators to those genes with either earlier or simultaneous expression changes (up- or down-regulation) in relation to their target genes. This allows us to limit the number of potential regulators and consequently reduce the search space. Furthermore, we use the time difference between the initial change in the expression of a given regulator gene and its potential target gene to estimate the transcriptional time lag between these two genes. This method of time lag estimation increases the accuracy of predicting gene regulatory networks. Our approach is evaluated using time-series expression data measured during the yeast cell cycle. The results demonstrate that this approach can predict regulatory networks with significantly improved accuracy and reduced computational time compared with existing DBN approaches. ::: ::: Availability: The programs described in this paper can be obtained from the corresponding author upon request. ::: ::: Contact: sconzen@medicine.bsd.uchicago.edu
---
paper_title: Learning Bayesian Networks

paper_content:

This chapter addresses the problem of learning the parameters from data. It also discusses score-based structure learning and constraint-based structure learning. The method for learning all parameters in a Bayesian network follows readily from the method for learning a single parameter. The chapter presents a method for learning the probability of a binomial variable and extends this method to multinomial variables. It also provides guidelines for articulating the prior beliefs concerning probabilities. The chapter illustrates the constraint-based approach by showing how to learn a directed acyclic graph (DAG) faithful to a probability distribution. Structure learning consists of learning the DAG in a Bayesian network from data. It is necessary to know which DAG satisfies the Markov condition with the probability distribution P that is generating the data. The process of learning such a DAG is called “model selection.” A DAG includes a probability distribution P if the DAG does not entail any conditional independencies that are not in P. In score-based structure learning, a score is assigned to each DAG based on the data such that in the limit. After scoring the DAGs, the score are used, possibly along with prior probabilities, to learn a DAG. The most straightforward score, the Bayesian score, is the probability of the data D given the DAG. Once a DAG is learnt from data, the parameters can be known. The result will be a Bayesian network that can be used to do inference. In the constraint-based approach, a DAG is found for which the Markov condition entails all and only those conditional independencies that are in the probability distribution P of the variables of interest. The chapter applies structure learning to inferring causal influences from data and presents learning packages. It presents examples of learning Bayesian networks and of causal learning.
---
paper_title: Modelling gene expression data using dynamic bayesian networks

paper_content:

Recently, there has been much interest in reverse engineering genetic networks from time series data. In this paper, we show that most of the proposed discrete time models — including the boolean network model [Kau93, SS96], the linear model of D’haeseleer et al. [DWFS99], and the nonlinear model of Weaver et al. [WWS99] — are all special cases of a general class of models called Dynamic Bayesian Networks (DBNs). The advantages of DBNs include the ability to model stochasticity, to incorporate prior knowledge, and to handle hidden variables and missing data in a principled way. This paper provides a review of techniques for learning DBNs.
---
paper_title: A Scoring Function for Learning Bayesian Networks based on Mutual Information and Conditional Independence Tests

paper_content:

We propose a new scoring function for learning Bayesian networks from data using score+search algorithms. This is based on the concept of mutual information and exploits some well-known properties of this measure in a novel way. Essentially, a statistical independence test based on the chi-square distribution, associated with the mutual information measure, together with a property of additive decomposition of this measure, are combined in order to measure the degree of interaction between each variable and its parent variables in the network. The result is a non-Bayesian scoring function called MIT (mutual information tests) which belongs to the family of scores based on information theory. The MIT score also represents a penalization of the Kullback-Leibler divergence between the joint probability distributions associated with a candidate network and with the available data set. Detailed results of a complete experimental evaluation of the proposed scoring function and its comparison with the well-known K2, BDeu and BIC/MDL scores are also presented.
---
paper_title: Inferring gene regulatory networks from multiple data sources via a dynamic bayesian network with structural EM

paper_content:

Using our dynamic Bayesian network with structural Expectation Maximization (SEM-DBN), we develop a new framework to model gene regulatory network from both gene expression data and transcriptional factor binding site data. Only based on mRNA expression data, it is not enough to accurately estimate a gene network. It is difficult for us to estimate a gene network accurately only with the mRNA expression data. In this paper, we use the transcription factor binding location data in order to introduce the prior knowledge to SEM-DBN model. Gene expression data are also exploited specifically for likelihood. Meanwhile, we incorporate the prior knowledge into every learning step by SEM rather than only learning from the very beginning, which can compensate the attenuation of the effect with location data. The effectiveness of our proposed method is demonstrated through the analysis of Saccharomyces cerevisiae cell cycle data. The combination of heterogeneous data from multiple sources ensures that our results are more accurate than those recovered from only gene expression data alone.
---
paper_title: Bayesian Orthogonal Least Squares (BOLS) algorithm for reverse engineering of gene regulatory networks

paper_content:

BackgroundA reverse engineering of gene regulatory network with large number of genes and limited number of experimental data points is a computationally challenging task. In particular, reverse engineering using linear systems is an underdetermined and ill conditioned problem, i.e. the amount of microarray data is limited and the solution is very sensitive to noise in the data. Therefore, the reverse engineering of gene regulatory networks with large number of genes and limited number of data points requires rigorous optimization algorithm.ResultsThis study presents a novel algorithm for reverse engineering with linear systems. The proposed algorithm is a combination of the orthogonal least squares, second order derivative for network pruning, and Bayesian model comparison. In this study, the entire network is decomposed into a set of small networks that are defined as unit networks. The algorithm provides each unit network with P(D|Hi), which is used as confidence level. The unit network with higher P(D|Hi) has a higher confidence such that the unit network is correctly elucidated. Thus, the proposed algorithm is able to locate true positive interactions using P(D|Hi), which is a unique property of the proposed algorithm.The algorithm is evaluated with synthetic and Saccharomyces cerevisiae expression data using the dynamic Bayesian network. With synthetic data, it is shown that the performance of the algorithm depends on the number of genes, noise level, and the number of data points. With Yeast expression data, it is shown that there is remarkable number of known physical or genetic events among all interactions elucidated by the proposed algorithm.The performance of the algorithm is compared with Sparse Bayesian Learning algorithm using both synthetic and Saccharomyces cerevisiae expression data sets. The comparison experiments show that the algorithm produces sparser solutions with less false positives than Sparse Bayesian Learning algorithm.ConclusionFrom our evaluation experiments, we draw the conclusion as follows: 1) Simulation results show that the algorithm can be used to elucidate gene regulatory networks using limited number of experimental data points. 2) Simulation results also show that the algorithm is able to handle the problem with noisy data. 3) The experiment with Yeast expression data shows that the proposed algorithm reliably elucidates known physical or genetic events. 4) The comparison experiments show that the algorithm more efficiently performs than Sparse Bayesian Learning algorithm with noisy and limited number of data.
---
paper_title: Probabilistic Modeling in Bioinformatics and Medical Informatics

paper_content:

Probabilistic Modelling in Bioinformatics and Medical Informatics has been written for researchers and students in statistics, machine learning, and the biological sciences. The first part of this book provides a self-contained introduction to the methodology of Bayesian networks. The following parts demonstrate how these methods are applied in bioinformatics and medical informatics. All three fields - the methodology of probabilistic modeling, bioinformatics, and medical informatics - are evolving very quickly. The text should therefore be seen as an introduction, offering both elementary tutorials as well as more advanced applications and case studies.
---
paper_title: A Structure Learning Algorithm for Inference of Gene Networks from Microarray Gene Expression Data Using Bayesian Networks

paper_content:

Estimation of gene networks based on microarray gene expression data is an important problem in systems biology. In this paper we use Bayesian networks as a mathematical model for reverse-engineering gene networks from microarray data. In such a case, structural learning of Bayesian networks is known as an NP-hard problem and we need to use heuristic algorithms to find better network structures. Recently, several algorithms have been proposed to estimate optimal Bayesian network structure, but the number of genes included in the network is limited less than 30 or so. In order to apply Bayesian network approach to drug target gene discovery, we need to consider gene networks with several hundreds of genes. Therefore we need to develop more efficient algorithms to learn Bayesian network structure based on observed data. In this paper we propose an efficient structural learning algorithm for Bayesian networks by extending K2 algorithm that is one of the standard learning algorithms in Bayesian networks. We conduct Monte Carlo simulations to examine the effectiveness of the proposed algorithm by comparing with greedy hill-climbing algorithm. We also show the application of yeast gene network estimation based on the proposed algorithm.
---
paper_title: Using dynamic bayesian networks to infer gene regulatory networks from expression profiles

paper_content:

Two major challenges in inferring the sparse topological architecture of Gene Regulatory Networks using computational methods are 1) the low accuracy of predicting connections between genes and 2) the excessive computational cost. In order to address these challenges, we have exploited some biological features of yeast cell cycle. One such feature is that, a high proportion of Cell Cycle Regulated genes are periodically expressed; that is genes are maximally expressed to affect and control the regulation of other genes and on completing certain tasks; they are repressed by some other regulator genes. Thus the whole cell cycle progresses systematically through the successive activation and inactivation of CCR genes. To use this feature, we have calculated the peak time of individual genes which falls into one/more phases of the cell cycle. Therefore, genes that peak in the interval of the same phase of the cell cycle have been grouped together. Finally, we have applied the Dynamic Bayesian Network (DBN) algorithm within distinct phases of genes. As a consequence, both the accuracy and the computational cost of our learning algorithm have been improved in comparison with the existing DBN algorithms.
---
paper_title: Improving the structure MCMC sampler for Bayesian networks by introducing a new edge reversal move

paper_content:

Applications of Bayesian networks in systems biology are computationally demanding due to the large number of model parameters. Conventional MCMC schemes based on proposal moves in structure space tend to be too slow in mixing and convergence, and have recently been superseded by proposal moves in the space of node orders. A disadvantage of the latter approach is the intrinsic inability to specify the prior probability on network structures explicitly. The relative paucity of different experimental conditions in contemporary systems biology implies a strong influence of the prior probability on the posterior probability and, hence, the outcome of inference. Consequently, the paradigm of performing MCMC proposal moves in order rather than structure space is not entirely satisfactory. In the present article, we propose a new and more extensive edge reversal move in the original structure space, and we show that this significantly improves the convergence of the classical structure MCMC scheme.
---
paper_title: Gene regulatory network inference: Data integration in dynamic models - A review

paper_content:

Systems biology aims to develop mathematical models of biological systems by integrating experimental and theoretical techniques. During the last decade, many systems biological approaches that base on genome-wide data have been developed to unravel the complexity of gene regulation. This review deals with the reconstruction of gene regulatory networks (GRNs) from experimental data through computational methods. Standard GRN inference methods primarily use gene expression data derived from microarrays. However, the incorporation of additional information from heterogeneous data sources, e.g. genome sequence and protein–DNA interaction data, clearly supports the network inference process. This review focuses on promising modelling approaches that use such diverse types of molecular biological information. In particular, approaches are discussed that enable the modelling of the dynamics of gene regulatory systems. The review provides an overview of common modelling schemes and learning algorithms and outlines current challenges in GRN modelling.
---
paper_title: Modeling Gene Regulatory Networks with Piecewise Linear Differential Equations Abstract

paper_content:

Abstract Microarray chips generate large amounts of data about a cell’s state. In our work we want to analyze these data in order to describe the regulation processes within a cell. Therefore, we build a model which is capable of capturing the most relevant regulating interactions and present an approach how to calculate the parameters for the model from time-series data. This approach uses the discrete approximation method of least squares to solve a data fitting modeling problem. Furthermore, we analyze the features of our proposed system, i.e., which kinds of dynamical behaviors the system is able to show.
---
paper_title: Gene regulatory network inference: Data integration in dynamic models - A review

paper_content:

Systems biology aims to develop mathematical models of biological systems by integrating experimental and theoretical techniques. During the last decade, many systems biological approaches that base on genome-wide data have been developed to unravel the complexity of gene regulation. This review deals with the reconstruction of gene regulatory networks (GRNs) from experimental data through computational methods. Standard GRN inference methods primarily use gene expression data derived from microarrays. However, the incorporation of additional information from heterogeneous data sources, e.g. genome sequence and protein–DNA interaction data, clearly supports the network inference process. This review focuses on promising modelling approaches that use such diverse types of molecular biological information. In particular, approaches are discussed that enable the modelling of the dynamics of gene regulatory systems. The review provides an overview of common modelling schemes and learning algorithms and outlines current challenges in GRN modelling.
---
paper_title: ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context

paper_content:

BackgroundElucidating gene regulatory networks is crucial for understanding normal cell physiology and complex pathologic phenotypes. Existing computational methods for the genome-wide "reverse engineering" of such networks have been successful only for lower eukaryotes with simple genomes. Here we present ARACNE, a novel algorithm, using microarray expression profiles, specifically designed to scale up to the complexity of regulatory networks in mammalian cells, yet general enough to address a wider range of network deconvolution problems. This method uses an information theoretic approach to eliminate the majority of indirect interactions inferred by co-expression methods.ResultsWe prove that ARACNE reconstructs the network exactly (asymptotically) if the effect of loops in the network topology is negligible, and we show that the algorithm works well in practice, even in the presence of numerous loops and complex topologies. We assess ARACNE's ability to reconstruct transcriptional regulatory networks using both a realistic synthetic dataset and a microarray dataset from human B cells. On synthetic datasets ARACNE achieves very low error rates and outperforms established methods, such as Relevance Networks and Bayesian Networks. Application to the deconvolution of genetic networks in human B cells demonstrates ARACNE's ability to infer validated transcriptional targets of the cMYC proto-oncogene. We also study the effects of misestimation of mutual information on network reconstruction, and show that algorithms based on mutual information ranking are more resilient to estimation errors.ConclusionARACNE shows promise in identifying direct transcriptional interactions in mammalian cellular networks, a problem that has challenged existing reverse engineering algorithms. This approach should enhance our ability to use microarray data to elucidate functional mechanisms that underlie cellular processes and to identify molecular targets of pharmacological compounds in mammalian cellular networks.
---
paper_title: Learning Large-Scale Graphical Gaussian Models from Genomic Data

paper_content:

The inference and modeling of network‐like structures in genomic data is of prime importance in systems biology. Complex stochastic associations and interdependencies can very generally be described as a graphical model. However, the paucity of available samples in current high‐throughput experiments renders learning graphical models from genome data, such as microarray expression profiles, a challenging and very hard problem. Here we review several recently developed approaches to small‐sample inference of graphical Gaussian modeling and discuss strategies to cope with the high dimensionality of functional genomics data. Particular emphasis is put on regularization methods and an empirical Bayes network inference procedure.
---
paper_title: A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics

paper_content:

Inferring large-scale covariance matrices from sparse genomic data is an ubiquitous problem in bioinformatics. Clearly, the widely used standard covariance and correlation estimators are ill-suited for this purpose. As statistically efficient and computationally fast alternative we propose a novel shrinkage covariance estimator that exploits the Ledoit-Wolf (2003) lemma for analytic calculation of the optimal shrinkage intensity. Subsequently, we apply this improved covariance estimator (which has guaranteed minimum mean squared error, is well-conditioned, and is always positive definite even for small sample sizes) to the problem of inferring large-scale gene association networks. We show that it performs very favorably compared to competing approaches both in simulations as well as in application to real expression data.
---
paper_title: Reconstructing Gene Regulatory Networks with Bayesian Networks by Combining Expression Data with Multiple Sources of Prior Knowledge

paper_content:

There have been various attempts to reconstruct gene regulatory networks from microarray expression data in the past. However, owing to the limited amount of independent experimental conditions and noise inherent in the measurements, the results have been rather modest so far. For this reason it seems advisable to include biological prior knowledge, related, for instance, to transcription factor binding locations in promoter regions or partially known signalling pathways from the literature. In the present paper, we consider a Bayesian approach to systematically integrate expression data with multiple sources of prior knowledge. Each source is encoded via a separate energy function, from which a prior distribution over network structures in the form of a Gibbs distribution is constructed. The hyperparameters associated with the different sources of prior knowledge, which measure the influence of the respective prior relative to the data, are sampled from the posterior distribution with MCMC. We have evaluated the proposed scheme on the yeast cell cycle and the Raf signalling pathway. Our findings quantify to what extent the inclusion of independent prior knowledge improves the network reconstruction accuracy, and the values of the hyperparameters inferred with the proposed scheme were found to be close to optimal with respect to minimizing the reconstruction error.
---
paper_title: MicroRNA function in cancer: oncogene or a tumor suppressor?

paper_content:

MicroRNAs (miRNAs) are small noncoding, double-stranded RNA molecules that can mediate the expression of target genes with complementary sequences. About 5,300 human genes have been implicated as targets for miRNAs, making them one of the most abundant classes of regulatory genes in humans. MiRNAs recognize their target mRNAs based on sequence complementarity and act on them to cause the inhibition of protein translation by degradation of mRNA. Besides contributing to development and normal function, microRNAs have functions in various human diseases. Given the importance of miRNAs in regulating cellular differentiation and proliferation, it is not surprising that their misregulation is linked to cancer. In cancer, miRNAs function as regulatory molecules, acting as oncogenes or tumor suppressors. Amplification or overexpression of miRNAs can down-regulate tumor suppressors or other genes involved in cell differentiation, thereby contributing to tumor formation by stimulating proliferation, angiogenesis, and invasion; i.e., they act as oncogenes. Similarly, miRNAs can down-regulate different proteins with oncogenic activity; i.e., they act as tumor suppressors. This review will highlight the recent discoveries regarding miRNAs and their importance in cancer.
---
paper_title: Integrate Qualitative Biological Knowledge to Build Gene Networks by Parallel Dynamic Bayesian Network Structure Learning

paper_content:

Dynamic Bayesian Network has become a popular tool for reconstructing gene regulatory networks from microarray data. Developing high performance method is crucial to deal with the huge computational workload of Bayesian Network structure learning. Also noisy and under-sampled microarray data requires data integration mechanism to make use of legacy biological knowledge for more accurate gene network prediction. In this paper, we introduce a software system targeting on building large-scale gene networks realized by both parallel computing, and a novel data integration model which fuses qualitative gene interaction information with quantitative microarray data under the Dynamic Bayesian Networks framework. The experimental study shows that our method can accelerate the computation by using multiple CPUs, while still maintain its advantage in accuracy over non-integrative methods.
---
paper_title: CF-GeNe: Fuzzy Framework for Robust Gene Regulatory Network Inference

paper_content:

Most Gene Regulatory Network (GRN) studies ignore the impact of the noisy nature of gene expression data despite its significant influence upon inferred results. This paper presents an innovative Collateral-Fuzzy Gene Regulatory Network Reconstruction (CF-GeNe) framework for Gene Regulatory Network (GRN) inference. The approach uses the Collateral Missing Value Estimation (CMVE) algorithm as its core to estimate missing values in microarray gene expression data. CF-GeNe also mimics the inherent fuzzy nature of gene co-regulation by applying fuzzy clustering principles using the well-established fuzzy cmeans algorithm, with the model adapting to the data distribution by automatically determining key parameters, like the number of clusters. Empirical results confirm that the CMVE-based CF-GeNe paradigm infers the majority of co-regulated links even in the presence of large numbers of missing values, compared to other data imputation methods including: Least Square Impute (LSImpute), K-Nearest Neighbour Impute (KNN), Bayesian Principal Component Analysis Impute (BPCA) and ZeroImpute. The statistical significance of this improved performance has been underscored by gene selection and also by applying the Wilcoxon Ranksum Significance Test, with results corroborating the ability of CF-GeNe to successfully infer GRN interactions in noisy gene expression data.
---
paper_title: Using a state-space model with hidden variables to infer transcription factor activities

paper_content:

Motivation: In a gene regulatory network, genes are typically regulated by transcription factors (TFs). Transcription factor activity (TFA) is more difficult to measure than gene expression levels are. Other models have extracted information about TFA from gene expression data, but without explicitly modeling feedback from the genes. We present a state-space model (SSM) with hidden variables. The hidden variables include regulatory motifs in the gene network, such as feedback loops and auto-regulation, making SSM a useful complement to existing models. ::: ::: Results: A gene regulatory network incorporating, for example, feed-forward loops, auto-regulation and multiple-inputs was constructed with an SSM model. First, the gene expression data were simulated by SSM and used to infer the TFAs. The ability of SSM to infer TFAs was evaluated by comparing the profiles of the inferred and simulated TFAs. Second, SSM was applied to gene expression data obtained from Escherichia coli K12 undergoing a carbon source transition and from the Saccharomyces cerevisiae cell cycle. The inferred activity profile for each TF was validated either by measurement or by activity information from the literature. The SSM model provides a probabilistic framework to simulate gene regulatory networks and to infer activity profiles of hidden variables. ::: ::: Availability: Supplementary data and Matlab code will be made available at the URL below. ::: ::: Contact: krischan@egr.msu.edu ::: ::: Supplementary information: http://www.chems.msu.edu/groups/chan/ssm.zip
---
paper_title: A New Approach Combined Fuzzy Clustering and Bayesian Networks for Modeling Gene Regulatory Networks

paper_content:

The appearance of microarray technologies makes it possible to simultaneously monitor the expression levels for tens of thousands of genes. Bayesian Network (BN) is an important approach for predicting gene regulatory networks from expression data. However, two fundamental problems greatly reduce the effectiveness of current BN methods. The first problem is much less samples than genes, it makes that to find a "best" network is very difficult. The second is the excessive computational time, the search space of possible Bayesian network is very large because genes are numerous. In this paper, we introduce a new method to learn Bayesian networks which combines fuzzy clustering algorithm to reduce the search space. From the view of systems biology, modularity and hierarchy are key features of biological networks. This allows us to gain global network by assembling local components. A local component is composed of genes with same function. Those genes may have same or very similar expression pattern. Besides, those inter-related genes may be bridge of different functional modules. Consequently, in a process of learning of local networks, the gap between the number of samples and genes is shortened, and the search space is reduced.Our approach is evaluated using artificial data and expression data measured during the yeast cell cycle. Results demonstrate that this approach can predict regulatory networks with significantly improved accuracy and reduced computational time compared with existing BN approaches.
---
paper_title: Modeling gene expression regulatory networks with the sparse vector autoregressive model

paper_content:

BackgroundTo understand the molecular mechanisms underlying important biological processes, a detailed description of the gene products networks involved is required. In order to define and understand such molecular networks, some statistical methods are proposed in the literature to estimate gene regulatory networks from time-series microarray data. However, several problems still need to be overcome. Firstly, information flow need to be inferred, in addition to the correlation between genes. Secondly, we usually try to identify large networks from a large number of genes (parameters) originating from a smaller number of microarray experiments (samples). Due to this situation, which is rather frequent in Bioinformatics, it is difficult to perform statistical tests using methods that model large gene-gene networks. In addition, most of the models are based on dimension reduction using clustering techniques, therefore, the resulting network is not a gene-gene network but a module-module network. Here, we present the Sparse Vector Autoregressive model as a solution to these problems.ResultsWe have applied the Sparse Vector Autoregressive model to estimate gene regulatory networks based on gene expression profiles obtained from time-series microarray experiments. Through extensive simulations, by applying the SVAR method to artificial regulatory networks, we show that SVAR can infer true positive edges even under conditions in which the number of samples is smaller than the number of genes. Moreover, it is possible to control for false positives, a significant advantage when compared to other methods described in the literature, which are based on ranks or score functions. By applying SVAR to actual HeLa cell cycle gene expression data, we were able to identify well known transcription factor targets.ConclusionThe proposed SVAR method is able to model gene regulatory networks in frequent situations in which the number of samples is lower than the number of genes, making it possible to naturally infer partial Granger causalities without any a priori information. In addition, we present a statistical test to control the false discovery rate, which was not previously possible using other gene regulatory network models.
---
paper_title: Inferring transcriptional and microRNA-mediated regulatory programs in glioblastoma

paper_content:

Large-scale cancer genomics projects are profiling hundreds of tumors at multiple molecular layers, including copy number, mRNA and miRNA expression, but the mechanistic relationships between these layers are often excluded from computational models. We developed a supervised learning framework for integrating molecular profiles with regulatory sequence information to reveal regulatory programs in cancer, including miRNA-mediated regulation. We applied our approach to 320 glioblastoma profiles and identified key miRNAs and transcription factors as common or subtype-specific drivers of expression changes. We confirmed that predicted gene expression signatures for proneural subtype regulators were consistent with in vivo expression changes in a PDGF-driven mouse model. We tested two predicted proneural drivers, miR-124 and miR-132, both underexpressed in proneural tumors, by overexpression in neurospheres and observed a partial reversal of corresponding tumor expression changes. Computationally dissecting the role of miRNAs in cancer may ultimately lead to small RNA therapeutics tailored to subtype or individual.
---
paper_title: Network component analysis: Reconstruction of regulatory signals in biological systems

paper_content:

High-dimensional data sets generated by high-throughput technologies, such as DNA microarray, are often the outputs of complex networked systems driven by hidden regulatory signals. Traditional statistical methods for computing low-dimensional or hidden representations of these data sets, such as principal component analysis and independent component analysis, ignore the underlying network structures and provide decompositions based purely on a priori statistical constraints on the computed component signals. The resulting decomposition thus provides a phenomenological model for the observed data and does not necessarily contain physically or biologically meaningful signals. Here, we develop a method, called network component analysis, for uncovering hidden regulatory signals from outputs of networked systems, when only a partial knowledge of the underlying network topology is available. The a priori network structure information is first tested for compliance with a set of identifiability criteria. For networks that satisfy the criteria, the signals from the regulatory nodes and their strengths of influence on each output node can be faithfully reconstructed. This method is first validated experimentally by using the absorbance spectra of a network of various hemoglobin species. The method is then applied to microarray data generated from yeast Saccharamyces cerevisiae and the activities of various transcription factors during cell cycle are reconstructed by using recently discovered connectivity information for the underlying transcriptional regulatory networks.
---
paper_title: COMBINING MICROARRAYS AND BIOLOGICAL KNOWLEDGE FOR ESTIMATING GENE NETWORKS VIA BAYESIAN NETWORKS

paper_content:

We propose a statistical method for estimating a gene network based on Bayesian networks from microarray gene expression data together with biological knowledge including protein-protein interactions, protein-DNA interactions, binding site information, existing literature and so on. Microarray data do not contain enough information for constructing gene networks accurately in many cases. Our method adds biological knowledge to the estimation method of gene networks under a Bayesian statistical framework, and also controls the trade-off between microarray information and biological knowledge automatically. We conduct Monte Carlo simulations to show the effectiveness of the proposed method. We analyze Saccharomyces cerevisiae gene expression data as an application.
---
paper_title: Reverse engineering of gene regulatory networks: a finite state linear model

paper_content:

We propose a new model for describing gene regulatory networks that can capture discrete (Boolean) and continuous (differential) aspects of gene regulation. After giving some illustrations of the model, we study the problem of the reverse engineering of such networks, i.e., how to construct a network from gene expression data. We prove that for our model there exists an algorithm finding a network compatible with the given data. We demonstrate the model by simulating lambda-phage. We also describe some generalizations of the model, discuss their relevance to the real-world gene networks and formulate a number of open problems.
---
paper_title: Modelling and analysing genetic networks: From Boolean networks to Petri nets

paper_content:

In order to understand complex genetic regulatory networks researchers require automated formal modelling techniques that provide appropriate analysis tools. In this paper we propose a new qualitative model for genetic regulatory networks based on Petri nets and detail a process for automatically constructing these models using logic minimization. We take as our starting point the Boolean network approach in which regulatory entities are viewed abstractly as binary switches. The idea is to extract terms representing a Boolean network using logic minimization and to then directly translate these terms into appropriate Petri net control structures. The resulting compact Petri net model addresses a number of shortcomings associated with Boolean networks and is particularly suited to analysis using the wide range of Petri net tools. We demonstrate our approach by presenting a detailed case study in which the genetic regulatory network underlying the nutritional stress response in Escherichia coli is modelled and analysed.
---
paper_title: Gene network reconstruction from microarray data

paper_content:

BackgroundOften, software available for biological pathways reconstruction rely on literature search to find links between genes. The aim of this study is to reconstruct gene networks from microarray data, using Graphical Gaussian models.ResultsThe GeneNet R package was applied to the Eadgene chicken infection data set. No significant edges were found for the list of differentially expressed genes between conditions MM8 and MA8. On the other hand, a large number of significant edges were found among 85 differentially expressed genes between conditions MM8 and MM24.ConclusionMany edges were inferred from the microarray data. Most of them could, however, not be validated using other pathway reconstruction software. This was partly due to the fact that a quite large proportion of the differentially expressed genes were not annotated. Further biological validation is therefore needed for these networks, using for example in vitro invalidation of genes.
---
paper_title: Using Higher-Order Dynamic Bayesian Networks to Model Periodic Data from the Circadian Clock of Arabidopsis Thaliana

paper_content:

Modelling gene regulatory networks in organisms is an important task that has recently become possible due to large scale assays using technologies such as microarrays. In this paper, the circadian clock of Arabidopsis thaliana is modelled by fitting dynamic Bayesian networks to luminescence data gathered from experiments. This work differs from previous modelling attempts by using higher-order dynamic Bayesian networks to explicitly model the time lag between the various genes being expressed. In order to achieve this goal, new techniques in preprocessing the data and in evaluating a learned model are proposed. It is shown that it is possible, to some extent, to model these time delays using a higher-order dynamic Bayesian network.
---
paper_title: Using AUC and accuracy in evaluating learning algorithms

paper_content:

The area under the ROC (receiver operating characteristics) curve, or simply AUC, has been traditionally used in medical diagnosis since the 1970s. It has recently been proposed as an alternative single-number measure for evaluating the predictive ability of learning algorithms. However, no formal arguments were given as to why AUC should be preferred over accuracy. We establish formal criteria for comparing two different measures for learning algorithms and we show theoretically and empirically that AUC is a better measure (defined precisely) than accuracy. We then reevaluate well-established claims in machine learning based on accuracy using AUC and obtain interesting and surprising new results. For example, it has been well-established and accepted that Naive Bayes and decision trees are very similar in predictive accuracy. We show, however, that Naive Bayes is significantly better than decision trees in AUC. The conclusions drawn in this paper may make a significant impact on machine learning and data mining applications.
---
paper_title: Boosting Signal-to-Noise in Complex Biology: Prior Knowledge Is Power

paper_content:

A major difficulty in the analysis of complex biological systems is dealing with the low signal-to-noise inherent to nearly all large biological datasets. We discuss powerful bioinformatic concepts for boosting signal-to-noise through external knowledge incorporated in processing units we call filters and integrators. These concepts are illustrated in four landmark studies that have provided model implementations of filters, integrators, or both.
---
paper_title: The TRANSFAC system on gene expression regulation

paper_content:

The TRANSFAC database on transcription factors and their DNA-binding sites and profiles (http://www.gene-regulation.de/) has been quantitatively extended and supplemented by a number of modules. These modules give information about pathologically relevant mutations in regulatory regions and transcription factor genes (PathoDB), scaffold/matrix attached regions (S/MARt DB), signal transduction (TRANSPATH) and gene expression sources (CYTOMER). Altogether, these distinct database modules constitute the TRANSFAC system. They are accompanied by a number of program routines for identifying potential transcription factor binding sites or for localizing individual components in the regulatory network of a cell.
---
paper_title: Inferring Gene Regulatory Networks from Expression Data

paper_content:

Summary. Gene regulatory networks describe how cells control the expression of genes, which, together with some additional regulation further downstream, determines the production of proteins essential for cellular function. The level of expression of each gene in the genome is modified by controlling whether and how vigorously it is transcribed to RNA, and subsequently translated to protein. RNA and protein expression will influence expression rates of other genes, thus giving rise to a complicated network structure. An analysis of regulatory processes within the cell will significantly further our understanding of cellular dynamics. It will shed light on normal and abnormal, diseased cellular events, and may provide information on pathways in dire diseases such as cancer. These pathways can provide information on how the disease develops, and what processes are involved in progression. Ultimately, we can hope that this will provide us with new therapeutic approaches and targets for drug design. It is thus no surprise that many efforts have been undertaken to reconstruct gene regulatory networks from gene expression measurements. In this chapter, we will provide an introductory overview over the field. In particular, we will present several different approaches to gene regulatory network inference, discuss their strengths and weaknesses, and provide guidelines on which models are appropriate under what circumstances. In addition, we sketch future developments and open problems.
---
paper_title: A new Dynamic Bayesian Network ( DBN ) approach for identifying gene regulatory networks from time course microarray data

paper_content:

Motivation: Signaling pathways are dynamic events that take place over a given period of time. In order to identify these pathways, expression data over time are required. Dynamic Bayesian network (DBN) is an important approach for predicting the gene regulatory networks from time course expression data. However, two fundamental problems greatly reduce the effectiveness of current DBN methods. The first problem is the relatively low accuracy of prediction, and the second is the excessive computational time. ::: ::: Results: In this paper, we present a DBN-based approach with increased accuracy and reduced computational time compared with existing DBN methods. Unlike previous methods, our approach limits potential regulators to those genes with either earlier or simultaneous expression changes (up- or down-regulation) in relation to their target genes. This allows us to limit the number of potential regulators and consequently reduce the search space. Furthermore, we use the time difference between the initial change in the expression of a given regulator gene and its potential target gene to estimate the transcriptional time lag between these two genes. This method of time lag estimation increases the accuracy of predicting gene regulatory networks. Our approach is evaluated using time-series expression data measured during the yeast cell cycle. The results demonstrate that this approach can predict regulatory networks with significantly improved accuracy and reduced computational time compared with existing DBN approaches. ::: ::: Availability: The programs described in this paper can be obtained from the corresponding author upon request. ::: ::: Contact: sconzen@medicine.bsd.uchicago.edu
---
paper_title: Analysis of feedback loops and robustness in network evolution based on Boolean models

paper_content:

BackgroundMany biological networks such as protein-protein interaction networks, signaling networks, and metabolic networks have topological characteristics of a scale-free degree distribution. Preferential attachment has been considered as the most plausible evolutionary growth model to explain this topological property. Although various studies have been undertaken to investigate the structural characteristics of a network obtained using this growth model, its dynamical characteristics have received relatively less attention.ResultsIn this paper, we focus on the robustness of a network that is acquired during its evolutionary process. Through simulations using Boolean network models, we found that preferential attachment increases the number of coupled feedback loops in the course of network evolution. Whereas, if networks evolve to have more coupled feedback loops rather than following preferential attachment, the resulting networks are more robust than those obtained through preferential attachment, although both of them have similar degree distributions.ConclusionThe presented analysis demonstrates that coupled feedback loops may play an important role in network evolution to acquire robustness. The result also provides a hint as to why various biological networks have evolved to contain a number of coupled feedback loops.
---
paper_title: CF-GeNe: Fuzzy Framework for Robust Gene Regulatory Network Inference

paper_content:

Most Gene Regulatory Network (GRN) studies ignore the impact of the noisy nature of gene expression data despite its significant influence upon inferred results. This paper presents an innovative Collateral-Fuzzy Gene Regulatory Network Reconstruction (CF-GeNe) framework for Gene Regulatory Network (GRN) inference. The approach uses the Collateral Missing Value Estimation (CMVE) algorithm as its core to estimate missing values in microarray gene expression data. CF-GeNe also mimics the inherent fuzzy nature of gene co-regulation by applying fuzzy clustering principles using the well-established fuzzy cmeans algorithm, with the model adapting to the data distribution by automatically determining key parameters, like the number of clusters. Empirical results confirm that the CMVE-based CF-GeNe paradigm infers the majority of co-regulated links even in the presence of large numbers of missing values, compared to other data imputation methods including: Least Square Impute (LSImpute), K-Nearest Neighbour Impute (KNN), Bayesian Principal Component Analysis Impute (BPCA) and ZeroImpute. The statistical significance of this improved performance has been underscored by gene selection and also by applying the Wilcoxon Ranksum Significance Test, with results corroborating the ability of CF-GeNe to successfully infer GRN interactions in noisy gene expression data.
---
paper_title: Learning Bayesian Networks

paper_content:

This chapter addresses the problem of learning the parameters from data. It also discusses score-based structure learning and constraint-based structure learning. The method for learning all parameters in a Bayesian network follows readily from the method for learning a single parameter. The chapter presents a method for learning the probability of a binomial variable and extends this method to multinomial variables. It also provides guidelines for articulating the prior beliefs concerning probabilities. The chapter illustrates the constraint-based approach by showing how to learn a directed acyclic graph (DAG) faithful to a probability distribution. Structure learning consists of learning the DAG in a Bayesian network from data. It is necessary to know which DAG satisfies the Markov condition with the probability distribution P that is generating the data. The process of learning such a DAG is called “model selection.” A DAG includes a probability distribution P if the DAG does not entail any conditional independencies that are not in P. In score-based structure learning, a score is assigned to each DAG based on the data such that in the limit. After scoring the DAGs, the score are used, possibly along with prior probabilities, to learn a DAG. The most straightforward score, the Bayesian score, is the probability of the data D given the DAG. Once a DAG is learnt from data, the parameters can be known. The result will be a Bayesian network that can be used to do inference. In the constraint-based approach, a DAG is found for which the Markov condition entails all and only those conditional independencies that are in the probability distribution P of the variables of interest. The chapter applies structure learning to inferring causal influences from data and presents learning packages. It presents examples of learning Bayesian networks and of causal learning.
---
paper_title: A clustering-based approach for inferring recurrent neural networks as gene regulatory networks

paper_content:

Constructing genetic regulatory networks is one of the most important issues in system biology research. Yet, building regulatory models manually is a tedious task, especially when the number of genes involved increases with the complexity of regulation. To automate the procedure of network construction, in this work we establish a clustering-based approach to infer recurrent neural networks as regulatory systems. Our approach also deals with the scalability problem by developing a clustering method with several data analysis techniques. To verify the presented approach, experiments have been conducted and the results show that it can be used to infer gene regulatory networks successfully.
---
paper_title: Using a state-space model with hidden variables to infer transcription factor activities

paper_content:

Motivation: In a gene regulatory network, genes are typically regulated by transcription factors (TFs). Transcription factor activity (TFA) is more difficult to measure than gene expression levels are. Other models have extracted information about TFA from gene expression data, but without explicitly modeling feedback from the genes. We present a state-space model (SSM) with hidden variables. The hidden variables include regulatory motifs in the gene network, such as feedback loops and auto-regulation, making SSM a useful complement to existing models. ::: ::: Results: A gene regulatory network incorporating, for example, feed-forward loops, auto-regulation and multiple-inputs was constructed with an SSM model. First, the gene expression data were simulated by SSM and used to infer the TFAs. The ability of SSM to infer TFAs was evaluated by comparing the profiles of the inferred and simulated TFAs. Second, SSM was applied to gene expression data obtained from Escherichia coli K12 undergoing a carbon source transition and from the Saccharomyces cerevisiae cell cycle. The inferred activity profile for each TF was validated either by measurement or by activity information from the literature. The SSM model provides a probabilistic framework to simulate gene regulatory networks and to infer activity profiles of hidden variables. ::: ::: Availability: Supplementary data and Matlab code will be made available at the URL below. ::: ::: Contact: krischan@egr.msu.edu ::: ::: Supplementary information: http://www.chems.msu.edu/groups/chan/ssm.zip
---
paper_title: Reveal, A General Reverse Engineering Algorithm For Inference Of Genetic Network Architectures

paper_content:

Given the immanent gene expression mapping covering whole genomes during development, health and disease, we seek computational methods to maximize functional inference from such large data sets. Is it possible, in principle, to completely infer a complex regulatory network architecture from input/output patterns of its variables? We investigated this possibility using binary models of genetic networks. Trajectories, or state transition tables of Boolean nets, resemble time series of gene expression. By systematically analyzing the mutual information between input states and output states, one is able to infer the sets of input elements controlling each element or gene in the network. This process is unequivocal and exact for complete state transition tables. We implemented this REVerse Engineering ALgorithm (REVEAL) in a C program, and found the problem to be tractable within the conditions tested so far. For n = 50 (elements) and k = 3 (inputs per element), the analysis of incomplete state transition tables (100 state transition pairs out of a possible 10(15)) reliably produced the original rule and wiring sets. While this study is limited to synchronous Boolean networks, the algorithm is generalizable to include multi-state models, essentially allowing direct application to realistic biological data sets. The ability to adequately solve the inverse problem may enable in-depth analysis of complex dynamic systems in biology and other fields.
---
paper_title: Modeling gene expression regulatory networks with the sparse vector autoregressive model

paper_content:

BackgroundTo understand the molecular mechanisms underlying important biological processes, a detailed description of the gene products networks involved is required. In order to define and understand such molecular networks, some statistical methods are proposed in the literature to estimate gene regulatory networks from time-series microarray data. However, several problems still need to be overcome. Firstly, information flow need to be inferred, in addition to the correlation between genes. Secondly, we usually try to identify large networks from a large number of genes (parameters) originating from a smaller number of microarray experiments (samples). Due to this situation, which is rather frequent in Bioinformatics, it is difficult to perform statistical tests using methods that model large gene-gene networks. In addition, most of the models are based on dimension reduction using clustering techniques, therefore, the resulting network is not a gene-gene network but a module-module network. Here, we present the Sparse Vector Autoregressive model as a solution to these problems.ResultsWe have applied the Sparse Vector Autoregressive model to estimate gene regulatory networks based on gene expression profiles obtained from time-series microarray experiments. Through extensive simulations, by applying the SVAR method to artificial regulatory networks, we show that SVAR can infer true positive edges even under conditions in which the number of samples is smaller than the number of genes. Moreover, it is possible to control for false positives, a significant advantage when compared to other methods described in the literature, which are based on ranks or score functions. By applying SVAR to actual HeLa cell cycle gene expression data, we were able to identify well known transcription factor targets.ConclusionThe proposed SVAR method is able to model gene regulatory networks in frequent situations in which the number of samples is lower than the number of genes, making it possible to naturally infer partial Granger causalities without any a priori information. In addition, we present a statistical test to control the false discovery rate, which was not previously possible using other gene regulatory network models.
---
paper_title: Learning Large-Scale Graphical Gaussian Models from Genomic Data

paper_content:

The inference and modeling of network‐like structures in genomic data is of prime importance in systems biology. Complex stochastic associations and interdependencies can very generally be described as a graphical model. However, the paucity of available samples in current high‐throughput experiments renders learning graphical models from genome data, such as microarray expression profiles, a challenging and very hard problem. Here we review several recently developed approaches to small‐sample inference of graphical Gaussian modeling and discuss strategies to cope with the high dimensionality of functional genomics data. Particular emphasis is put on regularization methods and an empirical Bayes network inference procedure.
---
paper_title: Learning Bayesian Networks with Local Structure

paper_content:

We examine a novel addition to the known methods for learning Bayesian networks from data that improves the quality of the learned networks. Our approach explicitly represents and learns the local structure in the conditional probability distributions (CPDs) that quantify these networks. This increases the space of possible models, enabling the representation of CPDs with a variable number of parameters. The resulting learning procedure induces models that better emulate the interactions present in the data. We describe the theoretical foundations and practical aspects of learning local structures and provide an empirical evaluation of the proposed learning procedure. This evaluation indicates that learning curves characterizing this procedure converge faster, in the number of training instances, than those of the standard procedure, which ignores the local structure of the CPDs. Our results also show that networks learned with local structures tend to be more complex (in terms of arcs), yet require fewer parameters.
---
paper_title: Bayesian Orthogonal Least Squares (BOLS) algorithm for reverse engineering of gene regulatory networks

paper_content:

BackgroundA reverse engineering of gene regulatory network with large number of genes and limited number of experimental data points is a computationally challenging task. In particular, reverse engineering using linear systems is an underdetermined and ill conditioned problem, i.e. the amount of microarray data is limited and the solution is very sensitive to noise in the data. Therefore, the reverse engineering of gene regulatory networks with large number of genes and limited number of data points requires rigorous optimization algorithm.ResultsThis study presents a novel algorithm for reverse engineering with linear systems. The proposed algorithm is a combination of the orthogonal least squares, second order derivative for network pruning, and Bayesian model comparison. In this study, the entire network is decomposed into a set of small networks that are defined as unit networks. The algorithm provides each unit network with P(D|Hi), which is used as confidence level. The unit network with higher P(D|Hi) has a higher confidence such that the unit network is correctly elucidated. Thus, the proposed algorithm is able to locate true positive interactions using P(D|Hi), which is a unique property of the proposed algorithm.The algorithm is evaluated with synthetic and Saccharomyces cerevisiae expression data using the dynamic Bayesian network. With synthetic data, it is shown that the performance of the algorithm depends on the number of genes, noise level, and the number of data points. With Yeast expression data, it is shown that there is remarkable number of known physical or genetic events among all interactions elucidated by the proposed algorithm.The performance of the algorithm is compared with Sparse Bayesian Learning algorithm using both synthetic and Saccharomyces cerevisiae expression data sets. The comparison experiments show that the algorithm produces sparser solutions with less false positives than Sparse Bayesian Learning algorithm.ConclusionFrom our evaluation experiments, we draw the conclusion as follows: 1) Simulation results show that the algorithm can be used to elucidate gene regulatory networks using limited number of experimental data points. 2) Simulation results also show that the algorithm is able to handle the problem with noisy data. 3) The experiment with Yeast expression data shows that the proposed algorithm reliably elucidates known physical or genetic events. 4) The comparison experiments show that the algorithm more efficiently performs than Sparse Bayesian Learning algorithm with noisy and limited number of data.
---
paper_title: Network component analysis: Reconstruction of regulatory signals in biological systems

paper_content:

High-dimensional data sets generated by high-throughput technologies, such as DNA microarray, are often the outputs of complex networked systems driven by hidden regulatory signals. Traditional statistical methods for computing low-dimensional or hidden representations of these data sets, such as principal component analysis and independent component analysis, ignore the underlying network structures and provide decompositions based purely on a priori statistical constraints on the computed component signals. The resulting decomposition thus provides a phenomenological model for the observed data and does not necessarily contain physically or biologically meaningful signals. Here, we develop a method, called network component analysis, for uncovering hidden regulatory signals from outputs of networked systems, when only a partial knowledge of the underlying network topology is available. The a priori network structure information is first tested for compliance with a set of identifiability criteria. For networks that satisfy the criteria, the signals from the regulatory nodes and their strengths of influence on each output node can be faithfully reconstructed. This method is first validated experimentally by using the absorbance spectra of a network of various hemoglobin species. The method is then applied to microarray data generated from yeast Saccharamyces cerevisiae and the activities of various transcription factors during cell cycle are reconstructed by using recently discovered connectivity information for the underlying transcriptional regulatory networks.
---
paper_title: Reverse engineering of gene regulatory networks: a finite state linear model

paper_content:

We propose a new model for describing gene regulatory networks that can capture discrete (Boolean) and continuous (differential) aspects of gene regulation. After giving some illustrations of the model, we study the problem of the reverse engineering of such networks, i.e., how to construct a network from gene expression data. We prove that for our model there exists an algorithm finding a network compatible with the given data. We demonstrate the model by simulating lambda-phage. We also describe some generalizations of the model, discuss their relevance to the real-world gene networks and formulate a number of open problems.
---

Title: A survey of models for inference of gene regulatory networks
Section 1: Introduction
Description 1: Provide an overview of the paper and outline its structure.

Section 2: Biological and experimental backgrounds
Description 2: Discuss the biological foundations and experimental data essential for gene regulatory networks (GRNs) inference.

Section 3: Computational backgrounds of GRNs inference
Description 3: Describe the theoretical and computational approaches used for inferencing GRNs, including historical context.

Section 4: Gene regulatory networks models
Description 4: Present the different models used to infer GRNs, including Boolean networks, Bayesian networks, dynamic Bayesian networks, differential equations models, and association networks.

Section 5: Current trends - GRNs inference by integration of prior knowledge
Description 5: Detail the recent advancements and trends in inferring GRNs by integrating various types of prior knowledge.

Section 6: Model validation
Description 6: Discuss the methodologies used to validate inferred GRNs and the criteria used to assess the quality of these networks.

Section 7: Models comparison
Description 7: Compare the various GRNs models based on their advantages, disadvantages, and performance metrics.

Section 8: Conclusions and further works
Description 8: Summarize the survey's findings and suggest directions for future research in the field of GRNs inference.