Methods and techniques of complex systems science: An overview
---
paper_title: Robust perfect adaptation in bacterial chemotaxis through integral feedback control

paper_content:

‡Integral feedback control is a basic engineering strategy for ensuring that the output of a system robustly tracks its desired value independent of noise or variations in system parameters. In biological systems, it is common for the response to an extracellular stimulus to return to its prestimulus value even in the continued presence of the signal—a process termed adaptation or desensitization. Barkai, Alon, Surette, and Leibler have provided both theoretical and experimental evidence that the precision of adaptation in bacterial chemotaxis is robust to dramatic changes in the levels and kinetic rate constants of the constituent proteins in this signaling network [Alon, U., Surette, M. G., Barkai, N. & Leibler, S. (1998) Nature (London) 397, 168 ‐171]. Here we propose that the robustness of perfect adaptation is the result of this system possessing the property of integral feedback control. Using techniques from control and dynamical systems theory, we demonstrate that integral control is structurally inherent in the Barkai‐ Leibler model and identify and characterize the key assumptions of the model. Most importantly, we argue that integral control in some form is necessary for a robust implementation of perfect adaptation. More generally, integral control may underlie the robustness of many homeostatic mechanisms.
---
paper_title: The chemical basis of morphogenesis

paper_content:

It is suggested that a system of chemical substances, called morphogens, reacting together and diffusing through a tissue, is adequate to account for the main phenomena of morphogenesis. Such a system, although it may originally be quite homogeneous, may later develop a pattern or structure due to an instability of the homogeneous equilibrium, which is triggered off by random disturbances. Such reaction-diffusion systems are considered in some detail in the case of an isolated ring of cells, a mathematically convenient, though biologically unusual system. The investigation is chiefly concerned with the onset of instability. It is found that there are six essentially different forms which this may take. In the most interesting form stationary waves appear on the ring. It is suggested that this might account, for instance, for the tentacle patterns on Hydra and for whorled leaves. A system of reactions and diffusion on a sphere is also considered. Such a system appears to account for gastrulation. Another reaction system in two dimensions gives rise to patterns reminiscent of dappling. It is also suggested that stationary waves in two dimensions could account for the phenomena of phyllotaxis. The purpose of this paper is to discuss a possible mechanism by which the genes of a zygote may determine the anatomical structure of the resulting organism. The theory does not make any new hypotheses; it merely suggests that certain well-known physical laws are sufficient to account for many of the facts. The full understanding of the paper requires a good knowledge of mathematics, some biology, and some elementary chemistry. Since readers cannot be expected to be experts in all of these subjects, a number of elementary facts are explained, which can be found in text-books, but whose omission would make the paper difficult reading.
---
paper_title: Adaptation and Optimal Chemotactic Strategy for E. Coli

paper_content:

Extending the classic works of Berg and Purcell on the biophysics of bacterial chemotaxis, we find the optimal chemotactic strategy for the peritrichous bacterium {ital E. coli} in the high and low signal to noise ratio limits. The optimal strategy depends on properties of the environment and properties of the individual bacterium, and is therefore highly adaptive. We review experiments relevant to testing both the form of the proposed strategy and its adaptability, and propose extensions of them which could test the limits of the adaptability in this simplest sensory processing system. {copyright} {ital 1998} {ital The American Physical Society}
---
paper_title: Complexity: Hierarchical Structures and Scaling in Physics

paper_content:

Part I. Phenomenology and Models: 1. Introduction 2. Examples of complex behaviour 3. Mathematical models Part II: 4. Symbolic representations of physical systems 5. Probability, ergodic theory, and information 6. Thermodynamic formalism Part III. Formal Characterization of Complexity: 7. Physical and computational analysis of symbolic signals 8. Algorithmic and grammatical complexities 9. Hierarchical scaling complexities 10. Summary and perspectives.
---
paper_title: The self-made tapestry: pattern formation in nature

paper_content:

From the Publisher: ::: Why do similar patterns and forms appear in nature in settings that seem to bear no relation to one another? The Windblown ripples of desert sand follow a sinuous course that resembles the stripes of a zebra or a marine fish. In the trellis-like shells of microscopic sea creatures we see the same geometry as in the bubble walls of foam, Forks of lightning mirror the branches of a river network or a tree. ::: This book explains why these are not coincidences. Nature commonly weaves its tapestry without any master plan or blueprint. Instead, these designs build themselves by self-organization. The interactions between the component parts -- whether they be grains of sand, molecules or living cells -- give rise to spontaneous patterns that are at the same time complex and beautiful. Many of these patterns are universal, recurring again and again in the natural order: spirals, spots, stripes, branches, honeycombs. Philip Ball conducts a profusely illustrated tour of this gallery, and reveals the secrets of how nature's patterns are made.
---
paper_title: The Theory of Evolution and Dynamical Systems: Mathematical Aspects of Selection

paper_content:

Preface Part I. Selection Dynamics and Population Genetics: A Discrete Introduction: 1. The biological background 2. The Hardy-Weinberg law 3. Selection and the fundamental theorem 4. Mutation and recombination Part II. Growth Rates and Ecological Models: An ABC on ODE: 5. The ecology of populations 6. The logistic equation 7. Lotka-Volterra equations for predator prey-systems 8. Lotka-Volterra equations for two competing species 9. Lotka-Volterra equations for more than two populations Part III. Test Tube Evolution and Hypercycles: A Prebiotic Primer: 10. Prebiotic evolution 11. Branching processes and the complexity threshold 12. Catalytic groth f selfreproducing molecules 13. Permanence and the evolution of hypercycles Part IV. Strategies and Stability: An Opening in Game Dynamics: 14. Some aspects of sociobiology 15. Evolutionarily stable strategies 16. Game dynamics 17. Asymmetric conflicts Interlude.
---
paper_title: Robustness in bacterial chemotaxis.

paper_content:

Networks of interacting proteins orchestrate the responses of living cells to a variety of external stimuli, but how sensitive is the functioning of these protein networks to variations in their biochemical parameters? One possibility is that to achieve appropriate function, the reaction rate constants and enzyme concentrations need to be adjusted in a precise manner, and any deviation from these 'fine-tuned' values ruins the network's performance. An alternative possibility is that key properties of biochemical networks are robust; that is, they are insensitive to the precise values of the biochemical parameters. Here we address this issue in experiments using chemotaxis of Escherichia coli, one of the best-characterized sensory systems. We focus on how response and adaptation to attractant signals vary with systematic changes in the intracellular concentration of the components of the chemotaxis network. We find that some properties, such as steady-state behaviour and adaptation time, show strong variations in response to varying protein concentrations. In contrast, the precision of adaptation is robust and does not vary with the protein concentrations. This is consistent with a recently proposed molecular mechanism for exact adaptation, where robustness is a direct consequence of the network's architecture.
---
paper_title: Optimization Based on Bacterial Chemotaxis

paper_content:

We present an optimization algorithm based on a model of bacterial chemotaxis. The original biological model is used to formulate a simple optimization algorithm, which is evaluated on a set of standard test problems. Based on this evaluation, several features are added to the basic algorithm using evolutionary concepts in order to obtain an improved optimization strategy, called the bacteria chemotaxis (BC) algorithm. This strategy is evaluated on a number of test functions for local and global optimization, compared with other optimization techniques, and applied to the problem of inverse airfoil design. The comparisons show that on average, BC performs similar to standard evolution strategies and worse than evolution strategies with enhanced convergence properties.
---
paper_title: Lattice-gas automata for the Navier-Stokes equation.

paper_content:

A very brief presentation of how lattice gas hydrodynamics is made. It includes key references.
---
paper_title: Complexity, Entropy And The Physics Of Information

paper_content:

A water gel explosive composition and process for preparing the same is provided which comprises an oxidizer, water, a gelling agent, and a crosslinker, and wherein the improvement comprises including therein from about 1 to about 10 weight percent of at least one amine nitrate sensitizer selected from the group comprising lower alkyl and alkanol amine nitrates and from about 1 to about 10 weight percent of an aluminum sensitizer having a surface area per unit weight of from about 3 to about 9 square meters per gram, said weight percentages based upon the total weight of the explosive composition.
---
paper_title: Emergence: From Chaos to Order

paper_content:

From the Publisher: ::: From one of today's most innovative thinkers comes the first book to carefully explore emergence - a surprisingly simple notion (the whole is more than the sum of its parts) with enormous implications for science, business, and the arts. In this work, John Holland, a leader in the study of complexity at the Santa Fe Institute, dramatically shows that a theory of emergence can predict many complex behaviors, and has much to teach us about life, the mind, and organizations. In Emergence, Holland demonstrates that a small number of rules of laws can generate systems of surprising complexity. Board games provide an ancient and direct example: Chess is defined by fewer than two dozen rules, but the myriad patterns that result lead to perpetual novelty and emergence. It took centuries of study to recognize certain patterns of play, such as the control of pawn formations. But once recognized, these patterns greatly enhance the possibility of winning the game. The discovery of similar patterns in other facets of our world opens the way to a deeper understanding of the complexity of life, answering such questions as: How does a fertilized egg program the development of a trillion-cell organism? How can we build human organizations that respond rapidly to change through innovation? Throughout the book, Holland compares different systems and models that exhibit emergence in the quest for common rules or laws.
---
paper_title: Information Theory and an Extension of the Maximum Likelihood Principle

paper_content:

In this paper it is shown that the classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion. This observation shows an extension of the principle to provide answers to many practical problems of statistical model fitting.
---
paper_title: A theory of the learnable

paper_content:

Humans appear to be able to learn new concepts without needing to be programmed explicitly in any conventional sense. In this paper we regard learning as the phenomenon of knowledge acquisition in the absence of explicit programming. We give a precise methodology for studying this phenomenon from a computational viewpoint. It consists of choosing an appropriate information gathering mechanism, the learning protocol, and exploring the class of concepts that can be learnt using it in a reasonable (polynomial) number of steps. We find that inherent algorithmic complexity appears to set serious limits to the range of concepts that can be so learnt. The methodology and results suggest concrete principles for designing realistic learning systems.
---
paper_title: Nonparametric Time Series Prediction Through Adaptive Model Selection

paper_content:

We consider the problem of one-step ahead prediction for time series generated by an underlying stationary stochastic process obeying the condition of absolute regularity, describing the mixing nature of process. We make use of recent results from the theory of empirical processes, and adapt the uniform convergence framework of Vapnik and Chervonenkis to the problem of time series prediction, obtaining finite sample bounds. Furthermore, by allowing both the model complexity and memory size to be adaptively determined by the data, we derive nonparametric rates of convergence through an extension of the method of structural risk minimization suggested by Vapnik. All our results are derived for general L error measures, and apply to both exponentially and algebraically mixing processes.
---
paper_title: Measuring the VC-Dimension Using Optimized Experimental Design

paper_content:

VC-dimension is the measure of model complexity (capacity) used in VC-theory. The knowledge of the VC-dimension of an estimator is necessary for rigorous complexity control using analytic VC generalization bounds. Unfortunately, it is not possible to obtain the analytic estimates of the VC-dimension in most cases. Hence, a recent proposal is to measure the VC-dimension of an estimator experimentally by fitting the theoretical formula to a set of experimental measurements of the frequency of errors on artificially generated data sets of varying sizes (Vapnik, Levin, & Le Cun, 1994). However, it may be difficult to obtain an accurate estimate of the VC-dimension due to the variability of random samples in the experimental procedure proposed by Vapnik et al. (1994). We address this problem by proposing an improved design procedure for specifying the measurement points (i.e., the sample size and the number of repeated experiments at a given sample size). Our approach leads to a nonuniform design structure as opposed to the uniform design structure used in the original article (Vapnik et al., 1994). Our simulation results show that the proposed optimized design structure leads to a more accurate estimation of the VC-dimension using the experimental procedure. The results also show that a more accurate estimation of VC-dimension leads to improved complexity control using analytic VC-generalization bounds and, hence, better prediction accuracy.
---
paper_title: Neural Network Learning: Theoretical Foundations

paper_content:

This important work describes recent theoretical advances in the study of artificial neural networks. It explores probabilistic models of supervised learning problems, and addresses the key statistical and computational questions. Chapters survey research on pattern classification with binary-output networks, including a discussion of the relevance of the Vapnik Chervonenkis dimension, and of estimates of the dimension for several neural network models. In addition, Anthony and Bartlett develop a model of classification by real-output networks, and demonstrate the usefulness of classification with a "large margin." The authors explain the role of scale-sensitive versions of the Vapnik Chervonenkis dimension in large margin classification, and in real prediction. Key chapters also discuss the computational complexity of neural network learning, describing a variety of hardness results, and outlining two efficient, constructive learning algorithms. The book is self-contained and accessible to researchers and graduate students in computer science, engineering, and mathematics.
---
paper_title: An Introduction to Support Vector Machines and Other Kernel-based Learning Methods

paper_content:

From the publisher: This is the first comprehensive introduction to Support Vector Machines (SVMs), a new generation learning system based on recent advances in statistical learning theory. SVMs deliver state-of-the-art performance in real-world applications such as text categorisation, hand-written character recognition, image classification, biosequences analysis, etc., and are now established as one of the standard tools for machine learning and data mining. Students will find the book both stimulating and accessible, while practitioners will be guided smoothly through the material required for a good grasp of the theory and its applications. The concepts are introduced gradually in accessible and self-contained stages, while the presentation is rigorous and thorough. Pointers to relevant literature and web sites containing software ensure that it forms an ideal starting point for further study. Equally, the book and its associated web site will guide practitioners to updated literature, new applications, and on-line software.
---
paper_title: Learning Kernel Classifiers: Theory and Algorithms

paper_content:

From the Publisher: ::: Linear classifiers in kernel spaces have emerged as a major topic within the field of machine learning. The kernel technique takes the linear classifier--a limited, but well-established and comprehensively studied model--and extends its applicability to a wide range of nonlinear pattern-recognition tasks such as natural language processing, machine vision, and biological sequence analysis. This book provides the first comprehensive overview of both the theory and algorithms of kernel classifiers, including the most recent developments. It begins by describing the major algorithmic advances: kernel perceptron learning, kernel Fisher discriminants, support vector machines, relevance vector machines, Gaussian processes, and Bayes point machines. Then follows a detailed introduction to learning theory, including VC and PAC-Bayesian theory, data-dependent structural risk minimization, and compression bounds. Throughout, the book emphasizes the interaction between theory and algorithms: how learning algorithms work and why. The book includes many examples, complete pseudo code of the algorithms presented, and an extensive source code library.
---
paper_title: The Helmholtz Machine

paper_content:

Discovering the structure inherent in a set of patterns is a fundamental aim of statistical inference or learning. One fruitful approach is to build a parameterized stochastic generative model, independent draws from which are likely to produce the patterns. For all but the simplest generative models, each pattern can be generated in exponentially many ways. It is thus intractable to adjust the parameters to maximize the probability of the observed patterns. We describe a way of finessing this combinatorial explosion by maximizing an easily computed lower bound on the probability of the observations. Our method can be viewed as a form of hierarchical self-supervised learning that may relate to the function of bottom-up and top-down cortical processing pathways.
---
paper_title: Causality: Models, Reasoning, and Inference

paper_content:

1. Introduction to probabilities, graphs, and causal models 2. A theory of inferred causation 3. Causal diagrams and the identification of causal effects 4. Actions, plans, and direct effects 5. Causality and structural models in the social sciences 6. Simpson's paradox, confounding, and collapsibility 7. Structural and counterfactual models 8. Imperfect experiments: bounds and counterfactuals 9. Probability of causation: interpretation and identification Epilogue: the art and science of cause and effect.
---
paper_title: Causation, prediction, and search

paper_content:

What assumptions and methods allow us to turn observations into causal knowledge, and how can even incomplete causal knowledge be used in planning and prediction to influence and control our environment? In this book Peter Spirtes, Clark Glymour, and Richard Scheines address these questions using the formalism of Bayes networks, with results that have been applied in diverse areas of research in the social, behavioral, and physical sciences. The authors show that although experimental and observational study designs may not always permit the same inferences, they are subject to uniform principles. They axiomatize the connection between causal structure and probabilistic independence, explore several varieties of causal indistinguishability, formulate a theory of manipulation, and develop asymptotically reliable procedures for searching over equivalence classes of causal models, including models of categorical data and structural equation models with and without latent variables. The authors show that the relationship between causality and probability can also help to clarify such diverse topics in statistics as the comparative power of experimentation versus observation, Simpson's paradox, errors in regression models, retrospective versus prospective sampling, and variable selection. The second edition contains a new introduction and an extensive survey of advances and applications that have appeared since the first edition was published in 1993.
---
paper_title: Empirical Processes in M-Estimation

paper_content:

A coating composition suitable for coating the interior of tins comprises a solvent having dissolved therein at least 18% of two copolymers A and B in a weight ratio between 98.2 and 75.25 together with 2 to 10% (by weight of A+B) of an esterified aminoplast resin C. The solvent must comprise at least 75% of a liquid mononuclear aromatic hydrocarbon; copolymer A is a copolymer of 55 to 75% vinyl chloride with a C6- 24 diester of maleic, fumaric and/or chloromaleic acid which is completely soluble at 25% solids in toluene and has a relative viscosity of 1.3 to 1.7 as a 1% solution in cyclohexanone at 20 DEG C.; B is a copolymer of 60 to 92% vinylchloride with a vinyl fatty acid ester saponified to 2 to 10% vinyl alcohol content; and resin C is soluble in the solvent and is obtained by etherifying a condensate of urea, melamine or benzoguanamine and excess formaldehyde with a C3- 8 primary saturated monohydric alcohol. The solvent may contain up to 25% of a polar solvent e.g. ketones, esters, cyclic oxygen compounds, ether alcohols and their esters, nitrocompounds, amides and nitriles. To obtain copolymers A having the specified properties (indicating low molecular weight) copolymerisation may be effected in the presence of 1 to 6.5% of a non-polymerising halohydrocarbon, many examples being given. Copolymers B should have a molecular weight of 5,000 to 20,000 and corrected iodine number 2 to 10. In examples the compositions are coated on primed electrolytic tinplate. Primers specified are oleoresinous phenol aldehyde varnishes, epoxy resins mixed with a urea-formaldehyde and a phenol-formaldehyde resin, and copolymers of butadiene 1, 2-with styrene. Specification 883,070 is referred to.ALSO:A coating composition suitable for coating metal to be subsequently formed into tins comprises a solvent having dissolved therein at least 18% of two copolymers A and B in a weight ratio between 98,2 and 75,25 together with 2-10% (by weight of A+B) of an esterified aminoplast resin C. The solvent must comprise at least 75% of a liquid mononuclear aromatic hydrocarbon; copolymer A is a copolymer of 55-75% vinyl chloride with a C6-24 diester of maleic, fumaric and/or chloromaleic acid which is completely soluble at 25% solids in toluene and has a relative viscosity of 1,3-1,7 as a 1% solution in cyclohexanone at 20 DEG C.; B is a copolymer of 60-92% vinyl-chloride with a vinyl fatty acid ester saponified to 2-10% vinyl alcohol content; and resin C is soluble in the solvent and is obtained by etherifying a condensate of urea, melamine or benzoguanamine and excess formaldehyde with a C3-8 primary saturated monohydric alcohol. The solvent may contain up to 25% of a polar solvent, e.g. ketones, esters, cyclic oxygen compounds, ether alcohols and their esters, nitrocompounds, amides and nitriles. Copolymers B should have a molecular weight of 5000-20,000 and corrected iodine number of 2-10. In examples the compositions are coated on primed electrolytic tinplate and baked. Primers specified are oleoresinous phenol aldehyde varnishes, epoxy resins mixed with a ureaformaldehyde and a phenol-formaldehyde resin, and copolymers of butadiene-1,2 with styrene. Specification 883,070 is referred to.
---
paper_title: The Role of Occam's Razor in Knowledge Discovery

paper_content:

Many KDD systems incorporate an implicit or explicit preference for simpler models, but this use of “Occam‘s razor” has been strongly criticized by several authors (e.g., Schaffer, 1993s Webb, 1996). This controversy arises partly because Occam‘s razor has been interpreted in two quite different ways. The first interpretation (simplicity is a goal in itself) is essentially correct, but is at heart a preference for more comprehensible models. The second interpretation (simplicity leads to greater accuracy) is much more problematic. A critical review of the theoretical arguments for and against it shows that it is unfounded as a universal principle, and demonstrably false. A review of empirical evidence shows that it also fails as a practical heuristic. This article argues that its continued use in KDD risks causing significant opportunities to be missed, and should therefore be restricted to the comparatively few applications where it is appropriate. The article proposes and reviews the use of domain constraints as an alternative for avoiding overfitting, and examines possible methods for handling the accuracy–comprehensibility trade-off.
---
paper_title: Error And The Growth Of Experimental Knowledge

paper_content:

Thank you very much for downloading error and the growth of experimental knowledge. Maybe you have knowledge that, people have search numerous times for their chosen readings like this error and the growth of experimental knowledge, but end up in malicious downloads. Rather than reading a good book with a cup of coffee in the afternoon, instead they are facing with some malicious bugs inside their laptop.
---
paper_title: Asymptotic optimal inference for non-ergodic models

paper_content:

0. An Over-view.- 1. Introduction.- 2. The Classical Fisher-Rao Model for Asymptotic Inference.- 3. Generalisation of the Fisher-Rao Model to Non-ergodic Type Processes.- 4. Mixture Experiments and Conditional Inference.- 5. Non-local Results.- 1. A General Model and Its Local Approximation.- 1. Introduction.- 2. LAMN Families.- 3. Consequences of the LAMN Condition.- 4. Sufficient Conditions for the LAMN Property.- 5. Asymptotic Sufficiency.- 6. An Example (Galton-Watson Branching Process).- 7. Bibliographical Notes.- 2. Efficiency of Estimation.- 1. Introduction.- 2. Asymptotic Structure of Limit Distributions of Sequences of Estimators.- 3. An Upper Bound for the Concentration.- 4. The Existence and Optimality of the Maximum Likelihood Estimators.- 5. Optimality of Bayes Estimators.- 6. Bibliographical Notes.- 3. Optimal Asymptotic Tests.- 1. Introduction.- 2. The Optimality Criteria: Definitions.- 3. An Efficient Test of Simple Hypotheses: Contiguous Alternatives.- 4. Local Efficiency and Asymptotic Power of the Score Statistic.- 5. Asymptotic Power of the Likelihood Ratio Test: Simple Hypothesis.- 6. Asymptotic Powers of the Score and LR Statistics for Composite Hypotheses with Nuisance Parameters.- 7. An Efficient Test of Composite Hypotheses with Contiguous Alternatives.- 8. Examples.- 9. Bibliographical Notes.- 4. Mixture Experiments and Conditional Inference.- 1. Introduction.- 2. Mixture of Exponential Families.- 3. Some Examples.- 4. Efficient Conditional Tests with Reference to L.- 5. Efficient Conditional Tests with Reference to L?.- 6. Efficient Conditional Tests with Reference to LC: Bahadur Efficiency.- 7. Efficiency of Conditional Maximum Likelihood Estimators.- 8. Conditional Tests for Markov Sequences and Their Mixtures.- 9. Some Heuristic Remarks about Conditional Inference for the General Model.- 10. Bibliographical Notes.- 5. Some Non-local Results.- 1. Introduction.- 2. Non-local Behaviour of the Likelihood Ratio.- 3. Examples.- 4. Non-local Efficiency Results for Simple Likelihood Ratio Tests.- 5. Bibiographical Notes.- Appendices.- A.1 Uniform and Continuous Convergence.- A.2 Contiguity of Probability Measures.- References.
---
paper_title: Entropy and Information Theory

paper_content:

This book is an updated version of the information theory classic, first published in 1990. About one-third of the book is devoted to Shannon source and channel coding theorems; the remainder addresses sources, channels, and codes and on information and distortion measures and their properties. New in this edition:Expanded treatment of stationary or sliding-block codes and their relations to traditional block codesExpanded discussion of results from ergodic theory relevant to information theoryExpanded treatment of B-processes -- processes formed by stationary coding memoryless sourcesNew material on trading off information and distortion, including the Marton inequalityNew material on the properties of optimal and asymptotically optimal source codesNew material on the relationships of source coding and rate-constrained simulation or modeling of random processesSignificant material not covered in other information theory texts includes stationary/sliding-block codes, a geometric view of information theory provided by process distance measures, and general Shannon coding theorems for asymptotic mean stationary sources, which may be neither ergodic nor stationary, and d-bar continuous channels.
---
paper_title: Numerical Recipes In C The Art Of Scientific Computing

paper_content:

Thank you very much for reading numerical recipes in c the art of scientific computing. Maybe you have knowledge that, people have search hundreds times for their favorite readings like this numerical recipes in c the art of scientific computing, but end up in malicious downloads. Rather than enjoying a good book with a cup of tea in the afternoon, instead they are facing with some infectious bugs inside their desktop computer.
---
paper_title: Time Series Analysis and Its Applications

paper_content:

Characteristics of time series.- Time series regression and exploratory data analysis.- ARIMA models.- Spectral analysis and filtering.- Additional time domain topics.- State-space models.- Statistical methods in the frequency domain.
---
paper_title: Stochastic Dynamical Systems: Concepts, Numerical Methods, Data Analysis

paper_content:

A textbook designed for those commencing study in stochastic dynamical systems. It is addressed to students of physics, mathematics, chemistry, also those engaged in medical research, and aims to bridge mathematics and the applied sciences. It provides numerical methods and the tools to analyze complex systems. The book contains various stochastic and mathematical methods which are necessary for the analysis of complex systems such as polymeric melts, the human body and the atmosphere. Data analysis is treated, as well as simulation methods for given circumstances.
---
paper_title: Time Series Analysis and Its Applications

paper_content:

Characteristics of time series.- Time series regression and exploratory data analysis.- ARIMA models.- Spectral analysis and filtering.- Additional time domain topics.- State-space models.- Statistical methods in the frequency domain.
---
paper_title: Predictive Turbulence Modeling by Variational Closure

paper_content:

We show that a variational implementation of probability density function (PDF) closures has the potential to make predictions of general turbulence mean statistics for which a priori knowledge of the incorrectness is possible. This possibility exists because of realizability conditions on “effective potential” functions for general turbulence statistics. These potentials measure the cost for fluctuations to occur away from the ensemble-mean value in empirical time-averages of the given variable, and their existence is a consequence of a refined ergodic hypothesis for the governing dynamical system (Navier–Stokes dynamics). Approximations of the effective potentials can be calculated within PDF closures by an efficient Rayleigh–Ritz algorithm. The failure of realizability within a closure for the approximate potential of any chosen statistic implies a priori that the closure prediction for that statistic is not converged. The systematic use of these novel realizability conditions within PDF closures is shown in a simple 3-mode system of Lorenz to result in a statistically improved predictive ability. In certain cases the variational method allows an a priori optimum choice of free parameters in the closure to be made.
---
paper_title: Parametric modelling of turbulence

paper_content:

Some steps are taken towards a parametric statistical model for the velocity and velocity derivative fields in stationary turbulence, building on the background of existing theoretical and empirical knowledge of such fields. While the ultimate goal is a model for the three-dimensional velocity components, and hence for the corresponding velocity derivatives, we concentrate here on the stream wise velocity component. Discrete and continuous time stochastic processes of the first-order autoregressive type and with one-dimensional marginals having log-linear tails are constructed and compared with two large data-sets. It turns out that a first-order autoregression that fits the local correlation structure well is not capable of describing the correlations over longer ranges. A good fit locally as well as at longer ranges is achieved by using a process that is the sum of two independent autoregressions. We study this type of model in some detail. We also consider a model derived from the above-mentioned autoregressions and with dependence structure on the borderline to long-range dependence. This model is obtained by means of a general method for construction of processes with long-range dependence. Some suggestions for future empirical and theoretical work are given.
---
paper_title: A Variational Formulation of Optimal Nonlinear Estimation

paper_content:

We propose a variational method to solve all three estimation problems for nonlinear stochastic dynamical systems: prediction, filtering, and smoothing. Our new approach is based upon a proper choice of cost function, termed the {\it effective action}. We show that this functional of time-histories is the unique statistically well-founded cost function to determine most probable histories within empirical ensembles. The ensemble dispersion about the sample mean history can also be obtained from the Hessian of the cost function. We show that the effective action can be calculated by a variational prescription, which generalizes the ``sweep method'' used in optimal linear estimation. An iterative numerical scheme results which converges globally to the variational estimator. This scheme involves integrating forward in time a ``perturbed'' Fokker-Planck equation, very closely related to the Kushner-Stratonovich equation for optimal filtering, and an adjoint equation backward in time, similarly related to the Pardoux-Kushner equation for optimal smoothing. The variational estimator enjoys a somewhat weaker property, which we call ``mean optimality''. However, the variational scheme has the principal advantage---crucial for practical applications---that it admits a wide variety of finite-dimensional moment-closure approximations. The moment approximations are derived reductively from the Euler-Lagrange variational formulation and preserve the good structural properties of the optimal estimator.
---
paper_title: The Lure of Modern Science: Fractal Thinking

paper_content:

Lure of modern science linear spaces and geometry in natural philosophy noise in natural philosophy self-similarity, fractals and measurements maps and dynamics dynamics in fractal dimensions.
---
paper_title: Nonparametric statistics for stochastic processes : estimation and prediction

paper_content:

Synopsis.- 1. Inequalities for mixing processes.- 2. Density estimation for discrete time processes.- 3. Regression estimation and prediction for discrete time processes.- 4. Kernel density estimation for continuous time processes.- 5. Regression estimation and prediction in continuous time.- 6. The local time density estimator.- 7. Implementation of nonparametric method and numerical applications.- References.
---
paper_title: Intrinsic limits on dimension calculations

paper_content:

The combined influences of boundary effects at large scales and nonzero nearest neighbor separations at small scales are used to compute intrinsic limits on the minimum size of a data set required for calculation of scaling exponents. A lower bound on the number of points required for a reliable estimation of the correlation exponent is given in terms of the dimension of the object and the desired accuracy. A method of estimating the correlation integral computed from a finite sample of a white noise signal is given.
---
paper_title: Investigating nonlinear dynamics from time series: The influence of symmetries and the choice of observables

paper_content:

When a dynamical system is investigated from a time series, one of the most challenging problems is to obtain a model that reproduces the underlying dynamics. Many papers have been devoted to this problem but very few have considered the influence of symmetries in the original system and the choice of the observable. Indeed, it is well known that there are usually some variables that provide a better representation of the underlying dynamics and, consequently, a global model can be obtained with less difficulties starting from such variables. This is connected to the problem of observing the dynamical system from a single time series. The roots of the nonequivalence between the dynamical variables will be investigated in a more systematic way using previously defined observability indices. It turns out that there are two important ingredients which are the complexity of the coupling between the dynamical variables and the symmetry properties of the original system. As will be mentioned, symmetries and the choice of observables also has important consequences in other problems such as synchronization of nonlinear oscillators. (c) 2002 American Institute of Physics.
---
paper_title: Nonlinear Time Series Analysis

paper_content:

Part I. Basic Concepts: 1. Introduction: why nonlinear methods? 2. Linear tools and general considerations 3. Phase space methods 4. Determinism and predictability 5. Instability: Lyapunov exponents 6. Self-similarity: dimensions 7. Using nonlinear methods when determinism is weak 8. Selected nonlinear phenomena Part II. Advanced Topics: 9. Advanced embedding methods 10. Chaotic data and noise 11. More about invariant quantities 12. Modeling and forecasting 13. Chaos control 14. Other selected topics Appendix 1. Efficient neighbour searching Appendix 2. Program listings Appendix 3. Description of the experimental data sets.
---
paper_title: Lectures on Discrete Time Filtering

paper_content:

This text is based on a course given at the University of Southern California, at the University of Nice, and at Cheng Kung University in Taiwan. It discusses linear and nonlinear sequential filtering theory: that is, the problem of estimating the process underlying a stochastic signal. For the linear coloured-noise problem, the theory is due to Kalman, and in the case of white noise it is the continuous Kalman-Bucy theory. The techniques considered have applications in fields as diverse as economics (prediction of the money supply), geophysics (processing of sonar signals), electrical engineering (detection of radar signals), and numerical analysis (in integration packages). The nonlinear theory is treated thoroughly, along with some novel synthesis methods for this computationally demanding problem. The author also discusses the Burg technique, and gives a detailed analysis of the matrix Riccati equation.
---
paper_title: A Variational Formulation of Optimal Nonlinear Estimation

paper_content:

We propose a variational method to solve all three estimation problems for nonlinear stochastic dynamical systems: prediction, filtering, and smoothing. Our new approach is based upon a proper choice of cost function, termed the {\it effective action}. We show that this functional of time-histories is the unique statistically well-founded cost function to determine most probable histories within empirical ensembles. The ensemble dispersion about the sample mean history can also be obtained from the Hessian of the cost function. We show that the effective action can be calculated by a variational prescription, which generalizes the ``sweep method'' used in optimal linear estimation. An iterative numerical scheme results which converges globally to the variational estimator. This scheme involves integrating forward in time a ``perturbed'' Fokker-Planck equation, very closely related to the Kushner-Stratonovich equation for optimal filtering, and an adjoint equation backward in time, similarly related to the Pardoux-Kushner equation for optimal smoothing. The variational estimator enjoys a somewhat weaker property, which we call ``mean optimality''. However, the variational scheme has the principal advantage---crucial for practical applications---that it admits a wide variety of finite-dimensional moment-closure approximations. The moment approximations are derived reductively from the Euler-Lagrange variational formulation and preserve the good structural properties of the optimal estimator.
---
paper_title: Complexity: Hierarchical Structures and Scaling in Physics

paper_content:

Part I. Phenomenology and Models: 1. Introduction 2. Examples of complex behaviour 3. Mathematical models Part II: 4. Symbolic representations of physical systems 5. Probability, ergodic theory, and information 6. Thermodynamic formalism Part III. Formal Characterization of Complexity: 7. Physical and computational analysis of symbolic signals 8. Algorithmic and grammatical complexities 9. Hierarchical scaling complexities 10. Summary and perspectives.
---
paper_title: Hidden Markov Models: Estimation and Control

paper_content:

Hidden Markov Model Processing.- Discrete-Time HMM Estimation.- Discrete States and Discrete Observations.- Continuous-Range Observations.- Continuous-Range States and Observations.- A General Recursive Filter.- Practical Recursive Filters.- Continuous-Time HMM Estimation.- Discrete-Range States and Observations.- Markov Chains in Brownian Motion.- Two-Dimensional HMM Estimation.- Hidden Markov Random Fields.- HMM Optimal Control.- Discrete-Time HMM Control.- Risk-Sensitive Control of HMM.- Continuous-Time HMM Control.
---
paper_title: A View Of The Em Algorithm That Justifies Incremental, Sparse, And Other Variants

paper_content:

The EM algorithm performs maximum likelihood estimation for data in which some variables are unobserved. We present a function that resembles negative free energy and show that the M step maximizes this function with respect to the model parameters and the E step maximizes it with respect to the distribution over the unobserved variables. From this perspective, it is easy to justify an incremental variant of the EM algorithm in which the distribution for only one of the unobserved variables is recalculated in each E step. This variant is shown empirically to give faster convergence in a mixture estimation problem. A variant of the algorithm that exploits sparse conditional distributions is also described, and a wide range of other variant algorithms are also seen to be possible.
---
paper_title: Variable length Markov chains

paper_content:

We study estimation in the class of stationary variable length Markov chains (VLMC) on a finite space. The processes in this class are still Markovian of high order, but with memory of variable length yielding a much bigger and structurally richer class of models than ordinary high-order Markov chains. From an algorithmic view, the VLMC model class has attracted interest in information theory and machine learning, but statistical properties have not yet been explored. Provided that good estimation is available, the additional structural richness of the model class enhances predictive power by finding a better trade-off between model bias and variance and allowing better structural description which can be of specific interest. The latter is exemplified with some DNA data. A version of the tree-structured context algorithm, proposed by Rissanen in an information theoretical set-up is shown to have new good asymptotic properties for estimation in the class of VLMCs. This remains true even when the underlying model increases in dimensionality. Furthermore, consistent estimation of minimal state spaces and mixing properties of fitted models are given. We also propose a new bootstrap scheme based on fitted VLMCs. We show its validity for quite general stationary categorical time series and for a broad range of statistical procedures.
---
paper_title: The Context Tree Weighting Method: Basic Properties

paper_content:

Describes a sequential universal data compression procedure for binary tree sources that performs the "double mixture." Using a context tree, this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources, and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters. Computational and storage complexity of the proposed procedure are both linear in the source sequence length. The authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences. The three terms in this bound can be identified as coding, parameter, and model redundancy, The bound holds for all source sequence lengths, not only for asymptotically large lengths. The analysis that leads to this bound is based on standard techniques and turns out to be extremely simple. The upper bound on the redundancy shows that the proposed context-tree weighting procedure is optimal in the sense that it achieves the Rissanen (1984) lower bound. >
---
paper_title: Context-tree modeling of observed symbolic dynamics.

paper_content:

Modern techniques invented for data compression provide efficient automated algorithms for the modeling of the observed symbolic dynamics. We demonstrate the relationship between coding and modeling, motivating the well-known minimum description length (MDL) principle, and give concrete demonstrations of the ``context-tree weighting'' and ``context-tree maximizing'' algorithms. The predictive modeling technique obviates many of the technical difficulties traditionally associated with the correct MDL analyses. These symbolic models, representing the symbol generating process as a finite-state automaton with probabilistic emission probabilities, provide excellent and reliable entropy estimations. The resimulations of estimated tree models satisfying the MDL model-selection criterion are faithful to the original in a number of measures. The modeling suggests that the automated context-tree model construction could replace fixed-order word lengths in many traditional forms of empirical symbolic analysis of the data. We provide an explicit pseudocode for implementation of the context-tree weighting and maximizing algorithms, as well as for the conversion to an equivalent Markov chain.
---
paper_title: Complexity: Hierarchical Structures and Scaling in Physics

paper_content:

Part I. Phenomenology and Models: 1. Introduction 2. Examples of complex behaviour 3. Mathematical models Part II: 4. Symbolic representations of physical systems 5. Probability, ergodic theory, and information 6. Thermodynamic formalism Part III. Formal Characterization of Complexity: 7. Physical and computational analysis of symbolic signals 8. Algorithmic and grammatical complexities 9. Hierarchical scaling complexities 10. Summary and perspectives.
---
paper_title: Estimating good discrete partitions from observed data: symbolic false nearest neighbors.

paper_content:

A symbolic analysis of observed time series requires a discrete partition of a continuous state space containing the dynamics. A particular kind of partition, called "generating," preserves all deterministic dynamical information in the symbolic representation, but such partitions are not obvious beyond one dimension. Existing methods to find them require significant knowledge of the dynamical evolution operator. We introduce a statistic and algorithm to refine empirical partitions for symbolic state reconstruction. This method optimizes an essential property of a generating partition, avoiding topological degeneracies, by minimizing the number of "symbolic false nearest neighbors." It requires only the observed time series and is sensible even in the presence of noise when no truly generating partition is possible.
---
paper_title: Validity of threshold-crossing analysis of symbolic dynamics from chaotic time series

paper_content:

A practical and popular technique to extract the symbolic dynamics from experimentally measured chaotic time series is the threshold-crossing method, by which an arbitrary partition is utilized for determining the symbols. We address to what extent the symbolic dynamics so obtained can faithfully represent the phase-space dynamics. Our principal result is that such a practice can lead to a severe misrepresentation of the dynamical system. The measured topological entropy is a Devil's staircase-like, but surprisingly nonmonotone, function of a parameter characterizing the amount of misplacement of the partition.
---
paper_title: Symbolic Dynamics: One-sided, Two-sided and Countable State Markov Shifts

paper_content:

1. Background and Basics.- 1.1 Subshifts of Finite Type.- 1.2 Examples.- 1.3 Perron-Frobenius Theory.- 1.4 Basic Dynamics.- Notes.- References.- 2. Topology Conjugacy.- 2.1 Decomposition of Topological Conjugacies.- 2.2 Algebraic Consequences of Topological Conjugacy.- Notes.- References.- 3. Automorphisms.- 3.1 Automorphisms.- 3.2 Automorphisms as Conjugacies.- 3.3 Subgroups of the Automorphism Group.- 3.4 Actions of Automorphisms.- 3.5 Summary.- Notes.- References.- 4. Embeddinggs and Factor Maps.- 4.1 Factor Maps.- 4.2 Finite-to-one Factor Maps.- 4.3 Special Constructions Involving Factor Maps.- 4.4 Subsystems and Infinite-to-One Factor Maps.- Notes.- References.- 5. Almost-Topological Conjugacy.- 5.1 Reducible Subshifts of Finite Type.- 5.2 Almost-Topological Conjugacy.- Notes.- References.- 6. Further Topics.- 6.1 Sofic Systems.- 6.2 Markov Measures and the Maximal Measure.- 6.3 Markov Subgroups.- 6.4 Cellular Automata.- 6.5 Channnel Codes.- Notes.- References.- 7. Countable State Markov Shifts.- 7.1 Perron-Frobenius Theory.- 7.2 Basic Symbolic Dynamics.- Notes.- References.- Name Index.
---
paper_title: Symbolic dynamics of noisy chaos

paper_content:

Abstract One model of randomness observed in physical systems is that low-dimensional deterministic chaotic attractors underly the observations. A phenomenological theory of chaotic dynamics requires an accounting of the information flow from the observed system to the observer, the amount of information available in observations, and just how this information affects predictions of the system's future behavior. In an effort to develop such a description, we discuss the information theory of highly discretized observations of random behavior. Metric entropy and topological entropy are well-defined invariant measures of such an attractor's “level of chaos”, and are computable using symbolic dynamics. Real physical systems that display low dimensional dynamics are, however, inevitably coupled to high-dimensional randomless, e.g. thermal noise. We investigate the effects of such fluctuations coupled to deterministic chaotic systems, in particular, the metric entropy's response to the fluctuations. We find that the entropy increases with a power law in the noise level, and that the convergence of the entropy and the effect of fluctuations can be cast as a scaling theory. We also argue that in addition to the metric entropy, there is a second scaling invariant quantity that characterizes a deterministic system with added fluctuations: I0, the maximum average information obtainable about the initial condition that produces a particular sequence of measurements (or symbols).
---
paper_title: Unreconstructible At Any Radius

paper_content:

Abstract Modeling pattern data series with cellular automata fails for a wide range of deterministic nonlinear spatial processes. If the latter have finite spatially-local memory, reconstructed cellular automata with infinite radius may be required. In some cases, even this is not adequate: an irreducible stochasticity remains on the shortest time scales. The underlying problem is illustrated and quantitatively analyzed using an alternative model class called cellular transducers.
---
paper_title: Lattice Gas Prediction is P-Complete

paper_content:

We show that predicting the HPP or FHP III lattice gas for finite time is equivalent to calculating the output of an arbitrary Boolean circuit, and is therefore P-complete: that is, it is just as hard as any other problem solvable by a serial computer in polynomial time. It is widely believed in computer science that there are inherently sequential problems, for which parallel processing gives no significant speedup. Unless this is false, it is impossible even with highly parallel processing to predict lattice gases much faster than by explicit simulation. More precisely, we cannot predict t time-steps of a lattice gas in parallel computation time O(log^{k}t) for any k, or O(t^{\alpha}) for
---
paper_title: Majority-vote cellular automata, Ising dynamics, and P-completeness

paper_content:

We study cellular automata where the state at each site is decided by a majority vote of the sites in its neighborhood. These are equivalent, for a restricted set of initial conditions, to nonzero probability transitions in single spin-flip dynamics of the Ising model at zero temperature. We show that in three or more dimensions these systems can simulate Boolean circuits of AND and OR gates, and are therefore P-complete. That is, predicting their state t time-steps in the future is at least as hard as any other problem that takes polynomial time on a serial computer. Therefore, unless a widely believed conjecture in computer science is false, it is impossible even with parallel computation to predict majority-vote cellular automata, or zero-temperature single spin-flip Ising dynamics, qualitatively faster than by explicit simulation.
---
paper_title: Threshold-range scaling of excitable cellular automata

paper_content:

Each cell of a two-dimensional lattice is painted one of κ colors, arranged in a ‘color wheel’. The colors advance (k tok+1 mod κ) either automatically or by contact with at least a threshold number of successor colors in a prescribed local neighborhood. Discrete-time parallel systems of this sort in which color 0 updates by contact and the rest update automatically are called Greenberg-Hastings (GH) rules. A system in which all colors update by contact is called a cyclic cellular automation (CCA). Started from appropriate initial conditions, these models generate periodic traveling waves. Started from random configurations the same rules exhibit complex self-organization, typically characterized by nucleation of locally periodic ‘ram's horns’ or spirals. Corresponding random processes give rise to a variety of ‘forest fire’ equilibria that display large-scale stochastic wave fronts. This paper describes a framework, theoretically based, but relying on extensive interactive computer graphics experimentation, for investigation of the complex dynamics shared by excitable media in a broad spectrum of scientific contexts. By focusing on simple mathematical prototypes we hope to obtain a better understanding of the basic organizational principles underlying spatially distributed oscillating systems.
---
paper_title: Lattice-gas automata for the Navier-Stokes equation.

paper_content:

A very brief presentation of how lattice gas hydrodynamics is made. It includes key references.
---
paper_title: Lattice-Gas Cellular Automata: Simple Models of Complex Hydrodynamics

paper_content:

Preface Acknowledgements 1. A simple model of fluid mechanics 2. Two routes to hydrodynamics 3. Inviscid two-dimensional lattice-gas hydrodynamics 4. Viscous two-dimensional hydrodynamics 5. Some simple 3D models 6. The lattice-Boltzmann method 7. Using the Boltzmann method 8. Miscible fluids 9. Immiscible lattice gases 10. Lattice-Boltzmann method for immiscible fluids 11. Immiscible lattice gases in three dimensions 12. Liquid-gas models 13. Flow through porous media 14. Equilibrium statistical mechanics 15. Hydrodynamics in the Boltzmann approximation 16. Phase separation 17. Interfaces 18. Complex fluids and patterns Appendices Author Index Subject Index.
---
paper_title: Molecular dynamics of a classical lattice gas: Transport properties and time correlation functions

paper_content:

A study of the dynamics of a discrete two-dimensional system of classical particles is presented. In this model, dynamics and computations may be done exactly, by definition. The equilibrium state is investigated and the Navier-Stokes hydrodynamical equations are derived. Two hydrodynamical modes exist in the model: the sound waves and a kind of vorticity diffusion. In the Navier-Stokes equations one obtains a transport coefficient which is given by a Green-Kubo formula. The related time correlation function has been calculated in a numerical simulation up to a time of the order of 50 mean free flights. After a short time of exponential decay this time correlation behaves like ${t}^{\ensuremath{-}S}$, the exponent being compared to theoretical predictions.
---
paper_title: Understanding Object-Oriented Programming with Java

paper_content:

From the Publisher: ::: Timothy Budd, leading author, educator and researcher in the object-oriented programming community, provides a deep understanding of object-oriented programming and Java. ::: Understanding Object-Oriented Programming with Java teaches readers why the Java language works the way it does, as opposed to many other books that focus on how Java works. ::: Readers learn about the development decisions that went into making the Java language, and leave with a more sophisticated knowledge of Java and how it fits in the context of object-oriented programming. Throughout the text, the focus remains on teaching readers to master the necessary object-oriented programming concepts. Dr. Budd explains to the reader in clear and simple terms the fundamental principles of object-oriented programming, illustrating these principles with extensive examples from the Java standard library. In short, he thoughtfully created this book, not as a reference manual for the Java language, but as a tool for understanding Java and the object-oriented programming philosophy. ::: Highlights: ::: Provides several graduated example programs in Part II (i.e., cannon and pinball games) for readers to work through and successively learn object-oriented programming features. ::: Includes extensive examples from the Java standard library so that readers can better understand the many design patterns found in the AWT, the multiple purposes for which inheritance is used in the standard classes, and more. Discusses features of Java in Part V that are important for students to understand, but not necessarily notable for their object-oriented features. Instructors have the flexibility to omit altogether, or introduce in parallel with earlier material.
---
paper_title: Turtles, termites, and traffic jams: explorations in massively parallel microworlds

paper_content:

Part 1 Foundations: introduction the era of decentralization. Part 2 Constructions: constructionism LEGO/logo StarLogo objects and parallelism. Part 3 Explorations: simulations and stimulations slime mould artificial ants traffic jams termites turtles and frogs turtle ecology new turtle geometry forest fire recursive trees. Part 4 Reflections: the centralized mindset beyond the centralized mindset. Part 5 Projections: growing up. Appendices: student participants StarLogo overview.
---
paper_title: Co-ordination in Artificial Agent Societies: Social Structures and Its Implications for Autonomous Problem-Solving Agents

paper_content:

Advances in Computer Science often arise from new ideas and concepts, that prove to be advantageous for the design of complex software systems. The con ception of multi agent systems is particularly attractive, as it prommodul ises arity based on the conceptual speciality of an agent, as well as flexibility in their inte gration through appropriate interaction models. While early systems drew upon co operative agents, recent developments have realised the importance of the notion of autonomy in the design of agent based applications. The emergence of systems of autonomous problem solving agents paves the way for complex Artificial Intelligence applications that allow fosca r lability and at the same time foster the reusability of their components. In consequence, an intelligent multi agent application can be seen as a collec tion of autonomous agents, usually specialised in different tasks, together with a social model of their interactions. This approach implies a dynamic generation of complex relational structures, that agents need to be knowledgeable of in order to successfully achieve their goals. Therefore, a multi agent system designer needs to think carefully about conceptualisation, representation and enactment of the different types of knowledge that its agents rely on, for individual problem solving as well as for mutual co ordination.
---
paper_title: Reasoning about Rational Agents

paper_content:

One goal of modern computer science is to engineer computer programs that can act as autonomous, rational agents; software that can independently make good decisions about what actions to perform on our behalf and execute those actions. Applications range from small programs that intelligently search the Web buying and selling goods via electronic commerce, to autonomous space probes. This book focuses on the belief-desire-intention (BDI) model of rational agents, which recognizes the primacy of beliefs, desires, and intentions in rational action. The BDI model has three distinct strengths: an underlying philosophy based on practical reasoning in humans, a software architecture that is implementable in real systems, and a family of logics that support a formal theory of rational agency.The book introduces a BDI logic called LORA (Logic of Rational Agents). In addition to the BDI component, LORA contains a temporal component, which allows one to represent the dynamics of how agents and their environments change over time, and an action component, which allows one to represent the actions that agents perform and the effects of the actions. The book shows how LORA can be used to capture many components of a theory of rational agency, including such notions as communication and cooperation.
---
paper_title: Putting Intentions into Cell Biochemistry: An Artificial Intelligence Perspective

paper_content:

The living cell exists by virtue of thousands of nonlinearly interacting processes. This complexity greatly impedes its understanding. The standard approach to the calculation of the behaviour of the living cell, or part thereof, integrates all the rate equations of the individual processes. If successful extremely intensive calculations often lead the calculation of coherent, apparently simple, cellular "decisions" taken in response to a signal: the complexity of the behavior of the cell is often smaller than it might have been. The "decisions" correspond to the activation of entire functional units of molecular processes, rather than individual ones. The limited complexity of signal and response suggests that there might be a simpler way to model at least some important aspects of cell function. In the field of Artificial Intelligence, such simpler modelling methods for complex systems have been developed. In this paper, it is shown how the Artificial Intelligence description method for deliberative agents functioning on the basis of beliefs, desires and intentions as known in Artificial Intelligence, can be used successfully to describe essential aspects of cellular regulation. This is demonstrated for catabolite repression and substrate induction phenomena in the bacterium Escherichia coli. The method becomes highly efficient when the computation is automated in a Prolog implementation. By defining in a qualitative way the food supply of the bacterium, the make-up of its catabolic pathways is readily calculated for cases that are sufficiently complex to make the traditional human reasoning tedious and error prone.
---
paper_title: Principles of Condensed Matter Physics

paper_content:

Preface 1. Overview 2. Structure and scattering 3. Thermodynamics and statistical mechanics 4. Mean-field theory 5. Field theories, critical phenomena, and the renormalization group 6. Generalized elasticity 7. Dynamics: correlation and response 8. Hydrodynamics 9. Topological defects 10. Walls, kinks and solitons Glossary Index.
---
paper_title: Equation of state calculations by fast computing machines

paper_content:

A general method, suitable for fast computing machines, for investigating such properties as equations of state for substances consisting of interacting individual molecules is described. The method consists of a modified Monte Carlo integration over configuration space. Results for the two‐dimensional rigid‐sphere system have been obtained on the Los Alamos MANIAC and are presented here. These results are compared to the free volume equation of state and to a four‐term virial coefficient expansion.
---
paper_title: A Theory of Learning and Generalization: With Applications to Neural Networks and Control Systems

paper_content:

Contents: Preface.- Introduction.- Preliminaries.- Problem Formulations.- Vapnik-Chervonenkis and Pollard (Pseudo-) Dimensions.- Uniform Convergence of Empirical Means.- Learning Under a Fixed Probability Measure.- Distribution-ree Learning.- Learning Under an Intermediate Family of Probabilities.- Alternate Models of Learning.- Applications to Neural Networks.- Applications to Control Systems.- Some Open Problems.
---
paper_title: Technology and Market Structure: Theory and History

paper_content:

Economists have traditionally taken two very different approaches to studying market structure. One looks to "industry characteristics" to explain why different industries develop in different ways; the other looks to the pattern of firm growth within a "typical" industry to describe the evolution of the size distribution of firms. In his new book, John Sutton sets out a unified theory that encompasses both approaches, while generating a series of novel predictions as to how markets evolve. Using statistical analysis and a detailed examination of industry histories, he rigorously tests these new predictions. Data in the paperback edition have been revised and updated.
---
paper_title: Stochastic Dynamical Systems: Concepts, Numerical Methods, Data Analysis

paper_content:

A textbook designed for those commencing study in stochastic dynamical systems. It is addressed to students of physics, mathematics, chemistry, also those engaged in medical research, and aims to bridge mathematics and the applied sciences. It provides numerical methods and the tools to analyze complex systems. The book contains various stochastic and mathematical methods which are necessary for the analysis of complex systems such as polymeric melts, the human body and the atmosphere. Data analysis is treated, as well as simulation methods for given circumstances.
---
paper_title: Learning in graphical models

paper_content:

Part 1 Inference: introduction to inference for Bayesian networks, Robert Cowell advanced inference in Bayesian networks, Robert Cowell inference in Bayesian networks using nested junction trees, Uffe Kjoerulff bucket elimination - a unifying framework for probabilistic inference, R. Dechter an introduction to variational methods for graphical models, Michael I. Jordan et al improving the mean field approximation via the use of mixture distributions, Tommi S. Jaakkola and Michael I. Jordan introduction to Monte Carlo methods, D.J.C. MacKay suppressing random walls in Markov chain Monte Carlo using ordered overrelaxation, Radford M. Neal. Part 2 Independence: chain graphs and symmetric associations, Thomas S. Richardson the multiinformation function as a tool for measuring stochastic dependence, M. Studeny and J. Vejnarova. Part 3 Foundations for learning: a tutorial on learning with Bayesian networks, David Heckerman a view of the EM algorithm that justifies incremental, sparse and other variants, Radford M. Neal and Geoffrey E. Hinton. Part 4 Learning from data: latent variable models, Christopher M. Bishop stochastic algorithms for exploratory data analysis - data clustering and data visualization, Joachim M. Buhmann learning Bayesian networks with local structure, Nir Friedman and Moises Goldszmidt asymptotic model selection for directed networks with hidden variables, Dan Geiger et al a hierarchical community of experts, Geoffrey E. Hinton et al an information-theoretic analysis of hard and soft assignment methods for clustering, Michael J. Kearns et al learning hybrid Bayesian networks from data, Stefano Monti and Gregory F. Cooper a mean field learning algorithm for unsupervised neural networks, Lawrence Saul and Michael Jordan edge exclusion tests for graphical Gaussian models, Peter W.F. Smith and Joe Whittaker hepatitis B - a case study in MCMC, D.J. Spiegelhalter et al prediction with Gaussian processes - from linear regression to linear prediction and beyond, C.K.I. Williams.
---
paper_title: Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues

paper_content:

Preface * 1 Probability Review * 2 Discrete Time Markov Models * 3 Recurrence and Ergodicity * 4 Long Run Behavior * 5 Lyapunov Functions and Martingales * 6 Eigenvalues and Nonhomogeneous Markov Chains * 7 Gibbs Fields and Monte Carlo Simulation * 8 Continuous-Time Markov Models 9 Poisson Calculus and Queues * Appendix * Bibliography * Author Index * Subject Index
---
paper_title: Adaptive Cooperative Systems

paper_content:

From the Publisher: ::: This book presents a unified treatment of self-organizing processes, drawing upon examples from physics, spatial statistics, image processing, and brain science. It offers a rigorous theory of cooperative computation as applied to problems in perceptual inferencing. The problems addressed include the integration of multiple sensory information (multiuser function), figure ground segregation, the segmentation of visual images, attention, the self-organization of feature detecting neurons, and short-term synaptic plasticity.
---
paper_title: Error And The Growth Of Experimental Knowledge

paper_content:

Thank you very much for downloading error and the growth of experimental knowledge. Maybe you have knowledge that, people have search numerous times for their chosen readings like this error and the growth of experimental knowledge, but end up in malicious downloads. Rather than reading a good book with a cup of coffee in the afternoon, instead they are facing with some malicious bugs inside their laptop.
---
paper_title: When Time Breaks Down: The Three‐Dimensional Dynamics of Electrochemical Waves and Cardiac Arrhythmias

paper_content:

The Description for this book, When Time Breaks Down: The Three-Dimensional Dynamics of Electrochemical Waves and Cardiac Arrhythmias, will be forthcoming.
---
paper_title: Revisiting the Edge of Chaos: Evolving Cellular Automata to Perform Computations

paper_content:

Author(s): Mitchell, M; Hraber, P; Crutchfield, JP | Abstract: We present results from an experiment similar to one performed by Packard (1988), in which a genetic algorithm is used to evolve cellular automata (CA) to perform a particular computational task. Packard examined the frequency of evolved CA rules as a function of Langton's lambda parameter (Langton, 1990), and interpreted the results of his experiment as giving evidence for the following two hypotheses: (1) CA rules able to perform complex computations are most likely to be found near ``critical'' lambda values, which have been claimed to correlate with a phase transition between ordered and chaotic behavioral regimes for CA; (2) When CA rules are evolved to perform a complex computation, evolution will tend to select rules with lambda values close to the critical values. Our experiment produced very different results, and we suggest that the interpretation of the original results is not correct. We also review and discuss issues related to lambda, dynamical-behavior classes, and computation in CA. The main constructive results of our study are identifying the emergence and competition of computational strategies and analyzing the central role of symmetries in an evolutionary system. In particular, we demonstrate how symmetry breaking can impede the evolution toward higher computational capability.
---
paper_title: Ecological inference

paper_content:

Ecological inference is the process of ::: drawing conclusions about individual-level behavior from ::: aggregate-level data. Recent advances involve the combination of ::: statistical and deterministic means to produce such inferences.
---
paper_title: Optimum experimental designs

paper_content:

Part I. Fundamentals Introduction Some key ideas Experimental strategies The choice of a model Models and least squares Criteria for a good experiment Standard designs The analysis of experiments Part II. Theory and applications Optimum design theory Criteria of optimality Experiments with both qualitative and quantitative factors Blocking response surface designs Restricted region designs Failure of the experiment and design augmentation Non-linear models Optimum Bayesian design Discrimination between models Composite design criteria Further topics.
---
paper_title: Recursive nonlinear estimation: A geometric approach

paper_content:

Abstract The paper deals with approximation of nonlinear estimation within the Bayesian framework. The underlying idea is to project the true posterior density orthogonally onto a prespecified approximation family. The problem to be coped with is that the true posterior density is not typically at disposal when estimation is implemented recursively . It is shown that there exists a Bayes-closed description of the posterior density which is recursively computable without complete knowledge of the true posterior. We study a mutual relationship between the equivalence class, composed of densities matching the current description, and a prespecified parametric family. It is proved that if the approximation family is of the mixture type, the equivalence classes can be made orthogonal to this family. Then the approximating density done by the orthogonal projection of the true posterior density minimizes the Kullback-Leibler distance between both densities. On the contrary, if the approximation family is of the exponential type, the analogous result holds at most locally. To be able to give a sensible definition of the orthogonal projection, we have been forced to introduce a Riemannian geometry on the family of probability distributions. Being aware that the differential-geometric concepts and tools do not belong to common knowledge of control engineers, we include necessary preliminary information.
---
paper_title: Spikes: Exploring the Neural Code

paper_content:

Our perception of the world is driven by input from the sensory nerves. This input arrives encoded as sequences of identical spikes. Much of neural computation involves processing these spike trains. What does it mean to say that a certain set of spikes is the right answer to a computational problem? In what sense does a spike train convey information about the sensory world? Spikes begins by providing precise formulations of these and related questions about the representation of sensory signals in neural spike trains. The answers to these questions are then pursued in experiments on sensory neurons.The authors invite the reader to play the role of a hypothetical observer inside the brain who makes decisions based on the incoming spike trains. Rather than asking how a neuron responds to a given stimulus, the authors ask how the brain could make inferences about an unknown stimulus from a given neural response. The flavor of some problems faced by the organism is captured by analyzing the way in which the observer can make a running reconstruction of the sensory stimulus as it evolves in time. These ideas are illustrated by examples from experiments on several biological systems. Intended for neurobiologists with an interest in mathematical analysis of neural data as well as the growing number of physicists and mathematicians interested in information processing by "real" nervous systems, Spikes provides a self-contained review of relevant concepts in information theory and statistical decision theory. A quantitative framework is used to pose precise questions about the structure of the neural code. These questions in turn influence both the design and analysis of experiments on sensory neurons.
---
paper_title: Learning in graphical models

paper_content:

Part 1 Inference: introduction to inference for Bayesian networks, Robert Cowell advanced inference in Bayesian networks, Robert Cowell inference in Bayesian networks using nested junction trees, Uffe Kjoerulff bucket elimination - a unifying framework for probabilistic inference, R. Dechter an introduction to variational methods for graphical models, Michael I. Jordan et al improving the mean field approximation via the use of mixture distributions, Tommi S. Jaakkola and Michael I. Jordan introduction to Monte Carlo methods, D.J.C. MacKay suppressing random walls in Markov chain Monte Carlo using ordered overrelaxation, Radford M. Neal. Part 2 Independence: chain graphs and symmetric associations, Thomas S. Richardson the multiinformation function as a tool for measuring stochastic dependence, M. Studeny and J. Vejnarova. Part 3 Foundations for learning: a tutorial on learning with Bayesian networks, David Heckerman a view of the EM algorithm that justifies incremental, sparse and other variants, Radford M. Neal and Geoffrey E. Hinton. Part 4 Learning from data: latent variable models, Christopher M. Bishop stochastic algorithms for exploratory data analysis - data clustering and data visualization, Joachim M. Buhmann learning Bayesian networks with local structure, Nir Friedman and Moises Goldszmidt asymptotic model selection for directed networks with hidden variables, Dan Geiger et al a hierarchical community of experts, Geoffrey E. Hinton et al an information-theoretic analysis of hard and soft assignment methods for clustering, Michael J. Kearns et al learning hybrid Bayesian networks from data, Stefano Monti and Gregory F. Cooper a mean field learning algorithm for unsupervised neural networks, Lawrence Saul and Michael Jordan edge exclusion tests for graphical Gaussian models, Peter W.F. Smith and Joe Whittaker hepatitis B - a case study in MCMC, D.J. Spiegelhalter et al prediction with Gaussian processes - from linear regression to linear prediction and beyond, C.K.I. Williams.
---
paper_title: Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids

paper_content:

Probablistic models are becoming increasingly important in analyzing the huge amount of data being produced by large-scale DNA-sequencing efforts such as the Human Genome Project. For example, hidden Markov models are used for analyzing biological sequences, linguistic-grammar-based probabilistic models for identifying RNA secondary structure, and probabilistic evolutionary models for inferring phylogenies of sequences from different organisms. This book gives a unified, up-to-date and self-contained account, with a Bayesian slant, of such methods, and more generally to probabilistic methods of sequence analysis. Written by an interdisciplinary team of authors, it is accessible to molecular biologists, computer scientists, and mathematicians with no formal knowledge of the other fields, and at the same time presents the state of the art in this new and important field.
---
paper_title: Recursive nonlinear estimation: A geometric approach

paper_content:

Abstract The paper deals with approximation of nonlinear estimation within the Bayesian framework. The underlying idea is to project the true posterior density orthogonally onto a prespecified approximation family. The problem to be coped with is that the true posterior density is not typically at disposal when estimation is implemented recursively . It is shown that there exists a Bayes-closed description of the posterior density which is recursively computable without complete knowledge of the true posterior. We study a mutual relationship between the equivalence class, composed of densities matching the current description, and a prespecified parametric family. It is proved that if the approximation family is of the mixture type, the equivalence classes can be made orthogonal to this family. Then the approximating density done by the orthogonal projection of the true posterior density minimizes the Kullback-Leibler distance between both densities. On the contrary, if the approximation family is of the exponential type, the analogous result holds at most locally. To be able to give a sensible definition of the orthogonal projection, we have been forced to introduce a Riemannian geometry on the family of probability distributions. Being aware that the differential-geometric concepts and tools do not belong to common knowledge of control engineers, we include necessary preliminary information.
---
paper_title: Complexity: Hierarchical Structures and Scaling in Physics

paper_content:

Part I. Phenomenology and Models: 1. Introduction 2. Examples of complex behaviour 3. Mathematical models Part II: 4. Symbolic representations of physical systems 5. Probability, ergodic theory, and information 6. Thermodynamic formalism Part III. Formal Characterization of Complexity: 7. Physical and computational analysis of symbolic signals 8. Algorithmic and grammatical complexities 9. Hierarchical scaling complexities 10. Summary and perspectives.
---
paper_title: COMPLEXITY AS THERMODYNAMIC DEPTH

paper_content:

Abstract A measure of complexity for the macroscopic states of physical systems is defined. Called depth, the measure is universal: it applies to all physical systems. The form of the measure is uniquely fixed by the requirement that it be a continuous, additive function of the processes that can result in a state. Applied to a Hamiltonian system, the measure is equal to the difference between the system's coarse- and fine-grained entropy, a quantity that we call thermodynamic depth. The measure satisfies the intuitive requirements that wholly ordered and wholly random systems are not thermodynamically deep and that a complex object together with a copy is not much deeper than the object alone. Applied to systems capable of computation, the measure yields a conventional computational measure of complexity as a special case. The relation of depth and thermodynamic depth to previously proposed definitions of complexity is discussed, and applications to physical, chemical, and mathematical problems are proposed.
---
paper_title: HLA and HIV Infection Progression: Application of the Minimum Description Length Principle to Statistical Genetics

paper_content:

The minimum description length (MDL) principle was developed in the context of computational complexity and coding theory. It states that the best model to account for some data minimizes the sum of the lengths, in bits, of the descriptions of the model and the data as encoded via the model. The MDL principle gives a criterion for parameter selection, by using the description length as a test statistic. Class I HLA genes play a major role in the immune response to HIV, and are known to be associated with rates of progression to AIDS. However, these genes are extremely polymorphic, making it difficult to associate alleles with disease outcome, given statistical issues of multiple testing. Application of the MDL principle to immunogenetic data from a longitudinal cohort study (Chicago MACS) enables classification of alleles associated with plasma HIV RNA abundance, an indicator of infection progression. Variation in progression is strongly associated with HLA-B. Allele associations with viral levels support and extend previous studies. In particular, individuals without B58s supertype alleles average viral RNA levels 3.6 times greater than individuals with them. Mechanisms for these associations include variation in epitope specificity and selection that favors rare alleles.
---
paper_title: Toward a Quantitative Theory of Self-Generated Complexity

paper_content:

Quantities are defined operationally which qualify as measures of complexity of patterns arising in physical situations. Their main features, distinguishing them from previously used quantities, are the following: (1) they are measuretheoretic concepts, more closely related to Shannon entropy than to computational complexity; and (2) they are observables related to ensembles of patterns, not to individual patterns. Indeed, they are essentially Shannon information needed to specify not individual patterns, but either measure-theoretic or algebraic properties of ensembles of patterns arising ina priori translationally invariant situations. Numerical estimates of these complexities are given for several examples of patterns created by maps and by cellular automata.
---
paper_title: Predictability, Complexity, and Learning

paper_content:

We define predictive information Ipred(T) as the mutual information between the past and the future of a time series. Three qualitatively different behaviors are found in the limit of large observation times T:Ipred(T) can remain finite, grow logarithmically, or grow as a fractional power law. If the time series allows us to learn a model with a finite number of parameters, then Ipred(T) grows logarithmically with a coefficient that counts the dimensionality of the model space. In contrast, power-law growth is associated, for example, with the learning of infinite parameter (or nonparametric) models such as continuous functions with smoothness constraints. There are connections between the predictive information and measures of complexity that have been defined both in learning theory and the analysis of physical systems through statistical mechanics and dynamical systems theory. Furthermore, in the same way that entropy provides the unique measure of available information consistent with some simple and plausible conditions, we argue that the divergent part of Ipred(T) provides the unique measure for the complexity of dynamics underlying a time series. Finally, we discuss how these ideas may be useful in problems in physics, statistics, and biology.
---
paper_title: Computational mechanics: Pattern and prediction, structure and simplicity

paper_content:

Computational mechanics, an approach to structural complexity, defines a process's causal states and gives a procedure for finding them. We show that the causal-state representation--an e-machine--is the minimal one consistent with accurate prediction. We establish several results on e-machine optimality and uniqueness and on how e-machines compare to alternative representations. Further results relate measures of randomness and structural complexity obtained from e-machines to those from ergodic and information theories.
---
paper_title: Elements of the Theory of Computation

paper_content:

From the Publisher: ::: Lewis and Papadimitriou present this long awaited Second Edition of their best-selling theory of computation. The authors are well-known for their clear presentation that makes the material accessible to a a broad audience and requires no special previous mathematical experience. In this new edition, the authors incorporate a somewhat more informal, friendly writing style to present both classical and contemporary theories of computation. Algorithms, complexity analysis, and algorithmic ideas are introduced informally in Chapter 1, and are pursued throughout the book. Each section is followed by problems.
---
paper_title: Unreconstructible At Any Radius

paper_content:

Abstract Modeling pattern data series with cellular automata fails for a wide range of deterministic nonlinear spatial processes. If the latter have finite spatially-local memory, reconstructed cellular automata with infinite radius may be required. In some cases, even this is not adequate: an irreducible stochasticity remains on the shortest time scales. The underlying problem is illustrated and quantitatively analyzed using an alternative model class called cellular transducers.
---
paper_title: Self-Organized Criticality: Emergent Complex Behavior in Physical and Biological Systems

paper_content:

Preface 1. Introduction 2. Characterization of the SOC state 3. Systems exhibiting SOC 4. Computer models 5. The search for a formalism 6. Is it SOC or is it not? Appendices.
---
paper_title: Optimal Design, Robustness, and Risk Aversion

paper_content:

Highly optimized tolerance is a model of optimization in engineered systems, which gives rise to power-law distributions of failure events in such systems. The archetypal example is the highly optimized forest fire model. Here we give an analytic solution for this model which explains the origin of the power laws. We also generalize the model to incorporate risk aversion, which results in truncation of the tails of the power law so that the probability of catastrophically large events is dramatically lowered, giving the system more robustness.
---
paper_title: Highly optimized tolerance: A mechanism for power laws in designed systems

paper_content:

We introduce a mechanism for generating power law distributions, referred to as highly optimized tolerance (HOT), which is motivated by biological organisms and advanced engineering technologies. Our focus is on systems which are optimized, either through natural selection or engineering design, to provide robust performance despite uncertain environments. We suggest that power laws in these systems are due to tradeoffs between yield, cost of resources, and tolerance to risks. These tradeoffs lead to highly optimized designs that allow for occasional large events. We investigate the mechanism in the context of percolation and sand pile models in order to emphasize the sharp contrasts between HOT and self-organized criticality (SOC), which has been widely suggested as the origin for power laws in complex systems. Like SOC, HOT produces power laws. However, compared to SOC, HOT states exist for densities which are higher than the critical density, and the power laws are not restricted to special values of the density. The characteristic features of HOT systems include: (1) high efficiency, performance, and robustness to designed-for uncertainties; (2) hypersensitivity to design flaws and unanticipated perturbations; (3) nongeneric, specialized, structured configurations; and (4) power laws. The first three of these are in contrast to the traditional hallmarks of criticality, and are obtained by simply adding the element of design to percolation and sand pile models, which completely changes their characteristics.
---
paper_title: From gene families and genera to incomes and internet file sizes: Why power laws are so common in nature

paper_content:

We present a simple explanation for the occurrence of power-law tails in statistical distributions by showing that if stochastic processes with exponential growth in expectation are killed (or observed) randomly, the distribution of the killed or observed state exhibits power-law behavior in one or both tails. This simple mechanism can explain power-law tails in the distributions of the sizes of incomes, cities, internet files, biological taxa, and in gene family and protein family frequencies.
---
paper_title: Measuring complexity using information fluctuation

paper_content:

Abstract A method for analyzing deterministic dynamical systems is presented. New measures of complexity are proposed, based on fluctuation in net information gain and its dependence on system size. These measures are applied to one-dimensional cellular automata and shown to be useful in selecting rules that support slow-moving gliders in quiescent backgrounds.
---
paper_title: Complexity: Hierarchical Structures and Scaling in Physics

paper_content:

Part I. Phenomenology and Models: 1. Introduction 2. Examples of complex behaviour 3. Mathematical models Part II: 4. Symbolic representations of physical systems 5. Probability, ergodic theory, and information 6. Thermodynamic formalism Part III. Formal Characterization of Complexity: 7. Physical and computational analysis of symbolic signals 8. Algorithmic and grammatical complexities 9. Hierarchical scaling complexities 10. Summary and perspectives.
---
paper_title: Classes of network connectivity and dynamics

paper_content:

Many kinds of complex systems exhibit characteristic patterns of temporal correlations that emerge as the result of functional interactions within a structured network. One such complex system is the brain, composed of numerous neuronal units linked by synaptic connections. The activity of these neuronal units gives rise to dynamic states that are characterized by specific patterns of neuronal activation and co-activation. These patterns, called functional connectivity, are possible neural correlates of perceptual and cognitive processes. Which functional connectivity patterns arise depends on the anatomical structure of the underlying network, which in turn is modified by a broad range of activity-dependent processes. Given this intricate relationship between structure and function, the question of how patterns of anatomical connectivity constrain or determine dynamical patterns is of considerable theoretical importance. The present study develops computational tools to analyze networks in terms of their structure and dynamics. We identify different classes of network, including networks that are characterized by high complexity. These highly complex networks have distinct structural characteristics such as clustered connectivity and short wiring length similar to those of large-scale networks of the cerebral cortex. 2002 Wiley Periodicals, Inc.
---
paper_title: Measures of Statistical Complexity: Why?

paper_content:

We review several statistical complexity measures proposed over the last decade and a half as general indicators of structure or correlation. Recently, L\`opez-Ruiz, Mancini, and Calbet [Phys. Lett. A 209 (1995) 321] introduced another measure of statistical complexity $C_{\rm LMC}$ that, like others, satisfies the ``boundary conditions'' of vanishing in the extreme ordered and disordered limits. We examine some properties of $C_{\rm LMC}$ and find that it is neither an intensive nor an extensive thermodynamic variable. It depends nonlinearly on system size and vanishes exponentially in the thermodynamic limit for all one-dimensional finite-range spin systems. We propose a simple alteration of $C_{\rm LMC}$ that renders it extensive. however, this remedy results in a quantity that is a trivial function of the entropy density and hence of no use as a measure of structure or memory. We conclude by suggesting that a useful ``statistical complexity'' must not only obey the ordered-random bounary conditions of vanishing, it must also be defined in a setting that gives a clear interpretation to what structures are quantified.
---
paper_title: Complexity: Hierarchical Structures and Scaling in Physics

paper_content:

Part I. Phenomenology and Models: 1. Introduction 2. Examples of complex behaviour 3. Mathematical models Part II: 4. Symbolic representations of physical systems 5. Probability, ergodic theory, and information 6. Thermodynamic formalism Part III. Formal Characterization of Complexity: 7. Physical and computational analysis of symbolic signals 8. Algorithmic and grammatical complexities 9. Hierarchical scaling complexities 10. Summary and perspectives.
---
paper_title: Computational mechanics: Pattern and prediction, structure and simplicity

paper_content:

Computational mechanics, an approach to structural complexity, defines a process's causal states and gives a procedure for finding them. We show that the causal-state representation--an e-machine--is the minimal one consistent with accurate prediction. We establish several results on e-machine optimality and uniqueness and on how e-machines compare to alternative representations. Further results relate measures of randomness and structural complexity obtained from e-machines to those from ergodic and information theories.
---
paper_title: Principles of Data Mining

paper_content:

The growing interest in data mining is motivated by a common problem across disciplines: how does one store, access, model, and ultimately describe and understand very large data sets? Historically, different aspects of data mining have been addressed independently by different disciplines. This is the first truly interdisciplinary text on data mining, blending the contributions of information science, computer science, and statistics. The book consists of three sections. The first, foundations, provides a tutorial overview of the principles underlying data mining algorithms and their application. The presentation emphasizes intuition rather than rigor. The second section, data mining algorithms, shows how algorithms are constructed to solve specific problems in a principled manner. The algorithms covered include trees and rules for classification and regression, association rules, belief networks, classical statistical models, nonlinear models such as neural networks, and local "memory-based" models. The third section shows how all of the preceding analysis fits together when applied to real-world data mining problems. Topics include the role of metadata, how to handle missing data, and data preprocessing.
---
paper_title: An information-geometric approach to a theory of pragmatic structuring

paper_content:

1.1. The motivation of the approach. In the field of neural networks, so-called infomax principles like the principle of “maximum information preservation” by Linsker [20] are formulated to derive learning rules that improve the information processing properties of neural systems (see [12]). These principles, which are based on information-theoretic measures, are intended to describe the mechanism of learning in the brain. There, the starting point is a low-dimensional and biophysiologically motivated parametrization of the neural system, which need not necessarily be compatible with the given optimization principle. In contrast to this, we establish theoretical results about the low complexity of optimal solutions for the optimization problem of frequently used measures like the mutual information in an unconstrained and more theoretical setting. In the present paper, we do not comment on applications to modeling neural networks. This is intended to be done in a further step, where the results can be used for the characterization of “good” parameter sets that, on the one hand, are compatible with the underlying optimization and, on the other hand, are biologically motivated.
---
paper_title: A Variational Formulation of Optimal Nonlinear Estimation

paper_content:

We propose a variational method to solve all three estimation problems for nonlinear stochastic dynamical systems: prediction, filtering, and smoothing. Our new approach is based upon a proper choice of cost function, termed the {\it effective action}. We show that this functional of time-histories is the unique statistically well-founded cost function to determine most probable histories within empirical ensembles. The ensemble dispersion about the sample mean history can also be obtained from the Hessian of the cost function. We show that the effective action can be calculated by a variational prescription, which generalizes the ``sweep method'' used in optimal linear estimation. An iterative numerical scheme results which converges globally to the variational estimator. This scheme involves integrating forward in time a ``perturbed'' Fokker-Planck equation, very closely related to the Kushner-Stratonovich equation for optimal filtering, and an adjoint equation backward in time, similarly related to the Pardoux-Kushner equation for optimal smoothing. The variational estimator enjoys a somewhat weaker property, which we call ``mean optimality''. However, the variational scheme has the principal advantage---crucial for practical applications---that it admits a wide variety of finite-dimensional moment-closure approximations. The moment approximations are derived reductively from the Euler-Lagrange variational formulation and preserve the good structural properties of the optimal estimator.
---
paper_title: Cellular Automata And Complexity: Collected Papers

paper_content:

Primary Papers * Statistical Mechanics of Cellular Automata * Algebraic Properties of Cellular Automata * Universality and Complexity in Cellular Automata * Computation Theory of Cellular Automata * Undecidability and Intractability in Theoretical Physics * Two-Dimensional Cellular Automata * Origins of Randomness in Physical Systems * Thermodynamics and Hydrodynamics of Cellular Automata * Random Sequence Generation by Cellular Automata * Approaches to Complexity Engineering * Minimal Cellular Automaton Approximations to Continuum Systems * Cellular Automaton Fluids: Basic Theory Additional And Survey Papers * Cellular Automata * Computers in Science and Mathematics * Geometry of Binomial Coefficients * Twenty Problems in the Theory of Cellular Automata * Cryptography with Cellular Automata * Complex Systems Theory * Cellular Automaton Supercomputing Appendices * Tables of Cellular Automaton Properties * Scientific Bibliography of Stephen Wolfram
---
paper_title: Cellular Automata: Theory and Experiment

paper_content:

Part 1 Mathematical analysis of cellular automata: aperiodicity in one-dimensional cellular automata, E. Jen cyclic cellular automata and related processes, R. Fisch nearest neighbour cellular automata over Z2 with periodic boundary conditions, B. Voorhees cellular automaton ruled by an eccentric conservation law, A.M. Barbe Boolean derivatives on cellular automata, G. Vichniac. Part 2 Structure of the space of cellular automata: transition phenomena in cellular automata: transition phenomena in cellular automata rule space, W. Li, et al is there a sharp phase transition for deterministic cellular automata?, W.W. Wootters and C.G. Langton Wolfram's class IV automata and a good life, H.V. McIntosh criticality in cellular automata, H. Chate and P. Manneville a hierarchical classification of cellular automata, H.A. Gutowitz. Part 3 Learning rules with specified properties: adaptive stochastic cellular automata - theory, Y.C. Lee, et al adaptive stochastic cellular automata - experiment, S. Qian, et al extracting cellular automaton rules directly from experimental data, F.C. Richards, et al. Part 4 Cellular automata and the natural sciences - biology: what can automaton theory tell us about the brain?, J.D. Victor simulation of HIV-infection in artificial immune systems, H. Sieburg, et al physics and chemistry - invertible cellular automata - a review, T. Toffoli and N. Margolus digital mechanics - an informational process based on reversible universal cellular automata, E. Fredkin representations of geometrical and topological quantities in cellular automata, M.A. Smith relaxation properties of elementary reversible cellular automata, S. Takesue a comparison of spin exchange and cellular automaton models for diffusion-controlled reactions, A. Canning and M. Droz reversible cellular automata and chemical turbulence, H. Hartman and P. Tamayo soliton turbulence in one-dimensional cellular automata, Y. Aizawa, et al knot invariants and cellular automata, B. Hasslacher and D.A. Meyer critical dynamics of one-dimensional irreversible systems, O. Martin. Part 5 comutation theory of cellular automata, computation theoretic aspects of cellular automata, K. Culik II reversibility of 2D cellular automata is undecidable, J. Kari classifying circular cellular automata, K. Sutner formal languages and global cellular automaton behaviour, K. Culik II a characterization of constant-time cellular automata computation, S. Kim and R. McCloskey constructive chaos by cellular automata and possible sources of an arrow to time, K. Svozil. Part 6 Generalizations of cellular automata: cellular automata and discrete neural networks, M. Garzon attractor dominance patterns in sparsely connected Boolean nets, C.C. Walker periodic orbits and log transients in coupled map lattices, R. Livi. Appendices: a brief review of cellular automata packages, D. Hiebeler maps of recent cellular automata and lattice gas automata literature, H.A. Gutowitz.
---
paper_title: From classical models of morphogenesis to agent-based models of pattern formation

paper_content:

An extremely large body of theoretical work exists on pattern formation, but very few experimental results have confirmed the relevance of theoretical models. It is argued in this article that the notion of agent-based pattern formation, which is introduced and exemplified, can serve as a basis to study pattern formation in nature, especially because pattern-forming systems based on agents are (relatively) more easily amenable to experimental observations. Moreover, understanding agent-based pattern formation is a necessary step if one wishes to design distributed artificial pattern-forming systems. But, to achieve this goal, a theory of agent-based pattern formation is needed. This article suggests that it can certainly be derived from existing theories of pattern formation.
---
paper_title: Agent-based computational models and generative social science

paper_content:

Agent-based computational modeling is changing the face of social science. In Generative Social Science , Joshua Epstein argues that this powerful, novel technique permits the social sciences to meet a fundamentally new standard of explanation, in which one "grows" the phenomenon of interest in an artificial society of interacting agents: heterogeneous, boundedly rational actors, represented as mathematical or software objects. After elaborating this notion of generative explanation in a pair of overarching foundational chapters, Epstein illustrates it with examples chosen from such far-flung fields as archaeology, civil conflict, the evolution of norms, epidemiology, retirement economics, spatial games, and organizational adaptation. In elegant chapter preludes, he explains how these widely diverse modeling studies support his sweeping case for generative explanation. This book represents a powerful consolidation of Epstein's interdisciplinary research activities in the decade since the publication of his and Robert Axtell's landmark volume, Growing Artificial Societies . Beautifully illustrated, Generative Social Science includes a CD that contains animated movies of core model runs, and programs allowing users to easily change assumptions and explore models, making it an invaluable text for courses in modeling at all levels.
---
paper_title: Transmission of information

paper_content:

A quantitative measure of “information” is developed which is based on physical as contrasted with psychological considerations. How the rate of transmission of this information over a system is limited by the distortion resulting from storage of energy is discussed from the transient viewpoint. The relation between the transient and steady state viewpoints is reviewed. It is shown that when the storage of energy is used to restrict the steady state transmission to a limited range of frequencies the amount of information that can be transmitted is proportional to the product of the width of the frequency-range by the time it is available. Several illustrations of the application of this principle to practical systems are included. In the case of picture transmission and television the spacial variation of intensity is analyzed by a steady state method analogous to that commonly used for variations with time.
---
paper_title: Entropy and Information Theory

paper_content:

This book is an updated version of the information theory classic, first published in 1990. About one-third of the book is devoted to Shannon source and channel coding theorems; the remainder addresses sources, channels, and codes and on information and distortion measures and their properties. New in this edition:Expanded treatment of stationary or sliding-block codes and their relations to traditional block codesExpanded discussion of results from ergodic theory relevant to information theoryExpanded treatment of B-processes -- processes formed by stationary coding memoryless sourcesNew material on trading off information and distortion, including the Marton inequalityNew material on the properties of optimal and asymptotically optimal source codesNew material on the relationships of source coding and rate-constrained simulation or modeling of random processesSignificant material not covered in other information theory texts includes stationary/sliding-block codes, a geometric view of information theory provided by process distance measures, and general Shannon coding theorems for asymptotic mean stationary sources, which may be neither ergodic nor stationary, and d-bar continuous channels.
---
paper_title: Computational mechanics: Pattern and prediction, structure and simplicity

paper_content:

Computational mechanics, an approach to structural complexity, defines a process's causal states and gives a procedure for finding them. We show that the causal-state representation--an e-machine--is the minimal one consistent with accurate prediction. We establish several results on e-machine optimality and uniqueness and on how e-machines compare to alternative representations. Further results relate measures of randomness and structural complexity obtained from e-machines to those from ergodic and information theories.
---

Title: Methods and techniques of complex systems science: An overview

Section 1: INTRODUCTION
Description 1: This section discusses the general nature of complex systems and introduces the goals of complex systems science.

Section 2: Outline of This Chapter
Description 2: This section provides an overview of the chapter structure and the main topics that will be covered.

Section 3: TIME SERIES ANALYSIS
Description 3: This section delves into methods for analyzing time series data, including both traditional statistical approaches and nonlinear dynamics approaches.

Section 4: The State-Space Picture
Description 4: This section describes the state-space model as a common framework for analyzing time series.

Section 5: General Properties of Time Series
Description 5: This section covers properties such as stationarity, ergodicity, and autocorrelation functions, as well as frequency-domain properties.

Section 6: The Traditional Statistical Approach
Description 6: This section explains classical time series models including moving average, autoregressive, and autoregressive moving average models.

Section 7: Applicability of Linear Statistical Models
Description 7: This section discusses the application of linear models to nonlinear dynamical systems.

Section 8: Extensions
Description 8: This section explores extensions to standard linear models including long memory processes, volatility models, and nonlinear models.

Section 9: The Nonlinear Dynamics Approach
Description 9: This section discusses approaches to time series analysis inspired by nonlinear dynamics, including the Takens Embedding Theorem.

Section 10: Filtering and State Estimation
Description 10: This section covers techniques for estimating the state of a system from observed data, including linear and nonlinear filters.

Section 11: Symbolic or Categorical Time Series
Description 11: This section examines time series composed of discrete symbols, and the application of concepts from automata theory.

Section 12: Hidden Markov Models
Description 12: This section introduces Hidden Markov Models as a generalization of Markov chains to the state-space picture.

Section 13: Variable-Length Markov Models
Description 13: This section explains context trees or probabilistic suffix trees and their application to symbolic dynamics.

Section 14: Cellular Automata
Description 14: This section provides an overview of cellular automata as models of complex systems, including their use in pattern formation and other applications.

Section 15: A Basic Explanation of CAs
Description 15: This section gives a basic explanation of how cellular automata work, including their structure and rules.

Section 16: Cellular Automata as Parallel Computers
Description 16: This section discusses the computational power of cellular automata and their equivalence to Turing machines.

Section 17: Cellular Automata as Discrete Field Theories
Description 17: This section explores the use of cellular automata to simulate classical field theories and physical phenomena.

Section 18: AGENT-BASED MODELS
Description 18: This section introduces agent-based models as a way to simulate the behavior of individual agents and their interactions within a system.

Section 19: Computational Implementation: Agents are Objects
Description 19: This section discusses the implementation of agent-based models using object-oriented programming.

Section 20: Three Things Which Are Not Agent-Based Models
Description 20: This section clarifies misconceptions about what constitutes an agent-based model.

Section 21: The Simplicity of Complex Systems Models
Description 21: This section discusses the reasons for using simple models in complex systems science.

Section 22: EVALUATING MODELS OF COMPLEX SYSTEMS
Description 22: This section covers methods for evaluating models of complex systems, including simulation and comparison to real-world data.

Section 23: Simulation
Description 23: This section discusses the use of direct simulation to understand the behavior of complex system models.

Section 24: Monte Carlo Methods
Description 24: This section explains how Monte Carlo methods can be used to estimate properties of complex systems.

Section 25: Analytical Techniques
Description 25: This section explores analytical techniques for studying complex systems, particularly those based on Markov processes.

Section 26: General issues
Description 26: This section talks about the general issues in comparing models to real-world data.

Section 27: Two Stories and Some Morals
Description 27: This section provides two examples illustrating fundamental points about model evaluation in complex systems science.

Section 28: Comparing Macro-data and Micro-models
Description 28: This section discusses how to compare individual-level models to aggregate data.

Section 29: Comparison to Other Models
Description 29: This section discusses methods to compare different models of complex systems.

Section 30: INFORMATION THEORY
Description 30: This section covers the basic concepts and applications of information theory in complex systems science.

Section 31: Basic Definitions
Description 31: This section provides definitions of key information-theoretic quantities such as entropy and mutual information.

Section 32: Optimal Coding
Description 32: This section covers the basics of coding theory and its relevance to information theory.

Section 33: Applications of Information Theory
Description 33: This section discusses the applications of information theory in various scientific domains.

Section 34: COMPLEXITY MEASURES
Description 34: This section discusses different measures of complexity and their applications to complex systems.

Section 35: Algorithmic Complexity
Description 35: This section introduces the concept of Kolmogorov complexity and its limitations.

Section 36: Refinements of Algorithmic Complexity
Description 36: This section explores modifications of algorithmic complexity such as logical depth and algorithmic statistics.

Section 37: Statistical Measures of Complexity
Description 37: This section describes statistical approaches to measuring complexity, including the minimum description length principle.

Section 38: Stochastic Complexity and the Minimum Description Length
Description 38: This section explains how to quantify complexity using the minimum description length principle.

Section 39: Complexity via Prediction I: Forecast Complexity and Predictive Information
Description 39: This section discusses the concept of forecast complexity and its relationship to predictive information.

Section 40: Complexity via Prediction II: The Crutchfield-Young Statistical Complexity
Description 40: This section details the Crutchfield-Young method for measuring statistical complexity based on causal states.

Section 41: Power Law Distributions
Description 41: This section discusses the misconception that power law distributions are indicative of complexity.

Section 42: Other Measures of Complexity
Description 42: This section surveys other measures of complexity not covered in the main text.

Section 43: Relevance of Complexity Measures
Description 43: This section discusses the importance of choosing relevant complexity measures and their applications.

Section 44: GUIDE TO FURTHER READING
Description 44: This section provides references for further reading on the topics covered in the chapter.

Section 45: General
Description 45: This section lists general references and surveys on complex systems science.

Section 46: Data Mining and Statistical Learning
Description 46: This section provides reading recommendations on statistical learning and data mining.

Section 47: Time Series
Description 47: This section lists recommended readings on time series analysis.

Section 48: Filtering
Description 48: This section provides references on filtering methods and state estimation.

Section 49: Symbolic Dynamics and Hidden Markov Models
Description 49: This section discusses further reading on symbolic dynamics and hidden Markov models.

Section 50: Cellular Automata
Description 50: This section lists recommended readings on cellular automata, including applications and theory.

Section 51: Agent-Based Modeling
Description 51: This section provides references for further reading on agent-based modeling techniques and applications.

Section 52: Evaluating Models of Complex Systems
Description 52: This section suggests references on evaluating and comparing models of complex systems.

Section 53: Monte Carlo
Description 53: This section lists further reading on Monte Carlo methods.

Section 54: Experimental design
Description 54: This section provides references on the principles and methodologies of experimental design.

Section 55: Information Theory
Description 55: This section suggests further reading on information theory and its applications.

Section 56: Complexity Measures
Description 56: This section details references and surveys on different measures of complexity.