Regression techniques for subspace-based black-box state-space system identification: an overview
---
paper_title: PREDICTION ERROR VS SUBSPACE METHODS IN CLOSED LOOP IDENTIFICATION

paper_content:

Abstract In this paper we investigate the role of the output predictor in subspace identification. We shall see that subspace identification of the predictor model can ideally yield consistent estimates regardless of the presence of feedback. This solves a longstanding open question in system identification.
---
paper_title: System Identification: Theory for the User

paper_content:

Das Buch behandelt die Systemidentifizierung in dem theoretischen Bereich, der direkte Auswirkungen auf Verstaendnis und praktische Anwendung der verschiedenen Verfahren zur Identifizierung hat. Da ...
---
paper_title: Continuous-time subspace identification in closed-loop using Laguerre filters

paper_content:

This paper deals with the problem of continuous-time model identification and presents a subspace-based algorithm capable of dealing with data generated by systems operating in closed-loop. The algorithm is developed by reformulating the identification problem from the time-domain to the so-called all-pass domain by using a bank of Laguerre filters applied to the input-output data. In this framework, the PBSID subspace identification algorithm, originally developed in the case of discrete-time systems, can be reformulated for the continuous-time case. Simulation results are presented to illustrate the performance of the algorithm.
---
paper_title: Filtering and System Identification: A Least Squares Approach

paper_content:

Preface 1. Introduction 2. Linear algebra 3. Discrete-time signals and systems 4. Random variables and signals 5. Kalman filtering 6. Estimation of spectra and frequency response functions 7. Output-error parametric model estimation 8. Prediction-error parametric model estimation 9. Subspace model identification 10. The system identification cycle Notation and symbols List of abbreviations References Index.
---
paper_title: Parallel QR Implementation of Subspace Identification with Parsimonious Models

paper_content:

In this paper we reveal that the typical subspace identification algorithms use non-parsimonious model formulations, with extra terms in the model that appear to be non-causal. These terms are the ...
---
paper_title: Choice of state-space basis in combined deterministic-stochastic subspace identification

paper_content:

This paper describes how the state-space basis of models identified with subspace identification algorithms can be determined. It is shown that this basis is determined by the input spectrum and by user-defined input and output weightings. Through the connections between subspace identification and frequency-weighted balancing, the state-space basis of the subspace-identified models is shown to coincide with a frequency-weighted balanced basis.
---
paper_title: Fully Parametrized State-Space Models In System Identification

paper_content:

Abstract In this paper we consider identification of multivariable linear systems using state-space models. A new model structure which is fully parametrized is introduced. All systems of a given order. can be described with this model structure and thus relieve us from all the internal structural issues otherwise inherent in the multivariable state-space identification problem. We present an identification algorithm which minimize a regularized prediction error criterion. We show that the proposed model structure retains the statistical properties of the standard identifiable model structures. The proposed identification algorithm is shown to locally converge to the set of true systems. Examples are given illustrating the results as well as showing the practical use of the proposed model structure using real data.
---
paper_title: Subspace identification method incorporating prior information

paper_content:

Subspace identification methods proved to be a powerful tool, which can further benefit from the incorporation of prior information. In the industrial environment, there is often strong prior information about the identified system, that can be used to improve the model quality and its compliance with physical reality. Such prior information can be the known static gains, the dominant time constants, the impulse response smoothness, etc. An idea comes from the possibility to consider the subspace identification as an optimization problem of finding a model with the optimal multi-step predictions on the experimental data. Further, the problem is reformulated to the Bayesian framework allowing to combine available prior information with the information contained in the experimental data by covariance matrix shaping. The paper is completed with an application to experimental data from an oil firing steam boiler with the rated effective power of 100 MW.
---
paper_title: N4SID: Subspace Algorithms for the Identification of Combined Deterministic-Stochastic Systems

paper_content:

Abstract Recently a great deal of attention has been given to numerical algorithms for subspace state space system identification (N4SID). In this paper, we derive two new N4SID algorithms to identify mixed deterministic-stochastic systems. Both algorithms determine state sequences through the projection of input and output data. These state sequences are shown to be outputs of non-steady state Kalman filter banks. From these it is easy to determine the state space system matrices. The N4SID algorithms are always convergent (non-iterative) and numerically stable since they only make use of QR and Singular Value Decompositions. Both N4SID algorithms are similar, but the second one trades off accuracy for simplicity. These new algorithms are compared with existing subspace algorithms in theory and in practice.
---
paper_title: Subspace identification with prior steady-state information

paper_content:

In system identification, the quality of data is important for obtaining good models, but there are situations where the available data are highly corrupted with noise. However, some prior information about the system to be identified, such as dc gain and settling time, may be available to obtain improved model identification despite data noise. In this paper, a subspace identification scheme incorporating known dc gain is investigated. The prior process information is incorporated into system identification through using the least square with the equality constraint that the sum of impulse response parameters is equal to the dc gain. In comparison with the existing approaches, the proposed identification strategy provides unbiased parameter estimations, is applicable to Multi-Input Multi-Output (MIMO) systems, and is computationally efficient.
---
paper_title: Analysis of the asymptotic properties of the MOESP type of subspace algorithms

paper_content:

The MOESP type of subspace algorithms are used for the identification of linear, discrete time, finite-dimensional state-space systems. They are based on the geometric structure of covariance matrices and exploit the properties of the state vector extensively. In this paper the asymptotic properties of the algorithms are examined. The main results include consistency and asymptotic normality for the estimates of the system matrices, under suitable assumptions on the noise sequence, the input process and the underlying true system.
---
paper_title: COMPARISONS OF SUBSPACE IDENTIFICATION METHODS FOR SYSTEMS OPERATING ON CLOSED-LOOP

paper_content:

In this paper, we analyze two recently proposed closed-loop subspace identification methods, referred to as innovation estimation method and whitening filter approach respectively. The similarity and difference between them are investigated in detail. It turns out that all closed-loop subspace identification methods can be classified as one-step, two-step, or multi-stage projection methods. A SISO closed-loop simulation shows that to identify a consistent model the whitening filter approach might require longer future and past horizons than the innovation estimation method.
---
paper_title: A linear regression approach to state-space subspace system identification

paper_content:

Abstract Recently, state-space subspace system identification (4SID) has been suggested as an alternative to the more traditional prediction error system identification. The aim of this paper is to analyze the connections between these two different approaches to system identification. The conclusion is that 4SID can be viewed as a linear regression multistep-ahead prediction error method with certain rank constraints. This allows us to describe 4SID methods within the standard framework of system identification and linear regression estimation. For example, this observation is used to compare different cost-functions which occur rather implicitly in the ordinary framework of 4SID. From the cost-functions, estimates of the extended observability matrix are derived and related to previous work. Based on the estimates of the observability matrix, the asymptotic properties of two pole estimators, namely the shift invariance method and a weighted subspace fitting method, are analyzed. Expressions for the asymptotic variances of the pole estimation error are given. From these expressions, difficulties in choosing userspecified parameters are pointed out. Furthermore, it is found that a row-weighting in the subspace estimation step does not affect the pole estimation error asymptotically.
---
paper_title: Solving Least Squares Problems

paper_content:

This disclosure is directed to a powered cutting tool and a cutting head adapted for use therewith comprising a housing for containing a motor means and an actuator for controlling the operation of the motor means. A cutting head is connected in driving relationship with the motor means. The cutting head includes a pair of relatively movable jaws having a by-pass disposed in substantial alignment with the shear line of the jaws so as to prohibit curling of the material or binding of the tool during a cutting operation.
---
paper_title: Identification of the deterministic part of MIMO state space models given in innovations form from input-output data

paper_content:

The problem of Linear Multivariable State Space model identification from input-output data can under the presence of process- and measurement noise be solved in a non-iterative way when incorporating instrumental variables constructed from both input and output sequences in the recently developed class of multivariable output-error state space model class of subspace model identification schemes. Abstract--In this paper we describe two algorithms to identify a linear, time-invariant, finite dimensional state space model from input-output data. The system to be identified is assumed to be excited by a measurable input and an unknown process tloise and the measurements are disturbed by unknown measurement noise. Both noise sequences are discrete zero-mean white noise, The first algorithm gives consistent estimates only for the case where the input also is zero-mean white noise, while the same result is obtained with the second algorithm without this constraint. For the special case where the input signal is discrete zero-mean white noise, it is explicitly shown that this second algorithm is a special case of the recently developed Multivariable Output-Error State Space (MOESP) class of algorithms based on instrumental variables. The usefulness of the presented schemes is highlighted in a realistic simulation study.
---
paper_title: A least squares interpretation of sub-space methods for system identification

paper_content:

So called subspace methods for direct identification of linear models in state space form have drawn considerable interest. The algorithms consist of series of quite complex projections, and it is not so easy to intuitively understand how they work. They have also defied, so far, complete asymptotic analysis of their stochastic properties. This contribution describes an interpretation of how they work. It specifically deals with how consistent estimates of the dynamics can be achieved, even though correct predictors are not used. We stress how the basic idea is to focus on the estimation of the state-variable candidates-the k-step ahead output predictors.
---
paper_title: Subspace Identification from Closed Loop Data

paper_content:

The so-called subspace methods for direct identification of linear models in state space form have drawn considerable interest recently. They have been found to work well in many cases but have one drawback — they do not yield consistent estimates for data collected under output feedback. The present paper points to the reasons for this. We stress how the basic idea is to focus on the estimation of the state-variable candidates — the k-step ahead output predictors. By recomputing these from a ‘non-parametric’ (or, rather, high order ARX) one-step ahead predictor model, closed loop data can be handled.
---
paper_title: Algorithms for deterministic balanced subspace identification

paper_content:

New algorithms for identification of a balanced state space representation are proposed. They are based on a procedure for the estimation of impulse response and sequential zero input responses directly from data. The proposed algorithms are more efficient than the existing alternatives that compute the whole Hankel matrix of Markov parameters. It is shown that the computations can be performed on Hankel matrices of the input-output data of various dimensions. By choosing wider matrices, we need persistency of excitation of smaller order. Moreover, this leads to computational savings and improved statistical accuracy when the data is noisy. Using a finite amount of input-output data, the existing algorithms compute finite time balanced representation and the identified models have a lower bound on the distance to an exact balanced representation. The proposed algorithm can approximate arbitrarily closely an exact balanced representation. Moreover, the finite time balancing parameter can be selected automatically by monitoring the decay of the impulse response. We show what is the optimal in terms of minimal identifiability condition partition of the data into ''past'' and ''future''.
---
paper_title: Subspace-based methods for the identification of linear time-invariant systems

paper_content:

Subspace-based methods for system identification have attracted much attention during the past few years. This interest is due to the ability of providing accurate state-space models for multivariable linear systems directly from input-output data. The methods have their origin in classical state-space realization theory as developed in the 1960s. The main computational tools are the QR and the singular-value decompositions. Here, an overview of existing subspace-based techniques for system identification is given. The methods are grouped into the classes of realization-based and direct techniques. Similarities between different algorithms are pointed out, and their applicability is commented upon. We also discuss some recent ideas for improving and extending the methods. A simulation example is included for comparing different algorithms. The subspace-based approach is found to perform competitive with respect to prediction-error methods, provided the system is properly excited.
---
paper_title: Subspace Identification and ARX Modeling

paper_content:

Abstract In this paper we present a new identification method that points at the close relationship between high order ARX modeling and subspace identification. A high order ARX model is utilized to obtain initial estimates of certain Markov parameters. These parameters are then used to restructure the data model used for subspace identification to facilitate the estimation of the state sequence. Based on the estimated state sequence, the system parameters are estimated by linear regression. The method is shown to be competitive to existing subspace methods by a simulation example. The method can also be used, without modification, on closed loop data in contrast to most previously published subspace identification methods.
---
paper_title: System identification using balanced parametrizations

paper_content:

Some general issues in the "black-box" identification of multivariable systems are first discussed. It is then suggested that balanced parametrizations can be used to give identifiable forms. A particular advantage is that balanced parametrizations are known for several useful classes of linear dynamic models, including stable minimal models, minimum-phase models, positive-real models, and normalized coprime factor models. Before optimizing the parameters of balanced parametrizations, an initial model must be found. We use realization-based methods and so-called "subspace" methods for this purpose. These methods are very effective at finding accurate initial models without preliminary estimation of various structural indexes. The paper ends with two simulation examples, which compare the use of balanced parametrizations with more traditional ones, and three "real" examples based on practical problems: a distillation column, an industrial dryer, and the (irrational) spectrum or sea waves.
---
paper_title: Subspace Identification for Linear Systems: Theory ― Implementation ― Applications

paper_content:

Subspace Identification for Linear Systems focuses on the theory, implementation and applications of subspace identification algorithms for linear time-invariant finitedimensional dynamical systems. These algorithms allow for a fast, straightforward ::: and accurate determination of linear multivariable models from measured inputoutput data. The theory of subspace identification algorithms is presented in detail. Several chapters are devoted to deterministic, stochastic and combined deterministicstochastic ::: subspace identification algorithms. For each case, the geometric properties are stated in a main 'subspace' Theorem. Relations to existing algorithms and literature are explored, as are the interconnections between different subspace algorithms. The subspace identification theory is linked to the theory of frequency weighted model reduction, which leads to new interpretations and insights. The implementation of subspace identification algorithms is discussed in terms of the robust and computationally efficient RQ and singular value decompositions, which are well-established algorithms from numerical linear algebra. The algorithms are implemented in combination with a whole set of classical identification algorithms,processing and validation tools in Xmath's ISID, a commercially available graphical user interface toolbox. The basic subspace algorithms in the book are also implemented in a set of MATLABÂ® files accompanying the book. An application of ISID to an industrial glass tube manufacturing process is presented in detail, illustrating the power and user-friendliness of the subspace identification ::: algorithms and of their implementation in ISID. The identified model allows for an optimal control of the process, leading to a significant enhancement of the production quality. The applicability of subspace identification algorithms in industry is further illustrated with the application of the MATLABÂ® files to ten practical problems. Since all necessary data and MATLABÂ® files are included, the reader can easily step through these applications, and thus get more insight in the algorithms. ::: Subspace Identification for Linear Systems is an important reference for all researchers in system theory, control theory, signal processing, automization,mechatronics, chemical, electrical, mechanical and aeronautical engineering.
---
paper_title: Continuous-time predictor-based subspace identification using laguerre filters

paper_content:

This study deals with the problem of continuous-time model identification and presents two subspace-based algorithms capable of dealing with data generated by systems operating in closed loop. The algorithms are developed by reformulating the identification problem from the continuous-time model to equivalent ones to which discrete-time subspace identification techniques can be applied. More precisely, two approaches are considered, the former leading to the so-called all-pass domain by using a bank of Laguerre filters applied to the input-output data and the latter corresponding to the projection of the input-output data onto an orthonormal basis, again defined in terms of Laguerre filters. In both frameworks, the Predictor-Based Subspace Identification, originally developed in the case of discrete-time systems, can be reformulated for the continuous-time case. Simulation results are used to illustrate the achievable performance of the proposed approaches with respect to existing methods available in the literature.
---
paper_title: Continuous-time subspace identification in closed-loop

paper_content:

This paper deals with the problem of continuoustime model identification and presents a subspace-based algorithm capable of dealing with data generated by systems operating in closed-loop. The algorithm is developed by reformulating the identification problem from the continuous-time model to an equivalent one to which discrete-time subspace identification techniques can be applied. More precisely, the considered approach corresponds to the projection of the input-output data onto an orthonormal basis, defined in terms of Laguerre filters. In this framework, the PBSID subspace identification algorithm, originally developed in the case of discrete-time systems, can be reformulated for the continuoustime case. Simulation results are used to illustrate the achievable performance of the proposed approach with respect to existing methods available in the literature.
---
paper_title: Subspace like identification incorporating prior information

paper_content:

The subspace identification methods have proved to be a powerful tool, which can further benefit from the prior information incorporation algorithm proposed in this note. In the industrial environment, there is often some knowledge about the identified system (known static gains, dominant time constants, low frequency character, etc.), which can be used to improve model quality and its compliance with first principles. The proposed algorithm has two stages. The first one is similar to the subspace methods as it uses their interpretation as an optimization problem of finding parameters of an optimal multi-step linear predictor for the experimental data. This problem is reformulated in the Bayesian framework allowing prior information incorporation in the form of the mean value and the covariance of the impulse response, which is shown to be useful for the incorporation of several prior information types. The second stage with state space model realization from the posterior impulse response estimate is different from the standard subspace methods as it is based on the structured weighted lower rank approximation, which is necessary to preserve the prior information incorporated in the first stage.
---
paper_title: Parameterization and identification of multivariable state-space systems: A canonical approach

paper_content:

In this paper, the problem of determining a canonical state-space representation for multivariable systems is revisited. A method is derived to build a canonical state-space representation directly from data generated by a linear time-invariant system. Contrary to the classic construction methods of canonical parameterizations, the technique developed in this paper does not assume the availability of any observability or controllability indices. However, it requires the A-matrix of any minimal realization of the system to be non-derogatory. A subspace-based identification algorithm is also introduced to estimate such a canonical state-space parameterization directly from input-output data.
---
paper_title: Closed-loop subspace identification with innovation estimation

paper_content:

Abstract Most subspace identification algorithms are not applicable to closed-loop identification because they require future input to be uncorrelated with past innovation. In this paper, we propose a new subspaee identification method that remove this requirement by using a parsimonious model formulation with innovation estimation. A simulation example is included to show the effectiveness of the proposed method.
---
paper_title: Subspace model identification Part 2. Analysis of the elementary output-error state-space model identification algorithm

paper_content:

The elementary MOESP algorithm presented in the first part of this series of papers is analysed in this paper. This is done in three different ways. First, we study the asymptotic properties of the estimated state-space model when only considering zero-mean white noise perturbations on the output sequence. It is shown that, in this case, the MOESPl implementation yields asymptotically unbiased estimates. An important constraint to this result is that the underlying system must have a finite impulse response and subsequently the size of the Hankel matrices, constructed from the input and output data at the beginning of the computations, depends on the number of non-zero Markov parameters. This analysis, however, leads to a second implementation of the elementary MOESP scheme, namely MOESP2. The latter implementation has the same asymptotic properties without the finite impulse response constraint. Secondly, we compare the MOESP2 algorithm with a classical state space model identification scheme. The latter...
---
paper_title: A novel subspace identification approach with enforced causal models

paper_content:

Subspace identification methods (SIMs) for estimating state-space models have been proven to be very useful and numerically efficient. They exist in several variants, but all have one feature in common: as a first step, a collection of high-order ARX models are estimated from vectorized input-output data. In order not to obtain biased estimates, this step must include future outputs. However, all but one of the submodels include non-causal input terms. The coefficients of them will be correctly estimated to zero as more data become available. They still include extra model parameters which give unnecessarily high variance, and also cause bias for closed-loop data. In this paper, a new model formulation is suggested that circumvents the problem. Within the framework, the system matrices (A,B,C,D) and Markov parameters can be estimated separately. It is demonstrated through analysis that the new methods generally give smaller variance in the estimate of the observability matrix and it is supported by simulation studies that this gives lower variance also of the system invariants such as the poles.
---
paper_title: Canonical variate analysis in identification, filtering, and adaptive control

paper_content:

The canonical variate analysis (CVA) approach for system identification, filtering, and adaptive control is developed. The past/future Markov property provides a starting point for defining a reduced-order prediction problem. The solution is a canonical variate analysis that is characterized by a generalized singular value decomposition. State-space model estimation requires only simple regression, and state order selection involves the optimal Akaike information criterion procedure. The CVA method extends to time-varying and abruptly changing systems. A reduce-rank stochastic model predictive control problem is shown to be equivalent to the CVA problem. Also discussed are computational aspects, applications, and an example illustrating the method. New extensions to the identification of general nonlinear systems are briefly discussed. The CVA method provides an approach giving reliable, automatic implementation of identification, filtering, and control for online adaptive control. >
---
paper_title: Consistency analysis of some closed-loop subspace identification methods

paper_content:

We study statistical consistency of two recently proposed subspace identification algorithms for closed-loop systems. These algorithms may be seen as implementations of an abstract state-space construction procedure described by the authors in previous work on stochastic realization of closed-loop systems. A detailed error analysis is undertaken which shows that both algorithms are biased due to an unavoidable mishandling of initial conditions which occurs in closed-loop identification. Instability of the open loop system may also be a cause of trouble.
---
paper_title: On the Asymptotic Properties of Closed-Loop CCA-Type Subspace Algorithms: Equivalence Results and Role of the Future Horizon

paper_content:

In this paper, we shall consider a class of subspace algorithms for identification of linear time invariant systems operating in "closed loop." In particular we study algorithms based on the so-called "state-sequence" approach; we first show that the ADAPTx algorithm by Larimore is asymptotically equivalent to a number of recently developed algorithms, which we call CCA-type algorithms. Based on this equivalence result, we then study the effect of the "future horizon," which is one of the principal "user choices" in subspace identification. It is well known that for the CCA algorithm the asymptotic variance of any system invariant is a non increasing function of the future horizon when input signals are white (or absent). In particular we extend this result, valid for white noise input signals to a slightly more general class of input signals, which include proportional (output or state) feedback controllers and LQG controllers, provided the reference input is white. The condition on the input will be expressed in terms of its state space, which we regard as a rather natural condition in this framework. For the situations not covered by the above result, we shall also describe a computational procedure, based on some recently derived asymptotic variance formulas, which allows to optimize the choice of the future horizon. Some simulation results are included.
---
paper_title: Subspace model identification Part 1. The output-error state-space model identification class of algorithms

paper_content:

In this paper, we present two novel algorithms to realize a finite dimensional, linear time-invariant state-space model from input-output data. The algorithms have a number of common features. They are classified as one of the subspace model identification schemes, in that a major part of the identification problem consists of calculating specially structured subspaces of spaces defined by the input-output data. This structure is then exploited in the calculation of a realization. Another common feature is their algorithmic organization: an RQ factorization followed by a singular value decomposition and the solution of an overdetermined set (or sets) of equations. The schemes assume that the underlying system has an output-error structure and that a measurable input sequence is available. The latter characteristic indicates that both schemes are versions of the MIMO Output-Error State Space model identification (MOESP) approach. The first algorithm is denoted in particular as the (elementary MOESP scheme)...
---
paper_title: 4SID linear regression

paper_content:

State-space subspace system identification (4SID) has been suggested as an alternative to more traditional prediction error system identification, such as ARX least squares estimation. The aim of this note is to analyse the connections between these two different approaches to system identification. The conclusion is that 4SID can be viewed as a linear regression multistep ahead prediction error method, with certain rank constraints. This allows us to analyse 4SID methods within the standard framework of system identification and linear regression estimation. For example, it is shown that ARX models have nice properties in terms of 4SID identification. From a linear regression model, estimates of the extended observability matrix are found. Results from an asymptotic analysis are presented, i.e. explicit formulas for the asymptotic variances of the pole estimation error are given. From these expressions, some difficulties in choosing user specified parameters are pointed out in an example. >
---
paper_title: A linear regression approach to state-space subspace system identification

paper_content:

Abstract Recently, state-space subspace system identification (4SID) has been suggested as an alternative to the more traditional prediction error system identification. The aim of this paper is to analyze the connections between these two different approaches to system identification. The conclusion is that 4SID can be viewed as a linear regression multistep-ahead prediction error method with certain rank constraints. This allows us to describe 4SID methods within the standard framework of system identification and linear regression estimation. For example, this observation is used to compare different cost-functions which occur rather implicitly in the ordinary framework of 4SID. From the cost-functions, estimates of the extended observability matrix are derived and related to previous work. Based on the estimates of the observability matrix, the asymptotic properties of two pole estimators, namely the shift invariance method and a weighted subspace fitting method, are analyzed. Expressions for the asymptotic variances of the pole estimation error are given. From these expressions, difficulties in choosing userspecified parameters are pointed out. Furthermore, it is found that a row-weighting in the subspace estimation step does not affect the pole estimation error asymptotically.
---
paper_title: Subspace Identification from Closed Loop Data

paper_content:

The so-called subspace methods for direct identification of linear models in state space form have drawn considerable interest recently. They have been found to work well in many cases but have one drawback — they do not yield consistent estimates for data collected under output feedback. The present paper points to the reasons for this. We stress how the basic idea is to focus on the estimation of the state-variable candidates — the k-step ahead output predictors. By recomputing these from a ‘non-parametric’ (or, rather, high order ARX) one-step ahead predictor model, closed loop data can be handled.
---
paper_title: Filtering and System Identification: A Least Squares Approach

paper_content:

Preface 1. Introduction 2. Linear algebra 3. Discrete-time signals and systems 4. Random variables and signals 5. Kalman filtering 6. Estimation of spectra and frequency response functions 7. Output-error parametric model estimation 8. Prediction-error parametric model estimation 9. Subspace model identification 10. The system identification cycle Notation and symbols List of abbreviations References Index.
---
paper_title: Kalman filtering for positioning and heading control of ships and offshore rigs

paper_content:

In this article, we have described the main components of a ship motion-control system and two particular motion-control problems that require wave filtering, namely, dynamic positioning and heading autopilot. Then, we discussed the models commonly used for vessel response and showed how these models are used for Kalman filter design. We also briefly discussed parameter and noise covariance estimation, which are used for filter tuning. To illustrate the performance, a case study based on numerical simulations for a ship autopilot was considered. The material discussed in this article conforms to modern commercially available ship motion-control systems. Most of the vessels operating in the offshore industry worldwide use Kalman filters for velocity estimation and wave filtering. Thus, the article provides an up-to-date tutorial and overview of Kalman-filter-based wave filtering.
---
paper_title: Adaptive Filtering And Change Detection

paper_content:

INTRODUCTION Extended Summary. Applications. SIGNAL ESTIMATION On--Line Approaches. Off--Line Approaches. PARAMETER ESTIMATION Adaptive Filtering. Change Detection Based on Sliding Windows Change Detection Based on Filter Banks STATE ESTIMATION Kalman Filtering Change Detection Based on Likelihood Ratios Change Detection Based on Multiple Models Change Detection Based on Algebraical Consistency Tests THEORY Evaluation Theory Linear Estimation A. Signal models and notation B. Fault detection terminology
---
paper_title: Constructing the state of random processes with feedback

paper_content:

Abstract Subspace identification, is based on state space construction, i.e. stochastic realization theory. However state space construction in the presence of feedback is still an open problem. In this paper we provide a geometric construction based on a oblique predictor space which yields a bona-fide (stationary) state space in the presence of feedback, if the open loop transfer functions of the system satisfy certain stability conditions.
---
paper_title: Applications of Kalman Filtering in Aerospace 1960 to the Present

paper_content:

Nothing speaks more eloquently of the impact of Kalman fi ltering on society than the 2008 Draper Prize awarded by the National Academy of Engineering to Rudolf Emil Kalman “for the development and dissemination of the optimal digital technique (known as the Kalman fi lter) that is pervasively used to control a vast array of consumer, health, commercial, and defense products.” To quote further from the announcement of the Awards Committee, “The Kalman fi lter revolutionized the fi eld of control theory and has become pervasive in engineering systems.” The latter statement is especially pertinent within the aerospace community. In the 1960s, the Kalman filter was applied to navigation for the Apollo Project, which required estimates of the trajectories of manned spacecraft going to the Moon and back. With the lives of the astronauts at stake, it was essential that the Kalman filter be proven effective and reliable before it could be used. This article is about the lead up to Kalman’s work, key discoveries in the development and maturation of the filter, a sampling of its many applications in aerospace, and recognition of some who played key roles in that history.
---
paper_title: Optimal Control: Linear Quadratic Methods

paper_content:

Part 1 Theory of the optimal regulator: the standard regulator problems I and II tracking systems. Part 2 Properties and application of the optimal regulator: properties of regulator systems with a classical control interpretation asymptotic properties and quadratic weights selection state estimator design system design using state estimators frequency shaping controller reduction digital controllers.
---
paper_title: System Modeling and Identification

paper_content:

Black box models signals and systems spectrum analysis linear regression identification of time-series models modelling the experimental procedures model approximation real-time identification continous-time models multidimentional identification adaptive systems. Appendices: basic matrix algebra statistical inference numerical optimization statistical properties of time series a case study.
---
paper_title: System Identification: Theory for the User

paper_content:

Das Buch behandelt die Systemidentifizierung in dem theoretischen Bereich, der direkte Auswirkungen auf Verstaendnis und praktische Anwendung der verschiedenen Verfahren zur Identifizierung hat. Da ...
---
paper_title: N4SID: Subspace Algorithms for the Identification of Combined Deterministic-Stochastic Systems

paper_content:

Abstract Recently a great deal of attention has been given to numerical algorithms for subspace state space system identification (N4SID). In this paper, we derive two new N4SID algorithms to identify mixed deterministic-stochastic systems. Both algorithms determine state sequences through the projection of input and output data. These state sequences are shown to be outputs of non-steady state Kalman filter banks. From these it is easy to determine the state space system matrices. The N4SID algorithms are always convergent (non-iterative) and numerically stable since they only make use of QR and Singular Value Decompositions. Both N4SID algorithms are similar, but the second one trades off accuracy for simplicity. These new algorithms are compared with existing subspace algorithms in theory and in practice.
---
paper_title: Subspace State Space System Identification For Industrial Processes

paper_content:

Abstract We give a general overview of the state-of-the-art in subspace system identification methods. We have restricted ourselves to the most important ideas and developments since the methods appeared in the late eighties. First, the basics of linear subspace identification are summarized. Different algorithms one finds in literature (such as N4SID, IV-4SID, MOESP, CVA) are discussed and put into a unifying framework. Further, a comparison between subspace identification and prediction error methods is made on the basis of computational complexity and precision of the methods by applying them on 10 industrial data sets.
---
paper_title: Subspace-based methods for the identification of linear time-invariant systems

paper_content:

Subspace-based methods for system identification have attracted much attention during the past few years. This interest is due to the ability of providing accurate state-space models for multivariable linear systems directly from input-output data. The methods have their origin in classical state-space realization theory as developed in the 1960s. The main computational tools are the QR and the singular-value decompositions. Here, an overview of existing subspace-based techniques for system identification is given. The methods are grouped into the classes of realization-based and direct techniques. Similarities between different algorithms are pointed out, and their applicability is commented upon. We also discuss some recent ideas for improving and extending the methods. A simulation example is included for comparing different algorithms. The subspace-based approach is found to perform competitive with respect to prediction-error methods, provided the system is properly excited.
---
paper_title: Subspace Identification and ARX Modeling

paper_content:

Abstract In this paper we present a new identification method that points at the close relationship between high order ARX modeling and subspace identification. A high order ARX model is utilized to obtain initial estimates of certain Markov parameters. These parameters are then used to restructure the data model used for subspace identification to facilitate the estimation of the state sequence. Based on the estimated state sequence, the system parameters are estimated by linear regression. The method is shown to be competitive to existing subspace methods by a simulation example. The method can also be used, without modification, on closed loop data in contrast to most previously published subspace identification methods.
---
paper_title: Subspace Identification for Linear Systems: Theory ― Implementation ― Applications

paper_content:

Subspace Identification for Linear Systems focuses on the theory, implementation and applications of subspace identification algorithms for linear time-invariant finitedimensional dynamical systems. These algorithms allow for a fast, straightforward ::: and accurate determination of linear multivariable models from measured inputoutput data. The theory of subspace identification algorithms is presented in detail. Several chapters are devoted to deterministic, stochastic and combined deterministicstochastic ::: subspace identification algorithms. For each case, the geometric properties are stated in a main 'subspace' Theorem. Relations to existing algorithms and literature are explored, as are the interconnections between different subspace algorithms. The subspace identification theory is linked to the theory of frequency weighted model reduction, which leads to new interpretations and insights. The implementation of subspace identification algorithms is discussed in terms of the robust and computationally efficient RQ and singular value decompositions, which are well-established algorithms from numerical linear algebra. The algorithms are implemented in combination with a whole set of classical identification algorithms,processing and validation tools in Xmath's ISID, a commercially available graphical user interface toolbox. The basic subspace algorithms in the book are also implemented in a set of MATLABÂ® files accompanying the book. An application of ISID to an industrial glass tube manufacturing process is presented in detail, illustrating the power and user-friendliness of the subspace identification ::: algorithms and of their implementation in ISID. The identified model allows for an optimal control of the process, leading to a significant enhancement of the production quality. The applicability of subspace identification algorithms in industry is further illustrated with the application of the MATLABÂ® files to ten practical problems. Since all necessary data and MATLABÂ® files are included, the reader can easily step through these applications, and thus get more insight in the algorithms. ::: Subspace Identification for Linear Systems is an important reference for all researchers in system theory, control theory, signal processing, automization,mechatronics, chemical, electrical, mechanical and aeronautical engineering.
---
paper_title: Optimal Control: Linear Quadratic Methods

paper_content:

Part 1 Theory of the optimal regulator: the standard regulator problems I and II tracking systems. Part 2 Properties and application of the optimal regulator: properties of regulator systems with a classical control interpretation asymptotic properties and quadratic weights selection state estimator design system design using state estimators frequency shaping controller reduction digital controllers.
---
paper_title: Consistency analysis of some closed-loop subspace identification methods

paper_content:

We study statistical consistency of two recently proposed subspace identification algorithms for closed-loop systems. These algorithms may be seen as implementations of an abstract state-space construction procedure described by the authors in previous work on stochastic realization of closed-loop systems. A detailed error analysis is undertaken which shows that both algorithms are biased due to an unavoidable mishandling of initial conditions which occurs in closed-loop identification. Instability of the open loop system may also be a cause of trouble.
---
paper_title: Improvement of Observer/Kalman Filter Identification (OKID) by Residual Whitening

paper_content:

This paper presents a time-domain method to identify a state space model of a linear system and its corresponding observer/Kalman filter from a given set of general input-output data. The identified filter has the properties that its residual is minimized in the least squares sense, orthogonal to the time-shifted versions of itself, and to the given input-output data sequence. The connection between the state space model and a particular auto-regressive moving average description of a linear system is made in terms of the Kalman filter and a deadbeat gain matrix. The procedure first identifies the Markov parameters of an observer system, from which a state space model of the system and the filter gain are computed. The developed procedure is shown to improve results obtained by an existing observer/Kalman filter identification method, which is based on an auto-regressive model without the moving average terms. Numerical and experimental results are presented to illustrate the proposed method.
---
paper_title: System Identification: Theory for the User

paper_content:

Das Buch behandelt die Systemidentifizierung in dem theoretischen Bereich, der direkte Auswirkungen auf Verstaendnis und praktische Anwendung der verschiedenen Verfahren zur Identifizierung hat. Da ...
---
paper_title: Matrix analysis

paper_content:

Linear algebra and matrix theory are fundamental tools in mathematical and physical science, as well as fertile fields for research. This new edition of the acclaimed text presents results of both classic and recent matrix analyses using canonical forms as a unifying theme, and demonstrates their importance in a variety of applications. The authors have thoroughly revised, updated, and expanded on the first edition. The book opens with an extended summary of useful concepts and facts and includes numerous new topics and features, such as: - New sections on the singular value and CS decompositions - New applications of the Jordan canonical form - A new section on the Weyr canonical form - Expanded treatments of inverse problems and of block matrices - A central role for the Von Neumann trace theorem - A new appendix with a modern list of canonical forms for a pair of Hermitian matrices and for a symmetric-skew symmetric pair - Expanded index with more than 3,500 entries for easy reference - More than 1,100 problems and exercises, many with hints, to reinforce understanding and develop auxiliary themes such as finite-dimensional quantum systems, the compound and adjugate matrices, and the Loewner ellipsoid - A new appendix provides a collection of problem-solving hints.
---
paper_title: 4SID linear regression

paper_content:

State-space subspace system identification (4SID) has been suggested as an alternative to more traditional prediction error system identification, such as ARX least squares estimation. The aim of this note is to analyse the connections between these two different approaches to system identification. The conclusion is that 4SID can be viewed as a linear regression multistep ahead prediction error method, with certain rank constraints. This allows us to analyse 4SID methods within the standard framework of system identification and linear regression estimation. For example, it is shown that ARX models have nice properties in terms of 4SID identification. From a linear regression model, estimates of the extended observability matrix are found. Results from an asymptotic analysis are presented, i.e. explicit formulas for the asymptotic variances of the pole estimation error are given. From these expressions, some difficulties in choosing user specified parameters are pointed out in an example. >
---
paper_title: AUTOMATIC INFERENCE FOR INFINITE ORDER VECTOR AUTOREGRESSIONS

paper_content:

Infinite order vector autoregressive (VAR) models have been used in a number of applications ranging from spectral density estimation, impulse response analysis, and tests for cointegration and unit roots, to forecasting. For estimation of such models it is necessary to approximate the infinite order lag structure by finite order VARs. In practice, the order of approximation is often selected by information criteria or by general-to-specific specification tests. Unlike in the finite order VAR case these selection rules are not consistent in the usual sense, and the asymptotic properties of parameter estimates of the infinite order VAR do not follow as easily as in the finite order case. In this paper it is shown that the parameter estimates of the infinite order VAR are asymptotically normal with zero mean when the model is approximated by a finite order VAR with a data dependent lag length. The requirement for the result to hold is that the selected lag length satisfies certain rate conditions with probability tending to one. Two examples of selection rules satisfying these requirements are discussed. Uniform rates of convergence for the parameters of the infinite order VAR are also established. Very helpful comments by the editor and two referees led to a substantial improvement of the manuscript. I am particularly indebted to one of the referees for pointing out an error in the proofs. All remaining errors are my own. Financial support from NSF grant SES−0095132 is gratefully acknowledged.
---
paper_title: A linear regression approach to state-space subspace system identification

paper_content:

Abstract Recently, state-space subspace system identification (4SID) has been suggested as an alternative to the more traditional prediction error system identification. The aim of this paper is to analyze the connections between these two different approaches to system identification. The conclusion is that 4SID can be viewed as a linear regression multistep-ahead prediction error method with certain rank constraints. This allows us to describe 4SID methods within the standard framework of system identification and linear regression estimation. For example, this observation is used to compare different cost-functions which occur rather implicitly in the ordinary framework of 4SID. From the cost-functions, estimates of the extended observability matrix are derived and related to previous work. Based on the estimates of the observability matrix, the asymptotic properties of two pole estimators, namely the shift invariance method and a weighted subspace fitting method, are analyzed. Expressions for the asymptotic variances of the pole estimation error are given. From these expressions, difficulties in choosing userspecified parameters are pointed out. Furthermore, it is found that a row-weighting in the subspace estimation step does not affect the pole estimation error asymptotically.
---
paper_title: Identification of observer/Kalman filter Markov parameters - Theory and experiments

paper_content:

This paper discusses an algorithm to compute the Markov parameters of an observer or Kalman filter from experimental input and output data. The Markov parameters can then be used for identification of a state-space representation, with associated Kalman or observer gain, for the purpose of controller design. The algorithm is a nonrecursive matrix version of two recursive algorithms developed in previous works for different purposes, and the relationship between these other algorithms is developed. The new matrix formulation here gives insight into the existence and uniqueness of solutions of certain equations and offers bounds on the proper choice of observer order. It is shown that if one uses data containing noise and seeks the fastest possible deterministic observer, the deadbeat observer, one instead obtains the Kalman filter, which is the fastest possible observer in the stochastic environment. The results of the paper are demonstrated in numerical studies and experiments on the Bubble space telescope.
---
paper_title: Subspace-based methods for the identification of linear time-invariant systems

paper_content:

Subspace-based methods for system identification have attracted much attention during the past few years. This interest is due to the ability of providing accurate state-space models for multivariable linear systems directly from input-output data. The methods have their origin in classical state-space realization theory as developed in the 1960s. The main computational tools are the QR and the singular-value decompositions. Here, an overview of existing subspace-based techniques for system identification is given. The methods are grouped into the classes of realization-based and direct techniques. Similarities between different algorithms are pointed out, and their applicability is commented upon. We also discuss some recent ideas for improving and extending the methods. A simulation example is included for comparing different algorithms. The subspace-based approach is found to perform competitive with respect to prediction-error methods, provided the system is properly excited.
---
paper_title: On Consistency of Subspace Methods for System Identification

paper_content:

Subspace methods for identification of linear time-invariant dynamical systems typically consist of two main steps. First, a so-called subspace estimate is constructed. This first step usually consists of estimating the range space of the extended observability matrix. Secondly, an estimate of system parametersis obtained, based on the subspace estimate. In this paper, the consistency of a large class of methods for estimating the extended observability matrix is analyzed. Persistence of excitation conditions on the input signal are given which guarantee consistent estimates for systems with only measurement noise. For systems with process noise, it is shown that a persistence of excitation condition on the input is not sufficient. More precisely, an example for which the subspace methods fail to give a consistent estimate of the transfer function is given. This failure occurs even if the input is persistently exciting of any order. It is also shown that this problem can be eliminated if stronger conditions on the input signal are imposed.
---
paper_title: System Identification: Theory for the User

paper_content:

Das Buch behandelt die Systemidentifizierung in dem theoretischen Bereich, der direkte Auswirkungen auf Verstaendnis und praktische Anwendung der verschiedenen Verfahren zur Identifizierung hat. Da ...
---
paper_title: Filtering and System Identification: A Least Squares Approach

paper_content:

Preface 1. Introduction 2. Linear algebra 3. Discrete-time signals and systems 4. Random variables and signals 5. Kalman filtering 6. Estimation of spectra and frequency response functions 7. Output-error parametric model estimation 8. Prediction-error parametric model estimation 9. Subspace model identification 10. The system identification cycle Notation and symbols List of abbreviations References Index.
---
paper_title: Application of a recursive subspace identification algorithm to change detection

paper_content:

Abstract This paper presents a new change detection method with the aid of subspace identification. The proposed method is based on monitoring a change in variance of a statistic generated by a recursive subspace identification algorithm. An asymptotic property of the statistic is presented. Without changes during sampling, it is shown that, under relevant assumptions. the statistic converges in probability to a stack of noise vectors multiplied from the left side by a Toeplitz matrix. A numerical example illustrates that the proposed method can detect changes in the dynamics of a system, without being disturbed by changes in the dynamics of an input signal which are not our concern.
---
paper_title: Asymptotic variance of subspace estimates

paper_content:

Abstract We give new simple general expressions for the asymptotic covariance of the estimated system parameters (A,B,C,D) in subspace identification. The formulas can be applied to a whole class of subspace methods including N4SID, MOESP, CVA, etc. The asymptotic expressions highlight how the conditioning of the estimation problem influences the accuracy of the estimates.
---
paper_title: Asymptotic variance of subspace methods by data orthogonalization and model decoupling: a comparative analysis

paper_content:

This is a companion of the paper Chiuso and Picci (2004d) where we do asymptotic error analysis of a weighted PI-MOESP type method and compare accuracy with respect to estimates obtained by customary ''joint'' subspace methods. The analysis shows that, under certain conditions, methods based on orthogonal decomposition of the input-output data and block-decoupled parametrization perform better than traditional joint-model based methods in the circumstance of nearly parallel regressors.
---
paper_title: Constructing the state of random processes with feedback

paper_content:

Abstract Subspace identification, is based on state space construction, i.e. stochastic realization theory. However state space construction in the presence of feedback is still an open problem. In this paper we provide a geometric construction based on a oblique predictor space which yields a bona-fide (stationary) state space in the presence of feedback, if the open loop transfer functions of the system satisfy certain stability conditions.
---
paper_title: A linear regression approach to state-space subspace system identification

paper_content:

Abstract Recently, state-space subspace system identification (4SID) has been suggested as an alternative to the more traditional prediction error system identification. The aim of this paper is to analyze the connections between these two different approaches to system identification. The conclusion is that 4SID can be viewed as a linear regression multistep-ahead prediction error method with certain rank constraints. This allows us to describe 4SID methods within the standard framework of system identification and linear regression estimation. For example, this observation is used to compare different cost-functions which occur rather implicitly in the ordinary framework of 4SID. From the cost-functions, estimates of the extended observability matrix are derived and related to previous work. Based on the estimates of the observability matrix, the asymptotic properties of two pole estimators, namely the shift invariance method and a weighted subspace fitting method, are analyzed. Expressions for the asymptotic variances of the pole estimation error are given. From these expressions, difficulties in choosing userspecified parameters are pointed out. Furthermore, it is found that a row-weighting in the subspace estimation step does not affect the pole estimation error asymptotically.
---
paper_title: Identification of the deterministic part of MIMO state space models given in innovations form from input-output data

paper_content:

The problem of Linear Multivariable State Space model identification from input-output data can under the presence of process- and measurement noise be solved in a non-iterative way when incorporating instrumental variables constructed from both input and output sequences in the recently developed class of multivariable output-error state space model class of subspace model identification schemes. Abstract--In this paper we describe two algorithms to identify a linear, time-invariant, finite dimensional state space model from input-output data. The system to be identified is assumed to be excited by a measurable input and an unknown process tloise and the measurements are disturbed by unknown measurement noise. Both noise sequences are discrete zero-mean white noise, The first algorithm gives consistent estimates only for the case where the input also is zero-mean white noise, while the same result is obtained with the second algorithm without this constraint. For the special case where the input signal is discrete zero-mean white noise, it is explicitly shown that this second algorithm is a special case of the recently developed Multivariable Output-Error State Space (MOESP) class of algorithms based on instrumental variables. The usefulness of the presented schemes is highlighted in a realistic simulation study.
---
paper_title: On the Relation Between CCA and Predictor-Based Subspace Identification

paper_content:

In this paper, we investigate the relation between a recently proposed subspace method based on predictor identification (PBSID), known also as ldquowhitening filter algorithm,rdquo and the classical CCA algorithm. The comparison is motivated by i) the fact that CCA is known to be asymptotically efficient for time series identification and optimal for white measured inputs and ii) some recent results showing that a number of recently developed algorithms are very closely related to PBSID. We show that PBSID is asymptotically equivalent to CCA precisely in the situations in which CCA is optimal while an ldquooptimizedrdquo version of PBSID behaves no worse than CCA also for nonwhite inputs. Even though PBSID (and its optimized version) are consistent regardless of the presence of feedback, in this paper we work under the assumption that there is no feedback to make the comparison with CCA meaningful. The results of this paper imply that the ldquooptimizedrdquo PBSID, besides being able to handle feedback, is to be preferred to CCA also when there is no feedback; only in very specific cases (white or no inputs) are the two algorithms (asymptotically) equivalent.
---
paper_title: Subspace-based methods for the identification of linear time-invariant systems

paper_content:

Subspace-based methods for system identification have attracted much attention during the past few years. This interest is due to the ability of providing accurate state-space models for multivariable linear systems directly from input-output data. The methods have their origin in classical state-space realization theory as developed in the 1960s. The main computational tools are the QR and the singular-value decompositions. Here, an overview of existing subspace-based techniques for system identification is given. The methods are grouped into the classes of realization-based and direct techniques. Similarities between different algorithms are pointed out, and their applicability is commented upon. We also discuss some recent ideas for improving and extending the methods. A simulation example is included for comparing different algorithms. The subspace-based approach is found to perform competitive with respect to prediction-error methods, provided the system is properly excited.
---
paper_title: Subspace Identification for Linear Systems: Theory ― Implementation ― Applications

paper_content:

Subspace Identification for Linear Systems focuses on the theory, implementation and applications of subspace identification algorithms for linear time-invariant finitedimensional dynamical systems. These algorithms allow for a fast, straightforward ::: and accurate determination of linear multivariable models from measured inputoutput data. The theory of subspace identification algorithms is presented in detail. Several chapters are devoted to deterministic, stochastic and combined deterministicstochastic ::: subspace identification algorithms. For each case, the geometric properties are stated in a main 'subspace' Theorem. Relations to existing algorithms and literature are explored, as are the interconnections between different subspace algorithms. The subspace identification theory is linked to the theory of frequency weighted model reduction, which leads to new interpretations and insights. The implementation of subspace identification algorithms is discussed in terms of the robust and computationally efficient RQ and singular value decompositions, which are well-established algorithms from numerical linear algebra. The algorithms are implemented in combination with a whole set of classical identification algorithms,processing and validation tools in Xmath's ISID, a commercially available graphical user interface toolbox. The basic subspace algorithms in the book are also implemented in a set of MATLABÂ® files accompanying the book. An application of ISID to an industrial glass tube manufacturing process is presented in detail, illustrating the power and user-friendliness of the subspace identification ::: algorithms and of their implementation in ISID. The identified model allows for an optimal control of the process, leading to a significant enhancement of the production quality. The applicability of subspace identification algorithms in industry is further illustrated with the application of the MATLABÂ® files to ten practical problems. Since all necessary data and MATLABÂ® files are included, the reader can easily step through these applications, and thus get more insight in the algorithms. ::: Subspace Identification for Linear Systems is an important reference for all researchers in system theory, control theory, signal processing, automization,mechatronics, chemical, electrical, mechanical and aeronautical engineering.
---
paper_title: Closed-loop subspace identification with innovation estimation

paper_content:

Abstract Most subspace identification algorithms are not applicable to closed-loop identification because they require future input to be uncorrelated with past innovation. In this paper, we propose a new subspaee identification method that remove this requirement by using a parsimonious model formulation with innovation estimation. A simulation example is included to show the effectiveness of the proposed method.
---
paper_title: Subspace model identification Part 2. Analysis of the elementary output-error state-space model identification algorithm

paper_content:

The elementary MOESP algorithm presented in the first part of this series of papers is analysed in this paper. This is done in three different ways. First, we study the asymptotic properties of the estimated state-space model when only considering zero-mean white noise perturbations on the output sequence. It is shown that, in this case, the MOESPl implementation yields asymptotically unbiased estimates. An important constraint to this result is that the underlying system must have a finite impulse response and subsequently the size of the Hankel matrices, constructed from the input and output data at the beginning of the computations, depends on the number of non-zero Markov parameters. This analysis, however, leads to a second implementation of the elementary MOESP scheme, namely MOESP2. The latter implementation has the same asymptotic properties without the finite impulse response constraint. Secondly, we compare the MOESP2 algorithm with a classical state space model identification scheme. The latter...
---
paper_title: Consistency analysis of some closed-loop subspace identification methods

paper_content:

We study statistical consistency of two recently proposed subspace identification algorithms for closed-loop systems. These algorithms may be seen as implementations of an abstract state-space construction procedure described by the authors in previous work on stochastic realization of closed-loop systems. A detailed error analysis is undertaken which shows that both algorithms are biased due to an unavoidable mishandling of initial conditions which occurs in closed-loop identification. Instability of the open loop system may also be a cause of trouble.
---
paper_title: On the Asymptotic Properties of Closed-Loop CCA-Type Subspace Algorithms: Equivalence Results and Role of the Future Horizon

paper_content:

In this paper, we shall consider a class of subspace algorithms for identification of linear time invariant systems operating in "closed loop." In particular we study algorithms based on the so-called "state-sequence" approach; we first show that the ADAPTx algorithm by Larimore is asymptotically equivalent to a number of recently developed algorithms, which we call CCA-type algorithms. Based on this equivalence result, we then study the effect of the "future horizon," which is one of the principal "user choices" in subspace identification. It is well known that for the CCA algorithm the asymptotic variance of any system invariant is a non increasing function of the future horizon when input signals are white (or absent). In particular we extend this result, valid for white noise input signals to a slightly more general class of input signals, which include proportional (output or state) feedback controllers and LQG controllers, provided the reference input is white. The condition on the input will be expressed in terms of its state space, which we regard as a rather natural condition in this framework. For the situations not covered by the above result, we shall also describe a computational procedure, based on some recently derived asymptotic variance formulas, which allows to optimize the choice of the future horizon. Some simulation results are included.
---
paper_title: Subspace model identification Part 1. The output-error state-space model identification class of algorithms

paper_content:

In this paper, we present two novel algorithms to realize a finite dimensional, linear time-invariant state-space model from input-output data. The algorithms have a number of common features. They are classified as one of the subspace model identification schemes, in that a major part of the identification problem consists of calculating specially structured subspaces of spaces defined by the input-output data. This structure is then exploited in the calculation of a realization. Another common feature is their algorithmic organization: an RQ factorization followed by a singular value decomposition and the solution of an overdetermined set (or sets) of equations. The schemes assume that the underlying system has an output-error structure and that a measurable input sequence is available. The latter characteristic indicates that both schemes are versions of the MIMO Output-Error State Space model identification (MOESP) approach. The first algorithm is denoted in particular as the (elementary MOESP scheme)...
---
paper_title: Parallel QR Implementation of Subspace Identification with Parsimonious Models

paper_content:

In this paper we reveal that the typical subspace identification algorithms use non-parsimonious model formulations, with extra terms in the model that appear to be non-causal. These terms are the ...
---
paper_title: Constructing the state of random processes with feedback

paper_content:

Abstract Subspace identification, is based on state space construction, i.e. stochastic realization theory. However state space construction in the presence of feedback is still an open problem. In this paper we provide a geometric construction based on a oblique predictor space which yields a bona-fide (stationary) state space in the presence of feedback, if the open loop transfer functions of the system satisfy certain stability conditions.
---
paper_title: Consistency analysis of some closed-loop subspace identification methods

paper_content:

We study statistical consistency of two recently proposed subspace identification algorithms for closed-loop systems. These algorithms may be seen as implementations of an abstract state-space construction procedure described by the authors in previous work on stochastic realization of closed-loop systems. A detailed error analysis is undertaken which shows that both algorithms are biased due to an unavoidable mishandling of initial conditions which occurs in closed-loop identification. Instability of the open loop system may also be a cause of trouble.
---
paper_title: Parallel QR Implementation of Subspace Identification with Parsimonious Models

paper_content:

In this paper we reveal that the typical subspace identification algorithms use non-parsimonious model formulations, with extra terms in the model that appear to be non-causal. These terms are the ...
---
paper_title: Identification of the deterministic part of MIMO state space models given in innovations form from input-output data

paper_content:

The problem of Linear Multivariable State Space model identification from input-output data can under the presence of process- and measurement noise be solved in a non-iterative way when incorporating instrumental variables constructed from both input and output sequences in the recently developed class of multivariable output-error state space model class of subspace model identification schemes. Abstract--In this paper we describe two algorithms to identify a linear, time-invariant, finite dimensional state space model from input-output data. The system to be identified is assumed to be excited by a measurable input and an unknown process tloise and the measurements are disturbed by unknown measurement noise. Both noise sequences are discrete zero-mean white noise, The first algorithm gives consistent estimates only for the case where the input also is zero-mean white noise, while the same result is obtained with the second algorithm without this constraint. For the special case where the input signal is discrete zero-mean white noise, it is explicitly shown that this second algorithm is a special case of the recently developed Multivariable Output-Error State Space (MOESP) class of algorithms based on instrumental variables. The usefulness of the presented schemes is highlighted in a realistic simulation study.
---
paper_title: Filtering and System Identification: A Least Squares Approach

paper_content:

Preface 1. Introduction 2. Linear algebra 3. Discrete-time signals and systems 4. Random variables and signals 5. Kalman filtering 6. Estimation of spectra and frequency response functions 7. Output-error parametric model estimation 8. Prediction-error parametric model estimation 9. Subspace model identification 10. The system identification cycle Notation and symbols List of abbreviations References Index.
---
paper_title: Asymptotic variance of subspace methods by data orthogonalization and model decoupling: a comparative analysis

paper_content:

This is a companion of the paper Chiuso and Picci (2004d) where we do asymptotic error analysis of a weighted PI-MOESP type method and compare accuracy with respect to estimates obtained by customary ''joint'' subspace methods. The analysis shows that, under certain conditions, methods based on orthogonal decomposition of the input-output data and block-decoupled parametrization perform better than traditional joint-model based methods in the circumstance of nearly parallel regressors.
---
paper_title: The role of Vector AutoRegressive Modeling in Predictor Based Subspace Identification

paper_content:

Subspace identification for closed loop systems has been recently studied by several authors. A class of new and consistent closed-loop subspace algorithms is based on identification of a predictor model, in a way similar as prediction error methods (PEM) do. Experimental evidence suggests that these methods have a behavior which is very close to PEM in certain examples. The asymptotical statistical properties of one of these methods have been studied recently allowing to show (i) its relation with CCA and (ii) that Cramer-Rao lower bound is not reached in general. Very little, however, is known concerning their relative performance. In this paper we shall discuss the link between these ''predictor-based'' methods; to this purpose we exploit the role which Vector Auto Regressive with eXogenous input models play in all these algorithms. The results of this paper provide a unifying framework under which all these algorithms can be viewed; also the link with VARX modeling have important implications as to computational complexity is concerned, leading to very computationally attractive implementations. We also hope that this framework, and in particular the relation with VARX modeling followed by model reduction will turn out to be useful in future developments of subspace identification, such as the quest for efficient procedures and the statistical analysis with finite-data.
---
paper_title: N4SID: Subspace Algorithms for the Identification of Combined Deterministic-Stochastic Systems

paper_content:

Abstract Recently a great deal of attention has been given to numerical algorithms for subspace state space system identification (N4SID). In this paper, we derive two new N4SID algorithms to identify mixed deterministic-stochastic systems. Both algorithms determine state sequences through the projection of input and output data. These state sequences are shown to be outputs of non-steady state Kalman filter banks. From these it is easy to determine the state space system matrices. The N4SID algorithms are always convergent (non-iterative) and numerically stable since they only make use of QR and Singular Value Decompositions. Both N4SID algorithms are similar, but the second one trades off accuracy for simplicity. These new algorithms are compared with existing subspace algorithms in theory and in practice.
---
paper_title: Closed loop subspace system identification

paper_content:

We present a general framework for closed loop subspace system identification. This framework consists of two new projection theorems which allow the extraction of non-steady state Kalman filter states and of system related matrices directly from input output data. Three algorithms for the identification of the state space matrices can be derived from these theorems. The similarities between the theorems and algorithms, and the corresponding open loop theorems and algorithms in the literature are remarked on.
---
paper_title: ASYMPTOTIC EQUIVALENCE OF CERTAIN CLOSED LOOP SUBSPACE IDENTIFICATION METHODS

paper_content:

Abstract Subspace identification for closed loop systems has been recently studied by several authors. Even though results are available which allows to compute the asymptotic variance of the estimated parameters for several algorithms, less clear is the situation as to relative performance is concerned. In this paper we partly answer this last question showing that the SSARX algorithm introduced by Jansson, which requires preliminary ARX modeling, and its “geometric version„ called PBSID in the literature, which does not require any preliminary estimation step, are asymptotically equivalent. The question as to which is to be preferred in practice when working with finite data size remains open.
---
paper_title: Some algorithmic aspects of subspace identification with inputs

paper_content:

It has been experimentally verified that most commonly used subspace methods for identification of linear state-space systems with exogenous inputs may, in certain experimental conditions, run into ill-conditioning and lead to ambiguous results. An analysis of the critical situations has lead us to propose a new algorithmic structure which could be used either to test difficult cases and/or to implement a suitable combination of new and old algorithms presented in the literature to help fixing the problem.
---
paper_title: COMPARISONS OF SUBSPACE IDENTIFICATION METHODS FOR SYSTEMS OPERATING ON CLOSED-LOOP

paper_content:

In this paper, we analyze two recently proposed closed-loop subspace identification methods, referred to as innovation estimation method and whitening filter approach respectively. The similarity and difference between them are investigated in detail. It turns out that all closed-loop subspace identification methods can be classified as one-step, two-step, or multi-stage projection methods. A SISO closed-loop simulation shows that to identify a consistent model the whitening filter approach might require longer future and past horizons than the innovation estimation method.
---
paper_title: Identification of the deterministic part of MIMO state space models given in innovations form from input-output data

paper_content:

The problem of Linear Multivariable State Space model identification from input-output data can under the presence of process- and measurement noise be solved in a non-iterative way when incorporating instrumental variables constructed from both input and output sequences in the recently developed class of multivariable output-error state space model class of subspace model identification schemes. Abstract--In this paper we describe two algorithms to identify a linear, time-invariant, finite dimensional state space model from input-output data. The system to be identified is assumed to be excited by a measurable input and an unknown process tloise and the measurements are disturbed by unknown measurement noise. Both noise sequences are discrete zero-mean white noise, The first algorithm gives consistent estimates only for the case where the input also is zero-mean white noise, while the same result is obtained with the second algorithm without this constraint. For the special case where the input signal is discrete zero-mean white noise, it is explicitly shown that this second algorithm is a special case of the recently developed Multivariable Output-Error State Space (MOESP) class of algorithms based on instrumental variables. The usefulness of the presented schemes is highlighted in a realistic simulation study.
---
paper_title: Subspace Identification from Closed Loop Data

paper_content:

The so-called subspace methods for direct identification of linear models in state space form have drawn considerable interest recently. They have been found to work well in many cases but have one drawback — they do not yield consistent estimates for data collected under output feedback. The present paper points to the reasons for this. We stress how the basic idea is to focus on the estimation of the state-variable candidates — the k-step ahead output predictors. By recomputing these from a ‘non-parametric’ (or, rather, high order ARX) one-step ahead predictor model, closed loop data can be handled.
---
paper_title: On the Relation Between CCA and Predictor-Based Subspace Identification

paper_content:

In this paper, we investigate the relation between a recently proposed subspace method based on predictor identification (PBSID), known also as ldquowhitening filter algorithm,rdquo and the classical CCA algorithm. The comparison is motivated by i) the fact that CCA is known to be asymptotically efficient for time series identification and optimal for white measured inputs and ii) some recent results showing that a number of recently developed algorithms are very closely related to PBSID. We show that PBSID is asymptotically equivalent to CCA precisely in the situations in which CCA is optimal while an ldquooptimizedrdquo version of PBSID behaves no worse than CCA also for nonwhite inputs. Even though PBSID (and its optimized version) are consistent regardless of the presence of feedback, in this paper we work under the assumption that there is no feedback to make the comparison with CCA meaningful. The results of this paper imply that the ldquooptimizedrdquo PBSID, besides being able to handle feedback, is to be preferred to CCA also when there is no feedback; only in very specific cases (white or no inputs) are the two algorithms (asymptotically) equivalent.
---
paper_title: Subspace Identification and ARX Modeling

paper_content:

Abstract In this paper we present a new identification method that points at the close relationship between high order ARX modeling and subspace identification. A high order ARX model is utilized to obtain initial estimates of certain Markov parameters. These parameters are then used to restructure the data model used for subspace identification to facilitate the estimation of the state sequence. Based on the estimated state sequence, the system parameters are estimated by linear regression. The method is shown to be competitive to existing subspace methods by a simulation example. The method can also be used, without modification, on closed loop data in contrast to most previously published subspace identification methods.
---
paper_title: Subspace Identification for Linear Systems: Theory ― Implementation ― Applications

paper_content:

Subspace Identification for Linear Systems focuses on the theory, implementation and applications of subspace identification algorithms for linear time-invariant finitedimensional dynamical systems. These algorithms allow for a fast, straightforward ::: and accurate determination of linear multivariable models from measured inputoutput data. The theory of subspace identification algorithms is presented in detail. Several chapters are devoted to deterministic, stochastic and combined deterministicstochastic ::: subspace identification algorithms. For each case, the geometric properties are stated in a main 'subspace' Theorem. Relations to existing algorithms and literature are explored, as are the interconnections between different subspace algorithms. The subspace identification theory is linked to the theory of frequency weighted model reduction, which leads to new interpretations and insights. The implementation of subspace identification algorithms is discussed in terms of the robust and computationally efficient RQ and singular value decompositions, which are well-established algorithms from numerical linear algebra. The algorithms are implemented in combination with a whole set of classical identification algorithms,processing and validation tools in Xmath's ISID, a commercially available graphical user interface toolbox. The basic subspace algorithms in the book are also implemented in a set of MATLABÂ® files accompanying the book. An application of ISID to an industrial glass tube manufacturing process is presented in detail, illustrating the power and user-friendliness of the subspace identification ::: algorithms and of their implementation in ISID. The identified model allows for an optimal control of the process, leading to a significant enhancement of the production quality. The applicability of subspace identification algorithms in industry is further illustrated with the application of the MATLABÂ® files to ten practical problems. Since all necessary data and MATLABÂ® files are included, the reader can easily step through these applications, and thus get more insight in the algorithms. ::: Subspace Identification for Linear Systems is an important reference for all researchers in system theory, control theory, signal processing, automization,mechatronics, chemical, electrical, mechanical and aeronautical engineering.
---
paper_title: Application of a subspace model identification technique to identify LTI systems operating in closed-loop

paper_content:

In this paper we reformulate the identification of linear time-invariant (LTI) systems operating in a closed-loop with an LTI compensator to an open-loop multi-input-multi-output (MIMO) (state space model) identification problem, followed by a model reduction step. The open-loop identification problem consists of the identification of a MIMO state space model, describing the deterministic transfer from freely excitable inputs to measurable (constrained) signals in the loop. This problem is solved by the recently proposed MOESP (MIMO output-error state spac]e model) identification technique, described in Verhaegen and Dewilde (Int. J. Control,56, 1187–1210, 1992). The described state space approach does not constrain the delay structure of the controller-plant combination; it delivers an estimate of the controller and it allows an analytical reduction of the obtained state space model of the system (or controller) to half its original system order. The open-loop identification problem only requires the selection of the combined model and controller order; the individual model and controller order selection is performed in the subsequent model reduction step. Furthermore, specializing the presented approach to the identification of a SISO system provided useful information regarding where to inject the signal in the loop in order to enhance numerical robustness. ::: ::: The reliability of the outlined approach and the usefulness of the obtained insights are highlighted in two illustrative numerical examples.
---
paper_title: A personal view of the development of system identification: A 30-year journey through an exciting field

paper_content:

In this article the author describes the development of system identification in the control community as he has observed it over the last 30 years, both as a student of the subject eager to learn and understand the work of his colleagues and as an active participant in these developments. This article attempts to exhibit both the continuity and the motivation for developments in system identification in the last 30 years and also the significant new departures and insights that came as the result of some important breakthroughs
---
paper_title: IDENTIFICATION FOR CONTROL

paper_content:

Abstract We present a conceptual characterization of the various ways of addressing the problem of identification for control. This leads us to distinguish between a dual control approach, an optimal experiment design approach and a robust control approach. The connections and distinctions between these three viewpoints are discussed, and recent results for each approach are briefly presented.
---
paper_title: Closed-loop subspace identification with innovation estimation

paper_content:

Abstract Most subspace identification algorithms are not applicable to closed-loop identification because they require future input to be uncorrelated with past innovation. In this paper, we propose a new subspaee identification method that remove this requirement by using a parsimonious model formulation with innovation estimation. A simulation example is included to show the effectiveness of the proposed method.
---
paper_title: Subspace model identification Part 2. Analysis of the elementary output-error state-space model identification algorithm

paper_content:

The elementary MOESP algorithm presented in the first part of this series of papers is analysed in this paper. This is done in three different ways. First, we study the asymptotic properties of the estimated state-space model when only considering zero-mean white noise perturbations on the output sequence. It is shown that, in this case, the MOESPl implementation yields asymptotically unbiased estimates. An important constraint to this result is that the underlying system must have a finite impulse response and subsequently the size of the Hankel matrices, constructed from the input and output data at the beginning of the computations, depends on the number of non-zero Markov parameters. This analysis, however, leads to a second implementation of the elementary MOESP scheme, namely MOESP2. The latter implementation has the same asymptotic properties without the finite impulse response constraint. Secondly, we compare the MOESP2 algorithm with a classical state space model identification scheme. The latter...
---
paper_title: A novel subspace identification approach with enforced causal models

paper_content:

Subspace identification methods (SIMs) for estimating state-space models have been proven to be very useful and numerically efficient. They exist in several variants, but all have one feature in common: as a first step, a collection of high-order ARX models are estimated from vectorized input-output data. In order not to obtain biased estimates, this step must include future outputs. However, all but one of the submodels include non-causal input terms. The coefficients of them will be correctly estimated to zero as more data become available. They still include extra model parameters which give unnecessarily high variance, and also cause bias for closed-loop data. In this paper, a new model formulation is suggested that circumvents the problem. Within the framework, the system matrices (A,B,C,D) and Markov parameters can be estimated separately. It is demonstrated through analysis that the new methods generally give smaller variance in the estimate of the observability matrix and it is supported by simulation studies that this gives lower variance also of the system invariants such as the poles.
---
paper_title: Canonical variate analysis in identification, filtering, and adaptive control

paper_content:

The canonical variate analysis (CVA) approach for system identification, filtering, and adaptive control is developed. The past/future Markov property provides a starting point for defining a reduced-order prediction problem. The solution is a canonical variate analysis that is characterized by a generalized singular value decomposition. State-space model estimation requires only simple regression, and state order selection involves the optimal Akaike information criterion procedure. The CVA method extends to time-varying and abruptly changing systems. A reduce-rank stochastic model predictive control problem is shown to be equivalent to the CVA problem. Also discussed are computational aspects, applications, and an example illustrating the method. New extensions to the identification of general nonlinear systems are briefly discussed. The CVA method provides an approach giving reliable, automatic implementation of identification, filtering, and control for online adaptive control. >
---
paper_title: Consistency analysis of some closed-loop subspace identification methods

paper_content:

We study statistical consistency of two recently proposed subspace identification algorithms for closed-loop systems. These algorithms may be seen as implementations of an abstract state-space construction procedure described by the authors in previous work on stochastic realization of closed-loop systems. A detailed error analysis is undertaken which shows that both algorithms are biased due to an unavoidable mishandling of initial conditions which occurs in closed-loop identification. Instability of the open loop system may also be a cause of trouble.
---
paper_title: On the Asymptotic Properties of Closed-Loop CCA-Type Subspace Algorithms: Equivalence Results and Role of the Future Horizon

paper_content:

In this paper, we shall consider a class of subspace algorithms for identification of linear time invariant systems operating in "closed loop." In particular we study algorithms based on the so-called "state-sequence" approach; we first show that the ADAPTx algorithm by Larimore is asymptotically equivalent to a number of recently developed algorithms, which we call CCA-type algorithms. Based on this equivalence result, we then study the effect of the "future horizon," which is one of the principal "user choices" in subspace identification. It is well known that for the CCA algorithm the asymptotic variance of any system invariant is a non increasing function of the future horizon when input signals are white (or absent). In particular we extend this result, valid for white noise input signals to a slightly more general class of input signals, which include proportional (output or state) feedback controllers and LQG controllers, provided the reference input is white. The condition on the input will be expressed in terms of its state space, which we regard as a rather natural condition in this framework. For the situations not covered by the above result, we shall also describe a computational procedure, based on some recently derived asymptotic variance formulas, which allows to optimize the choice of the future horizon. Some simulation results are included.
---
paper_title: Subspace model identification Part 1. The output-error state-space model identification class of algorithms

paper_content:

In this paper, we present two novel algorithms to realize a finite dimensional, linear time-invariant state-space model from input-output data. The algorithms have a number of common features. They are classified as one of the subspace model identification schemes, in that a major part of the identification problem consists of calculating specially structured subspaces of spaces defined by the input-output data. This structure is then exploited in the calculation of a realization. Another common feature is their algorithmic organization: an RQ factorization followed by a singular value decomposition and the solution of an overdetermined set (or sets) of equations. The schemes assume that the underlying system has an output-error structure and that a measurable input sequence is available. The latter characteristic indicates that both schemes are versions of the MIMO Output-Error State Space model identification (MOESP) approach. The first algorithm is denoted in particular as the (elementary MOESP scheme)...
---
paper_title: The role of Vector AutoRegressive Modeling in Predictor Based Subspace Identification

paper_content:

Subspace identification for closed loop systems has been recently studied by several authors. A class of new and consistent closed-loop subspace algorithms is based on identification of a predictor model, in a way similar as prediction error methods (PEM) do. Experimental evidence suggests that these methods have a behavior which is very close to PEM in certain examples. The asymptotical statistical properties of one of these methods have been studied recently allowing to show (i) its relation with CCA and (ii) that Cramer-Rao lower bound is not reached in general. Very little, however, is known concerning their relative performance. In this paper we shall discuss the link between these ''predictor-based'' methods; to this purpose we exploit the role which Vector Auto Regressive with eXogenous input models play in all these algorithms. The results of this paper provide a unifying framework under which all these algorithms can be viewed; also the link with VARX modeling have important implications as to computational complexity is concerned, leading to very computationally attractive implementations. We also hope that this framework, and in particular the relation with VARX modeling followed by model reduction will turn out to be useful in future developments of subspace identification, such as the quest for efficient procedures and the statistical analysis with finite-data.
---
paper_title: On the Relation Between CCA and Predictor-Based Subspace Identification

paper_content:

In this paper, we investigate the relation between a recently proposed subspace method based on predictor identification (PBSID), known also as ldquowhitening filter algorithm,rdquo and the classical CCA algorithm. The comparison is motivated by i) the fact that CCA is known to be asymptotically efficient for time series identification and optimal for white measured inputs and ii) some recent results showing that a number of recently developed algorithms are very closely related to PBSID. We show that PBSID is asymptotically equivalent to CCA precisely in the situations in which CCA is optimal while an ldquooptimizedrdquo version of PBSID behaves no worse than CCA also for nonwhite inputs. Even though PBSID (and its optimized version) are consistent regardless of the presence of feedback, in this paper we work under the assumption that there is no feedback to make the comparison with CCA meaningful. The results of this paper imply that the ldquooptimizedrdquo PBSID, besides being able to handle feedback, is to be preferred to CCA also when there is no feedback; only in very specific cases (white or no inputs) are the two algorithms (asymptotically) equivalent.
---
paper_title: Consistency analysis of some closed-loop subspace identification methods

paper_content:

We study statistical consistency of two recently proposed subspace identification algorithms for closed-loop systems. These algorithms may be seen as implementations of an abstract state-space construction procedure described by the authors in previous work on stochastic realization of closed-loop systems. A detailed error analysis is undertaken which shows that both algorithms are biased due to an unavoidable mishandling of initial conditions which occurs in closed-loop identification. Instability of the open loop system may also be a cause of trouble.
---
paper_title: On the Asymptotic Properties of Closed-Loop CCA-Type Subspace Algorithms: Equivalence Results and Role of the Future Horizon

paper_content:

In this paper, we shall consider a class of subspace algorithms for identification of linear time invariant systems operating in "closed loop." In particular we study algorithms based on the so-called "state-sequence" approach; we first show that the ADAPTx algorithm by Larimore is asymptotically equivalent to a number of recently developed algorithms, which we call CCA-type algorithms. Based on this equivalence result, we then study the effect of the "future horizon," which is one of the principal "user choices" in subspace identification. It is well known that for the CCA algorithm the asymptotic variance of any system invariant is a non increasing function of the future horizon when input signals are white (or absent). In particular we extend this result, valid for white noise input signals to a slightly more general class of input signals, which include proportional (output or state) feedback controllers and LQG controllers, provided the reference input is white. The condition on the input will be expressed in terms of its state space, which we regard as a rather natural condition in this framework. For the situations not covered by the above result, we shall also describe a computational procedure, based on some recently derived asymptotic variance formulas, which allows to optimize the choice of the future horizon. Some simulation results are included.
---
paper_title: System Identification: Theory for the User

paper_content:

Das Buch behandelt die Systemidentifizierung in dem theoretischen Bereich, der direkte Auswirkungen auf Verstaendnis und praktische Anwendung der verschiedenen Verfahren zur Identifizierung hat. Da ...
---
paper_title: On jointly stationary feedback-free stochastic processes

paper_content:

We consider stationary stochastic vector processes made up of two component processes y and u . Such processes arise, for example, in feedback processes. We consider the task of determining whether there is feedback from one process, say, y , to the other, say, u . A definition is proposed for the absence of feedback in terms of the spectrum \phi_{y u}(Z) of the joint process. Comparison with previous results on feedback-free processes shows that the proposed definition has some desirable properties which were absent in previous work. In particular, system structures other than canonical ones are shown to be feedback-free.
---
paper_title: Closed-loop subspace identification with innovation estimation

paper_content:

Abstract Most subspace identification algorithms are not applicable to closed-loop identification because they require future input to be uncorrelated with past innovation. In this paper, we propose a new subspaee identification method that remove this requirement by using a parsimonious model formulation with innovation estimation. A simulation example is included to show the effectiveness of the proposed method.
---
paper_title: PREDICTION ERROR VS SUBSPACE METHODS IN CLOSED LOOP IDENTIFICATION

paper_content:

Abstract In this paper we investigate the role of the output predictor in subspace identification. We shall see that subspace identification of the predictor model can ideally yield consistent estimates regardless of the presence of feedback. This solves a longstanding open question in system identification.
---
paper_title: The role of Vector AutoRegressive Modeling in Predictor Based Subspace Identification

paper_content:

Subspace identification for closed loop systems has been recently studied by several authors. A class of new and consistent closed-loop subspace algorithms is based on identification of a predictor model, in a way similar as prediction error methods (PEM) do. Experimental evidence suggests that these methods have a behavior which is very close to PEM in certain examples. The asymptotical statistical properties of one of these methods have been studied recently allowing to show (i) its relation with CCA and (ii) that Cramer-Rao lower bound is not reached in general. Very little, however, is known concerning their relative performance. In this paper we shall discuss the link between these ''predictor-based'' methods; to this purpose we exploit the role which Vector Auto Regressive with eXogenous input models play in all these algorithms. The results of this paper provide a unifying framework under which all these algorithms can be viewed; also the link with VARX modeling have important implications as to computational complexity is concerned, leading to very computationally attractive implementations. We also hope that this framework, and in particular the relation with VARX modeling followed by model reduction will turn out to be useful in future developments of subspace identification, such as the quest for efficient procedures and the statistical analysis with finite-data.
---
paper_title: ASYMPTOTIC EQUIVALENCE OF CERTAIN CLOSED LOOP SUBSPACE IDENTIFICATION METHODS

paper_content:

Abstract Subspace identification for closed loop systems has been recently studied by several authors. Even though results are available which allows to compute the asymptotic variance of the estimated parameters for several algorithms, less clear is the situation as to relative performance is concerned. In this paper we partly answer this last question showing that the SSARX algorithm introduced by Jansson, which requires preliminary ARX modeling, and its “geometric version„ called PBSID in the literature, which does not require any preliminary estimation step, are asymptotically equivalent. The question as to which is to be preferred in practice when working with finite data size remains open.
---
paper_title: COMPARISONS OF SUBSPACE IDENTIFICATION METHODS FOR SYSTEMS OPERATING ON CLOSED-LOOP

paper_content:

In this paper, we analyze two recently proposed closed-loop subspace identification methods, referred to as innovation estimation method and whitening filter approach respectively. The similarity and difference between them are investigated in detail. It turns out that all closed-loop subspace identification methods can be classified as one-step, two-step, or multi-stage projection methods. A SISO closed-loop simulation shows that to identify a consistent model the whitening filter approach might require longer future and past horizons than the innovation estimation method.
---
paper_title: Subspace Identification and ARX Modeling

paper_content:

Abstract In this paper we present a new identification method that points at the close relationship between high order ARX modeling and subspace identification. A high order ARX model is utilized to obtain initial estimates of certain Markov parameters. These parameters are then used to restructure the data model used for subspace identification to facilitate the estimation of the state sequence. Based on the estimated state sequence, the system parameters are estimated by linear regression. The method is shown to be competitive to existing subspace methods by a simulation example. The method can also be used, without modification, on closed loop data in contrast to most previously published subspace identification methods.
---
paper_title: Consistency analysis of some closed-loop subspace identification methods

paper_content:

We study statistical consistency of two recently proposed subspace identification algorithms for closed-loop systems. These algorithms may be seen as implementations of an abstract state-space construction procedure described by the authors in previous work on stochastic realization of closed-loop systems. A detailed error analysis is undertaken which shows that both algorithms are biased due to an unavoidable mishandling of initial conditions which occurs in closed-loop identification. Instability of the open loop system may also be a cause of trouble.
---
paper_title: The role of Vector AutoRegressive Modeling in Predictor Based Subspace Identification

paper_content:

Subspace identification for closed loop systems has been recently studied by several authors. A class of new and consistent closed-loop subspace algorithms is based on identification of a predictor model, in a way similar as prediction error methods (PEM) do. Experimental evidence suggests that these methods have a behavior which is very close to PEM in certain examples. The asymptotical statistical properties of one of these methods have been studied recently allowing to show (i) its relation with CCA and (ii) that Cramer-Rao lower bound is not reached in general. Very little, however, is known concerning their relative performance. In this paper we shall discuss the link between these ''predictor-based'' methods; to this purpose we exploit the role which Vector Auto Regressive with eXogenous input models play in all these algorithms. The results of this paper provide a unifying framework under which all these algorithms can be viewed; also the link with VARX modeling have important implications as to computational complexity is concerned, leading to very computationally attractive implementations. We also hope that this framework, and in particular the relation with VARX modeling followed by model reduction will turn out to be useful in future developments of subspace identification, such as the quest for efficient procedures and the statistical analysis with finite-data.
---
paper_title: Subspace Identification from Closed Loop Data

paper_content:

The so-called subspace methods for direct identification of linear models in state space form have drawn considerable interest recently. They have been found to work well in many cases but have one drawback — they do not yield consistent estimates for data collected under output feedback. The present paper points to the reasons for this. We stress how the basic idea is to focus on the estimation of the state-variable candidates — the k-step ahead output predictors. By recomputing these from a ‘non-parametric’ (or, rather, high order ARX) one-step ahead predictor model, closed loop data can be handled.
---
paper_title: The role of Vector AutoRegressive Modeling in Predictor Based Subspace Identification

paper_content:

Subspace identification for closed loop systems has been recently studied by several authors. A class of new and consistent closed-loop subspace algorithms is based on identification of a predictor model, in a way similar as prediction error methods (PEM) do. Experimental evidence suggests that these methods have a behavior which is very close to PEM in certain examples. The asymptotical statistical properties of one of these methods have been studied recently allowing to show (i) its relation with CCA and (ii) that Cramer-Rao lower bound is not reached in general. Very little, however, is known concerning their relative performance. In this paper we shall discuss the link between these ''predictor-based'' methods; to this purpose we exploit the role which Vector Auto Regressive with eXogenous input models play in all these algorithms. The results of this paper provide a unifying framework under which all these algorithms can be viewed; also the link with VARX modeling have important implications as to computational complexity is concerned, leading to very computationally attractive implementations. We also hope that this framework, and in particular the relation with VARX modeling followed by model reduction will turn out to be useful in future developments of subspace identification, such as the quest for efficient procedures and the statistical analysis with finite-data.
---
paper_title: Weighted low rank approximation and reduced rank linear regression

paper_content:

The weighted low-rank approximation (WLRA) problem is considered. The problem is that of approximating one matrix with another matrix of lower rank, such that the weighted norm of the difference is minimized. The problem is fundamental in a new method for reduced rank linear regression that is outlined, as well as in areas such as two-dimensional filter design and data mining. The WLRA problem has no known closed form solution in the general case, but iterative methods have previously been suggested. Non-iterative methods that are asymptotically optimal for the linear regression and related problems are developed. Computer simulations, where the new methods are compared to one step of the well-known alternating projections algorithm, show significantly improved performance.
---
paper_title: The role of Vector AutoRegressive Modeling in Predictor Based Subspace Identification

paper_content:

Subspace identification for closed loop systems has been recently studied by several authors. A class of new and consistent closed-loop subspace algorithms is based on identification of a predictor model, in a way similar as prediction error methods (PEM) do. Experimental evidence suggests that these methods have a behavior which is very close to PEM in certain examples. The asymptotical statistical properties of one of these methods have been studied recently allowing to show (i) its relation with CCA and (ii) that Cramer-Rao lower bound is not reached in general. Very little, however, is known concerning their relative performance. In this paper we shall discuss the link between these ''predictor-based'' methods; to this purpose we exploit the role which Vector Auto Regressive with eXogenous input models play in all these algorithms. The results of this paper provide a unifying framework under which all these algorithms can be viewed; also the link with VARX modeling have important implications as to computational complexity is concerned, leading to very computationally attractive implementations. We also hope that this framework, and in particular the relation with VARX modeling followed by model reduction will turn out to be useful in future developments of subspace identification, such as the quest for efficient procedures and the statistical analysis with finite-data.
---
paper_title: Towards Data-Driven Control for Modern Wind Turbines

paper_content:

Further developments in data-driven control techniques for the load reduction of modern wind turbines can achieve an increased lifetime of components and make the scaling to larger rotor diameters possible, and therefore improve the cost effectiveness of modern wind turbines. Also the success of future rotor designs will heavily depend for their operation on new developments in active control technologies. This thesis proposes a novel control algorithm for the rejection of periodic disturbances. The load disturbances seen by an individual rotor blade are to a large extent periodic and repetitive; for instance, tower shadow, wind shear, yawed error, and gravity are depending on the azimuth angle and rotational speed, and will change slowly over time. The contribution of these periodic disturbances will become even larger in the future, due to the increasing length and mass of the rotor blades and the possible downwind location of a two-bladed rotor. By compensating only the smooth periodic disturbances, the control action would not create to much fatigue damage on the actuators.
---
paper_title: Consistency analysis of some closed-loop subspace identification methods

paper_content:

We study statistical consistency of two recently proposed subspace identification algorithms for closed-loop systems. These algorithms may be seen as implementations of an abstract state-space construction procedure described by the authors in previous work on stochastic realization of closed-loop systems. A detailed error analysis is undertaken which shows that both algorithms are biased due to an unavoidable mishandling of initial conditions which occurs in closed-loop identification. Instability of the open loop system may also be a cause of trouble.
---
paper_title: On the Asymptotic Properties of Closed-Loop CCA-Type Subspace Algorithms: Equivalence Results and Role of the Future Horizon

paper_content:

In this paper, we shall consider a class of subspace algorithms for identification of linear time invariant systems operating in "closed loop." In particular we study algorithms based on the so-called "state-sequence" approach; we first show that the ADAPTx algorithm by Larimore is asymptotically equivalent to a number of recently developed algorithms, which we call CCA-type algorithms. Based on this equivalence result, we then study the effect of the "future horizon," which is one of the principal "user choices" in subspace identification. It is well known that for the CCA algorithm the asymptotic variance of any system invariant is a non increasing function of the future horizon when input signals are white (or absent). In particular we extend this result, valid for white noise input signals to a slightly more general class of input signals, which include proportional (output or state) feedback controllers and LQG controllers, provided the reference input is white. The condition on the input will be expressed in terms of its state space, which we regard as a rather natural condition in this framework. For the situations not covered by the above result, we shall also describe a computational procedure, based on some recently derived asymptotic variance formulas, which allows to optimize the choice of the future horizon. Some simulation results are included.
---

Title: Regression techniques for subspace-based black-box state-space system identification: an overview

Section 1: Motivations and problem formulation
Description 1: Describe the problem of state-space system identification, outline the basic LTI state-space form, and discuss the motivations behind subspace-based identification methods.

Section 2: Remark 1
Description 2: Clarify that all developments involve only discrete-time (DT) data and systems.

Section 3: Subspace-based identification involving linear regression
Description 3: Provide an overview of how linear regression methods are employed in subspace-based identification techniques, highlighting their main steps and characteristics.

Section 4: Construction of a state sequence from past input and past output data
Description 4: Explain the initial step of constructing a state sequence using past inputs and outputs, emphasizing different formulations and their relevance.

Section 5: State sequence construction from the Kalman filter
Description 5: Discuss the use of Kalman filters for state sequence construction, presenting its mathematical foundation and practical application.

Section 6: State sequence construction from infinite time series
Description 6: Introduce the method of constructing state sequences from infinite input and output time series and the mathematical basis for this approach.

Section 7: Approximated state sequence construction from finite time series
Description 7: Describe how to approximate state sequences using finite data and assess the quality of these estimates, including discussions on deterministic systems and user-defined observers.

Section 8: To sum up
Description 8: Summarize the methods discussed for constructing state sequences and transition to the next phase of linear regression-based subspace identification.

Section 9: Subspace-based identification using linear regression for systems operating in open-loop
Description 9: Present techniques for identifying systems in open-loop conditions using linear regression-based subspace methods.

Section 10: Unconstrained least-squares solutions
Description 10: Discuss the application of least-squares regression techniques in the subspace-based identification framework, including ordinary least-squares solutions.

Section 11: Constrained least-squares solutions
Description 11: Address issues with unconstrained solutions and introduce enhancements using Toeplitz structure and rank constraints in the least-squares optimization.

Section 12: Extraction of the state-space matrices
Description 12: Detail various methods for extracting state-space matrices from the estimated subspaces, including state subspace and observability subspace approaches.

Section 13: Subspace-based identification using linear regression for systems operating in closed-loop
Description 13: Explore techniques for identifying systems under closed-loop conditions, detailing the problems with traditional methods and recent advancements.

Section 14: The innovation estimation method
Description 14: Describe the innovation estimation method, its iterative pre-estimation of the innovation sequence, and subsequent steps.

Section 15: Closed-loop subspace-based identification with Markov parameter pre-estimation
Description 15: Introduce methods that use Markov parameter pre-estimation for identifying closed-loop systems, discussing the predictor form and state-space matrices extraction.

Section 16: The prediction-based subspace identification algorithm
Description 16: Present the prediction-based subspace identification (PBSID) algorithm, explaining how it estimates the state sequence and extended observability subspace.

Section 17: Conclusions
Description 17: Summarize the findings, highlighting the common points and differences among regression-based subspace identification methods and their applications to open-loop and closed-loop systems.