I'm a math postgrad student working through a paper on eigenvalue decompositions of matrices of FIR filters (used for stuff like total decorrelation, convolutive mixing, MIMO). Towards the beginning, when describing their signal model, they use the phrase:

If the PSD of the $l^{th}$ source is generated by a stable and causal innovation filter $\text{F}_{l}(z)$, ...

What does this mean? I understand a stable filter is one whose coefficients $a[n]$ approach 0 as $n \rightarrow \infty$, and a causal filter is one which does not use values from the future (i.e. no negative time lags). I have read that the innovation in terms of stochastic processes is defined as the difference between the actual next value of a time series, and the value given by an optimal prediction based on the information already available.

I cannot, however, find any description of an innovation filter. I have a mathematics background rather than a DSP/engineering background, and so sometimes some of the terminology trips me up, which is why I was hoping DSP stack exhange could kindly help!

  • 2
    $\begingroup$ The term innovations filter is usually encountered in Kalman Filtering, see Kailath, Thomas. "A view of three decades of linear filtering theory." IEEE Transactions on Information Theory 20.2 (1974): 146-181. $\endgroup$ – Stanley Pawlukiewicz Sep 21 '18 at 1:33
  • $\begingroup$ Hi: Stanley's reference is useful but more general than this one which focuses specifically on the innovations filter. metronu.ulb.ac.be/npauly/art_2014_2015/kailath_1968.pdf $\endgroup$ – mark leeds Sep 21 '18 at 5:23
  • $\begingroup$ Thank you both, I will have a look at these references shortly. $\endgroup$ – A. White Sep 22 '18 at 2:18

An innovations filter of a WSS process $X(t)$ is a causal and stable minimum phase filter that can be used to generate $X(t)$ from a white noise input $N(t)$:


where $h(t)$ is the impulse response of the innovations filter.

The (causal and stable) inverse filter of $h(t)$ is called the whitening filter of $X(t)$. Its response to the input $X(t)$ is white noise $N(t)$.

A process that can be represented by $(1)$ is called a regular process. Note that its power spectrum is given by


where $H(\omega)$ is the Fourier transform of the impulse response $h(t)$ of the innovations filter (assuming that $E\{N(t)N(t+\tau\}=c\delta(\tau)$ with $c=1$). The requirement of causality of the innovations filter implies that the power spectrum $S(\omega)$ of $X(t)$ satisfies the Paley-Wiener condition

$$\int_{-\infty}^{\infty}\frac{|\log S(\omega)|}{1+\omega^2}d\omega\lt\infty\tag{3}$$

This implies that $S(\omega)$ cannot contain spectral lines, and it cannot be band-limited. Unlike a singular process (consisting of spectral lines), a regular process cannot be parameterized by a finite set of random variables and it is not predictable, i.e., it is not completely determined in terms of its past.

Reference: Probability, Random Variables, and Stochastic Processes, A.Papoulis, Athanasios 1965. McGraw-Hill.

  • $\begingroup$ Great, this exactly what I needed to know, thank you very much. What implications for a signal does it have that its PSD is generated by such a filter? $\endgroup$ – A. White Sep 22 '18 at 2:17
  • $\begingroup$ @A.White: I've added some more information to my answer. $\endgroup$ – Matt L. Sep 22 '18 at 18:39
  • $\begingroup$ sorry to bring this up again after several months, but do you have any references for this information? Ideally I'd like to use what you've said here, and also to read further into it. $\endgroup$ – A. White Feb 17 at 4:30
  • $\begingroup$ @A.White: I've added a reference to my answer. $\endgroup$ – Matt L. Feb 17 at 7:17
  • $\begingroup$ Hi: just to add to this in case the OP does a search, an innovations filter is also referred to ( later in statistical world ) as the single source of error state space model. It's a kind of equivalent re-parameterization of the standard KF where there is only one source of noise. I'm quite confident about what Matt said but the Kalaiith articles are the classic references. The model allows for a larger parameter space than the standard KF and there are other positives also. $\endgroup$ – mark leeds Feb 19 at 0:28

Your Answer

