---
abstract: 'We consider the linear quadratic Gaussian control problem with a discounted cost functional for descriptor systems on the infinite time horizon. Based on recent results from the deterministic framework, we characterize the feasibility of this problem using a linear matrix inequality. In particular, conditions for existence and uniqueness of optimal controls are derived, which are weaker compared to the standard approaches in the literature. We further show that also for the stochastic problem, the optimal control is given in terms of the stabilizing solution of the Lur’e equation, which generalizes the algebraic Riccati equation.'
author:
- 'Hermann Mena, Lena-Maria Pfurtscheller, Matthias Voigt'
- '[^1]'
bibliography:
- 'biblio.bib'
title: Discounted Cost Linear Quadratic Gaussian Control for Descriptor Systems
---

differential-algebraic systems; descriptor systems; linear quadratic Gaussian control; Kalman-Yakubovich-Popov inequality; dissipation inequality

Introduction
============

We consider the stochastic system with additive noise $${\mathrm{d}}E x(t) = (Ax(t) + Bu(t)){\mathrm{d}}t + N(t){\mathrm{d}}w(t), \quad     Ex(0) = Ex_0, \label{eq:lqg}\\$$ where the matrices $E,\,A \in {\mathbb{R}}^{n\times n}$ are such that $\det(s 
E-A) \in {\mathbb{R}}[s]$ is not the zero polynomial. A matrix pencil $sE-A \in {\mathbb{R}}[s]^{n \times n}$ fulfilling this condition is called *regular*. Let $B \in {\mathbb{R}}^{n\times m}$, $N :{\mathbb{R}}\to {\mathbb{R}}^{n 
\times n_w}$ and $w$ be an $n_w$-dimensional Wiener process. The stochastic process $u:{\mathbb{R}}\times \Omega 
\to {\mathbb{R}}^m$ is called *input* and $x:{\mathbb{R}}\times \Omega \to {\mathbb{R}}^n$ *generalized state*. We assume that the initial value $Ex_0$ is a random variable that is independent of $w$. We denote the set of such systems  by $\Sigma_{n,m,n_w}$ and we write $[E,A,B,N] \in \Sigma_{n,m,n_w}$. Throughout this paper, the following notation is used: $(\Omega, \mathcal{A}, (\mathcal{F}_t)_{t\geq 0}, 
\mathbb{P})$ is a complete probability space with sample space $\Omega$, $\sigma$-algebra $\mathcal{A}$, filtration $(\mathcal{F}_t)_{t\geq 0}$ and probability measure $\mathbb{P}$. We denote by ${\mathbb{E}}[\cdot]$ the expectation operator with respect to the probability measure $\mathbb{P}$.

For an interval ${\mathcal{I}}:= [t_1,t_2]$ with $t_1,\,t_2 \in {\mathbb{R}}\cup \{-\infty,\infty\}$ and $t_1 \le t_2$, we consider stochastic processes $\varphi: {\mathcal{I}}\times \Omega 
\to \mathbb{R}^k$. Here we often use the short-hand $\varphi(t) := \varphi(t,\cdot)$. In particular, $\varphi(\cdot,\omega)$ is interpreted as a particular realization of the process while $\varphi(\cdot)$ contains information about its entire distribution. Let $$\begin{gathered}
L^{2,\rm loc}_{\mathcal{F}}({\mathcal{I}}, {\mathbb{R}}^k ) :=  \left\{ 
\vphantom{\int_1^2} \varphi : {\mathcal{I}}\times \Omega \to \mathbb{R}^k : 
\varphi := (\varphi(t))_{t \in {\mathcal{I}}} := (\varphi(t,\cdot))_{t \in {\mathcal{I}}} \; 
\text{is} \right. (\mathcal{F}_t)_{t \in {\mathcal{I}}}  
 \\ \left. \text{non-anticipating and }  
\int_\mathcal{K} {{\mathbb{E}}\left[\Vert \varphi(t) \Vert^2\right]}{\mathrm{d}t} < \infty \text{ for each compact subset } \mathcal{K} \subseteq {\mathcal{I}}\right\},\end{gathered}$$ where $\| \cdot \|$ denotes the Euclidean norm. For a subspace $\mathcal{V} \subseteq {\mathbb{R}}^k$, we also often make use of the notation $$\begin{gathered}
L^{2,\rm loc}_{\mathcal{F}}({\mathcal{I}}, \mathcal{V} ) := \left\{ \varphi \in L^{2,\rm loc}_{\mathcal{F}}({\mathcal{I}}, {\mathbb{R}}^k ) \; : \; \varphi(t,\omega) \in \mathcal{V} \right. \\  \text{for all } t \in {\mathcal{I}}\text{ and almost all } \omega \in \Omega \Big\}.\end{gathered}$$

Equations of the form  are called stochastic differential algebraic equations (SDAEs) or stochastic descriptor systems and the processes $(x,u) \in L^{2,\rm loc}_{\mathcal{F}}({\mathcal{I}}, {\mathbb{R}}^n ) \times 
L^{2,\rm loc}_{\mathcal{F}}({\mathcal{I}}, {\mathbb{R}}^m )$ are called *solutions* of $[E,A,B,N]$ on $[0,{t_{\rm f}}]$, if they solve  on the time interval $[0,{t_{\rm f}}]$. The latter means that $$\label{eq:solution}
 Ex(t,\omega) = Ex_0(\omega) + \int_{0}^t Ax(s,\omega) \mathrm{d}s + \int_{0}^t 
Bu(s,\omega) \mathrm{d}s + \int_{0}^t N \mathrm{d}w(s)$$ is satisfied for all $t \in [0,{t_{\rm f}}]$ and for almost all $\omega \in \Omega$. In , the first two integrals have to be understood in the Lebesgue sense, while the third integral is an Itô integral [@L74].

Note that we need to restrict the set of initial conditions to those that are independent of the noise $w$, so they live in $$L^2_{\mathcal{F}_0}({\mathbb{R}}^k) := \left\{ x_0 : \Omega \to {\mathbb{R}}^k \; : \; x_0 \text{ is } \mathcal{F}_0 \text{ non-anticipating and } {\mathbb{E}}\big[\|x_0\|^2\big] < \infty \right\},$$ and for a subspace $\mathcal{V} \subseteq {\mathbb{R}}^k$ we define $$L^2_{\mathcal{F}_0}(\mathcal{V}) := \left\{ x_0 \in L^2_{\mathcal{F}_0}({\mathbb{R}}^k) \, : \, x_0(\omega) \in \mathcal{V} \text{ for almost all } \omega \in \Omega \right\},$$ see for instance @L74 [Section 4.3].

We further consider the *discounted cost functional* $$\begin{aligned}
    J(x,u,[0,{t_{\rm f}}]) = {\mathbb{E}}\left[ {\int_{0}^{{t_{\rm f}}}{ {\mathrm{e}}^{2\beta t}\begin{pmatrix}
        x(t) \\ u(t) \end{pmatrix}{^\mathsf{T}}\begin{bmatrix} Q & S \\ S{^\mathsf{T}}& R \end{bmatrix} 
        \begin{pmatrix}
        x(t) \\ u(t) \end{pmatrix}}\,\mathrm{d}{t}} \right] \label{eq:ju}
    \end{aligned}$$ with $\beta < 0$, which is discussed e.g., in the works by @BWSV16 [@BS17] for the standard case $E = I_n$ . The goal is to minimize  with respect to the state equation , i.e., we want to minimize  over all solutions $(x,u)$ of $[E,A,B,N] \in 
\Sigma_{n,m,n_w}$ on $[0,{t_{\rm f}}]$. Such problems form a special class of *linear quadratic Gaussian (LQG) control problems*.

The deterministic counterpart of minimizing  with $\beta = 0$ with respect to the state equation $$\begin{aligned}
    \frac{{\mathrm{d}}}{{\mathrm{d}}t}Ex(t) = Ax(t) + Bu(t), \quad Ex(0) = Ex_0 \in {\mathbb{R}}^n \label{eq:det}
    \end{aligned}$$ is called the *linear quadratic regulator (LQR) problem* and has been widely studied in the literature. The case $E = I_n$ has been analyzed in [@Wil71] in which the connection of the LQR problem to a certain linear matrix inequality (the Kalman-Yakubovich-Popov inequality) and quadratic matrix equation (the algebraic Riccati equation) has been exposed. A difficulty in this context are so-called *singular LQR problems* for which optimal control functions do not exist or are not unique for some initial conditions of the ODE. In this case, the algebraic Riccati equation does not exist and one must resort to a linear matrix equation, called the Lur’e equation, see [@Rei11] and the references therein. Several attempts to generalize these concepts to differential algebraic equations have been undertaken, see e.g., [@Meh89; @Meh91; @Gee93; @Gee94; @KawTK99; @KawK02; @Mas06] for linear-time invariant LQR problems and [@KurM04; @KunM08; @KunM11] for time-varying and nonlinear optimal control problems. However, most of the analyses in the formerly mentioned works on the linear time-invariant LQR problem suffer from the fact that they are only valid if the system under consideration is impulse controllable or if the cost functional is nonnegative. A solution for this problem is presented in [@RRV15; @RV18] in which these conditions are not required anymore. This analysis is based on a generalization of the Lur’e equation to differential algebraic equations instead of an extension of the algebraic Riccati equation. This approach turns out be more beneficial even for regular control problems.

As any real-world system is influenced by uncertainties such as environmental influence, stochastic systems are often more suitable than purely deterministic ones. If we consider the case that $E$ is invertible then the system can be easily transformed to the standard stochastic linear quadratic regulator problem [@YZ99; @RMZ02; @SLY16; @D04]. For the ODE case with additive noise, in @DGP99 [@DPD13] have shown that the problem is solvable if the matrix $R$ in the cost functional  is positive definite and the optimal input is given in feedback form which depends on the solution of an algebraic Riccati equation. However, for deterministic descriptor systems, the problem may also be solvable even if $R$ is not positive definite, see @RV18 for some illustrative examples. One of the first works about descriptor systems with additive noise has been given by @D89lqg, where the system is first transferred into Weierstraß canonical form and an algorithm for solving the discrete LQG problem with noisy output is given. But, by first transferring the system into standard form and then solving the problem, one has to assume again that $R>0$ to ensure the solvability of the algebraic Riccati equation. @ZX14 considers a descriptor system with multiplicative noise and shows that the solution of this problem is given in terms of an algebraic Riccati equation, however, still with the positive definiteness restriction in the cost functional. @FCH13 considers the LQG problem in discrete-time with the assumption that $S=0$ and further $$\begin{bmatrix} Q & 0 \\ 0 & R \end{bmatrix} = \begin{bmatrix} G{^\mathsf{T}}\\ D{^\mathsf{T}}\end{bmatrix} 
\begin{bmatrix}  G & D\end{bmatrix}$$ with matrices $G$ and $D$ of appropriate dimensions. In particular, this means that $D{^\mathsf{T}}G = G{^\mathsf{T}}D = 0$. This restrictive assumption has been relaxed by @WL18, who further considers systems with positive semi-definite weight matrices $Q = DD{^\mathsf{T}}$, $R = GG{^\mathsf{T}}$, if $$\begin{aligned}
    \operatorname{rank}\begin{bmatrix}
    0&E&0\\E&A&B
    \end{bmatrix} &= \operatorname{rank}\begin{bmatrix} E &A & N& B \end{bmatrix} + \operatorname{rank}\, E ,\\
    \operatorname{rank}\begin{bmatrix}
    0&E&0\\E&A&B \\ 0 &G&0\\ 0&0&D  \end{bmatrix} &= n+m + \operatorname{rank}\, E.    
    \end{aligned}$$ However, the weight matrix $R$ in the deterministic case can be even negative definite [@RV18] while still ensuring existence of a feasible solution of the LQR problem.

Moreover, a common assumption for stochastic descriptor systems is that the uncontrolled system is impulse free, i.e., the DAE has at most index one; or the corresponding control system is impulse controllable. This means that there exists a feedback control $u(t) = Kx(t)$ such that the closed-loop system has index at most one. This assumption has been imposed, e.g., by @GS13, who has shown that the system is impulse free and mean-square exponentially stable if a certain linear matrix inequality is fulfilled. Similarly, the works @ZZS15 [@ZX14; @HM10] deal solely with the impulse free problem. Only @XZ16 has derived conditions for stability and exact observability of discrete systems, which do not need to be impulse free, using generalized Lyapunov equations. Also @RV18 have not used the assumption of impulse controllability.

In this work, we will derive conditions for feasibility, meaning that the control problem has a minimal value of the cost functional that will remain finite, and regularity, i.e., the control problem has a uniquely determined optimal solution trajectory for each consistent initial condition. In this work we will not impose any artificial conditions on the cost functional nor the index of the system just as in [@RV18]. We will use the dissipation inequality from @MP09 and extend it to stochastic descriptor systems, which in contrast to the works @ZZLMR16 [@RH16; @XDQ18] holds also for the additive noise case. With the help of this inequality we give conditions for feasibility of the problem and show its equivalence to the solvability of the Kalman-Yakubovich-Popov inequality. Moreover, we derive the optimal value of the cost functional as a function depending on the maximum solution of this inequality and show that also for the stochastic case, the proper extension of the algebraic Riccati equation is the Lur’e equation.

Formulation of the problem {#sec:formulation_nonsingular}
==========================

The aim of this section is to give a proper formulation of the problem and some important definitions used throughout this paper. Moreover, we present the results for the particular case of $E$ being nonsingular.

Assume that $N(t) \equiv N$, i.e., a constant matrix. We define the *space of consistent initial differential variables* of $[E,A,B,N]\in \Sigma_{n,m,n_w}$ by $$\begin{gathered}
    \mathcal{V}^{\rm diff}_{[E,A,B,N]} := \left\{ x_0(\omega) \in {\mathbb{R}}^{n}: \, \exists \, \text{a solution } \, (x,u) \; 
\text{of} \, [E,A,B,N] \right. \\ \left.
          \text{with} \, Ex(0) = Ex_0 \text{ and } (x(\cdot,\omega),u(\cdot,\omega)) \text{ satisfies \eqref{eq:solution}  on ${\mathbb{R}}_{\ge 0}$ with } \omega \in \Omega \right\} .
    \end{gathered}$$ Note that the space $\mathcal{V}^{\rm diff}_{[E,A,B,N]}$ is not well-defined in general. Conditions for its well-posedness will be devised in Section \[sec:stochastic\].

We further introduce a *value function* $$W_+: {L^2_{\mathcal{F}_0}\big(E{{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}\to {\mathbb{R}}\cup\{-\infty,\infty \},$$ which expresses the *optimal cost*, defined by $$\begin{gathered}
    W_+(Ex_0) = \inf\left\{ J(x,u,{\mathbb{R}}_{\ge 0}) \; : \; (x,u) \text{ is a 
solution of }  [E,A,B,N]  \text{ on } {\mathbb{R}}_{\geq 0} \right. \\ \left.  
\text{with } Ex(0) = Ex_0 \text{ and } {\mathbb{E}}[\Vert Ex(\infty)\Vert^2] = 0 \right\} , 
\label{eq:opcost}
    \end{gathered}$$ where ${\mathbb{R}}_{\geq 0}$ denotes the set of nonnegative real numbers.

Assume first that $E$ is nonsingular. Then, all initial conditions are consistent and the space $\mathcal{V}^{\rm diff}_{[E,A,B,N]}$ is well-defined, i.e., $\mathcal{V}^{\rm diff}_{[E,A,B,N]} = {\mathbb{R}}^n$ and one can left-multiply equation  by $E^{-1}$. If the system is stabilizable and $Q = Q{^\mathsf{T}}\geq 0$, $S=0$, and $R = R{^\mathsf{T}}> 0$, then the optimal cost for ${t_{\rm f}}= \infty$ is given by [@BS17] $$\begin{aligned}
    W_+(Ex_0) = \operatorname{tr}\big(P {\mathbb{E}}\big[Ex_0(Ex_0){^\mathsf{T}}\big]\big) - \frac{1}{2\beta} \operatorname{tr}\big( PNN{^\mathsf{T}}\big), 
\label{eq:cost_invE}
    \end{aligned}$$ where the matrix $P \in {\mathbb{R}}^{n\times n}$ is the stabilizing solution of the algebraic Riccati equation $$\begin{aligned}
    0 = E{^\mathsf{T}}P A_\beta + 
A_\beta{^\mathsf{T}}P E + Q - \big(E{^\mathsf{T}}P B ){R^{-1}}\big(E{^\mathsf{T}}P B \big){^\mathsf{T}}\label{eq:ARE}
    \end{aligned}$$ with $A_\beta := A + \beta E$.

Now for our problem with possibly singular $E$ we define $${\widetilde}{x}(t) := {\mathrm{e}}^{\beta t} x(t) , \quad {\widetilde}{u}(t) := {\mathrm{e}}^{\beta t} 
u(t),$$ and then we have $$\begin{aligned}
    {\mathrm{d}}E {\widetilde}{x}(t) &=  \beta {\mathrm{e}}^{\beta t} Ex(t) {\mathrm{d}}t +  {\mathrm{e}}^{\beta t} 
{\mathrm{d}}Ex(t) \\&=  \beta {\mathrm{e}}^{\beta t} Ex(t) {\mathrm{d}}t + {\mathrm{e}}^{\beta t} (A x(t) + 
Bu(t)){\mathrm{d}}t + N{\mathrm{e}}^{\beta t}{\mathrm{d}}w(t)  
    \\&= \left(A_\beta {\widetilde}{x}(t) + B {\widetilde}{u}(t)\right) {\mathrm{d}}t + N\mathrm{e}^{\beta t} {\mathrm{d}}w(t) . 
    \end{aligned}$$ This yields the modified problem of minimizing $$\begin{aligned}
    J({\widetilde}{x},{\widetilde}{u},{\mathbb{R}}_{\ge 0}) = {\mathbb{E}}\left[ {\int_{0}^{\infty}{\begin{pmatrix}
        {\widetilde}{x}(t) \\ {\widetilde}{u}(t) \end{pmatrix}{^\mathsf{T}}\begin{bmatrix} Q 
& S \\ S{^\mathsf{T}}& R \end{bmatrix} 
        \begin{pmatrix}
        {\widetilde}{x}(t) \\ {\widetilde}{u}(t) \end{pmatrix}}\,\mathrm{d}{t}} \right] 
\label{eq:ju1}
    \end{aligned}$$ subject to $$\begin{aligned}
    {\mathrm{d}}E{\widetilde}{x}(t) = (A_\beta {\widetilde}{x}(t) + B {\widetilde}{u}(t)){\mathrm{d}}t + N \mathrm{e}^{\beta t} {\mathrm{d}}w(t) . \label{eq:lqgs}
    \end{aligned}$$ This setting is closer to the deterministic setting, as we get rid of the exponential term in the cost functional. Thus, in the following we will study the problem – and write $x := {\widetilde}{x}, u := 
{\widetilde}{u}, N_{\beta}(t) := N\mathrm{e}^{\beta t}$, and $A := A_\beta$. Note, that the diffusion term in the equation is now time-dependent.

It should be emphasized that the algebraic Riccati equation  is independent of $N$ and coincides with the Riccati equation arising in the deterministic case, see e.g., @KS72. Only the optimal value  depends on $N$ and coincides with the deterministic value if $N = 0$ and the initial condition is deterministic. This motivates to consider the deterministic setting in @V15 [@RV18] and view the stochastic case as an extension of the deterministic problem. @V15 [@RV18] have shown that even if the matrices in the cost functional are not necessarily positive (semi-)definite, the optimal value is still a quadratic function depending on the initial condition, i.e., for a consistent initial differentiable variable $Ex_0 \in {\mathbb{R}}^n$, the optimal value is given by $$x_0{^\mathsf{T}}E{^\mathsf{T}}P E x_0,$$ but $P=P{^\mathsf{T}}\in{\mathbb{R}}^{n\times n}$ solves a Lur’e equation instead of an algebraic Riccati equation. In @V15 this result was deduced for impulse controllable systems and @RV18 extends it to systems which can also be non-impulse controllable.

Stochastic descriptor systems {#sec:stochastic}
=============================

In this section we give an overview on important definitions and basic concepts for both, stochastic calculus and the theory of descriptor systems.

Let us consider the uncontrolled stochastic differential equation $${\mathrm{d}}x(t) = Ax(t) {\mathrm{d}}t + N\mathrm{e}^{\beta t}{\mathrm{d}}w(t) , \quad x(0) = x_0. \label{eq:sde}$$ Then, the unique solution of this linear equation is given by $$x(t) = {\mathrm{e}}^{At} x_0 + {\int_{0}^{t}{{\mathrm{e}}^{A(t-s)} N\mathrm{e}^{\beta s} }\,\mathrm{d}{w(s)}},$$ where the stochastic integral is meant to be in the sense of Itô [@L74]. However, if we consider stochastic descriptor systems, the problem becomes more delicate, as their solutions are not always well-defined, as one can see in the following example.

\[ex:noise\] Consider $${\mathrm{d}}\begin{bmatrix} 0&1\\0&0 \end{bmatrix} \begin{pmatrix} x_1(t) \\ x_2 (t)    
\end{pmatrix} = \begin{bmatrix} 1&0\\0&1 \end{bmatrix} \begin{pmatrix} x_1(t) \\x_2(t)
\end{pmatrix} {\mathrm{d}}t + \begin{bmatrix}
        n_1\\ n_2
        \end{bmatrix} \mathrm{e}^{\beta t}  {\mathrm{d}}w(t) .$$ The equations then read as $$\begin{aligned}
        {\mathrm{d}}x_2(t) &= x_1(t) {\mathrm{d}}t + n_1\mathrm{e}^{\beta t}  {\mathrm{d}}w(t) ,\\
        0 &=x_2(t) + n_2 \mathrm{e}^{\beta t}  v(t) ,  
        \end{aligned}$$ where $v$ is the Gaussian white noise process defined by the relation [@L74] $$w(t) = {\int_{0}^{t}{v(s)}\,\mathrm{d}{s}}.$$ Inserting the second equation into the first leads to a derivation of the white noise $v$, which is nowhere differentiable and hence, the DAE is not well-posed. Moreover, the white noise process has to be understood in a distributional sense [@L74 Chap. 3], so $v(t)$ is only used as a symbol. Since distributions are excluded in our solution concept, we want to avoid the explicit occurrence of the process $v$ in our equations. In our example, this means that $n_2=0$ must hold. But then the first equation implies $$0 = x_1(t) \mathrm{d}t + n_1 \mathrm{e}^{\beta t}  {\mathrm{d}}w(t).$$ Thus, if $n_1 \neq 0$, then $x_1$ would be a white noise process as well. So, for well-posedness, we require $\left[\begin{smallmatrix} n_1 \\ n_2
\end{smallmatrix}\right] = 0$.

If no white noise (or its derivatives) appears in the solutions of the SDAE, then we call the SDAE well-posed. More precisely, an SDAE ${\mathrm{d}}x(t) = Ax(t){\mathrm{d}}t+ N\mathrm{e}^{\beta t}{\mathrm{d}}w(t)$ is well-posed, if and only if there exists an initial condition $Ex(0) = Ex_0$ such that the SDAE has a solution on ${\mathbb{R}}_{\ge 0}$. The following lemma - which states an equivalent condition for well-posedness - also makes clear, that if an SDAE is well-posed, then it is solvable for all consistent initial conditions.

\[lemma:noise\] Let $sE-A \in \mathbb{R}[s]^{n \times n}$ be regular and consider the SDAE $$\begin{aligned}
        \begin{aligned} 
        {\mathrm{d}}E x(t) &= Ax(t) {\mathrm{d}}t +  N \mathrm{e}^{\beta t} {\mathrm{d}}w(t). \\
                \end{aligned} 
        \end{aligned}$$ Then, the problem is well-posed, if and only if $$\label{eq:wpcond}
         \Pi_{\rm r} N = N,$$ where $\Pi_{\rm r} =  T^{-1}\left[\begin{smallmatrix} I_{n_1} &0 \\ 0&0 
\end{smallmatrix}\right] T$ is the spectral projector onto the right deflating subspace of the matrix pencil $s E-A$ and $T \in {\mathbb{R}}^{n\times n}$ is the right (nonsingular) transformation matrix for transforming $sE-A$ into quasi-Weierstra[ß]{} form [@BerIT12].

The proof is trivial by transforming the system such that the pencil $sE-A$ is in quasi-Weierstra[ß]{} form. This leads to the decoupled system $$\begin{aligned}
\label{eq:QWF}
     \begin{split}
      {\mathrm{d}}x_1(t) &= A_{11} x_1(t) {\mathrm{d}}t +N_1 \mathrm{e}^{\beta t} {\mathrm{d}}w(t), \\
      {\mathrm{d}}E_{22} x_2(t) &= x_2(t) {\mathrm{d}}t + N_2 \mathrm{e}^{\beta t} {\mathrm{d}}w(t),       \end{split}
     \end{aligned}$$ with $A_{11} \in {\mathbb{R}}^{n_1 \times n_1}$ and nilpotent $E_{22} \in {\mathbb{R}}^{n_2 
\times n_2}$, where $T^{-1} x(t) =: \left[\begin{smallmatrix} x_1(t) \\ x_2(t) 
\end{smallmatrix}\right]$. The first equation in is a standard SDE and is uniquely solvable for each choice of $N_1$ and each initial value. The second equation however has the (formal) solution $$x_2(t) = - \sum_{j=0}^{n_2-1} E_{22}^j N_2 (\mathrm{e}^{\beta t} v(t))^{(j)},$$ where $v$ is a Gaussian white noise process. Thus, the SDAE is well-posed, if and only if $N_2 = 0$ (and consequently $x_2(0) = 0$ is the only consistent initial condition) which is equivalent to condition .

As discussed in the previous subsection, one has to ensure that the white noise does not occur in the solution of the SDAE. We will now characterize well-posedness of the spaces of consistent initial differential variables and the system space of the stochastic descriptor *control* system $[E,A,B,N_{\beta}] \in \Sigma_{n,m,n_w}$ (where again $N_{\beta}(t):=N\mathrm{e}^{\beta t}$). Moreover, we analyze stabilizability of such a system.

First we consider the space of consistent initial differential variables $\mathcal{V}^{\rm diff}_{[E,A,B,N]}$ and derive a condition for its well-posedness. For this purpose, we make use of the feedback equivalence form given in [@RRV15].

The system $[E,A,B,N_\beta] \in \Sigma_{n,m,n_\omega}$ is *well-posed*, if there exists an initial value $Ex(0) = Ex_0 \in L^2_{\mathcal{F}_0}({\mathbb{R}}^n)$ for which the system has a solution $(x,u)$ on ${\mathbb{R}}_{\ge0}$.

\[lem:vdiff\] Let $[E,A,B,N_\beta]\in \Sigma_{n,m,n_\omega}$ with $N_\beta(t) = N\mathrm{e}^{\beta t}$ be given. Then there exist invertible matrices $W,\,T \in {\mathbb{R}}^{n\times n}$ and $F \in 
{\mathbb{R}}^{m\times n}$ such that $$\begin{aligned}
        W\begin{bmatrix} sE-A & B & N \end{bmatrix} \begin{bmatrix} T & 0 & 0 
\\ -FT & I_m & 0 \\ 0 & 0 & I_{n_\omega}
        \end{bmatrix} = \begin{bmatrix} sI_{n_1}-A_{11} & 0 & 0 
& B_1 & N_1 \\0&-I_{n_2} & sE_{23} & B_2 & N_2 \\ 0&0&sE_{33}-I_{n_3} & 0 & N_3 
    \end{bmatrix} \label{eq:feedback_transf}
    \end{aligned}$$ where $E_{33} \in {\mathbb{R}}^{n_3\times n_3}$ is nilpotent. Then the following statements are satisfied:

a)  The system $[E,A,B,N_\beta]$ with feedback equivalence form is well-posed, if and only if $N_2 = 0$ and $N_3 = 
    0$.

b)  If condition a) is satisfied, then the space of consistent initial differential variables $\mathcal{V}^{\rm diff}_{[E,A,B,N_\beta]}$ is well-defined and given by $$\begin{aligned}
            \mathcal{V}^{\rm diff}_{[E,A,B,,N_\beta]} = T\left( {\mathbb{R}}^{n_1+n_2} \times \ker\begin{bmatrix} E_{23} 
    \\E_{33} \end{bmatrix}\right). \label{eq:vdiff}
            \end{aligned}$$

The feedback equivalence form  is an immediate consequence of [@RRV15]. To show statement a), by Lemma \[lemma:noise\] we see that the equation obtained by the third row of is well-posed, if and only if $N_3 = 0$. If the state is partitioned according to the block structure of , i.e., $x(t) = 
T \left[\begin{smallmatrix} x_1(t) \\ x_2(t) \\ x_3(t) 
\end{smallmatrix}\right]$, then we directly see that $x_3 \equiv 0$. But then, from the second block row of , we get $$0 =  x_2(t) \mathrm{d}t + B_2 u(t) {\mathrm{d}}t + N_2\mathrm{e}^{\beta t}  \mathrm{d}w(t) \quad 
\Rightarrow \quad x_2(t) = -B_2 u(t) - N_2\mathrm{e}^{\beta t}  v(t).$$ In other words, well-posedness of this equation is equivalent to $N_2 = 0$.

Statement b) now follows immediately from statement a), since the structure is as in the deterministic setting in [@RRV15] except that there is a stochastic noise term in the differential part of the equation which does not affect the set of consistent initial differential variables.

a)  From Lemma \[lem:vdiff\] we see that the space $\mathcal{V}^{\rm diff}_{[E,A,B,N_\beta]}$ does not depend on $N_\beta$ in case of a well-posed system. Thus, from now on we will only write ${{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}$ instead of $\mathcal{V}^{\rm diff}_{[E,A,B,N_\beta]}$ to simplify notation.

b)  Lemma \[lem:vdiff\] makes clear, that if the system $[E,A,B,N_\beta] \in \Sigma_{n,m,n_w}$ has a solution for one initial condition $Ex(0) = Ex_0$, then it has a solution for all $Ex(0) = Ex_0 \in {L^2_{\mathcal{F}_0}\big(E{{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$.

Similarly as in the deterministic setting, we will impose a stabilizability condition in order to guarantee finiteness of the cost functional. Here we use an adaption of the concept of behavioral stabilizability to stochastic DAE from [@BR13].

A well-posed system $[E,A,B,N_\beta] \in \Sigma_{n,m,n_w}$ with $N_\beta(t) = N\mathrm{e}^{\beta t}$ is called *mean-square stabilizable (in the behavioral sense)* if for all solutions $(x,u)$ of $[E,A,B,N_\beta]$ there exists a solution $({\widetilde}{x},{\widetilde}{u}) $ of $[E,A,B,N_\beta]$ with $({\widetilde}{x}(t,\omega), {\widetilde}{u}(t,\omega)) = (x(t,\omega),u(t,\omega))$ for almost all $\omega \in \Omega$ and $t < 0$ and it holds that $$\lim\limits_{t \to \infty} {\mathbb{E}}\left[ \left\Vert \begin{pmatrix}{\widetilde}{x}(t) \\ {\widetilde}{u}(t) \end{pmatrix}\right\Vert^2\right] = 0.$$

The following characterization holds.

Let $[E,A,B,N_\beta] \in \Sigma_{n,m,n_w}$ with $N_\beta(t) = N\mathrm{e}^{\beta t}$ be well-posed for $t \in {\mathbb{R}}_{\ge 0}$ and $\beta < 0$. Then the system  is mean-square stabilizable, if and only if $$\begin{aligned}
                \operatorname{rank}\begin{bmatrix} \lambda E-A & B  \end{bmatrix} = n \quad \forall 
        \lambda \in \overline{\mathbb{C}^+},  \label{eq:rank_condition}
                \end{aligned}$$ where $\overline{\mathbb{C}^+} := \{ \lambda \in 
        \mathbb{C} \; : \; \operatorname{Re}(\lambda) \geq 0 \}$ is the closed right complex half-plane.

By using the transformation to feedback equivalence form of $[E,A,B,N_\beta]$ and due to the well-posedness of the system we can find invertible matrices $W,\,T \in {\mathbb{R}}^{n \times n}$ and $F \in {\mathbb{R}}^{m \times n}$ such that we obtain the transformed system with $N_2 = 0$ and $N_3 = 0$. Then $(x,u)$ is a solution of $[E,A,B,N_\beta]$, if and only if $\left( \left( \begin{smallmatrix} {\widetilde}{x}_1 \\ {\widetilde}{x}_2 \\ {\widetilde}{x}_3 \end{smallmatrix}\right), {\widetilde}{u}\right) := \left(T^{-1}x , Fx+u\right)$ solves the transformed system; and we have $$\begin{aligned}
 \operatorname{rank}\begin{bmatrix} \lambda E -A & B \end{bmatrix} = \operatorname{rank}\begin{bmatrix} \lambda I_{n_1} -A_{11} & B_1 \end{bmatrix} + n_2 + n_3  \quad \forall\,\lambda \in {\mathbb{C}}.\end{aligned}$$

Let be satisfied. Then $\operatorname{rank}\begin{bmatrix} \lambda I_{n_1} -A_{11} & B_1 \end{bmatrix} = n_1$ for all $\lambda \in \overline{{\mathbb{C}}^+}$. So we can find an $F_1 \in {\mathbb{R}}^{n_1 \times m}$ such that $\sigma(A_{11}+B_1F_1) \subset {\mathbb{C}}^-$. With ${\widehat}{A}_{11} := A_{11} + B_1F_1$, consider the SDE $${\mathrm{d}}{\widehat}{x}_1(t) = {\widehat}{A}_{11} {\widehat}{x}_1(t) {\mathrm{d}}t + N_1\mathrm{e}^{\beta t}  {\mathrm{d}}w(t)$$ which has the solution $${\widehat}{x}_1(t) = {\mathrm{e}}^{{\widehat}{A}_{11}t} {\widehat}{x}_1(0) + {\int_{0}^{t}{{\mathrm{e}}^{{\widehat}{A}_{11} 
    (t-s)}N_1e^{\beta s} }\,\mathrm{d}{w(s)}}.$$ In particular, it holds that $$\begin{aligned}
    {\mathbb{E}}\big[{\widehat}{x}_{1}(t) {\widehat}{x}_{1}(t){^\mathsf{T}}\big] &= {\mathrm{e}}^{{\widehat}{A}_{11} t} 
    {\mathbb{E}}\big[{\widehat}{x}_{1}(0) {\widehat}{x}_{1}(0){^\mathsf{T}}\big] 
    {\mathrm{e}}^{{\widehat}{A}_{11}{^\mathsf{T}}t} + {\int_{0}^{t}{{\mathrm{e}}^{2\beta s}{\mathrm{e}}^{{\widehat}{A}_{11}(t-s)}
    N_{1}N_{1}{^\mathsf{T}}{\mathrm{e}}^{{\widehat}{A}_{11}{^\mathsf{T}}(t-s)}}\,\mathrm{d}{s}} \\
            &= {\mathrm{e}}^{{\widehat}{A}_{11} t} {\mathbb{E}}\big[{\widehat}{x}_{1}(0){\widehat}{x}_{1}(0){^\mathsf{T}}\big] {\mathrm{e}}^{{\widehat}{A}_{11}{^\mathsf{T}}t}+ X(t),  
    \end{aligned}$$ where $X(\cdot)$ solves the differential Lyapunov equation $$\dot{X}(t)  = {\widehat}{A}_{11} X(t) + X(t) {\widehat}{A}_{11} {^\mathsf{T}}+ {\mathrm{e}}^{2\beta 
    t}N_{1}N_{1}{^\mathsf{T}}, \quad X(0) = 0,$$ see [@BWSV16]. In particular, $X(t) \to 0$ for $t \to \infty$, if $\sigma\big({\widehat}{A}_{11} \big) \subset {\mathbb{C}}^-$. Moreover, as ${\mathbb{E}}\big[{\widehat}{x}_{1}(t){^\mathsf{T}}{\widehat}{x}_{1}(t)\big] = \operatorname{tr}\left( {\mathbb{E}}\big[{\widehat}{x}_{1}(t){\widehat}{x}_{1}(t){^\mathsf{T}}\big] \right)$ it follows from the positive semi-definiteness of ${\mathbb{E}}\big[{\widehat}{x}_{1}(t) {\widehat}{x}_{1}(t){^\mathsf{T}}]$ that ${\mathbb{E}}\big[{\widehat}{x}_{1}(t){^\mathsf{T}}{\widehat}{x}_{1}(t)]  = 0$, if and only if ${\mathbb{E}}\big[{\widehat}{x}_{1}(t) {\widehat}{x}_{1}(t){^\mathsf{T}}] = 0$. As $\sigma({\widehat}{A}_{11}) \subset {\mathbb{C}}^-$, with the feedback ${\widetilde}{u}(t) := F_1{\widetilde}{x}_1(t)$ we obtain $\lim_{t \to \infty} {\mathbb{E}}\left[ \left\| \left(\begin{smallmatrix}{\widetilde}{x}_1(t) \\ {\widetilde}{u}(t)\end{smallmatrix}\right)\right\|^2 \right] = 0$ and consequently, $\lim_{t \to \infty} {\mathbb{E}}\big[ \big\| {\widetilde}{x}_2(t)\big\|^2 \big] = 0$ as well as $\lim_{t \to \infty} {\mathbb{E}}\big[ \big\|{\widetilde}{x}_3(t) \big\|^2 \big] = 0$. So the system $[E,A,B,N_\beta]$ is mean-square stabilizable.

Conversely assume that $ \operatorname{rank}\begin{bmatrix} \lambda E -A & B \end{bmatrix}< n$ for some $\lambda \in \overline{{\mathbb{C}}^+}$. Let $V \in {\mathbb{R}}^{n_1 \times n_1}$ be an orthogonal matrix leading to the Kalman decomposition $$V{^\mathsf{T}}A_{11} V = \begin{bmatrix} A_{11}^{(11)} &  A_{11}^{(12)} \\ 0 &  A_{11}^{(22)}  \end{bmatrix}, \quad V{^\mathsf{T}}B_1 = \begin{bmatrix} B_1^{(1)} \\ 0 \end{bmatrix}, \quad V{^\mathsf{T}}N_1 = \begin{bmatrix} N_1^{(1)} \\ N_1^{(2)} \end{bmatrix}$$ with $\sigma\big( A_{11}^{(22)} \big) \cap \overline{{\mathbb{C}}^+} \neq \emptyset$. But then by the considerations above, the input-independent SDE $${\mathrm{d}}{\widetilde}{x}_{1}^{(2)}(t) = A_{11}^{(22)} {\widetilde}{x}_{1}^{(2)}(t) {\mathrm{d}}t + N_1^{(2)}\mathrm{e}^{\beta t}  {\mathrm{d}}w(t)$$ is not mean-square stable, so $[E,A,B,N_\beta]$ is not mean-square stabilizable.

A crucial role in the study of descriptor systems plays the *system space*.

The *system space* of $[E,A,B,N_\beta]$ is the smallest subspace $\mathcal{V}^{\rm sys}_{[E,A,B,N_\beta]}\subseteq  {\mathbb{R}}^{n+m}$ such that $$\begin{gathered}
        \forall \text{ solutions } (x,u) \text{ of } [E,A,B,N_\beta]: \begin{pmatrix} x(t,\omega) \\ 
u(t,\omega)\end{pmatrix} \in \mathcal{V}^{\rm sys}_{[E,A,B,N_\beta]} \\ \text{ for all } t \in {\mathbb{R}}\text{ and all } \omega \in \Omega \text{ for which } (x(\cdot,\omega),u(\cdot,\omega)) \text{ satisfies } \eqref{eq:solution}.
    \end{gathered}$$

Since the noise only affects the dynamic part of our SDAE, it is easily verified that the space $\mathcal{V}^{\rm sys}_{[E,A,B,N_\beta]}$ is equal to system space for deterministic systems as in @RRV15 and hence, it does not depend on $N_\beta$. Therefore, for a more concise notation, we will only write ${{\mathcal{V}}^{\rm{sys}}_{[E,A,B]}}$ instead of $\mathcal{V}^{\rm sys}_{[E,A,B,N_\beta]}$.

The optimal control problem
===========================

In this section we return to the analysis of the optimal control problem which we state once completely. We consider the optimal control problem $$\begin{aligned}
 \label{eq:OCP}
     \begin{split}
      \text{minimize } & J(x,u,{\mathbb{R}}_{\ge 0}) = {\mathbb{E}}\left[ \int_0^\infty \begin{pmatrix} x(t) \\ u(t) \end{pmatrix}{^\mathsf{T}}\begin{bmatrix} Q & S \\ S{^\mathsf{T}}& R \end{bmatrix} \begin{pmatrix} x(t) \\ u(t) \end{pmatrix} {\mathrm{d}}t \right]   \\
      \text{subject to } & {\mathrm{d}}E x(t) = (Ax(t) + Bu(t)) {\mathrm{d}}t + N \mathrm{e}^{\beta t} {\mathrm{d}}w(t), \\ & Ex(0) = Ex_0, \quad \lim_{t \to \infty} {\mathbb{E}}\left[ \left\| Ex(t) \right\|^2\right] = 0. 
          \end{split}
    \end{aligned}$$ The aim of this section is to derive conditions for feasibility and regularity of the corresponding control problem. Recall that we say that $(x,u)$ is a solution of $[E,A,B,N$ on $[0,{t_{\rm f}}]$ if $(x,u) \in L^{2,\rm loc}_{\mathcal{F}}([0,{t_{\rm f}}], {\mathbb{R}}^n) \times L^{2,\rm loc}_{\mathcal{F}}([0,{t_{\rm f}}], {\mathbb{R}}^m )$ and it solves the stochastic DAE . The following definition has been adapted from [@RV18].

Let $[E,A,B,N_\beta] \in \Sigma_{n,m,n_w}$ with $N_\beta(t)=N\mathrm{e}^{\beta t}$ and $\beta < 0$ be well-posed and let $Q = Q{^\mathsf{T}}\in 
{\mathbb{R}}^{n\times 
n}$, $S \in {\mathbb{R}}^{n\times m}$, and $R = R{^\mathsf{T}}\in {\mathbb{R}}^{m\times m}$ be given.

a)  The optimal control problem  is called *feasible* if for all $x_0 \in {L^2_{\mathcal{F}_0}\big({{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$ it holds that $$-\infty < W_+(Ex_0) < \infty .$$

b)  A solution $(x{^\star}, u{^\star})$ of $[E,A,B,N_\beta] $ on ${\mathbb{R}}_{\geq 0}$ with initial condition $Ex(0) = Ex_0 \in {L^2_{\mathcal{F}_0}\big(E{{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$ and $\lim_{t \to \infty}{\mathbb{E}}\big[\Vert Ex(t)\Vert^2\big] = 0$ is called an *optimal control* for , if $$W_+(Ex_0) = J(x{^\star},u{^\star},{\mathbb{R}}_{\ge 0}) .$$

c)  The optimal control problem  is called *regular*, if for all $x_0 \in {L^2_{\mathcal{F}_0}\big({{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$, there exists a unique optimal control.

Feasibility of the problem {#sec:feas}
--------------------------

The following definition is an extension of dissipativity of stochastic differential equations. Our definition is a slight modification from the definitions in @MP09 [@ZJ10].

Let $[E,A,B,N_\beta] \in \Sigma_{n,m,n_w}$ with $N_\beta(t)=N\mathrm{e}^{\beta t}$ and $\beta < 0$ be well-posed and let $Q = Q{^\mathsf{T}}\in {\mathbb{R}}^{n\times 
n}$, $S \in {\mathbb{R}}^{n\times m}$, and $R = R{^\mathsf{T}}\in {\mathbb{R}}^{m\times m}$ be given. Then the system $[E,A,B,N_\beta]$ with the cost functional in  is said to be *dissipative* on ${\mathbb{R}}_{\ge 0}$ if there exist a twice continuously differentiable *storage function* $V:E {{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\to {\mathbb{R}}$ such that the *integral dissipation inequality* $$\begin{aligned}
        {\mathbb{E}}[V(Ex(t_1))] + {\mathbb{E}}\left[{\int_{t_1}^{t_2}{\mu_N(Ex(t))}\,\mathrm{d}{t}}\right] \leq  {\mathbb{E}}[V(Ex(t_2))] + 
J(x,u,[t_1,t_2])    \label{eq:dissipation}
        \end{aligned}$$ holds for any times $0 \leq t_1 \leq t_2 < \infty$ and any solution $(x,u)$ of $[E,A,B,N_\beta]$ on $[t_1,t_2]$ with $$\mu_N: E {{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\to {\mathbb{R}}, \quad Ex_0 \mapsto \frac{1}{2}e^{2\beta t} w(t){^\mathsf{T}}N{^\mathsf{T}}\nabla^2V(Ex_0) N w(t).$$\[def:dissipative\]

Note that the above definition boils down to the well-known dissipation inequality in the deterministic setting with $N = 0$ and deterministic initial condition.

From now on we also need the following notation: For two symmetric matrices $X,\,Y \in {\mathbb{R}}^{k \times k}$ and a subspace $\mathcal{V} \subseteq {\mathbb{R}}^k$ we write $$X =_{\mathcal{V}} (\leq_{\mathcal{V}}, \geq_{\mathcal{V}}) Y \iff v{^\mathsf{T}}Xv = 
(\le, \ge) v{^\mathsf{T}}Yv \text{ for all } v \in \mathcal{V}.$$ One can now check that any quadratic function $V : E{{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\to {\mathbb{R}}$ with $V(Ex_0) = (Ex_0){^\mathsf{T}}P Ex_0$ for which $P \in {\mathbb{R}}^{n \times n}$ is a solution of the *Kalman-Yakubovich-Popov (KYP) inequality* $$\begin{aligned}
            \begin{bmatrix}
            A{^\mathsf{T}}P E + E{^\mathsf{T}}P A + Q & E{^\mathsf{T}}PB+S\\B{^\mathsf{T}}PE+S{^\mathsf{T}}& R
            \end{bmatrix} \geq_{{{\mathcal{V}}^{\rm{sys}}_{[E,A,B]}}} 0, \quad P = P{^\mathsf{T}}\label{eq:kyp}
            \end{aligned}$$ is a storage function in our case. This can be seen together with Itô’s formula. We get $$\begin{aligned}
     & {\mathbb{E}}[V(Ex(t_2))] - {\mathbb{E}}[V(Ex(t_1))] \\&= {\mathbb{E}}\left[ \int_{t_1}^{t_2} {\mathrm{d}}V(Ex(t)) \right] + {\mathbb{E}}\left[\int_{t_1}^{t_2}  \frac{1}{2}e^{2\beta t} w(t){^\mathsf{T}}N{^\mathsf{T}}\nabla^2 V(Ex(t)) N w(t){\mathrm{d}}t \right]\\ 
     &= {\mathbb{E}}\left[ \int_{t_1}^{t_2} (\nabla V(Ex(t))){^\mathsf{T}}{\mathrm{d}}Ex(t) \right] + \int_{t_1}^{t_2}  \frac{1}{2}\mathrm{e}^{2\beta t}\operatorname{tr}\big(N{^\mathsf{T}}\nabla^2 V(Ex(t)) N\big) {\mathrm{d}}t \\ 
     &= {\mathbb{E}}\left[ \int_{t_1}^{t_2} 2x(t){^\mathsf{T}}E{^\mathsf{T}}P (Ax(t) + Bu(t)) {\mathrm{d}}t \right] + \int_{t_1}^{t_2} \mathrm{e}^{2\beta t}\operatorname{tr}\big(N{^\mathsf{T}}P N\big) {\mathrm{d}}t  \\
     &\ge -{\mathbb{E}}\left[ \int_{t_1}^{t_2} \begin{pmatrix}
        x(t) \\ u(t) \end{pmatrix}{^\mathsf{T}}\begin{bmatrix} Q & S \\ S{^\mathsf{T}}& R \end{bmatrix} 
        \begin{pmatrix}
        x(t) \\ u(t) \end{pmatrix} {\mathrm{d}}t \right] + {\mathbb{E}}\left[\int_{t_1}^{t_2} \mu_N(Ex(t)){\mathrm{d}}t\right].
    \end{aligned}$$ In the next theorem we will show analogously to the deterministic case that feasibility of the optimal control problem implies solvability of the KYP inequality.

Let $[E,A,B,N_\beta] \in \Sigma_{n,m,n_w}$ with $N_\beta(t)=N\mathrm{e}^{\beta t}$ and $\beta < 0$ be well-posed and let $Q = Q{^\mathsf{T}}\in {\mathbb{R}}^{n\times 
        n}$, $S \in {\mathbb{R}}^{n\times m}$, and $R = R{^\mathsf{T}}\in {\mathbb{R}}^{m\times m}$ be given. If the optimal control problem is feasible, then the system $[E,A,B,N_\beta]$ is mean-square stabilizable and the KYP inequality is feasible, i.e., there exists at least one matrix $P \in {\mathbb{R}}^{n \times n}$ satisfying . \[thm:wopt\]

Let $z := \left(\begin{smallmatrix} x \\ u\end{smallmatrix}\right) \in L^{2,\rm loc}_{\mathcal{F}}\big( {\mathbb{R}}_{\ge 0},{{\mathcal{V}}^{\rm{sys}}_{[E,A,B]}}\big)$ and ${\widetilde}{z}(t) := z(t) - {\mathbb{E}}[z(t)] := z(t)-{\widehat}{z}(t)$. Then $z(t) = {\widetilde}{z}(t) + {\widehat}{z}(t)$ and ${\mathbb{E}}[{\widetilde}{z}(t)] = 0$ for all $t \ge 0$. In particular, for $M := \left[ \begin{smallmatrix} Q & S \\ S{^\mathsf{T}}& R \end{smallmatrix}\right] \in {\mathbb{R}}^{(n+m)\times (n+m)}$ we have $$\begin{aligned}
    {\mathbb{E}}[z(t){^\mathsf{T}}M z(t)] &= {\mathbb{E}}\big[\big(z(t)-{\widehat}{z}(t) + {\widehat}{z}(t)\big){^\mathsf{T}}M \big(z(t)-{\widehat}{z}(t)+ {\widehat}{z}(t)\big)\big] \\
    &= {\mathbb{E}}\big[\big(z(t)-{\widehat}{z}(t)\big){^\mathsf{T}}M \big(z(t)-{\widehat}{z}(t)\big)\big] + {\mathbb{E}}\big[{\widehat}{z}(t){^\mathsf{T}}M {\widehat}{z}(t)\big] \\ & \qquad + 2 {\mathbb{E}}\big[{\widehat}{z}(t){^\mathsf{T}}M \big(z(t)-{\widehat}{z}(t)\big)\big]\\
    &={\mathbb{E}}\big[{\widetilde}{z}(t){^\mathsf{T}}M {\widetilde}{z}(t)\big] + {\widehat}{z}(t){^\mathsf{T}}M {\widehat}{z}(t) + 2 {\widehat}{z}(t){^\mathsf{T}}M {\mathbb{E}}\big[{\widetilde}{z}(t)\big] \\
    &= {\mathbb{E}}\big[{\widetilde}{z}(t){^\mathsf{T}}M {\widetilde}{z}(t)\big] + {\widehat}{z}(t){^\mathsf{T}}M {\widehat}{z}(t). 
    \end{aligned}$$ Thus with ${\widehat}{z}(\cdot)=:\left(\begin{smallmatrix} {\widehat}{x}(\cdot) \\ {\widehat}{u}(\cdot) \end{smallmatrix} \right)$ and ${\widetilde}{z}(\cdot)=:\left(\begin{smallmatrix} {\widetilde}{x}(\cdot) \\ {\widetilde}{u}(\cdot) \end{smallmatrix} \right)$ we can write $$J(x,u,{\mathbb{R}}_{\ge 0}) = J^{\rm d}\big({\widehat}{x},{\widehat}{u},{\mathbb{R}}_{\ge 0}\big) + J^{\rm s}\big({\widetilde}{x},{\widetilde}{u},{\mathbb{R}}_{\ge 0}\big),$$ where $$\begin{aligned}
    J^{\rm d}({\widehat}{x},{\widehat}{u},{\mathbb{R}}_{\ge 0}) &=\int_{0}^{\infty}\begin{pmatrix}{\widehat}{x}(t) \\ {\widehat}{u}(t) \end{pmatrix}{^\mathsf{T}}\begin{bmatrix} Q & S\\S{^\mathsf{T}}& R \end{bmatrix} \begin{pmatrix}{\widehat}{x}(t) \\ {\widehat}{u}(t) \end{pmatrix} {\mathrm{d}}t \\    J^{\rm s}({\widetilde}{x},{\widetilde}{u},{\mathbb{R}}_{\ge 0})  &= {\mathbb{E}}\left[ \int_{0}^{\infty}\begin{pmatrix}{\widetilde}{x}(t) \\ {\widetilde}{u}(t) \end{pmatrix}{^\mathsf{T}}\begin{bmatrix} Q & S\\S{^\mathsf{T}}& R \end{bmatrix} \begin{pmatrix}{\widetilde}{x}(t) \\ {\widetilde}{u}(t) \end{pmatrix} {\mathrm{d}}t \right]     \end{aligned}$$ subject to $$\begin{aligned}
    \frac{{\mathrm{d}}}{{\mathrm{d}}t}E{\widehat}{x}(t) &= A{\widehat}{x}(t) + B{\widehat}{u}(t), \quad E{\widehat}{x}(0) = {\mathbb{E}}[Ex_0], \label{eq:ode} \\ 
    {\mathrm{d}}E{\widetilde}{x}(t) & = (A{\widetilde}{x}(t) + B{\widetilde}{u}(t)) {\mathrm{d}}t + N\mathrm{e}^{\beta t}  {\mathrm{d}}w(t) , \quad E{\widetilde}{x}(0) = Ex_0 - {\mathbb{E}}[Ex_0]. \label{eq:sden}
    \end{aligned}$$ Consider the value functions $V_+^{\rm d}:E{{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\to {\mathbb{R}}\cup \{-\infty,\infty\}$ and $W_+^{\rm s} : {L^2_{\mathcal{F}_0}\big(E{{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}\to {\mathbb{R}}\cup \{-\infty, \infty\}$ with $$\begin{aligned}
    {V}_+^{\rm d}(Ex_0) &= \inf\left\{ J^{\rm d}({\widehat}{x},{\widehat}{u},{\mathbb{R}}_{\ge 0}): ({\widehat}{x},{\widehat}{u})  \text{ solves } \eqref{eq:ode} \text{ with } \right. \\ & \qquad\qquad\qquad\left. E {\widehat}{x}(0) = {\mathbb{E}}[Ex_0] \text{ and } \lim_{t \to \infty} E{\widehat}{x}(t) = 0 \right\}, \\     {W}_+^{\rm s}(Ex_0) &= \inf\Big\{ J^{\rm s}({\widetilde}{x},{\widetilde}{u},{\mathbb{R}}_{\ge 0}): ({\widetilde}{x},{\widetilde}{u})  \text{ solves } \eqref{eq:sden} \text{ with } \\ & \qquad\qquad\qquad \left. E {\widetilde}{x}(0) = Ex_0 - {\mathbb{E}}[Ex_0] \text{ and } \lim_{t \to \infty} {\mathbb{E}}\big[\left\|E{\widetilde}{x}(t)\right\|^2\big] = 0 \right\}
    \end{aligned}$$ Since both cost functionals can be minimized independently, we have $W_+(Ex_0) = V_+^{\rm d}(Ex_0) + W_+^{\rm s}(Ex_0)$ for each $x_0 \in {L^2_{\mathcal{F}_0}\big({{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$. Since is feasible and since and are simultaneously (mean-square) stabilizable in the behavioral sense or not, we must have $V_+^{\rm d}(Ex_0) < \infty$ and $W_+^{\rm s}(Ex_0) < \infty$ for each $x_0 \in {L^2_{\mathcal{F}_0}\big({{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$. Hence the system $[E,A,B,N_\beta]$ is mean-square stabilizable. Moreover, due to the feasibility of , we have $V_+^{\rm d}(Ex_0) >- \infty$ for each $x_0 \in {L^2_{\mathcal{F}_0}\big({{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$. Then according to @RV18 [Theorem 3.11], there exists a solution $P \in {\mathbb{R}}^{n \times n}$ of the KYP inequality .

If the optimal control problem  is feasible, then due to the stabilizability of in the behavioral sense and @RV18 [Theorem 3.11], we can even infer the existence of a maximal solution $P_+ \in {\mathbb{R}}^{n \times n}$ of , i.e., $$P_+ \ge_{E{{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}} P$$ for all solutions $P \in {\mathbb{R}}^{n\times n}$ of .

If $E = I_n$ and the weight matrix $R$ is invertible, then the solution of a feasible optimal control problem is given in terms of the solution of an algebraic Riccati equation, see Section \[sec:formulation\_nonsingular\]. A generalization of this algebraic Riccati equation for a possible singular matrix $E$ and a general weight $\left[ \begin{smallmatrix} Q & S \\ S{^\mathsf{T}}& R \end{smallmatrix} \right]$ is presented in the following.

Let $[E,A,B,N_\beta] \in \Sigma_{n,m,n_w}$ with $N_\beta(t)=N\mathrm{e}^{\beta t}$ and $\beta < 0$ be well-posed and let $Q = Q{^\mathsf{T}}\in {\mathbb{R}}^{n\times 
n}$, $S \in {\mathbb{R}}^{n\times m}$, and $R = R{^\mathsf{T}}\in {\mathbb{R}}^{m\times m}$ be given. A triple $(X,K,L) \in {\mathbb{R}}^{n\times n} \times {\mathbb{R}}^{q \times n} \times 
{\mathbb{R}}^{q \times m}$ that fulfills $$\begin{aligned}
        \begin{bmatrix}
        A{^\mathsf{T}}X E + E{^\mathsf{T}}XA+Q&E{^\mathsf{T}}XB+S\\B{^\mathsf{T}}XE+S{^\mathsf{T}}& R
        \end{bmatrix} =_{{{\mathcal{V}}^{\rm{sys}}_{[E,A,B]}}} \begin{bmatrix} K{^\mathsf{T}}\\L{^\mathsf{T}}\end{bmatrix}  \begin{bmatrix} K &L \end{bmatrix} \label{eq:lure} 
        \end{aligned}$$ and $$\begin{aligned}
        \operatorname{rank}_{{\mathbb{R}}[s]} \begin{bmatrix}
        -sE+A & B \\ K & L
        \end{bmatrix} = n+q
        \end{aligned}$$ is called solution of the *Lur’e equation*, where ${\mathbb{R}}[s]$ denotes the ring of polynomials with coefficients in ${\mathbb{R}}$.

A solution $(X,K,L) \in {\mathbb{R}}^{n\times n}\times {\mathbb{R}}^{q\times n} \times 
{\mathbb{R}}^{q\times m} $ of the Lur’e equation is called *stabilizing*, if additionally $$\begin{aligned}
        \operatorname{rank}\begin{bmatrix}
        -\lambda E +A & B \\ K & L
        \end{bmatrix} = n+q  \quad \forall \lambda \in \mathbb{C}^{+} .
        \end{aligned}$$\[def:lur\]

Note that if $(P_+,K,L)\in {\mathbb{R}}^{n\times n}\times {\mathbb{R}}^{q\times n} \times {\mathbb{R}}^{q\times m}$ is a stabilizing solution of the Lur’e equation , then $P_+$ is a maximal solution of the KYP inequality . Conversely, if $P_+ \in {\mathbb{R}}^{n \times n}$ is a maximal solution of , then there exist $K \in {\mathbb{R}}^{q \times n}$ and $L \in {\mathbb{R}}^{q \times m}$ such that $(P_+,K,L)$ is a stabilizing solution of  [@RRV15].

Now we can show the following result which is an adaptation of a result in @RV18.

Let $[E,A,B,N_\beta] \in \Sigma_{n,m,n_w}$ with $N_\beta(t)=N\mathrm{e}^{\beta t}$ and $\beta < 0$ be well-posed and let $Q = Q{^\mathsf{T}}\in {\mathbb{R}}^{n\times 
n}$, $S \in {\mathbb{R}}^{n\times m}$, and $R = R{^\mathsf{T}}\in {\mathbb{R}}^{m\times m}$ be given. Then the following statements are equivalent:

a)  The optimal control problem is feasible, i.e., $W_+(Ex_0) 
    \in {\mathbb{R}}$ for all $x_0 \in {L^2_{\mathcal{F}_0}\big({{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$.

b)  The system $[E,A,B,N_\beta]$ is mean-square stabilizable and the KYP inequality  has a maximal solution $P_+ = P_+{^\mathsf{T}}\in 
    {\mathbb{R}}^{n\times n}$.

c)  The system $[E,A,B,N_\beta]$ is mean-square stabilizable and there exists a storage function $V:E{{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\to {\mathbb{R}}$.

d)  There exist $q \in \mathbb{N}_0$, $K\in{\mathbb{R}}^{q\times n}$, and $L \in {\mathbb{R}}^{q\times m}$ such that $(P_+,K,L)$ is a stabilizing solution of the Lur’e equation .

In the case where the above statements are valid, we have

i)  $W_+(Ex_0) = \operatorname{tr}\big(P_+ {\mathbb{E}}\big[Ex_0 (Ex_0){^\mathsf{T}}]\big) - 
    \frac{1}{2\beta}\operatorname{tr}\big( P_+ NN{^\mathsf{T}}\big) \quad \forall x_0 \in {L^2_{\mathcal{F}_0}\big({{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$.

ii) For all $x_0 \in {L^2_{\mathcal{F}_0}\big({{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$ and solutions $(x,u)$ of $[E,A,B,N_\beta]$ on ${\mathbb{R}}_{\geq 0}$ with $Ex(0) = Ex_0$ and $\lim\limits_{t \to 
    \infty} {\mathbb{E}}\big[\Vert Ex(t) \Vert^2\big] = 0 $ it holds that $$\begin{gathered}
                J(x,u,{\mathbb{R}}_{\ge 0}) = \operatorname{tr}\big(P_+ {\mathbb{E}}\big[ E x_0 (Ex_0){^\mathsf{T}}\big]\big) - 
    \frac{1}{2\beta}\operatorname{tr}\big( P_+ NN{^\mathsf{T}}\big) \\ + {\int_{0}^{\infty}{{\mathbb{E}}\left[ \Vert K x(t) + 
    Lu(t) \Vert^2\right]}\,\mathrm{d}{t}} . \label{eq:jurepres}
                \end{gathered}$$

iii) A solution $(x{^\star},u{^\star})$ is an optimal control, if and only if ${\mathbb{E}}[\Vert Kx{^\star}+ Lu{^\star}\Vert^2] = 0$.

\[thm:eq\_feas\_lure\]

Assertion a) $\Rightarrow$ b) follows from Theorem \[thm:wopt\]. Statement b) $\Rightarrow$ c) is trivial and c) $\Rightarrow$ a) follows from the dissipation inequality since for every solution $(x,u)$ of $[E,A,B,N_\beta]$ with $x_0 \in {L^2_{\mathcal{F}_0}\big({{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$ and $\lim_{t \to \infty} {\mathbb{E}}\big[ \big\| Ex(t)\big\|^2\big] = 0$, we have $$-\infty < {\mathbb{E}}[V(Ex_0)] + {\mathbb{E}}\left[{\int_{0}^{\infty}{\mu_N(Ex(t))}\,\mathrm{d}{t}}\right] \leq J(x,u,{\mathbb{R}}_{\ge 0}).$$ The equivalence b) $\Leftrightarrow$ d) follows immediately from @RV18 [Theorem 3.13].

Next we show i)–iii): As $P_+ = P_+{^\mathsf{T}}\in {\mathbb{R}}^{n\times n}$ is the maximal solution of the KYP inequality, it follows from @RV18 [Theorem 4.5] that there exist a $q \in 
        \mathbb{N}_0$, $K\in {\mathbb{R}}^{q\times n}$, and $L \in {\mathbb{R}}^{q\times m}$ such that $(P_+,K,L)$ is stabilizing solution of the Lur’e equation . Moreover, from Itô’s formula we obtain $$\begin{aligned}
           \operatorname{tr}\big( P_+ & {\mathbb{E}}\big[E x_0 (Ex_0){^\mathsf{T}}\big]\big) - \frac{1}{2\beta}\operatorname{tr}\big( P_+ NN{^\mathsf{T}}\big) \\ 
          &= {\mathbb{E}}\big[ x_0{^\mathsf{T}}E{^\mathsf{T}}P_+ E x_0\big] - \frac{1}{2\beta}\operatorname{tr}\big( P_+ NN{^\mathsf{T}}\big) \\ 
            &= -{\mathbb{E}}\left[{\int_{0}^{\infty}{}\,\mathrm{d}{\left( (Ex(t)){^\mathsf{T}}P_+ (Ex(t)) 
        \right)}}\right] \\
            &= {\mathbb{E}}\left[ {\int_{0}^{\infty}{\begin{pmatrix}
                x(t) \\ u(t) \end{pmatrix}{^\mathsf{T}}\begin{bmatrix} -A{^\mathsf{T}}P_+E - 
        E{^\mathsf{T}}P_+ A & -E{^\mathsf{T}}P_+ B \\ -B{^\mathsf{T}}P_+ E& 0 \end{bmatrix} 
            \begin{pmatrix}
                x(t) \\ u(t) \end{pmatrix}}\,\mathrm{d}{t}} \right] \\
        &= {\mathbb{E}}\left[ {\int_{0}^{\infty}{\begin{pmatrix}
                x(t) \\ u(t) \end{pmatrix}{^\mathsf{T}}\left(  \begin{bmatrix} Q  & S \\ 
        S{^\mathsf{T}}& R \end{bmatrix} -
                \begin{bmatrix} K{^\mathsf{T}}K  & K{^\mathsf{T}}L \\ L{^\mathsf{T}}K& L{^\mathsf{T}}L 
        \end{bmatrix} \right) 
                \begin{pmatrix} 
                x(t) \\ u(t) \end{pmatrix}}\,\mathrm{d}{t}} \right] \\
            &= J(x,u,{\mathbb{R}}_{\ge 0}) - {\int_{0}^{\infty}{{\mathbb{E}}\left[ \Vert Kx(t) + Lu(t) 
        \Vert^2 \right]}\,\mathrm{d}{t}} , 
        \end{aligned}$$ which is assertion ii). In particular, a solution $(x{^\star},u{^\star})$ of $[E,A,B] $ on ${\mathbb{R}}_{\geq 0}$ with $Ex(0) = Ex_0$ and $\lim\limits_{t \to \infty} {\mathbb{E}}\big[\Vert Ex(t)\Vert^2\big] = 0$ is an optimal control, if and only if $ Kx{^\star}+ Lu{^\star}= 0$ almost surely (assertion iii)). The main difference to the deterministic case is the extra term $- \frac{1}{2\beta}\operatorname{tr}\big( P_+ NN{^\mathsf{T}}\big)$ due to the function $\mu_N(\cdot)$ in the dissipation inequality . Thus we can proceed analogously to [@RV18] by taking the infimum over all possible solutions $(x,u)$ with $Ex(0) = Ex_0$ and $\lim\limits_{t \to \infty} {\mathbb{E}}\big[\Vert Ex(t)\Vert^2\big] = 0$. It follows that $W_+(Ex_0) = \operatorname{tr}\big(P_+ {\mathbb{E}}\big[E x_0 (Ex_0){^\mathsf{T}}\big]\big)- \frac{1}{2\beta}\operatorname{tr}\big( P_+ NN{^\mathsf{T}}\big)$ for all $x_0 \in {L^2_{\mathcal{F}_0}\big({{\mathcal{V}}^{\rm{diff}}_{[E,A,B]}}\big)}$, since ${\int_{0}^{\infty}{{\mathbb{E}}\left[ \Vert Kx(t) + Lu(t) 
        \Vert^2 \right]}\,\mathrm{d}{t}}$ can be made arbitrarily small which gives assertion i).

Regularity of the problem {#sec:reg}
-------------------------

Next, we want to characterize the regularity of the optimal control problem. From Theorem \[thm:eq\_feas\_lure\], we have seen that the solution $(x{^\star},u{^\star})$ of $[E,A,B,N_\beta]$ with $Ex(0) = Ex_0$ and $\lim_{t \to \infty}{\mathbb{E}}\big[\Vert 
Ex(t)\Vert^2\big] = 0$ is an optimal control if and only if it fulfills the optimality DAE $$\begin{gathered}
    {\mathrm{d}}\begin{bmatrix}
    E &0\\0&0 \end{bmatrix} \begin{pmatrix}
    x{^\star}(t) \\u{^\star}(t) \end{pmatrix} = \begin{bmatrix} A & B \\ K & L \end{bmatrix} 
\begin{pmatrix}
    x{^\star}(t) \\u{^\star}(t) \end{pmatrix} {\mathrm{d}}t + \begin{bmatrix}
    N \\ 0
    \end{bmatrix}\mathrm{e}^{\beta t}  {\mathrm{d}}w(t), \\ Ex(0) = Ex_0,  \quad \lim_{t \to \infty}{\mathbb{E}}\big[\Vert 
Ex(t)\Vert^2\big] = 0. \label{eq:opt_dae} 
    \end{gathered}$$ Moreover, as no noise source appears in the algebraic constraints, this DAE is well-posed. Now we can immediately state a regularity result which directly follows from @RV18 [Theorem 4.7].

Let $[E,A,B,N_\beta] \in \Sigma_{n,m,n_w}$ with $N_\beta(t)=N\mathrm{e}^{\beta t}$ and $\beta < 0$ be well-posed and let $Q = Q{^\mathsf{T}}\in {\mathbb{R}}^{n\times 
n}$, $S \in {\mathbb{R}}^{n\times m}$, and $R = R{^\mathsf{T}}\in {\mathbb{R}}^{m\times m}$ be given. Let the optimal control problem  be feasible and $(P,K,L) \in {\mathbb{R}}^{n 
\times n} \times {\mathbb{R}}^{q\times n}\times {\mathbb{R}}^{q\times m}$ be a stabilizing solution of the Lur’e equation . Then the following two statements are equivalent:

a)  The optimal control problem is regular.

b)  The conditions $$\ker \begin{bmatrix}
                -\imath\omega E + A & B \\ K & L    \end{bmatrix} = \{0\} \quad \forall\,\omega \in {\mathbb{R}}, 
    \label{cond:pencil}$$ and $$\begin{gathered}
                \begin{bmatrix} E & 0\\ 0&0 \end{bmatrix} \cdot {{\mathcal{V}}^{\rm{sys}}_{[E,A,B]}}+ 
    \begin{bmatrix} A & B\\ K &L        \end{bmatrix} \cdot \left( \left(\ker E 
    \times {\mathbb{R}}^{m}\right) \cap {{\mathcal{V}}^{\rm{sys}}_{[E,A,B]}}\right)  \\   = \begin{bmatrix} 
    E & 0\\ 0&0 \end{bmatrix} \cdot {{\mathcal{V}}^{\rm{sys}}_{[E,A,B]}}+ \begin{bmatrix} A & B\\ K &L        
    \end{bmatrix} \cdot {{\mathcal{V}}^{\rm{sys}}_{[E,A,B]}}\label{cond:sys_space}
                \end{gathered}$$ are fulfilled.

We see that if $[E,A,B,N_\beta]$ is well-posed, then also the optimality DAE  is well-posed. We further know that  and  is equivalent to the unique solvability of the deterministic system. Thus, we conclude that there exists a unique solution of the stochastic DAE  and the optimal control problem is regular, if and only if  and  are fulfilled.

If the optimal control problem  is regular, then in the Lur’e equation  we have $q=m$. If further $L \in {\mathbb{R}}^{m \times m}$ is invertible, then the second row of the optimality DAE  can be resolved to obtain the explicit feedback control law $u(t) = -L^{-1}K x(t)$.

Conclusions and Outlook
=======================

In this paper, we have discussed feasibility and regularity of optimal control problems for linear stochastic descriptor system with additive noise by extending the results of the deterministic case to the stochastic problem. The feasibility of the control problem has been characterized by the dissipation inequality and the Kalman-Yakubovich-Popov inequality. We have constructed the solution of the problem using the stabilizing solution of the Lur’e equation and have derived conditions for existence and uniqueness of optimal controls.

Instead of the discounted cost functional , other functionals are often used. In the following we give a short overview of the used cost functionals and give an outlook on how to deal with these cost functionals in the same manner as with the discounted functional.

For example,  @KS72 used the time-average functional $$J^{\rm a}(x,u,[0,{t_{\rm f}}]) = \frac{1}{{t_{\rm f}}} {\mathbb{E}}\left[ {\int_{0}^{{t_{\rm f}}}{\begin{pmatrix}
        x(t) \\ u(t) \end{pmatrix}{^\mathsf{T}}\begin{bmatrix} Q & S \\ S{^\mathsf{T}}& 
R \end{bmatrix} 
        \begin{pmatrix}
        x(t) \\ u(t) \end{pmatrix}}\,\mathrm{d}{t}} \right]$$ and especially, for ${t_{\rm f}}= \infty$ we obtain $$J^{\rm a}(x,u,{\mathbb{R}}_{\ge 0}) = \lim\limits_{{t_{\rm f}}\to \infty}\frac{1}{{t_{\rm f}}} {\mathbb{E}}\left[ 
{\int_{0}^{{t_{\rm f}}}{\begin{pmatrix}
        x(t) \\ u(t) \end{pmatrix}{^\mathsf{T}}\begin{bmatrix} Q & S \\ S{^\mathsf{T}}& 
R \end{bmatrix} 
        \begin{pmatrix}
        x(t) \\ u(t) \end{pmatrix}}\,\mathrm{d}{t}} \right].$$ Then, for the standard case with $E = I_n$, $\left[\begin{smallmatrix} Q & S \\ S{^\mathsf{T}}& R \end{smallmatrix}\right]\geq 0$, and $R> 0$, the value function is given by $$W_+(x_0) = \operatorname{tr}\big(P^{\rm{a}}NN{^\mathsf{T}}\big),$$ where $P^{\rm{a}}$ solves the algebraic Riccati equation $$0 = P^{\rm{a}} A + 
A{^\mathsf{T}}P^{\rm{a}} + Q - (P^{\rm{a}} B + S){R^{-1}}(P^{\rm{a}} B + S){^\mathsf{T}},$$ which coincides with the Riccati equation arising from the discounted cost functional. Thus, one can probably show similar results as presented in this paper. Note that in this case, the value function only depends on the matrix $N$ and is independent of the initial condition. Moreover, the Lur’e equation and the KYP inequality have the same form as for the discounted functional.

The whole situation changes when considering DAEs with multiplicative noise, i.e., $${\mathrm{d}}Ex(t) = (A x(t) + Bu(t)) {\mathrm{d}}t + (Cx(t) + Du(t)) {\mathrm{d}}w(t) , \quad Ex(0) 
= Ex_0 ,$$ and with the cost functional $$J^{\rm m}(x,u,{\mathbb{R}}_{\ge 0}) = {\mathbb{E}}^0\left[ \int_{0}^{\infty} 
        \begin{pmatrix}
        x(t) \\ u(t) \end{pmatrix}{^\mathsf{T}}\begin{bmatrix} Q & S \\ S{^\mathsf{T}}& R 
        \end{bmatrix} 
        \begin{pmatrix}
        x(t) \\ u(t) \end{pmatrix}
        {\mathrm{d}}t \right],$$ where ${\mathbb{E}}^0[\cdot]:= {\mathbb{E}}[\cdot \,\vert\, \mathcal{F}_0]$ denotes the conditional expectation. For the standard case with (possibly) indefinite weight matrices in the cost functional, the optimal value is $$W^{\rm{m}}_+(x_0) = x_0{^\mathsf{T}}P^{\rm{m}} x_0 ,$$ where $P^{\rm{m}}$ solves the *bilinear* Riccati equation $$\begin{gathered}
    0 = P^{\rm{m}} A + A{^\mathsf{T}}P^{\rm{m}} + Q + C{^\mathsf{T}}P^{\rm{m}} C \\ - 
(BP^{\rm{m}} + S + D{^\mathsf{T}}P^{\rm{m}}C){^\mathsf{T}}(R+D{^\mathsf{T}}P^{\rm{m}} D)^{-1} 
(BP^{\rm{m}} + S + D{^\mathsf{T}}P^{\rm{m}}C), 
    \end{gathered}$$ where $R + D{^\mathsf{T}}P^{\rm{m}} D > 0$, see [@CLZ98] for further details. Thus, different linear matrix inequalities and Lur’e equations have to be derived and the theory developed for the deterministic case has to be extended. This is out of the scope of this paper and subject of future considerations.

Funding {#funding .unnumbered}
=======

L.-M. Pfurtscheller was supported by a scholarship of the Vizerektorat für Forschung, University of Innsbruck and by a scholarship of the German Academic Exchange Service (DAAD).

[^1]: CONTACT M. Voigt. Email: matthias.voigt@uni-hamburg.de
