---
abstract: 'We introduce a robust sensor design framework to provide defense against attackers that can bypass/hijack the existing defense mechanisms. For effective control, such attackers would still need to have access to the state of the system because of the presence of plant noise. We design “affine" sensor outputs to control their perception of the system so that their adversarial intentions would not be fulfilled or even inadvertently end up having a positive impact. The specific model we adopt is a Gauss-Markov process driven by a controller with a “private" malicious/benign quadratic control objective. We seek to defend against the worst possible distribution over the controllers’ objectives in a robust way. Under the solution concept of game-theoretic hierarchical equilibrium, we obtain a semi-definite programming problem equivalent to the problem faced by the sensor against a controller with an arbitrary, but known control objective even when the sensor has noisy measurements. Based on this equivalence relationship, we provide an algorithm to compute the optimal affine sensor outputs. Finally, we analyze the ensuing performance numerically for various scenarios.'
author:
- 'Muhammed O. Sayin and Tamer Başar,  [^1]'
bibliography:
- 'ref.bib'
title: Robust Sensor Design Against Multiple Attackers with Misaligned Control Objectives
---

=4

Stackelberg games, Stochastic control, Cyber-physical systems, Security, Advanced persistent threats, Sensor placement, Semi-definite programming.

Introduction
============

connectedness of physical systems makes them vulnerable against cyber attacks which could have undesirable physical outcomes, e.g., damages [@ref:Giraldo17; @ref:Humayed17]. Different from the vulnerability of computer systems in a cyber network, cyber connectedness of physical systems brings in new and distinct security challenges due to the inherent physical dynamics. Cyber attacks can be very strategic while disturbing the system to achieve certain malicious goal and therefore they differ from external disturbances that can be modeled, e.g., statistically or within certain bounds, for robust control system design. Cyber attacks can be advanced and very target specific by learning the system’s dynamics and tuning their attack specific to the underlying system and the existing defensive measures for success and stealthiness. To cite a recent occurrence of such an event, in 2014, Dragonfly Malware intervened the operation of many cyber-physical, e.g., process control, systems in energy and pharmaceutical industries across the world over a long period of time without being detected [@ref:Nelson16]. Therefore, it is crucial that we develop novel security mechanisms against such attackers for the security of cyber-physical systems.

Prior Literature
----------------

In a physical system, advance attackers can seek to evade detection mechanisms by manipulating the physical signals used by the detectors. For example, in [@ref:Liu09], the authors have introduced false data injection attacks, where the attackers can inject data into the sensor outputs, in the context of state estimation, and characterized undetectable attacks. The ensuing studies mainly focused on characterizing the vulnerabilities of control systems against such evasive attacks and designing counter measures to be able to detect them. In [@ref:Mo09], the authors have introduced replay attacks where the attacker records and replays the sensor outputs when the system is at steady state since they are expected to be similar. As a counter measure, an independent signal can be injected into the control input to detect such attacks at the expense of degradation in control performance [@ref:Mo09; @ref:Mo14]. In [@ref:Mo12; @ref:Mo16], the authors have characterized the reachable set that an evasive attacker can drive the system by injecting data into sensor outputs and control inputs jointly.

Within deterministic control scenarios, in [@ref:Teixeira12], the authors have analyzed open-loop stealthy attacks, and proposed to add new measurements as a counter measure, as in [@ref:Mo09]. In [@ref:Pasqualetti13], the authors have formulated the limitations of monitoring-based detection mechanisms against false data injection and replay attacks. In [@ref:Fawzi14], the authors have proposed decoding schemes to estimate the state based on sensor outputs while a subset of them could have been under attack. The attackers can also have adversarial control objectives. In [@ref:Chen16a; @ref:Chen16b], the authors have analyzed such attacks, where the attacker seeks to drive the state of the system according to his adversarial goal evasively by manipulating sensor outputs and control inputs jointly. In [@ref:Zhang17], the authors have analyzed optimal attack strategies to maximize the quadratic cost of a system with linear Gaussian dynamics without being detected. In [@ref:Miao17], the authors have proposed linear encoding schemes for sensor outputs of an LQG system in order to enhance detectability of false data injection attacks while the encoding matrix is assumed oblivious to the attackers.

Motivation
----------

Specifically, we address, in this paper, primarily the following two questions: “If we have already designed the sensor outputs, in a non-Bayesian setting, to what extent would we have secured the system against multiple type advanced and evasive attackers who can bypass/hijack the defensive measures to fulfill a certain malicious control objective?" And “what would be the best affine sensor outputs that can deceive such attackers about the underlying state of the system so that their actions/attacks would not lead to any degradation?" We consider attackers who have malicious control objectives misaligned with the normal operation of the system, but not completely opposite of it as in the framework of a zero-sum game. This implies that there is a part of the malicious objective that is benign. Correspondingly, the attacker would be acting in line with the normal operation of the system with respect to the aligned part of his objective. Our motivation is to restrain the attacker’s abilities so that he will not act along the misaligned part of the objectives while taking actions in line with the aligned part. To this end, we propose to design the information available to an attacker strategically since the attacker would be making decisions to fulfill his malicious objective based on the information available to him. By designing the sensor outputs strategically, our goal is to control the attacker’s perception about the underlying system, and correspondingly to persuade the attacker (without any explicit enforcement) to fulfill the aligned part of the objectives as much as possible without fulfilling the misaligned part.

We have partially addressed this challenge in [@ref:Sayin17b; @ref:Sayin17a; @ref:Sayin18a], in non-cooperative communication and control settings. For a discrete-time Gauss Markov process, and when the sender and the receiver in a non-cooperative communication setting have misaligned quadratic objectives, in [@ref:Sayin17b], we have shown the optimality of linear signaling rules within the general class of measurable policies and provided an algorithm to compute the optimal policies numerically. Also in [@ref:Sayin17b], we have formulated the optimal linear signaling rule in a non-cooperative LQG setting when the sensor and the controller have known misaligned control objectives. In [@ref:Sayin17a; @ref:Sayin18a], we have introduced a secure sensor design framework, where we addressed the optimal linear signaling rule again in a non-cooperative LQG setting when the sensor and private-type controller have misaligned control objectives in a Bayesian setting, i.e., the distribution over the private type of the controller is known. This paper differs from these earlier studies by addressing optimal affine signaling in a non-Bayesian setting, where the distribution over the private type of the controller is not known, in a robust way. Furthermore here we provide a comprehensive formulation by considering also the cases where the sensor could have partial or noisy information on the signal of interest and relevance. Further details on this will be given next as part of our description of the main contributions of this work, as well as throughout the paper.

Contributions
-------------

To obtain explicit results, we specifically consider systems with linear Gaussian dynamics and quadratic control objectives, which have various industrial applications [@ref:Zhang17] from manufacturing processes to aerospace control. We consider the possibility of adversarial intervention by multiple advanced and evasive attackers across control networks. The attackers have different long term control objectives. Due to the stochastic nature of the problem, i.e., due to the presence of state noise, any open-loop control strategy of an attacker could not drive the system along his desired path effectively. Therefore, regardless of whether the controller has an adversarial objective or not, it has to generate a closed-loop control input using the designed sensor outputs. We also consider the scenarios where the advanced attackers could learn the relationship between the sensor output and the state, i.e., the designed signaling rule[^2], in order to avoid any obscurity based defense, which can be bypassed once the advanced attacker learns the information in obscurity. This implies that the interaction between the sensor and the attackers could be modeled as a hierarchical dynamic game [@ref:Basar99], where the sensor leads the game by announcing its strategy in advance. Therefore, while designing the sensor outputs, we should consider the possibility of malicious or benign control inputs and defend against the worst possible distribution over them.

Specifically, we seek to determine optimal [*affine*]{} sensor strategies for controlled Gauss-Markov processes. The sensor can have partial or noisy measurements while in [@ref:Sayin17b; @ref:Sayin17a; @ref:Sayin18a] the sensor is considered to have access to the underlying state perfectly. We only consider affine signaling rules, since under such rules our setting entails a classical information model, whereas without such a structural restriction on the signaling rules, the underlying model features a non-classical information due to the asymmetry of information between the players and the dynamic interaction through closed-loop feedback signals. Different from our previous works [@ref:Sayin17b; @ref:Sayin17a; @ref:Sayin18a], here, the follower, i.e., the attacker, has a private type while the distribution over the types is not known by the leader, i.e., the sensor. Our goal is to defend against the worst possible distribution over these types. To this end, we provide an equivalent problem faced by the sensor in terms of the covariance of the posterior estimate of the (control-free) state by formulating the necessary and sufficient conditions on that covariance matrix. This new equivalent problem is linear in the optimization argument with a compact and convex constraint set.

We emphasize that what we have is an exact equivalence relation, and not like the equivalence in optimality, shown in [@ref:Sayin17b]. Based on this exact equivalence relation, we can provide an offline algorithm to compute the optimal affine sensor strategies. In particular, in order to determine the best signaling rule against multiple types of attackers, we introduce additional constraints on the equivalent problem, which implies that the equivalence in optimality is not a sufficient condition in that respect. We had noted in [@ref:Sayin17a; @ref:Sayin18a] for multiple attack types with [*known*]{} distribution over them that the optimum can be attained at the extreme points of the constraint set since the equivalent problem is linear in the optimization argument and the constraint set is compact and convex [@ref:Boyd04]. And, further the corresponding covariance matrices could be attained through certain [*linear*]{} signaling rules, where the sensor does not introduce any additional independent noise. However, when we defend against multiple types of attackers with the worst possible distribution over them as here, the new problem imposes additional linear constraints on the equivalent problem. With these new constraints, even though the optimum will be attained at the extreme points of this modified constraint set, there can be cases where the optimum may be attained only at non-extreme points of the original constraint set before the modification. Such covariance matrices could be attained through certain [*affine*]{} signaling rules, where the sensor can introduce additional independent noise.

We now list the main contributions of this paper as follows:

-   We introduce a robust sensor design agent that can craft the measurements sent to the noiseless communication network in order to defend against [*multiple*]{} advanced and evasive attackers with long term control objectives that are [*misaligned*]{} with the normal operation of the system.

-   We model the interaction between the sensor and the attackers, with private types, as a [*multi-stage Stackelberg game*]{}, where the sensor is the leader. Particularly, we suppose that the advanced attackers could be aware of the sensor’s strategies in order to avoid the vulnerability of obscurity based defenses.

-   We provide another problem [*equivalent*]{} to the problem faced by the sensor against any known type of attacker for compact computation of robust sensor outputs against the worst possible distribution over the attacker types.

-   We show the optimality of [*memoryless*]{} signaling rules within the general class of signaling rules with complete/bounded memory when the sensor has access to the underlying state of the system.

-   We show that certain affine signaling rules can lead to [*any*]{} covariance of the posterior estimate of the (control-free) state in between the extremes of disclosing nothing and disclosing the measurements fully according to a certain ordering of the covariance matrices.

-   We show that the optimal signaling rule dictates the sensor to possibly introduce additive independent noise into the sensor outputs if there are multiple types of attackers in a non-Bayesian setting.

-   We provide a numerical algorithm to compute the [*optimal*]{} affine robust sensor design strategies.

-   We extend the results to the cases where the sensor has partial or [*noisy*]{} information on the signal of interest and relevance.

-   We examine the performance of the proposed framework for various scenarios, displaying the significant impact of strategic design of sensor outputs on the system’s performance.

The paper is organized as follows: In Section \[sec:prob\], we formulate the robust sensor design game. In Section \[sec:robust\], we analyze the equilibrium of the robust sensor design game under perfect measurements. In Section \[sec:noisy\], we extend the results to the cases where there are partial or noisy measurements. In Section \[sec:examples\], we examine numerically the performance of the proposed scheme for various scenarios. We conclude the paper in Section \[sec:conclusion\] with several remarks and possible research directions. An appendix provides two technical results in support of some analyses in the main body of the paper.

[*Notation:*]{} For an ordered set of parameters, e.g., $x_1,\ldots,x_{\kappa}$, we define $x_{k:l} := x_k,\ldots,x_l$, where $1\leq k \leq l \leq \kappa$. $\N(0,.)$ denotes the multivariate Gaussian distribution with zero mean and designated covariance. We denote random variables by bold lower case letters, e.g., $\rx$. For a random vector, e.g., $\rx$, $\cov\{\rx\}$ denotes the corresponding covariance matrix. For a vector $x$ and a matrix $A$, $x'$ and $A'$ denote their transposes, and $\|x\|$ denotes the Euclidean ($L^2$) norm of the vector $x$. For a matrix $A$, $\trace\{A\}$ denotes its trace. We denote the identity and zero matrices with the associated dimensions by $I$ and $O$, respectively. For positive semi-definite matrices $A$ and $B$, $A\succeq B$ means that $A-B$ is also a positive semi-definite matrix. $\AS^m$ (or $\AS^m_+$) denotes the set of symmetric (or positive definite) matrices of dimensions $m$-by-$m$. $A\otimes B$ denotes the Kronecker product of the matrices $A$ and $B$.

[New\_model.pdf]{} (7,22)[$\rx_{k+1} = A\rx_k + B\ru_k + \rw_k$]{} (42,15)[$\ru_k= \gamma_k(\rs_{1:k})$]{} (13,48)[$\ry_k$]{} (27,38) [$\ry_{1:k-1}$]{} (41,48)[$\rs_k = \eta_k(\ry_{1:k})$]{}

Problem Formulation {#sec:prob}
===================

Consider a cyber-physical control system, seen in Fig. \[fig:model\], whose underlying state dynamics and sensor measurements are described, respectively, by: $$\begin{aligned}
&\rx_{k+1} = A\rx_k + B\ru_k + \rw_k, \label{eq:state}\\
&\ry_k = C\rx_k + \rv_k, \label{eq:measurement}\end{aligned}$$ for $k=1,\ldots,\kappa$, where[^3] $A\in\R^{m\times m}, B\in\R^{m\times r}$, and $C\in\R^{m\times m}$, and the initial state $\rx_1\sim\N(0,\Sigma_1)$. The additive state and measurement noise sequences $\{\rw_k\}$ and $\{\rv_k\}$, respectively, are white Gaussian vector processes, i.e., $\rw_k \sim \N(0,\Sigma_w)$ and $\rv_k\sim\N(0,\Sigma_v)$; and are independent of the initial state $\rx_1$ and of each other. As seen in Fig. \[fig:model\], the signal $\rs_k\in\R^{m}$, which can be different from the measurement $\ry_k\in\R^m$, is given by the affine signaling rule: $$\begin{aligned}
\label{eq:signal}
\rs_k &= \eta_k(\ry_{1:k})\\
&= L_{k,k}'\ry_k + \ldots + L_{k,1}'\ry_{1} + \rn_k,\end{aligned}$$ where $L_{k,j}\in\R^{m\times m}$, $j=1,\ldots,k$, can be any deterministic matrix, and $\rn_k \sim \N(0,\Theta_k)$ is independent of every other parameter. Let ${\Upsilon}_k$ denote the set of such affine signaling rules from $\R^{mk}$ to $\R^m$, i.e., $\eta_k\in{\Upsilon}_k$. Furthermore, the closed-loop control input $\ru_k\in\R^r$, which is constructed by the controller located in the cyber-part of the system, is given by $$\label{eq:control}
\ru_k = \gamma_k(\rs_{1:k}),$$ almost everywhere over $\R^r$, where $\gamma_k(\cdot)$ can be any Borel measurable function from $\R^{mk}$ to $\R^r$. Let $\Gamma_k$ denote the set of all Borel measurable functions from $\R^{mk}$ to $\R^r$, i.e., $\gamma_k\in\Gamma_k$.

Particularly, the dynamic system, measurement system, and actuators are located in the physical part while the controller is located in the cyber part through a connection over a noiseless communication channel. However, the connectivity over the channel is vulnerable to cyber attacks. We consider the scenarios where advanced and evasive attackers can intervene through the channel by injecting malicious control inputs to drive the underlying state according to a malicious long term control objective and can bypass or hijack the detection-based defensive measures. Our goal is to produce defense against such advanced evasive attacks by [*crafting*]{} the attacker’s perceptions about the underlying state of the system strategically so that their actions/attacks would not be harmful (to the extent possible) and would be along the desired objectives. To this end, we introduce a robust-secure-sensor-design component, denoted by , in the physical plant that gets the sensor measurement $\ry_k$ as an input and constructs the signal $\rs_k$, given by , and feeds this signal to the noiseless communication channel. Even if there is an attacker intervening over the channel, that attacker can only access that signal $\rs_k$ related to the underlying state of the system.

Note that if the signal $\rs_k$ is memoryless, then $\rs_k\in\R^m$ can be written as $$\begin{aligned}
\rs_k &= \underbrace{L_{k,k}'C}\rx_k + \underbrace{L_{k,k}'\rv_k + \rn_k}\\
&= \tC_k\rx_k + \rtv_k ,\end{aligned}$$ where $\rtv_k\sim\N(0,L_{k,k}'\Sigma_vL_{k,k} + \Theta_k)$, such that $\tC_k\in\R^{m\times m}$ is the gain matrix in the measurement while $\rtv_k$ is the white Gaussian measurement noise. Therefore, the optimal signaling rules $\eta_{1:\kappa}^*$ can provide a guideline to engineer (or designer) the placement of the physical sensors to monitor the underlying state of the system securely or to assess the resiliency against attacks with long-term control objectives. $\triangle$

The normal operation of the system, i.e., when there is no adversarial intervention by the attackers, is a stochastic control setting, where the controller, denoted by , constructs the control inputs $\ru_k\in\Gamma_k$ to minimize a finite horizon quadratic cost function given by $$\label{eq:Cobj}
\E\left\{\sum_{k=1}^{\kappa} \|\rx_{k+1}\|_Q^2 + \|\ru_k\|_R^2\right\},$$ where[^4] $Q\in\AS^m$ is positive-semi definite and $R\in\AS_+^r$ is positive definite.  selects the signaling rule $\eta_{1:\kappa}$ to minimize the same cost function with , i.e., , as a team. And if there were no possibility for an adversarial intervention,  could be as informative as disclosing the measurement directly to  since there is no cost for the disclosed information over the noiseless communication channel. However, as seen in Fig. \[fig:model\], there can be advanced and evasive adversarial interventions in the noiseless communication channel between the dynamic system and . We particularly consider multiple (finitely many) types of attackers who can inject malicious closed-loop control inputs with long-term control objectives. Let type-$\alpha$ attacker’s cost function be given by $$\label{eq:AAobj}
\E\left\{\sum_{k=1}^{\kappa} \|\rx_{k+1}\|_{Q_{\alpha}}^2 + \|\ru_k\|_{R_{\alpha}}^2\right\},$$ where $Q_{\alpha}\in\AS^m$ is positive semi-definite and $R_{\alpha}\in\AS^r$ is positive-definite.

$$\begin{aligned}
&U_{\playerC}^{\omega}(\eta_{1:\kappa},\gamma_{1:\kappa}^{\omega}) := \E\Bigg\{\sum_{k=1}^{\kappa}\|\rx_{k+1}^{\omega}\|_{Q_{\omega}}^2 + \|\gamma_k^{\omega}\big(\eta_k(\ry_{1:k}^{\omega}),\ldots,\eta_1(\ry_1^{\omega})\big)\|_{R_{\omega}}^2\Bigg\}\label{eq:typeCobj}\\
&U_{\playerS}(\eta_{1:\kappa},\{\gamma_{1:\kappa}^{\omega}\}_{\omega\in\Omega},p) := \sum_{\omega\in\Omega} p_{\omega} \E\Bigg\{\sum_{k=1}^{\kappa}\|\rx_{k+1}^{\omega}\|_{Q}^2 + \|\gamma_k^{\omega}\big(\eta_k(\ry_{1:k}^{\omega}),\ldots,\eta_1(\ry_1^{\omega})\big)\|_{R}^2\Bigg\}\label{eq:Sobj}\\
&U_{\playerA}(\eta_{1:\kappa},\{\gamma_{1:\kappa}^{\omega}\}_{\omega\in\Omega},p) := -\sum_{\omega\in\Omega} p_{\omega} \E\Bigg\{\sum_{k=1}^{\kappa}\|\rx_{k+1}^{\omega}\|_{Q}^2 + \|\gamma_k^{\omega}\big(\eta_k(\ry_{1:k}^{\omega}),\ldots,\eta_1(\ry_1^{\omega})\big)\|_{R}^2\Bigg\}\label{eq:Aobj}\end{aligned}$$

------------------------------------------------------------------------

The attacker objective also covers the special cases where the attacker seeks to regularize the underlying state $\{\rx_k\}$ around an external process, e.g., $\{\rz^{\alpha}_k\}$, rather than the zero vector. To this end, we can consider the augmented state vector $\begin{psmallmatrix} \rx_k' & (\rz^{\alpha}_k)' \end{psmallmatrix}'$ and set the associated weight matrix in the regularization, i.e., $Q_{\alpha}$ in , accordingly. $\triangle$

Since our aim is to defend against advanced and evasive attackers, we consider the scenarios where there exists a hierarchy between the defender, i.e., , and the attackers such that each type of attacker is aware of ’s signaling rules $\eta_{1:\kappa}$ by testing and learning the system’s dynamics once they are deployed publicly. Different from [@ref:Sayin17b; @ref:Sayin17a; @ref:Sayin18a], in this paper,  does not know the underlying distribution governing the attackers’ types and seeks to defend against the worst possible distribution. Particularly,  designs the secure sensor outputs such that ’s cost is minimized in expectation with respect to the worst possible [*true*]{} distribution of types within a robust setting.

This can be viewed as a game, where  designs the signaling rule $\eta_{1:\kappa}$ within a hierarchical setting, where the controllers,  with different benign/malicious types, and the adversary (), determining the distribution of types, are aware of the signaling rules. Therefore,  anticipates reactions of different types of controllers and the worst possible distribution over those types while selecting the signaling rule $\eta_{1:\kappa}$ to minimize . $\triangle$

Game Model
----------

We consider a game with three players: , , and .  can have different private types. Let $\Omega$ denote the finite set of all (benign/malicious) controller types. Correspondingly, depending on the type $\omega\in\Omega$,  selects the control rule $\gamma_{k}^{\omega}\in\Gamma_k$.  designs the signaling rule $\eta_{1:\kappa}$ to minimize the expected cost, where the expectation is taken over all the randomness (due to the initial state, and state and measurement noises), and the distribution of types, determined by . Let $p:=\{p_{\omega}\}_{\omega\in\Omega}$ denote the probabilities of types $\omega\in\Omega$. Then, type-$\omega$ ’s, ’s, and ’s cost functions are given by , , and , respectively, where we represent the dependence of the state on the controller’s type, $\omega$, due to the control input in the state recursion explicitly through $\rx_k^{\omega}$ (and the signal $\ry_k^{\omega}$) instead of $\rx_k$ (and $\ry_k$). Then, the robust sensor design game is defined as follows:

The robust sensor design game $$\calG := \left({\Upsilon}_{1:\kappa},\Gamma_{1:\kappa},\Delta^{|\Omega|},\rx_1,\rw_{1:\kappa},\rv_{1:\kappa}\right)$$ is a multi-stage Stackelberg game [@ref:Basar99] between , , and , with the following parameters:

-   $\kappa\in\Z$: denotes the number of stages, i.e., length of the horizon,

-   $\rx_1\sim\N(0,\Sigma_1)$: denotes the initial state,

-   $\{\rw_k\sim\N(0,\Sigma_w)\}$: denotes the white state noise process independent of other parameters,

-   $\{\rv_k\sim\N(0,\Sigma_v)\}$: denotes the white measurement noise process independent of other parameters.

In this hierarchical setting,  is the leader, who announces (and commits to) his strategies beforehand, while  and  are the followers, reacting to the leader’s announced strategies. ’s type is drawn according to ’s action $p\in\Delta^{|\Omega|}$ and his strategy space is $\Gamma_k$ at stage $k$. ’s strategy space is ${\Upsilon}_k$ at each stage $k$ and ’s action space is the simplex $\Delta^{|\Omega|}$. Objectives of , , and  are given by , and , respectively. The tuple of strategies $(\eta_{1:\kappa}^*,\{\gamma_{1:\kappa}^{\omega*}\}_{\omega\in\Omega},p^*)$ attains the Stackelberg equilibrium provided that

\[eq:equilibrium\] $$\begin{aligned}
&\eta_{1:\kappa}^* = \argmin\limits_{\substack{\eta_k\in{\Upsilon}_k\\k=1,\ldots,\kappa}} U_{\playerS}\big(\eta_{1:\kappa},\{\gamma_{1:\kappa}^{\omega*}(\eta_{1:\kappa})\}_{\omega\in\Omega},p^*(\eta_{1:\kappa})\big)\\
&\gamma_{1:\kappa}^{\omega*}(\eta_{1:\kappa}) = \argmin\limits_{\substack{\gamma_k^{\omega}\in\Gamma_k\\ k =1,\ldots,\kappa}} U_{\playerC}^{\omega}\big(\eta_{1:\kappa},\gamma_{1:\kappa}^{\omega}(\eta_{1:\kappa})\big),\label{eq:gammaStar}\\
&p^*(\eta_{1:\kappa}) = \argmin_{p\in\Delta^{|\Omega|}} U_{\playerA}\big(\eta_{1:\kappa},\{\gamma_{1:\kappa}^{\omega*}(\eta_{1:\kappa})\}_{\omega\in\Omega},p(\eta_{1:\kappa})\big)\end{aligned}$$

where, with abuse of notation, we denote type-$\omega$ ’s strategy $\gamma_k^{\omega}$ by $\gamma_{k}^{\omega}(\eta_{1:k})$ to show the dependence of type-$\omega$ ’s strategies on ’s signaling rules due to the hierarchy, explicitly, and we define $\gamma_{1:\kappa}^{\omega}(\eta_{1:\kappa}) := \{\gamma_1^{\omega}(\eta_1),\ldots,\gamma_{\kappa}^{\omega}(\eta_{1:\kappa})\}$.

The reaction set of type-$\omega$  is an equivalence class such that all $\gamma_{1:\kappa}^{\omega*}$ in the reaction set lead to the same control input $\ru_k^{\omega*}$ almost surely under certain convexity assumptions, e.g., $R_{\omega}\succ O$ is positive-definite, which will be shown in detail later. Furthermore, the reaction set of  is also an equivalence class due to the zero-sum relation between the cost functions $U_{\playerS}$ and $U_{\playerA}$. $\triangle$

Given ’s signaling rule, ’s objective $U_{\playerC}^{\omega}$, for $\omega\in\Omega$, is [*decoupled*]{} from the other follower ’s action while ’s objective $U_{\playerA}$ depends on different type ’s strategies. Therefore, this can be viewed as a sequential optimization between the two followers, where firstly  selects his strategy optimizing his objective given the leader’s strategy, and then  takes action corresponding to the leader’s strategy and the ’s optimal reaction to optimize his objective . $\triangle$

Robust Sensor Design Framework {#sec:robust}
==============================

We first assume, in this section, that  has access to perfect measurements, i.e., $\ry_k = \rx_k$ for $k=1,\ldots,\kappa$; the general noisy/partial measurements case will be addressed later in Section \[sec:noisy\]. Then, given $p\in\Delta^{|\Omega|}$,  faces the following optimization problem: $$\min\limits_{\substack{\eta_k\in{\Upsilon}_k\\k=1,\ldots,\kappa}} U_{\playerS}(\eta_{1:\kappa},\{\gamma_{1:\kappa}^{\omega*}(\eta_{1:\kappa})\}_{\omega\in\Omega},p),$$ where $\gamma_{1:\kappa}^{\omega*}(\eta_{1:\kappa})$ is given by . This is a highly nonlinear and non-convex problem. However, the following theorem provides an equivalent semi-definite programming (SDP) problem, which is not limited to equivalence in optimality as in [@ref:Sayin17b], so that we can address the equilibrium for the robust sensor design game $\calG$ in a compact way.

\[theorem:equivalent\] Let the convex and compact set $\Psi$ be defined as $$\label{eq:Psi}
\Psi := \left\{(S_k\in\AS^m)_{k=1}^{\kappa}\,|\, \Sigma_k^o \succeq S_k \succeq AS_{k-1}A', k=1,\ldots,\kappa, S_0 = O\right\},$$ where $\Sigma_k^o\in\AS^m$ is the covariance matrix of the control-free process: $$\rx_{k+1}^o = A\rx_k^o + \rw_k\mbox{ and }\rx_1^o = \rx_1,\label{eq:free}$$ i.e., $\rx_k^o\sim\N(0,\Sigma_k^o)$ and[^5] $\Sigma_k^o := \E\{\rx_k^o(\rx_k^o)'\}$. Then, given $p\in\Delta^{|\Omega|}$, for any signaling rule $\eta_{k}\in{\Upsilon}_k$, for $k=1,\ldots,\kappa$, there exists $S_{1:\kappa}\in\Psi$ such that

\[eq:toLeft\] $$\begin{aligned}
&U_{\playerS}(\eta_{1:\kappa},\{\gamma_{1:\kappa}^{\omega*}(\eta_{1:\kappa})\}_{\omega\in\Omega},p)\label{eq:left} \\
&\hspace{.4in}= \sum_{k=1}^{\kappa} \trace\left\{S_k\left(\sum_{\omega\in\Omega}p_{\omega}V_k(\omega)\right)\right\} + v_o,  \label{eq:right}\end{aligned}$$

where $V_{1:\kappa}(\omega)$ and $v_o$ are deterministic parameters that do not depend on $S_{1:\kappa}$ and derived in Appendix \[app:V\]. Furthermore, for any $S_{1:\kappa}\in\Psi$, there exists a signaling rule $\eta_{1:\kappa}$ such that $$\begin{aligned}
\label{eq:toRight}
\sum_{k=1}^{\kappa} \trace\left\{S_k\left(\sum_{\omega\in\Omega}p_{\omega}V_k(\omega)\right)\right\} + v_o= U_{\playerS}(\eta_{1:\kappa},\{\gamma_{1:\kappa}^{\omega*}(\eta_{1:\kappa})\}_{\omega\in\Omega},p). \end{aligned}$$

The proof proceeds as follows: $i)$ we first show that the optimization function can be written as a linear function of the covariance of the posterior control-free state, i.e., $$H_k := \cov\{\E\{\rx_k^o|\rs_{1:k}\}\},$$ for $k=1,\ldots,\kappa$; $ii)$ we then identify the necessary condition on $H_{1:\kappa}$, which is indeed the constraint set $\Psi$, i.e., $H_{1:\kappa}\in\Psi$; and finally $iii)$ we show that the constraint set $\Psi$ is also a sufficient condition for $H_{1:\kappa}$ since any point in $\Psi$ can be attained through certain affine signaling rule. We now provide details of each of these steps.

[*Step $i)$*]{} Our first goal is to isolate the underlying state from the control input by completing the squares in the cost functions and through change of variables, which yields the result that given positive semi-definite $Q_{\omega}\in\AS^{m}$ and positive definite $R_{\omega}\in\AS^{r}$ corresponding to the type-$\omega$ controller, we have $$\E\left\{\sum_{k=1}^{\kappa}\|\rx_{k+1}\|_{Q_{\omega}}^2 + \|\ru_k\|_{R_{\omega}}^2\right\} = \sum_{k=1}^{\kappa} \E\left\{\|\ru_k^o + K_k^{\omega} \rx_k^o\|_{\Delta_k^{\omega}}^2\right\} + \Delta_0^{\omega},\label{eq:control-free}$$ where $K_k^{\omega}\in\R^{r\times m}, \Delta_k^{\omega}\in\AS_{+}^m$, for $k=1,\ldots,\kappa$, and $\Delta_0^{\omega}\in\R$ are derived in Appendix \[app:computationA\]; and $\{\rx_k^o\}$ is the control-free process defined in and $$\begin{aligned}
\ru_k^o = \ru_k + K_k^{\omega}B\ru_{k-1} + \ldots + K_k^{\omega}A^{k-2}B\ru_1.\label{eq:transformed}\end{aligned}$$

Different from the cooperative settings, does [*not*]{} imply that the optimal transformed control input is $\ru_k^o = -K_k^{\omega}\E\{\rx_k^o|\rs_{1:k}\}$ since $\rs_k$ depends on previous control inputs $\ru_{1:k-1}$, as also pointed out in [@ref:Sayin17b]. Therefore, we cannot claim optimality of linear signaling rules within the general class of all measurable policies. $\triangle$

However, for affine signaling rules, e.g., $\rs_k = L_{k,k}'\rx_k + \ldots + L_{k,1}'\rx_{1}+ \rn_k$, we have $$\begin{aligned}
\E\{\rx_k^o | L_{1,1}'\rx_1 + \rn_1,\cdots,L_{k,k}'\rx_k + \ldots + L_{k,1}'\rx_{1}+\rn_k\} = \E\{\rx_k^o|L_{1,1}'\rx_1^o+\rn_1,\cdots,L_{k,k}'\rx^o_k + \ldots + L_{k,1}'\rx^o_{1}+\rn_k\},\label{eq:condexp}\end{aligned}$$ which holds since the signal $\rs_l$ for $l=1,\ldots,k$ can be written as $$\begin{aligned}
\label{eq:measurable}
L_{l,l}'\rx_l + \ldots + L_{l,1}'\rx_{1} + \rn_l = L_{l,l}'\rx_l^o + \ldots + L_{l,1}'\rx_{1}^o + \rn_l + \Big\{L_{l,l}'B\ru_{l-1}+\ldots+(L_{l,l}'A^{l-2} + \ldots + L_{l,2}')B\ru_{1}\Big\},\end{aligned}$$ where the term in-between $\{\cdot\}$ is $\sigma$-$\rs_{1:l-1}$ measurable since the control input is given by . This yields that the optimal transformed control input is given by $\ru_k^o = - K_k^{\omega}\E\{\rx_k^o|\rs_{1:k}\}$ since $\E\{\rx_k^o|\rs_{1:k}\}$ does not depend on $\ru_{1:k}$. Then, the corresponding optimal (original) control input $\ru_k^*$ can be computed based on and $\ru_k^*$ is linear in $\E\{\rx_k^o|\rs_{1:k}\}$ while $\E\{\rx_k^o|\rs_{1:k}\}$ does not depend on the type of the controller.

The result in Step $i)$ would also hold if the controllers have objectives (other than ), e.g., additional certain soft constraints, leading to optimal control inputs that are linear functions of $\E\{\rx_k^o|\rs_{1:k}\}$. $\triangle$

Due to the linearity of the controllers’ reactions in $\E\{\rx_k^o|\rs_{1:k}\}$, the quadratic objective can be written as $$\label{eq:short}
\sum_{k=1}^{\kappa} \trace\left\{H_k \left(\sum_{\omega\in\Omega}p_{\omega}V_k(\omega)\right)\right\} + v_o$$ where $V_{1:\kappa}(\omega)\in\AS^m$, for $\omega\in\Omega$, and $v_o\in\R$ are derived in Appendix \[app:V\].

[*Step $ii)$*]{} The covariance of the posterior control-free state $H_k\in\AS^m$ can be in-between two extremes: $\Sigma_k^o$ corresponding to full disclosure of the state, i.e., $\rs_k=\rx_k$; and $\E\{\E\{\rx_k^{o}|\rs_{1:k-1}\}\E\{\rx_k^{o}|\rs_{1:k-1}\}'\} = AH_{k-1}A'$ corresponding to sharing nothing, i.e., $\rs_k = 0$ [@ref:Sayin17b]. The inequality $$\Sigma_k^o\succeq H_k \succeq AH_{k-1}A'\label{eq:sub}$$ follows from the following covariance matrices: $$\begin{aligned}
&\cov\{\rx_k^o - \E\{\rx_k^o|\rs_{1:k}\}\} = \Sigma_k^o - H_k\succeq O,\\
&\cov\{\E\{\rx_k^o|\rs_{1:k}\} - \E\{\rx_k^o|\rs_{1:k-1}\}\} = H_k - AH_{k-1}A'\succeq O,\end{aligned}$$ since for arbitrary random variables $\ra$ and $\rb$, $$\begin{aligned}
\E\{\ra\E\{\ra|\rb\}\} = \E\{\E\{\ra|\rb\}\E\{\ra|\rb\}\}.\end{aligned}$$ We then arrive at based on .

[*Step $iii)$*]{} In order to show , we will be using the following lemma from [@ref:Sayin18e] to address the cases when $\Sigma_k^o - A\Sigma_{k-1}^oA' = \Sigma_w\succeq O$ can be singular.

\[lem:outsider0\] If we can partition a positive semi-definite matrix into blocks such that a block at the diagonal is a zero matrix, then we have $$\begin{bmatrix} A & B \\ B' & O \end{bmatrix} \succeq O \Leftrightarrow A\succeq O \mbox{ and } B = O.$$

Based on Lemma \[lem:outsider0\], the following lemma shows that any point in $\Psi$ can be attained by a certain affine signaling rule.

\[lem:affine\] Consider any $S_{1:\kappa}\in\Psi$, and let $$\Sigma_k^o -AS_{k-1}A' = \bU_k\begin{bmatrix} \bLambda_k & O \\ O & O\end{bmatrix}\bU_k'$$ be the eigen-decomposition such that $\bLambda_k \succ O$. Let $$\begin{aligned}
T_k:= \begin{bmatrix} \bLambda_k^{1/2}& O \end{bmatrix} \bU_k' (S_k-AS_{k-1}A')\bU_k \begin{bmatrix} \bLambda_k^{1/2} \\ O \end{bmatrix}\end{aligned}$$ have the eigen-decomposition $T_k = U_k\Lambda_kU_k'$ with eigenvalues, e.g.,[^6] $\lambda_{k,1},\ldots,\lambda_{k,t_k} \in [0,1]$, where $t_k = \rank\{\Sigma_k^o -AS_{k-1}A'\}$. Then, there exists a memoryless affine signaling rule $$\label{eq:affineSolution}
\ry_k = L_k' \rx_k + \rn_k,\;\mbox{for}\; k=1,\ldots,\kappa,$$ where $\rn_k\sim\N(0,\Theta_k)$ and $\Theta_k = \mathrm{diag}\{\theta_{1,1}^2,\ldots,\theta_{1,t_k}^2,0,\ldots,0\}$, $L_k$ is given by $$L_k = \bU_k\begin{bmatrix} \bLambda_k^{-1/2} U_k\Lambda_k^o & O \\ O & O \end{bmatrix}$$ where $\Lambda_k^o := \mathrm{diag}\{\lambda_{1,1}^o,\ldots,\lambda_{1,t_k}^o\}$ and $$\frac{(\lambda_{k,i}^o)^2}{(\lambda_{k,i}^o)^2 + \theta_{k,i}^2} = \lambda_{k,i} \in [0,1], \forall\, i=1.\ldots,t_k,$$ which leads to $S_{1:\kappa} = H_{1:\kappa}$.

The proof follows by induction. If $S_{1:\kappa}\in\Psi$, then $S_1\in\AS^m$ satisfies $$\label{eq:subConst1}
\Sigma_1^o \succeq S_1 \succeq O,$$ where $\Sigma_1^o\succeq O$ can be singular. Let $\Sigma_k^o = \bU_1 \begin{bmatrix} \bLambda_{1} & O \\ O & O\end{bmatrix}\bU_1'$ be the eigen-decomposition such that $\bLambda_{1} \succ O$. Then, we have $$\begin{bmatrix} \bLambda_{1} & O \\ O & O\end{bmatrix} \succeq \bU_1' S_1 \bU_1 \succeq O,$$ which implies that $$\begin{bmatrix} \bLambda_{1} & O \\ O & O\end{bmatrix} - \begin{bmatrix} M_{1,1} & M_{1,2} \\ M_{2,1} & M_{2,2} \end{bmatrix} \succeq O,\label{eq:this}$$ where we let $\bU_1' S_1 \bU_1 = \begin{bmatrix} M_{1,1} & M_{1,2} \\ M_{2,1} & M_{2,2} \end{bmatrix}$ be the corresponding partitioning. Note that since $\bU_1' S_1 \bU_1 \succeq O$, we have $M_{2,2}\succeq O$ [@ref:Horn85]. However, the bottom-right block of the positive semi-definite matrix (the whole term) on the left-hand-side of the inequality , i.e., $-M_{2,2}$, must also be a positive semi-definite matrix, which implies $O\succeq M_{2,2}$. Therefore we have $M_{2,2} = O$ and Lemma \[lem:outsider0\] yields that there exists a symmetric matrix $T_1\in\AS^{t_1}$, where $t_1:= \rank\{\Sigma_1^o\}$, such that $$\label{eq:ST}
S_1 = \bU_1 \begin{bmatrix} \bLambda_{1}^{1/2} T_1 \bLambda_{1}^{1/2} & O \\ O & O \end{bmatrix}\bU_1'.$$ Note that there exists a bijective relation between $S_1\in\AS^m$ and $T_1\in\AS^{t_1}$. Furthermore, and imply that $$I \succeq T_1 \succeq O,$$ and $T_1\in\AS^m$ has eigenvalues in the closed interval $[0,1]$ since the eigenvalues of $I$, i.e., the vector $\mathbf{1}\in\R^{t_1}$, weakly majorizes the eigenvalues of $T_1$ from below [@ref:Horn85]. Let $T_1 = U_1\Lambda_1U_1'$ be the eigen-decomposition and $\lambda_{1,1},\ldots,\lambda_{1,t_1}\in[0,1]$ be the associated eigenvalues.

Furthermore, consider the affine signaling rule $\rs_1 = L_1'\rx_1 + \rn_1$, where $\rn_1\sim\N(0,\Theta_1)$ is independent of all the other parameters. Then, the covariance of the posterior control-free state is given by $$H_1 = \Sigma_1^oL_1(L_1'\Sigma_1^oL_1 + \Theta_1)^{\dagger}L_1'\Sigma_1^o.$$ If we set $L_1 = \bU_1 \begin{bmatrix} \bLambda^{-1/2}_1U_1\Lambda_1^o & O \\ O & O \end{bmatrix}$ and $\Theta_1 \succeq O$ such that $$\Lambda_1^o := \begin{bmatrix} \lambda_{1,1}^o & & \\ & \ddots& \\ & & \lambda_{1,t_1}^o\end{bmatrix}, \Theta_1 := \begin{bmatrix} \theta_{1,1}^2 & & & \\  &\ddots & & \\ & & \theta_{1,t_1}^2 & \\ & & & \ddots \end{bmatrix}$$ and $$\frac{(\lambda_{1,i}^o)^2}{(\lambda_{1,i}^o)^2 + \theta_{1,i}^2} = \lambda_{1,i} \in [0,1], \mbox{ for } i=1,\ldots,t_1,$$ then, we would obtain $H_1 = S_1$ exactly.

Suppose that $H_j = S_j$ for $j<k$. Then, $S_k\in\AS^m$ satisfies $$\Sigma^o_k \succeq S_k \succeq AS_{k-1}A',$$ which is equivalent to $$\label{eq:asd}
\Sigma^o_k \succeq S_k \succeq AH_{k-1}A'.$$ Correspondingly, $\Sigma_k^o - AH_{k-1}A'\succeq O$ can be singular. Let $\Sigma_k^o - AH_{k-1}A' = \bU_k \begin{bmatrix} \bLambda_k & O \\ O & O\end{bmatrix}\bU_k'$ be the eigen-decomposition such that $\bLambda_k \succ O$. Then, we have $$\begin{bmatrix} \bLambda_k & O \\ O & O \end{bmatrix} \succeq \bU_k'(S_k-AH_{k-1}A')\bU_k \succeq O$$ and correspondingly Lemma \[lem:outsider0\] yields that there exists a symmetric matrix $T_k\in\AS^{t_k}$, where $t_k := \rank\{\Sigma_k^o - AH_{k-1}A'\}$, such that $$\label{eq:asd2}
S_k = AH_{k-1}A' + \bU_k \begin{bmatrix} \bLambda_k^{1/2} T_k \bLambda_k^{1/2} & O \\ O & O \end{bmatrix}\bU_k'.$$ Furthermore, and yield that $$I\succeq T_k \succeq O,$$ which implies that $T_k\in\AS^{t_k}$ has eigenvalues in the closed interval $[0,1]$. Let $T_k = U_k\Lambda_kU_k'$ be the eigen decomposition and $\lambda_{k,1},\ldots,\lambda_{k,t_k}\in[0,1]$ be the associated eigenvalues.

Furthermore, for the affine signaling rule $\rs_k = L_k'\rx_k + \rn_k$, where $\rn_k\sim\N(0,\Theta_k)$ is independent of all the other parameters, the covariance of the posterior control-free state is given by $$\begin{aligned}
H_k = AH_{k-1}A' + (\Sigma_k^o - AH_{k-1}A')L_k(L_k'(\Sigma_k^o - AH_{k-1}A')L_k + \Theta_k)^{\dagger}L_k'(\Sigma_k^o - AH_{k-1}A'),\end{aligned}$$ which follows since $$\begin{aligned}
\cov\{\E\{\rx_k^o|\rs_{1:k}\}\} = \cov\{\E\{\rx_k^o|\rs_{1:k-1}\}\} + \cov\{\E\{\rx_k^o | \rs_k - \E\{\rs_k|\rs_{1:k-1}\}\}\},\end{aligned}$$ due to the independence of the jointly Gaussian $\rs_{1:k-1}$ and $\rs_k-\E\{\rs_k|\rs_{1:k-1}\}$. If we set $L_k =  \bU_k\begin{bmatrix} \bLambda_k^{-1/2}U_k\Lambda_k^o & O \\ O & O \end{bmatrix}$ and $\Theta_k\succeq O$ such that $$\frac{(\lambda_{k,i}^o)^2}{(\lambda_{k,i}^o)^2 + \theta_{k,i}^2} = \lambda_{k,i} \in [0,1], \mbox{ for } i=1,\ldots,t_k,$$ then, we would obtain $H_k = S_k$ exactly. Therefore, by induction, we conclude that for any $S_{1:\kappa}\in\Psi$, there exists a certain affine signaling rule such that $H_k = S_k$ for $k=1,\ldots,\kappa$.

When  has perfect measurements, i.e., $\ry_k = \rx_k$, then the optimal signaling rules can be memoryless affine policies within the general class of affine policies with complete/bounded memory. $\triangle$

Lemma \[lem:affine\] implies the equality at , which completes the proof of Theorem \[theorem:equivalent\].

Henceforth, we will be working with instead of while analyzing the equilibrium of the game $\calG$.

[*New notation for compact presentation:*]{} Let $$S := \begin{bmatrix} S_{\kappa} & & \\ & \ddots & \\ & & S_1 \end{bmatrix},\; V(\omega) := \begin{bmatrix} V_{\kappa}(\omega) & & \\ & \ddots & \\ & & V_1(\omega) \end{bmatrix},$$ and $\bPsi \in \AS^{m\kappa}$ be the set corresponding to the constraint set $\Psi$ in this new high dimensional space, i.e., $\R^{m\kappa \times m\kappa}$. Furthermore, let $V_i = V(\omega_i)$ and $p_i := p_{\omega_i}$, where $i\in\calI$ and $\calI$ is certain index set of the type set $\Omega$.

Based on Theorem \[theorem:equivalent\], at the Stackelberg equilibrium, where  is the leader,  faces the following problem: $$\begin{aligned}
\label{eq:newProb}
\min_{S\in\bPsi}\max_{p\in\Delta^{|\Omega|}}\;\trace\left\{S\sum_{i\in\calI}p_iV_i\right\} + v_o,\end{aligned}$$ since  reacts to the committed signaling rule $\eta_{1:\kappa}$ and correspondingly reacts to $S\in\bPsi$. can also be written as $$\label{eq:maxmin}
\min_{S\in\bPsi}\max_{p\in\Delta^{|\Omega|}}\; \sum_{i\in\calI}p_i\trace\left\{SV_i\right\} + v_o.$$

The following proposition addresses the existence of an equilibrium for $\calG$.

\[prop:existence\] There exists at least one tuple of pure actions $(\eta_{1:\kappa}^*,\{\gamma_{1:\kappa}^{\omega*}(\eta_{1:\kappa})\}_{\omega\in\Omega}, p^*(\eta_{1:\kappa}))$ attaining the equilibrium of the Stackelberg game $\calG$, i.e., satisfying .

The proof follows from the equivalence between and , which yields that  faces . Since the objective function in is continuous in the optimization arguments and the constraint sets are decoupled and compact, the extreme value theorem and maximum theorem (showing the continuity of parametric maximization under certain conditions [@ref:Ok07]) yields the existence of a solution to , which completes the proof.

The objective function in is linear in the optimization argument $S\in\bPsi$, and the constraint set $\bPsi$ is compact and convex. Therefore, given $p\in\Delta^{|\Omega|}$, the solution could be attained at a certain extreme point of $\bPsi$. However, the following function: $$\max_{p\in\Delta^{|\Omega|}}\sum_{i\in\calI} p_i \trace\{SV_i\}$$ is convex in $S\in\bPsi$ since the maximum of any family of linear functions is a convex function [@ref:Boyd04]. Particularly, for $\mu\in[0,1]$, we have $$\begin{aligned}
\mu\max_{p\in\Delta^{|\Omega|}}\trace\left\{S\sum_{i}p_iV_i\right\}+ (1-\mu)\max_{p\in\Delta^{|\Omega|}}\trace\left\{\bS\sum_{i}p_{i}V_i\right\} \geq \max_{p\in\Delta^{|\Omega|}}\trace\left\{(\mu S+(1-\mu)\bS)\sum_{i}p_iV_i\right\}.\end{aligned}$$ Therefore, the solution can be a non-extreme point of the constraint set $\bPsi$. Correspondingly, Lemma \[lem:affine\] implies that the optimal signals would be affine in the underlying state rather than linear, i.e., there will be additional independent noise term $\rn_k\sim\N(0,\Theta_k)$, where $\Theta_k\neq O$.$\triangle$

Next, we seek to compute the equilibrium of $\calG$. To this end, we examine the equilibrium conditions further. In particular, according to , given $S\in\bPsi$, the best action for  is given by $$\begin{aligned}
\label{eq:postar}
p^{*} \in \Big\{p\in\Delta^{|\Omega|}\,|\,p_j = 0 \mbox{ if } \trace\{V_jS\}<\max_{i}\trace\{V_iS\}\Big\}\end{aligned}$$ since is linear in $p\in\Delta^{|\Omega|}$. Then, based on the observation , the following theorem provides an algorithm to compute the robust sensor outputs.

\[theorem:compute\] The value of the Stackelberg equilibrium is given by $\vartheta = \min_{j\in\calI}\{\vartheta_j\}$, where $$\begin{aligned}
\label{eq:vj}
\vartheta_j := \min_{S\in\bPsi}&\; \trace\big\{V_jS\big\} + v_o \\ 
\mathrm{s.t. }&  \;\trace\big\{(V_j-V_i)S\big\} \geq 0\; \forall i\in\calI.\nn\end{aligned}$$

Furthermore, let $\vartheta_{j^*} = \vartheta$ and $$\begin{aligned}
S^*\in \argmin_{S\in\bPsi}&\; \trace\big\{V_{j^*}S\big\} + v_o \label{eq:Sstarj} \\ 
\mathrm{s.t. }&  \;\trace\big\{(V_{j^*}-V_i)S\big\} \geq 0\; \forall i\in\calI.\nn\end{aligned}$$ Then, given $S^*\in\bPsi$, the optimal signaling rule $\eta_{1:\kappa}$ can be computed according to from Lemma \[lem:affine\].

Based on the existence result in Proposition \[prop:existence\], suppose that $(S^*,p^*)$ attains the Stackelberg equilibrium, i.e., solves . Since $p^*\in\Delta^{|\Omega|}$, there must be at least one type with positive weight. As an example, suppose positive weight for the type $\omega_j\in\Omega$, i.e., $p_j>0$. This implies that $$\trace\{V_jS^*\} \geq \trace\{V_iS^*\}\;\forall i$$ since $\trace\{V_jS^*\} = \max_{i\in\calI} \trace\{V_iS^*\}$ by . Furthermore, this also implies that $$\trace\{V_jS^*\} = \sum_{i\in\calI}p_{i}^{*}\,\trace\{V_iS^*\}$$ since if $p_{i}^*>0$, then we have $$\trace\{V_jS^*\} = \trace\{V_iS^*\}.$$ These necessary conditions yield that $$\begin{aligned}
\min_{S\in\bPsi}\max_{p\in\Delta^{|\Omega|}}\;\sum_{i\in\calI} \trace\{V_iS\}p_i = \min_{S\in\bPsi} &\; \trace\{V_jS\}\nn\\
\mathrm{s.t. }&\; \trace\{(V_j-V_i)S\} \geq 0 \;\forall i \end{aligned}$$ while the right-hand-side is an SDP problem isolated from ’s action. Therefore, by searching over the index set $\calI$, we can compute the left-hand-side, which is the minimum over $\calI$. Once the minimum value is computed, $S^*$ can be computed according to the corresponding index, i.e., .

In Theorem \[theorem:compute\], we search over the index set $\calI$ linearly, however, certain pruning operations can be conducted to speed up the computation. As an example, we can search over the extreme points of the convex hull of $V_j$, $j\in\calI$. $\triangle$

There might be multiple solutions for .  can be selective among those solutions. In particular, implies that  minimizes the cost given that  maximizes it. Therefore, if the true underlying distribution is not the worst possible distribution, then  would not get a cost more than the anticipated one. Any deviation from the worst distribution benefits . Furthermore, in the worst case,  assigns positive probabilities to the types leading to the maximum as in . Correspondingly, if  selects the solution $S^*\in\bPsi$ for such that the cardinality of $\argmax_{j}\trace\{V_jS^*\}$ is the smallest, then any positive probability on other types of attacks out of that set would lead to lower cost and would be desirable. $\triangle$

Robust Sensor Design with Noisy or Partial Measurements {#sec:noisy}
=======================================================

In this section, we obtain the optimal signaling rule when there are noisy or partial measurements of the type by turning the problem to the same structure with the case of perfect measurements based on a recent result from [@ref:Sayin18d] and then invoking the results from the previous section. There are several challenges in robust sensor design with noisy or partial measurements. As an example, the sufficiency result on the necessary conditions for the covariance of the posterior control-free state, i.e., $H_k$, does not hold in that case. Therefore, our focus will be on the necessary and sufficient conditions for the covariance of the posterior control-free [*measurements*]{}, i.e., $\cov\{\E\{\ry_k^o|\rs_{1:k}\}\}$, where $\ry_k^o := C\rx_k^o + \rv_k$. Similar to , we can show that $$\E\{\ry_k^o | \rs_{1:k}\} = \E\{\ry_k^o | \rs_{1:k}^o\},$$ where $\rs_k^o := L_{k,k}'\ry_k^o + \ldots + L_{k,1}'\ry_1^o + \rn_k$, since $\ru_{1:k-1}$ is $\sigma$-$\ry_{1:k-1}$ measurable. However, $\{\ry_k^o\}$ is not necessarily a Markov process. Therefore, we consider $$\begin{aligned}
\begin{bmatrix} \ry_k^o \\ \hdashline[2pt/2pt] \ry_{k-1}^o \\ \vdots \\ \ry_1^o\end{bmatrix} = \overbrace{\begin{bmatrix} \E\{\ry_k^o(\ry_{1:k-1}^o)'\}\E\{\ry_{1:k-1}^o(\ry_{1:k-1}^o)'\}^{\dagger} \\ \hdashline[2pt/2pt]  \\ I \\ \end{bmatrix}}^{=: A_k} \begin{bmatrix} \ry_{k-1}^o \\ \vdots \\ \ry_{1}^o\end{bmatrix} + \underbrace{\begin{bmatrix} \ry_k^o - \E\{\ry_k^o|\ry_{1:k-1}^o\} \\ \hdashline[2pt/2pt] \\ O \\ \end{bmatrix}}_{=:\re_k},\end{aligned}$$ which can also be written in a compact form as $$\ry_{1:k}^o = A_k \ry_{1:k-1}^o + \re_k,$$ where we denote the vector $\begin{bmatrix}(\ry_k^o)'& \cdots & (\ry_1^o)' \end{bmatrix}'$ by $\ry_{1:k}^o$ with some abuse of notation.

Furthermore, we note that $\rx_k^o$, $\ry_{1:k}^o$, and $\rs_{1:k}^o$ form a Markov chain in the order $\rx_k^o \rightarrow \ry_{1:k}^o \rightarrow \rs_{1:k}^o$. In that respect, the following lemma from [@ref:Sayin18d] shows that there exists a linear relation between the posterior estimates irrespective of the signal if they are jointly Gaussian and form a Markov chain in a certain order.

\[lem:outsider\] Given zero-mean jointly Gaussian random vectors forming a Markov chain, e.g., $\rx\rightarrow \ry \rightarrow \rs$ in this order, the posterior estimates of $\rx$ and $\ry$ given $\rs$ satisfy the following linear relation: $$\label{eq:outsider}
\E\{\rx|\rs\} = \E\{\rx\ry'\} \E\{\ry\ry'\}^{\dagger} \E\{\ry|\rs\},$$ which implies $\rs\rightarrow\E\{\ry|\rs\}\rightarrow\E\{\rx|\rs\}$ in this order.

Based on Lemma \[lem:outsider\], we have the following relation between $\E\{\rx_k^o|\rs_{1:k}^o\}$ and $\E\{\ry_{1:k}^o|\rs_{1:k}^o\}$: $$\E\{\rx_k^o|\rs_{1:k}^o\} = \underbrace{\E\{\rx_k^o(\ry_{1:k}^o)'\}\E\{\ry_{1:k}^o (\ry_{1:k}^o)'\}^{\dagger}}_{=:D_k} \E\{\ry_{1:k}^o|\rs_{1:k}^o\},\label{eq:condexp2}$$ where $D_k\in\R^{m\times mk}$ does not depend on the signaling rule $\eta_{1:k}(\cdot)$. We define $$Y_k := \cov\{\E\{\ry_{1:k}^o | \rs_{1:k}^o\}\}.$$ Then, yields that $$H_k = D_kY_kD_k'.$$ Correspondingly, , i.e., the problem faced by , can be written as $$\min\limits_{\substack{\eta_k\in {\Upsilon}_k\\k=1,\ldots,\kappa}} \sum_{k=1}^{\kappa} \trace\left\{Y_k \left(\sum_{\omega\in\Omega}p_{\omega}W_k(\omega)\right)\right\} + v_o,$$ where $W_k(\omega) := D_k'V(\omega)D_k$ is also a symmetric matrix. Furthermore, consider the following compact and convex set: $$\Phi := \{(S_k \in \AS^{mk})_{k=1}^{\kappa}| \Sigma_k^y \succeq S_k \succeq A_kS_{k-1}A_k', k=1,\ldots,\kappa,S_0=O\},$$ where $\Sigma_k^y := \E\{\ry_{1:k}^o(\ry_{1:k}^o)'\}$, $C\in\R^{m\times m}$ and $\Sigma_v \in \AS^{m}$.

Note that $\Sigma_k^y\in\AS^{mk}$, $A_k \in \R^{km\times(k-1)m}$, and $D_k\in\R^{m\times mk}$ can be written as $$\begin{aligned}
&\Sigma_k^y = \begin{bmatrix} O & I_k \otimes C\end{bmatrix}\Sigma^o \begin{bmatrix} O \\ I_k \otimes C' \end{bmatrix} + I_{k} \otimes \Sigma_v,\\
&A_k = \begin{bmatrix} \begin{bmatrix} O_{m\times (\kappa-k)m} & C & O_{m\times(k-1)m} \end{bmatrix}  \Sigma^o \begin{bmatrix} O \\ I_{k-1}\otimes C'\end{bmatrix} (\Sigma_{k-1}^y)^{\dagger} \\ I_{(k-1)m} \end{bmatrix},\\
&D_k = \begin{bmatrix} O_{m\times (\kappa-k)m} & I_m & O_{m\times(k-1)m} \end{bmatrix}  \Sigma^o \begin{bmatrix} O \\ I_k\otimes C'\end{bmatrix} (\Sigma_k^y)^{\dagger}\end{aligned}$$ in terms of $\Sigma^o\in\AS^{m\kappa}$, defined in Appendix \[app:V\] by . $\triangle$

Without loss of generality, suppose that $\rs_k \in \R^{mk}$ instead of $\rs_k\in\R^m$ such that  can [*disclose*]{} $\teta_k(\ry_{1:k}) = \ry_{1:k}$ with the affine signaling rule $\teta_k(\cdot)$ from $\R^{mk}$ to $\R^{mk}$. Particularly, in practice, we can always set the signaling rule $\eta_k(\cdot)$ from $\R^{mk}$ to $\R^{m}$ as $$\label{eq:wlg}
\eta_k(\ry_{1:k}) = \E\{\rx_k^o|\teta_1(\ry_1),\ldots,\teta_k(\ry_{1:k})\}.$$ For such a signaling rule $\teta_k(\cdot)$, by following similar lines in Step $ii)$ in the proof of Theorem \[theorem:equivalent\], we can show that a necessary condition on $Y_{1:\kappa}$ is that $Y_{1:\kappa}\in\Phi$. Furthermore, based on Lemma \[lem:affine\], a sufficient condition on $Y_{1:\kappa}$ is that for any $S_{1:\kappa}\in\Phi$, there exists a certain signaling rule such that $Y_{1:\kappa} = S_{1:\kappa}$. $\triangle$

The following corollary to Theorem \[theorem:equivalent\] provides an equivalent SDP problem for when there are noisy measurements.

\[corollary:lem\] Given $p\in\Delta^{|\Omega|}$, for any signaling rule $\eta_{1:\kappa}$, there exists $S_{1:\kappa}\in\Phi$ such that $$\begin{aligned}
&U_{\playerS}(\eta_{1:\kappa},\{\gamma_{1:\kappa}^{\omega*}(\eta_{1:\kappa})\}_{\omega\in\Omega},p)\\
&\hspace{.4in}=\sum_{k=1}^{\kappa} \trace\left\{S_k \left(\sum_{\omega\in\Omega}W_k(\omega)\right)\right\} + v_o.\end{aligned}$$ Furthermore, for any $S_{1:\kappa}\in\Phi$, there exists a signaling rule $\eta_{1:\kappa}$ such that $$\begin{aligned}
\label{eq:noisyToRight}
&\sum_{k=1}^{\kappa} \trace\left\{S_k \left(\sum_{\omega\in\Omega}W_k(\omega)\right)\right\} + v_o\\
&\hspace{.4in}=U_{\playerS}(\eta_{1:\kappa},\{\gamma_{1:\kappa}^{\omega*}(\eta_{1:\kappa})\}_{\omega\in\Omega},p).\end{aligned}$$

Based on Corollary \[corollary:lem\], the following corollary to Theorem \[theorem:compute\] provides an algorithm to compute the robust sensor outputs for the cases with noisy or partial measurements.

\[corollary:compute\] The value of the Stackelberg equilibrium $$\min_{S\in\bPhi}\max_{p\in\Delta^{|\Omega|}} \sum_{i\in\calI}p_i\trace\{W_iS\} + v_o,$$ where $W_i$ and $\bPhi$ are defined accordingly, is given by $\vartheta = \min_{j\in\calI}\{\vartheta_j\}$, where $$\begin{aligned}
\nn
\vartheta_j := \min_{S\in\bPhi}&\; \trace\big\{W_jS\big\} + v_o \\ 
\mathrm{s.t. }&  \;\trace\big\{(W_j-W_i)S\big\} \geq 0\; \forall i\in\calI.\nn\end{aligned}$$

Furthermore, let $\vartheta_{j^*} = \vartheta$ and $$\begin{aligned}
S^*\in \argmin_{S\in\bPhi}&\; \trace\big\{W_{j^*}S\big\} + v_o \nn \\ 
\mathrm{s.t. }&  \;\trace\big\{(W_{j^*}-W_i)S\big\} \geq 0\; \forall i\in\calI.\nn\end{aligned}$$ Then, given $S^*\in\bPhi$, the optimal signaling rule $\teta_{1:\kappa}$ can be computed according to from Lemma \[lem:affine\] with corresponding $\Sigma_k^y$ and $A_k$ instead of $\Sigma_k^o$ and $A$, for $k=1,\ldots,\kappa$, and then we can compute the actual signaling rules $\eta_{1:\kappa}$ via .

Illustrative Examples {#sec:examples}
=====================

As numerical illustrations, we compare the performance of the proposed secure sensor design framework with classical sensors that disclose the measurement to the controller directly. The controller can have three different types: type-$\omega_o$ corresponding to benign controller, and type-$\alpha$ and type-$\beta$ corresponding to malicious controllers. As an illustrative example, we set the time horizon $\kappa=10$, the state’s dimension $m=4$, and the control input’s dimension $r=2$. We consider that the state can be partitioned into the separate processes $\{\rt_k\in\R^2\}$ and $\{\rz_k\in\R^2\}$, i.e., $\rx_k' = \begin{bmatrix} \rt_k' & \rz_k' \end{bmatrix}$, and the state recursion is given by $$\begin{bmatrix} \rt_{k+1}\\ \rz_{k+1} \end{bmatrix} = \begin{bmatrix} A_t & O \\O & A_z\end{bmatrix} \begin{bmatrix} \rt_{k} \\ \rz_{k} \end{bmatrix} + \begin{bmatrix} B_t \\ O \end{bmatrix}\ru_k + \begin{bmatrix} \rw_k^t \\ \rw_k^z \end{bmatrix},$$ where $$A_t := \begin{bmatrix} 1/\sqrt{2} & 0\\0 & 1/2 \end{bmatrix},\, A_z := \begin{bmatrix} 1/3 & 1/10 \\ 1/10 & 1/\sqrt{2} \end{bmatrix},\, B_t := \begin{bmatrix} 1 & 1 \\  0 & 1 \end{bmatrix}.\nn$$ Furthermore, we let the initial state $\rx_1\sim\N(0,\Sigma_1)$ and the state noise $\rw_k \sim \N(0,\Sigma_w)$ have the covariance matrices: $$\begin{aligned}
\Sigma_1 := \begin{bmatrix} 1&0&0&0\\0&1&0&0\\ 0&0&1.5&0\\0&0&0&2\end{bmatrix},\,\Sigma_w:=\begin{bmatrix} 1&0&0&0\\ 0&2&0&0 \\ 0&0&0.98 & -0.228\\ 0&0&-0.228 & 0.985\end{bmatrix},\end{aligned}$$ which implies $\{\rz_k\}$ is a stationary exogenous process. The benign controller objective is given by $$\label{eq:simSobj}
\sum_{k=1}^{\kappa}\left\|\begin{bmatrix} \rt_{k+1} - \rz_{k+1} \\ \rt_{k+1} \end{bmatrix}\right\|^2 + \|\ru_k\|^2,$$ which implies that type-$\omega_o$ controller and  seek to regularize the controlled process $\{\rt_k\}$ around the zero vector and the exogenous process $\{\rz_k\}$. On the other hand, the other malicious type controllers’ objectives are misaligned with rather than being its complete opposite. Let $\rt_k = \begin{bmatrix} \rt_k^{(1)} & \rt_k^{(2)} \end{bmatrix}'$. Then, type-$\alpha$  seeks to regularize $\{\rt_k^{(1)}\in\R\}$ around zero and thus his control objective is given by $$\sum_{k=1}^{\kappa}\left\|\rt_{k+1}^{(1)}\right\|^2 + \|\ru_k\|^2.$$ Type-$\beta$  seeks to regularize the other component of $\rt_k$, $\{\rt_k^{(2)}\in\R\}$, again around zero and thus his control objective is given by $$\sum_{k=1}^{\kappa}\left\|\rt_{k+1}^{(2)}\right\|^2 + \|\ru_k\|^2.$$

We consider four different scenarios in terms of the measurements:

-   Scenario-$1$: Perfect Measurements, i.e., $\ry_k = \rx_k$.

-   Scenario-$2$: Noisy Measurements, i.e., $\ry_k = \rx_k + \rv_k$.

-   Scenario-$3$: Partial Measurements, i.e., $\ry_k = C\rx_k$.

-   Scenario-$4$: Partial Noisy Measurements, i.e., $$\ry_k = C\rx_k + \rv_k.$$

We let $\rv_k\sim\N(0,I_4)$ and $$C := \begin{bmatrix} 1&1&0&0 \\ 0&0&1&0 \\ 0&0&0&1 \\ 0&0&0&0 \end{bmatrix},$$ which is a singular matrix. In Tables \[tab:scenario1\]-\[tab:scenario4\], we compare the performance of the secure sensor design framework with the classical sensors in terms of the following performance metric: $$\label{eq:measure}
\max_{p\in\Delta^{|\Omega|}}\sum_{k=1}^{\kappa}\trace\left\{H_k \sum_{\omega\in\Omega}p_{\omega}V_k(\omega)\right\},$$ i.e., in terms of the impact of the sensor feedback and for the worst possible distribution over the controllers’ types, based on Theorem \[theorem:equivalent\] and Corollary \[corollary:lem\] while $V_{1:\kappa}(\omega)$, for $\omega\in\Omega$, is derived in Appendix \[app:V\].

Note that the performance metric excludes $v_o\in\R$ from the original cost function in perfect measurements case or in partial or noisy measurements case since $v_o\in\R$ does not depend on how the measurements have been shared with the controller, and therefore is fixed for all the scenarios. Correspondingly, the performance metric could be [*negative*]{} while the original cost function is always non-negative by definition. $\triangle$

Across Tables \[tab:scenario1\]-\[tab:scenario4\], i.e., across all Scenarios $1$-$4$, we have the following observations in common: $i)$ the proposed framework outperforms the classical sensors that disclose the (perfect or noisy) measurements directly without any crafting; $ii)$ the cost for the proposed framework and the classical full disclosure strategy is the same when there is only benign type almost surely; $iii)$ the benign type  is dominated by both type-$\alpha$  and type-$\beta$  in the worst distribution, i.e., benign type has zero probability almost surely in the worst distribution; $iv)$ type-$\beta$  is stronger attacker than type-$\alpha$  by leading to higher cost.

[C[.1in]{}C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}]{} & & &\
& & &\
& Secure & Full & Secure & Full & Secure & Full\
& & & & & &\
& & & & & &\
& & & & &\
& & & & &\
& & & & &\
& & & & & &\

\[tab:scenario1\]

As seen in Table \[tab:scenario1\], in Scenario-$1$, the cost of the proposed framework for all the cases, i.e., for different sets of types, is the smallest compared to the other scenarios, where there can be partial or noisy measurements. Particularly, the perfect measurements give the utmost [*freedom*]{} to  to select the signaling rule. Therefore, if there were any other case with partial or noisy measurements, where  achieves lower cost, then  could have selected that corresponding composed signaling rule in the case with perfect measurement. Partial or noisy measurements limit ’s ability to deceive the attackers. Furthermore, we observe that when all the types can exist with a positive probability, the cost is higher than the cases when only one attacker exists. This yields that the worst distribution to defend against is not dominated by the strongest attacker, who is type-$\beta$  in this scenario. In other words, the possibility of a mixture over the stronger and weaker attackers can be more powerful.

[C[.1in]{}C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}]{} & & &\
& & &\
& Secure & Full & Secure & Full & Secure & Full\
& & & & & &\
& & & & & &\
& & & & &\
& & & & &\
& & & & &\
& & & & & &\

\[tab:scenario2\]

[C[.1in]{}C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}]{} & & &\
& & &\
& Secure & Full & Secure & Full & Secure & Full\
& & & & & &\
& & & & & &\
& & & & &\
& & & & &\
& & & & &\
& & & & & &\

\[tab:scenario3\]

In Scenarios $2$ and $3$, as seen in Tables \[tab:scenario2\] and \[tab:scenario3\], the performance degrades compared to Scenario $1$ in the proposed framework. However, such a performance degradation is not the case for the full information disclosure in general. As an example, when there is only type-$\beta$  almost surely, the cost is higher in Scenario $3$ than the one in Scenario $1$ for full disclosure of the measurement. This is mainly because the objectives of the malicious type controller is not the complete opposite of the benign type controller. Therefore, even though in all the cases illustrated in this section, we have observed that there can even be examples, where the full disclosure of the measurement can lead to a [*positive*]{} cost, which would imply that disclosing even no information with the controller would lead to lower cost, i.e., $0$, since no information disclosure yields that the covariance of the posterior control-free state is $H_k = O$. Furthermore, we observe that the possibility of a mixture over the stronger and weaker attackers can also be more powerful in Scenarios $2$ and $3$.

[C[.1in]{}C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}|C[.35in]{}]{} & & &\
& & &\
& Secure & Full & Secure & Full & Secure & Full\
& & & & & &\
& & & & & &\
& & & & &\
& & & & &\
& & & & &\
& & & & & &\

\[tab:scenario4\]

In Scenario-$4$, as seen in Table \[tab:scenario4\], the cost for the case when there is only the benign type is the highest. However, while the costs for the cases when there is only type-$\alpha$ attacker is higher than the corresponding cases in Scenario-$2$, the costs for the cases when there is only type-$\beta$ attacker is [*lower*]{} than the corresponding cases in Scenario-$2$. Note that type-$\beta$ attacker is still stronger than type-$\alpha$ attacker by leading to higher cost. Furthermore, the costs for the case when all the types can exist with a positive probability also leads to a similar twist, where we observe a higher cost in Scenario-$2$. We also note that type-$\beta$ attacker dominates the worst distribution over all the types since the costs for the case where there is only type-$\beta$ is the same with the cost for the case when there are all types. Therefore, we can conclude that depending on the type of controllers and how informative the measurements are, the costs can vary in a complicated way while the proposed framework provides the optimal way to compute the robust sensor outputs and a performance assessment tool for, e.g., various sensor placement techniques.

Conclusion {#sec:conclusion}
==========

In this paper, we have proposed and addressed the robust sensor design problem for cyber-physical systems with linear Gaussian dynamics against multiple advanced and evasive attackers with quadratic control objectives. By designing sensor outputs cautiously in advance, we have sought to deceive the attackers about the underlying state of the system so that they would act/attack to the system in-line with the normal operation. Our goal has been to exploit the aligned part between the attackers’ and the system’s objectives so that the attackers would only have fulfilled the aligned part by crafting the information available to them. To this end, we have modeled the problem formally in a game-theoretical hierarchical setting, where the advanced attackers can be aware of the designed signaling rules.

We have formulated an equivalent problem to the problem faced by the sensor against any attacker with a known objective. This new problem was an SDP problem. We have introduced additional linear constraints on that equivalent problem and provided an SDP algorithm to compute the optimal robust sensor design strategies against multiple types of attackers. We have also extended the results to scenarios where the sensor could have access to partial or noisy measurements of the underlying state. Finally, we have examined the performance of the proposed framework across various scenarios and compared with the classical sensor outputs that disclose the measurements directly without any crafting.

Some future directions of research on this topic include formulation of secure sensor design strategies for robust control of systems, and considering scenarios where the attackers can have side information about the underlying state, which would limit the sensor’s ability to deceive the attackers. Another interesting research direction would be the application of the framework to sensor placement or sensor selection. Furthermore, even though we have motivated the framework by relating it to security, the framework could also address strategic information disclosure over multi-agent control networks with misaligned control objectives.

Computation of $K_{1:\kappa}^{\omega},\Delta_{1:\kappa}^{\omega}$, and $\Delta_0^{\omega}$ {#app:computationA}
------------------------------------------------------------------------------------------

A routine completion to square leads to [@ref:Bansal89; @ref:Kumar86] $$\E\left\{\sum_{k=1}^{\kappa}\|\rx_{k+1}\|_{Q_{\omega}}^2 + \|\ru_k\|_{R_{\omega}}^2\right\} = \sum_{k=1}^{\kappa}\E\left\{\|\ru_k+K_{k}^{\omega}\rx_k\|_{\Delta_{k}^{\omega}}^2\right\} + \Delta_{0}^{\omega},\label{eq:square}$$ where $$\begin{aligned}
&K_{k}^{\omega}=(\Delta_{k}^{\omega})^{-1}B'\tQ_{k+1}^{\omega}A\\
&\Delta_{k}^{\omega} = B'\tQ_{k+1}^{\omega}B + R_{\omega}\\
&\Delta_{0}^{\omega} = \trace\{Q_{\omega}\Sigma_1\} + \sum_{k=1}^{\kappa}\trace\{\tQ_{k+1}^{\omega}\Sigma_w\}\end{aligned}$$ and $\{\tQ_{k}^{\omega}\}$ is given by the discrete-time dynamic Riccati equation $$\tQ_{k}^{\omega} = Q_{\omega} + A'(\tQ_{k+1}^{\omega} - \tQ_{k+1}^{\omega}B(\Delta_{k}^{\omega})^{-1}B'\tQ_{k+1}^{\omega})A$$ and $\tQ_{\kappa+1}^{\omega} = Q_{\omega}$.

Computation of $V_{1:\kappa}(\omega)$ and $v_o$ {#app:V}
-----------------------------------------------

Let $\omega_o\in\Omega$ denote the type of the benign controller, who has the same objective with . We define $$\begin{aligned}
&\Phi(\omega) := \begin{bmatrix} 
I & K_{\kappa}^{\omega}B & K_{\kappa}^{\omega} AB & \cdots & K_{\kappa}^{\omega}A^{\kappa-2}B \\
& I & K_{\kappa-1}^{\omega}B & \cdots & K_{\kappa-1}^{\omega}A^{\kappa-3}B\\ 
& & I & \cdots & K_{\kappa -2}^{\omega}A^{\kappa-4}B\\
& & & \ddots & \vdots\\
& & & & I
\end{bmatrix}\\
&K(\omega) := \begin{bmatrix} K_{\kappa}^{\omega} & & \\ & \ddots & \\ & & K_1^{\omega}\end{bmatrix},\;\Delta(\omega) := \begin{bmatrix} \Delta_{\kappa}^{\omega} & & \\ & \ddots & \\ & & \Delta_1^{\omega}\end{bmatrix}.\end{aligned}$$ Then, can be written as $$\|\Phi(\omega)\ru + K(\omega)\rx^o\|_{\Delta(\omega)}^2 + \Delta^{\omega}_0$$ in terms of the augmented vectors $\ru,\rx^o\in\R^{m\kappa}$. Correspondingly, the optimal attack is $\ru^* = -\Phi(\omega)^{-1}K(\omega)\rhx^o$, where $\rhx^o := \begin{bmatrix} \E\{\rx_{\kappa}^o | \rs_{1:\kappa}\}' & \cdots & \E\{\rx_1^o|\rs_1\}' \end{bmatrix}'$, for type-$\omega$ controller. Therefore, , i.e., type-$\omega_o$, faces the following problem: $$\sum_{\omega\in\Omega}p_{\omega} \|K(\omega_o)\rx^o - T(\omega)\rhx^o\|_{\Delta(\omega_o)} + \Delta_{0}^{\omega_o},\label{eq:whole}$$ where $T(\omega) := \Phi(\omega_o)\Phi(\omega)^{-1}K(\omega)$. We introduce $$\begin{aligned}
&\Xi(\omega) := T(\omega)'\Delta(\omega_o) T(\omega) - T(\omega)'\Delta(\omega_o) K(\omega_o) - K(\omega_o)' \Delta(\omega_o) T(\omega)\\
&v_o := \trace\{\Sigma^o K(\omega_o)'\Delta(\omega_o)K(\omega_o)\} + \Delta_0^{\omega_o},\end{aligned}$$ where $\Sigma^o := \E\{\rx^o(\rx^o)'\}$ is given by $$\label{eq:Sigmao}
\Sigma^o := \begin{bmatrix} \Sigma_{\kappa}^o & A\Sigma_{\kappa-1}^o & \cdots & A^{\kappa-1}\Sigma_1^o \\ \Sigma_{\kappa-1}^oA' & \Sigma_{\kappa-1}^o & &A^{\kappa-2}\Sigma_1^o \\ 
\vdots & & \ddots & \vdots \\ \Sigma_1^o(A^{\kappa-1})' & \Sigma_1^o (A^{\kappa-2})' & \cdots & \Sigma_1^o \end{bmatrix}.$$ Note that $v_o\in\R$ does not depend on the types of the controllers. Then, can be written as $$\sum_{\omega\in\Omega} p_{\omega}\trace\{\E\{\rhx^o(\rhx^o)'\}\Xi(\omega)\} + v_o,$$ where we have $$\nn
\E\{\rhx^o(\rhx^o)'\} := \begin{bmatrix} H_{\kappa} & AH_{\kappa-1} & \cdots & A^{\kappa-1}H_1 \\ H_{\kappa-1}A' & H_{\kappa-1} & &A^{\kappa-2}H_1 \\ 
\vdots & & \ddots & \vdots \\ H_1(A^{\kappa-1})' & H_1 (A^{\kappa-2})' & \cdots & H_1 \end{bmatrix}.$$ Therefore, the corresponding $V_k(\omega)\in\AS^m$ in is given by $$V_k(\omega) = \Xi_{k,k}(\omega) + \sum_{l = k+1}^{\kappa} \Xi_{k,l}(\omega) A^{l-k} + (A^{l-k})' \Xi_{l,k}(\omega),$$ where $\Xi_{k,l}(\omega)\in\R^{m\times m}$ is an $m\times m$ block of $\Xi(\omega)$, with indexing from the right-bottom to the left-top.

[^1]: This research was supported by the U.S. Office of Naval Research (ONR) MURI grant N00014-16-1-2710. The authors are with the Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801 USA. E-mail: {sayin2,basar1}@illinois.edu

[^2]: In this paper, we use the terms “signaling rule" and “strategy" interchangeably.

[^3]: Even though we consider time invariant matrices $A, B$, and $C$, for notational simplicity, the provided results could be extended to time-variant cases rather routinely. Furthermore, we consider all the random parameters to have zero mean; however, the derivations can be extended to non-zero mean case in a straight-forward way.

[^4]: For notational simplicity, we consider time-invariant $Q$ and $R$. However, the results provided could be extended to the general time-variant case rather routinely.

[^5]: Note that $\Sigma_{k}^{o} = A\Sigma_{k-1}^oA' + \Sigma_w$.

[^6]: We do not assume that its eigenvalues are necessarily in $[0,1]$. But actually, they turn out to be in $[0,1]$.
