---
author:
- GIUSEPPE BOCCIGNONE
bibliography:
- 'levyeye.bib'
title: A probabilistic tour of visual attention and gaze shift computational models
---

This research was partially supported by the project “Interpreting emotions: a computational tool integrating facial expressions and biosignals based shape analysis and bayesian networks”, grant FIRB - *Future in Research* RBFR12VHR7\
Author’s address: G. Boccignone, Department of Computer Science, University of Milan, via Comelico 39/41, 20135 Milano, Italy; email: giuseppe.boccignone@unimi.it

Introduction
============

As the french philosopher Merleau-Ponty put it, “vision is a gaze at grips with a visible world” [@maurice1945phenomenologie]. Goals and purposes, either internal or external, press the observer to maximise his information intake over time, by moment-to-moment sampling the most informative parts of the world. In natural vision this endless endeavour is accomplished through a sequence of eye movements such as saccades and smooth pursuit, followed by fixations. Gaze shifts require visual attention to precede them to their goal, which has been shown to enhance the perception of selected part of the visual field (in turn related to the foveal structure of the human eye, see for an extensive discussion of these aspects).

The computational counterpart of using gaze shifts to enable a perceptual-motor analysis of the observed world can be traced back to pioneering work on active or animate vision [@aloimonos1988active; @Ballard; @bajcsy1992active]. The main concern at the time was to embody vision in the action-perception loop of an artificial agent that purposively acts upon the environment, an idea that grounds its roots in early cybernetics [@cordeschi2002discovery]. To such aim the sensory apparatus of the organism must be active and flexible, for instance, the vision system can manipulate the viewpoint of the camera(s) in order to investigate the environment and get better information from it. Surprisingly enough, the link between attention and active vision, notably when instantiated via gaze shifts (e.g, a moving camera), was overlooked in those early approaches, as lucidly remarked by . Indeed, active vision, as it has been proposed and used in computer vision, must include attention as a sub-problem [@rothenstein2008attention]. First and foremost when it must confront the computational load to achieve real-time processing (e.g., for autonomous robotics and videosurveillance).

Nevertheless, the mainstream of computer vision has not dedicated to attentive processes and, more generally, to active perception much consideration. This is probably due to the original sin of conceiving vision as a pure information-processing task, a reconstruction process creating representations at increasingly levels of abstraction, a land where action had no place: the “from pixels to predicates” paradigm [@aloimonos1988active].

To make a long story short, the research field had a sudden burst when the paper by was published. Their work provided a sound and neat computational model (and the software simulation) to contend with the problem: in a nutshell, derive a saliency map and generate gaze shifts as the result of a Winner-Take-All (WTA) sequential selection of most salient locations. Since then, proposals and techniques have flourished. Under these circumstances, a deceptively simple question arises: Where are we now?

A straight answer, which is the *leitmotiv* of this paper, is that whilst early active vision approaches overlooked attention, current approaches have betrayed purposive active perception.

In this perspective, here we provide a critical discussion of a number of models and techniques. It will be by no means exhaustive, and yet, to some extent, idiosyncratic. Our purpose is not to offer a review (there are excellent ones the reader is urged to consult, e.g., [@BorItti2012; @borji2014salient; @bruce2015computational; @bylinskii2015towards]), but rather to spell in a probabilistic framework the variety of approaches, so to discuss in a principled way current limitations and to envisage intriguing directions of research, e.g., the hitherto neglected link between oculomotor behavior and emotion.

In the following Section we highlight critical points of current approaches and open issues. In Section \[sec:prob\] we frame such problems in the language of probability. Section \[sec:action\] discusses possible routes to reconsider the problem of oculomotor behaviour within the action/perception loop. In Section \[sec:emo\] we explore the *terra incognita* of gaze shifts and emotion.

A Mini review and open issues
=============================

The aim of a computational model of attentive eye guidance is to answer the question *Where to Look Next?* by providing:

1.  at the *computational theory level* (following ), an account of the mapping from visual data of a natural scene, say $\mathbf{I}$ (raw image data representing either a static picture or a stream of images), to a sequence of time-stamped gaze locations $(\mathbf{r}_{F_1}, t_1), (\mathbf{r}_{F_2}, t_2),\cdots$, namely $$\mathbf{I} \mapsto \{\mathbf{r}_{F_1}, t_1; \mathbf{r}_{F_2}, t_2;\cdots \}, 
    \label{eq:mapping}$$

2.  at the *algorithmic level*, a procedure that simulates such mapping.

A simple example of the problem we are facing is shown in Figure \[Fig:variab\].

![Scan paths eye tracked from different human observers while viewing three pictures of different information content: outdoor (top row), indoor with meaningful objects (middle row), indoor with high semantic content (person and face, bottom row). The area of yellow disks marking fixations between saccades is proportional to fixation time (images freely available from the dataset).[]{data-label="Fig:variab"}](FigVariab.jpg)

Under this conceptualization, when the input $\mathbf{I}$ is a static scene (a picture), the fixation duration time and saccade (lengths and directions) sequence are the only observables of the underlying guidance mechanism. When $\mathbf{I}$ stands for a time varying scene (e.g. a video), pursuit needs to be taken into account, too. We will adopt the generic terms of gaze shifts for either pursuit, saccades and fixational movements.

In the following, for notational simplicity, we will write the time series $\{\mathbf{r}_{F_1}, t_1; \mathbf{r}_{F_2}, t_2;\cdots \}$ as the sequence $\{\mathbf{r}_{F}(1), \mathbf{r}_{F}(2),\cdots \}$, unless the expanded form is needed. Also, we will generically refer to such sequence as scan path, though this term has a historically precise meaning in the eye movement literature [@privitera2006scanpath].

The common practice of computational approaches to derive the mapping (\[eq:mapping\]) is to conceive it as a two step procedure:

1.  obtaining a suitable perceptual representation $\mathcal{W}$, i.e., $\mathbf{I} \mapsto  \mathcal{W}$;

2.  using $\mathcal{W}$ to generate the scan path, $\mathcal{W} \mapsto  \{\mathbf{r}_{F}(1), \mathbf{r}_{F}(2),\cdots \}$.

It is important to remark that each gaze position $\mathbf{r}_{F}(t)$ sets a new field of view for perceiving the world, thus $\mathcal{W}=\{\mathcal{W}_{\mathbf{r}_{F}(1)}, \mathcal{W}_{\mathbf{r}_{F}(2)},\cdots \}$ should be a time-varying representation, even in the case of a static image input. This feedback effect of the moving gaze is hardly considered at the modelling stage [@zelinsky2008theory; @TatlerBallard2011eye].

By overviewing the field [@TatlerBallard2011eye; @BorItti2012; @bruce2015computational; @bylinskii2015towards], computational modelling has been mainly concerned with the first step: deriving a representation $\mathcal{W}$, typically in the form of a salience map. Yet, such step has recently evolved in a parallel research program, in which gaze shift prediction and simulation is not the focus, but salient object detection (for an in-depth review of this “second wave” of saliency-centered methods, see ).

The second step, that is $\mathcal{W} \mapsto \{\mathbf{r}_{F}(1), \mathbf{r}_{F}(2),\cdots \}$, which actually brings in the question of *how* we look rather than *where*, is seldom taken into account. Surprisingly, in spite of the fact that the most cited work in the field [@IttiKoch98] clearly addressed the *how* issue (gaze shifts as the result of a WTA sequential selection of most salient locations), most models simply overlook the problem. As a matter of fact, the representation $\mathcal{W}$, once computed, is usually validated in respect of its capacity for predicting the image regions that would be explored by the overt attentional shifts of human observers (in a task designed to minimize the role of top-down factors). Predictability is assessed according to some established evaluation measures (see , and , for a recent discussion).

In other cases, if needed for practical purposes, e.g. for robotic applications, the problem of oculomotor action selection is solved by adopting some simple deterministic choice procedure that usually relies on selecting the gaze position $\mathbf{r}$ as the argument that maximizes a measure on the given representation $\mathcal{W}$. We will attack on the problem related to gaze shift generation in Sections \[sec:bias\] and \[sec:var\]. In the following Section we first discuss representational problems.

Levels of representation and control {#sec:levels}
------------------------------------

The guidance of eye movements is likely to be influenced by a hierarchy of several interacting control loops, operating at different levels of processing. Each processing step exploits the most suitable representation of the viewed scene for its own level of abstraction. , in a plausible portrayal, have sorted out the following representational levels: 1) *salience*, 2) *objects*, 3) *values*, and 4) *plans*.

Up to this date, the majority of computational models have retained a central place for low-level visual conspicuity [@TatlerBallard2011eye; @BorItti2012; @bruce2015computational]. The perceptual representation of the world $\mathcal{W}$ is usually epitomized in the form of a spatial saliency map, which is mostly derived bottom-up (early salience) following [@IttiKoch98]. The weakness of the bottom-up approach has been largely discussed (see, e.g. ). Indeed, the effect of early salience on attention is likely to be a correlational effect rather than an causal one [@foulsham2008], [@schutz2011eye]. Few examples are provided in Fig. \[Fig:itti\], where, as opposed to human scan paths (in free-viewing conditions), the scan path generated by using a salience-based representation [@IttiKoch98] does not spot semantically important objects (faces), the latter not being detected as regions of high contrast in colour, texture and luminance with respect to other regions of the picture.

Under these circumstances, early saliency can be modulated to improve its fixation prediction. has considered prior knowledge on the typical spatial location of the search target, as well as contextual information (the gist of a scene, ). Further, object knowledge can be used to top-down tune early salience. In particular, when dealing with faces, a face detection step [@cerf2008predicting], [@postma2011], [@marat2013improving] or a prior for Bayesian integration with low level features [@bocc08tcsvt], can provide a reliable cue to complement early conspicuity maps. Indeed, faces drive attention in a direct fashion [@cerf2009faces] and the same holds for text regions [@cerf2008predicting; @BocCOGN2014]. It has been argued that salience has only an indirect effect on attention by acting through recognised objects: observers attend to interesting objects and salience contributes little extra information to fixation prediction [@EinhauserSpainPerona2008]. As a matter of fact, in the real world, most fixations are on task-relevant objects and this may or may not correlate with the saliency of regions of the visual array [@canosa2009real; @rothkopfBallard2007]. Notwithstanding, object-based information has been scarcely taken into account in computational models [@TatlerBallard2011eye]. There are of course exceptions to this state of affairs, most notable ones those provided by , , the Bayesian models discussed by and .

The representational problem is just the light side of the eye guidance problem. When actual eye tracking data are considered, one has to confront with the dark side: regardless of the perceptual input, scan paths exhibit both systematic tendencies and notable inter- and intra-subject variability. As put it, where we choose to look next at any given moment in time is not completely deterministic, but neither is it completely random.

Biases in oculomotor behaviour {#sec:bias}
------------------------------

Systematic tendencies or “biases” in oculomotor behaviour can be thought of as regularities that are common across all instances of, and manipulations to, behavioural tasks [@tatler2008systematic; @tatler2009prominence]. In that case case useful information about how the observers will move their eyes can be found. One remarkable example is the amplitude distribution of saccades and microsaccades that typically exhibit a positively skewed, long-tailed shape [@TatlerBallard2011eye; @dorr2010variability; @tatler2008systematic; @tatler2009prominence]. Other paradigmatic examples of systematic tendencies in scene viewing are: initiating saccades in the horizontal and vertical directions more frequently than in oblique directions; small amplitude saccades tending to be followed by long amplitude ones and vice versa [@tatler2008systematic; @tatler2009prominence].

Indeed, biases affecting the manner in which we explore scenes with our eyes are well known in the psychological literature (see for a thorough review), albeit underexploited in computational models. Such biases may arise from a number of sources. have suggested the following: biomechanical factors, saccade flight time and landing accuracy, uncertainty, distribution of objects of interest in the environment, task parameters.

Understanding biases in eye guidance can provide powerful new insights into the decision about where to look in complex scenes. In a remarkable study, provided striking evidence that a model based solely on these biases and therefore blind to current visual information can outperform salience-based approaches. Further, the predictive performance of a salience-based model can be improved from $56\%$ to $80\%$ by including the probability of gaze shift directions and amplitudes. Failing to account properly for such characteristics results in scan patterns that are fairly different from those generated by human observers (which can be easily noticed in the example provided in Fig. \[Fig:itti\]) and eventually in distributions of saccade amplitudes and orientations that do not match those estimated from human eye behaviour.

Variability {#sec:var}
-----------

When looking at natural images or movies [@dorr2010variability] under a free-viewing or a general-purpose task, the relocation of gaze can be different among observers even though the same locations are taken into account. In practice, there is a small probability that two observers will fixate exactly the same location at exactly the same time. This effect is even more remarkable when free-viewing static images: consistency in fixation locations selected by observers decreases over the course of the first few fixations after stimulus onset [@TatlerBallard2011eye] and can become idiosyncratic. Such variations in individual scan paths (as regards chosen fixations, spatial scanning order, and fixation duration) still hold when the scene contains semantically rich “objects” (e.g., faces, see Figures \[Fig:variab\] and \[Fig:itti\]). Variability is also exhibited by the same subject along different trials on equal stimuli.

![A worst case analysis of scan path generation based on early salience (top left image, via ) vs. scan paths eye tracked from human subjects. Despite of inter-subject variability concerning the gaze relocation pattern and fixation duration time, human observers consistently fixate on the two faces. The simulated scan path fails to spot such semantically important objects that have not been highlighted as regions of high contrast in colour, texture and luminance. The area of yellow disks marking human fixations is proportional to fixation time ( dataset).[]{data-label="Fig:itti"}](FigItti.jpg)

Randomness in motor responses is likely to be originated from endogenous stochastic variations that affect each stage between a sensory event and the motor response: sensing, information processing, movement planning and executing [@vanBeers2007sources]. It is worth noting that uncertainty comes into play since the earliest stage of visual processing: the human retina evolved such that high quality vision is restricted to the small part of the retina (about $2^{0}-5^{0}$ degrees of visual angle) aligned with the visual axis, the fovea at the centre of vision. Thus, for many visually-guided behaviours the coarse information from peripheral vision is insufficient [@strasburger2011peripheral]. In certain circumstances, uncertainty may promote almost “blind" visual exploration strategies [@tatler2009prominence; @over2007coarse], much like the behaviour of a foraging animal exploring the environment under incomplete information; indeed when animals have limited information about where targets (e.g., resource patches) are located, different random search strategies may provide different chances to find them .

Indeed, few works have been trying to cope with the variability issue, after the early work by , . The glorious WTA scheme [@IttiKoch98], or variants such as the selection of the proto-object with the highest attentional weight [@anna] are deterministic procedures. Even when probabilistic frameworks are used to infer where to look next, the final decision is often taken via the maximum a posteriori (MAP) criterion which again is a deterministic procedure (technically, an $\arg\max$ operation, see ), or variants like the robust mean (arithmetic mean with maximum value) over candidate positions [@begum2010probabilistic]. As a result, for a chosen visual input $\mathbf{I}$ the mapping $\mathcal{W} \mapsto  \{\mathbf{r}_{F}(1), \mathbf{r}_{F}(2),\cdots \}$ will always generate the same scan path across different trials.

As a last remark, the variability of visual scan paths has been considered a nuisance rather than an opportunity from a modelling standpoint. Nevertheless, beside theoretical relevance for modelling human behavior, the randomness of the process can be an advantage in computer vision and learning tasks. For instance, have reported that a stochastic attention selection mechanism (a refinement of the algorithm proposed in ) enables the i-Cub robot to explore its environment up to three times faster compared to the standard WTA mechanism [@IttiKoch98]. Indeed, stochasticity makes the robot sensitive to new signals and flexibly change its attention, which in turn enables efficient exploration of the environment as a basis for action learning [@nagai2009stability; @nagai2009bottom].

There are few notable exceptions to this current state of affairs, which will be discussed in Section \[sec:prior\].

Framing models in a probabilistic setting {#sec:prob}
=========================================

We contend with the above issues by stating that observables such as fixation duration and gaze shift lengths and directions are random variables (RVs) that are generated by an underlying stochastic process. In other terms, the sequence $\{\mathbf{r}_{F}(1), \mathbf{r}_{F}(2),\cdots \}$ is the realization of a stochastic process, and the ultimate goal of a computational theory is to develop a mathematical model that describes statistical properties of eye movements as closely as possible. The problem of answering the question *Where to Look Next?* in a formal way can be conveniently set in a probabilistic Bayesian framework. have re-phrased this question in terms of the posterior probability density function (pdf) $P(\mathbf{r} \mid \mathcal{W})$, which accounts for the plausibility of generating the gaze shift $\mathbf{r} = \mathbf{r}_{F}(t) - \mathbf{r}_{F}(t-1)$, after the perceptual evaluation $\mathcal{W}$. Formally, via Bayes’ rule: $$P(\mathbf{r} \mid \mathcal{W})=  \frac{ P(\mathcal{W} \mid \mathbf{r} )}{P(\mathcal{W})}  P(\mathbf{r}).
\label{eq:BayesTatler}$$

In Eq. \[eq:BayesTatler\], the first term on the r.h.s. accounts for the likelihood $P(\mathcal{W} \mid \mathbf{r})$ of $\mathbf{r}$ when visual data (e.g., features, such as edges or colors) are observed under a gaze shift $\mathbf{r}_{F}(t) \rightarrow \mathbf{r}_{F}(t+1)$, normalized by $P(\mathcal{W})$, the evidence of the perceptual evaluation. As they put it, “The beauty of this approach is that the data could come from a variety of data sources such as simple feature cues, derivations such as Itti’s definition of salience, object-or other high-level sources”. The second term is the pdf $P(\mathbf{r})$ incorporating prior knowledge on gaze shift execution.

The generative model behind Eq. \[eq:BayesTatler\] is shown in Fig. \[fig:tata\] shaped in the form of a Probabilistic Graphical Model (PGM, see for an introduction). A PGM is a graph where nodes (e.g., $\mathbf{r}$ and $\mathcal{W}$) denote RVs and directed arcs (arrows) encode conditional dependencies between RVs, e.g $P(\mathcal{W} \mid \mathbf{r})$. A node with no input arcs (for example $\mathbf{r}$) is associated with a prior probability, e.g., $P(\mathbf{r})$. Technically, as a whole, the PGM specifies at a glance a chosen factorization of the joint probability of all nodes. Thus, in Fig. \[fig:tata\] we can promptly read that $P(\mathcal{W} , \mathbf{r}) = P(\mathcal{W} \mid \mathbf{r})P(\mathbf{r})$. The PGM in Fig. \[fig:tatb\] represents the PGM in Fig. \[fig:tata\], but unrolled in time. Note that now the arc $\mathbf{r}_{F}(t) \rightarrow \mathbf{r}_{F}(t+1)$ makes explicit the dynamics of the gaze shift occurring with probability $P(\mathbf{r}_{F}(t+1) \mid \mathbf{r}_{F}(t))$.

\[Fig:tat\]

![The dynamic PGM obtained by unrolling in time the PGM depicted in Fig. \[fig:tata\][]{data-label="fig:tatb"}](FigTatler.jpg)

![The dynamic PGM obtained by unrolling in time the PGM depicted in Fig. \[fig:tata\][]{data-label="fig:tatb"}](FigTatlerDyn.jpg)

The probabilistic model represented in Fig. \[Fig:tat\] is generative in the sense that if all pdfs involved were fully specified, the attentive process could be simulated (via ancestral sampling, ) as:

1.  Sampling the gaze shift from the prior: $$\mathbf{r}^{*} \sim P(\mathbf{r});
    \label{eq:priorTatlersamp}$$

2.  Sampling the observation of the world under the gaze shift: $$\mathcal{W} ^{*} \sim P(\mathcal{W} \mid \mathbf{r}^{*}).
    \label{eq:likeTatlersamp}$$

Inferring the gaze shift $\mathbf{r}$ when $\mathcal{W}$ is known boils down to the inverse probability problem (inverting the arrows), which is solved via Bayes’ rule (Eq. \[eq:BayesTatler\]). In the remainder of this paper we will largely use PGMs to simplify the presentation and discussion of probabilistic models.

We will see in brief (Section \[sec:like\]) that many current approaches previously mentioned can be accounted for by the likelihood term alone. But, crucial, and related to issues raised in Section \[sec:bias\], is the Bayesian prior $P(\mathbf{r})$.

The prior, first {#sec:prior}
----------------

The prior $P(\mathbf{r})$ can be defined *prima facie* as the probability of shifting the gaze to a location *irrespective of the visual information* at that location, although the term “irrespective” should be used with some caution [@le2016introducing]. Indeed, the prior is apt to encapsulate any systematic tendency in the manner in which we explore scenes with our eyes. The striking result obtained by is that if we learn $P(\mathbf{r})$ from the actual observer’s behavior, then we can stochastically sample gaze shifts (Eq. \[eq:priorTatlersamp\]) so to obtain scan paths that, blind to visual information, out-perform feature-based accounts of eye guidance.

Note that the apparent simplicity of the prior term $P(\mathbf{r})$ hides a number of subtleties. For instance, Tatler and Vincent expand the random vector $\mathbf{r}$ in terms of its components, amplitude $l$ and direction $\theta$. Thus, $P(\mathbf{r})= P(l, \theta)$. This simple statement paves the way to different options.

First easy option: such RVs are marginally independent, thus, $P(l, \theta) = P(l) P(\theta)$. In this case, gaze guidance, solely relying on biases, could be simulated by expanding Eq. \[eq:priorTatlersamp\] via independent sampling of both components, i.e. at each time $t$, $l(t) \sim P(l(t)), \theta(t) \sim P(\theta(t))$. Alternative option: conjecture some kind of dependency, e.g. amplitude on direction so that $P(l, \theta) = P(l \mid \theta) P(\theta)$. In this case, the gaze shift sampling procedure would turn into the sequence $\widehat{\theta}(t) \sim P(\theta(t)), l(t) \sim P(l(t) \mid \widehat{\theta}(t) )$. Further: assume that there is some persistence in the direction of the shift, which give rise to a stochastic process in which subsequent directions are correlated, i.e., $\theta(t) \sim P(\theta(t) \mid \theta(t-1))$, and so on.

To summarize, by simply taking into account the prior $P(\mathbf{r})$, a richness of possible behaviors and analyses are brought into the game. Unfortunately, most computational accounts of eye movements and visual attention have overlooked this opportunity, with some exceptions. For instance, propose a system model for saccade generation in a stochastic filtering framework. A prior on amplitude $P(l(t))$ is considered by learning a Gaussian mixture model from eye tracking data. This way one aspect of biases is indirectly taken into account. It is not clear if their model accounts for variability and whether and how oculomotor statistics compare to human data. In [@kimura2008dynamic], simple eye-movements patterns are straightforwardly incorporated as a prior of a dynamic Bayesian network to guide the sequence of eye focusing positions on videos.

In a different vein, have recently addressed in-depth the bias problem and made the interesting point that viewing tendencies are not universal, but modulated by the semantic visual category of the stimulus. They learn the joint pdf $P(l, \theta)$ of saccade amplitudes and orientations via kernel density estimation; fixation duration is not taken into account. The model also brings in variability [@le2015saccadic] by generating a number $N_c$ of random locations according to conditional probability $P(\mathbf{r}_{F}(t) \mid \mathbf{r}_{F}(t-1))$ and the location with the highest saliency gain is chosen as the next fixation point. $N_c$ controls the degree of stochasticity.

Others have tried to capture eye movements randomness [@keech2010eye1; @rutishauser2007probabilistic] but limiting to specific tasks such as conjunctive visual search. A few more exceptions can be found, but only in the very peculiar field of eye-movements in reading (see , for a discussion).

The variability and bias issues have been explicitly addressed from first principles in the theoretical context of Lévy flights [@brockgeis; @bfpha04]. The perceptual component was limited to a minimal core (e.g., based on a bottom-up salience map) sufficient enough to support the eye guidance component. In particular in [@bfpha04] the long tail, positively skewed distribution of saccade amplitudes was shaped as a prior in the form of a Cauchy distribution, whilst randomness was addressed at the algorithmic level by prior sampling $\mathbf{r} \sim P(\mathbf{r}(t))$ followed by a Metropolis-like acceptance rule based on a deterministic saliency potential field. The degree of stochasticity was controlled via the “temperature” parameter of the Metropolis algorithm. The underlying eye guidance model was that of a random walker exploring the potential landscape (salience) according to a Langevin-like stochastic differential equation (SDE). The merit of such equation is the joint treatment of both the deterministic and the stochastic (variability) components behind eye guidance[^1].

This basic mechanism has been refined and generalized in [@BocFerAnnals2012] to composite $\alpha$-stable or Lévy random walks (the Cauchy law is but one instance of the class of $\alpha$-stable distributions), where, inspired by animal foraging behaviour, a twofold regime can be distinguished: local exploitation (fixational movements following Brownian motion) and large exploration/relocation (saccade following Lévy motion). What is interesting, with respect to the early model [@bfpha04], is that the choice between the “feed” or “fly” states is made by sampling from a Bernoulli distribution, $Bern(z \mid \pi)$, with the parameter $\pi$ sampled from the conjugate prior $Beta(\pi \mid \alpha, \beta)$. In turn, the behaviour of the Beta prior can be shaped via its hyperparameters $(\alpha, \beta)$, which, in an Empirical Bayes approximation, can be tuned as a function of the class of perceptual data at hand (in the vein of ) and of time spent in feeding (fixation duration). Most important, this approach paves the way to the possibility of treating visual exploration strategies in terms of *foraging* strategies [@wolfe2013time; @cain2012bayesian; @BocFerSMCB2013; @BocCOGN2014; @napboc_TIP2015]. We will further expand on this in Section \[sec:action\].

The unbearable lightness of the likelihood {#sec:like}
------------------------------------------

We noticed before, by inspecting Eq. \[eq:BayesTatler\] that the term $\frac{ P(\mathcal{W} \mid \mathbf{r} )} {P(\mathcal{W})} $ could be related to many models proposed in the literature. This is an optimistic view. Most of the approaches actually discard the dynamics of gaze shifts implicitly captured by the shift vector $\mathbf{r}(t)$. In practice, they are more likely to be described by a simplified version of Eq. \[eq:BayesTatler\]: $$P(\mathbf{r}_{F} \mid \mathcal{W}) =  \frac{P(\mathcal{W} \mid \mathbf{r}_{F})} {P(\mathcal{W})}   P(\mathbf{r}_{F}).
\label{eq:BayesTatler2}$$

The difference between Eq. \[eq:BayesTatler\] and \[eq:BayesTatler2\] is subtle. The posterior $P(\mathbf{r}_{F} \mid \mathcal{W}) $ now answers the query “What is the probability of *fixating* at location $\mathbf{r}_{F}$ given visual data $\mathcal{W}$?” Further, the prior $P(\mathbf{r}_{F})$ simply accounts for the probability of spotting location $\mathbf{r}_{F}$. As a matter of fact, Eq. \[eq:BayesTatler2\] bears no dynamics.

In probabilistic terms we may re-phrase this result as the outcome of an assumption of independence: $P(\mathbf{r})   = P(\mathbf{r}_{F}(t) - \mathbf{r}_{F}(t-1)) \nonumber
  \simeq P(\mathbf{r}_{F}(t) \mid \mathbf{r}_{F}(t-1)) = P(\mathbf{r}_{F}(t))$. To make things even clearer, let us explicitly substitute $\mathbf{r}_{F}$ with a RV $\mathbf{L}$ denoting locations in the scene, and $\mathcal{W}$ with RV $\mathbf{F}$ denoting features (whatever they may be); then, Eq. \[eq:BayesTatler2\] boils down to $$P(\mathbf{L} \mid \mathbf{F}) =  \frac{P(\mathbf{F} \mid \mathbf{L})} {P(\mathbf{F})}  P(\mathbf{L}).
\label{eq:models}$$ The PGM underlying this inferential step is a very simple one and is represented in Figure \[fig:simple\]. A straightforward but principled use of Eq. \[eq:models\], which has been exploited by approaches that draw upon techniques borrowed from statistical machine learning [@murphy2012machine] is the following: consider $\mathbf{L}$ as a binary RV taking values in $\left[0,1\right]$ (or $\left[ -1, 1 \right]$), so that $P(\mathbf{L} = 1 \mid \mathbf{F})$ represents the probability for a pixel, a superpixel or a patch of being classified as salient.

![An extension of the PGM by , cfr. Fig. \[fig:poggio\], adopted in [@BocCOGN2014] and [@napboc_TIP2015]. In this case $\mathcal{W}$ explicitly depends on the current gaze position $\mathbf{r}_{F}$ and goal $\mathcal{G}$.[]{data-label="fig:myPGM"}](FigPGMsimple.jpg)

![An extension of the PGM by , cfr. Fig. \[fig:poggio\], adopted in [@BocCOGN2014] and [@napboc_TIP2015]. In this case $\mathcal{W}$ explicitly depends on the current gaze position $\mathbf{r}_{F}$ and goal $\mathcal{G}$.[]{data-label="fig:myPGM"}](FigPGMTorralba.jpg)

![An extension of the PGM by , cfr. Fig. \[fig:poggio\], adopted in [@BocCOGN2014] and [@napboc_TIP2015]. In this case $\mathcal{W}$ explicitly depends on the current gaze position $\mathbf{r}_{F}$ and goal $\mathcal{G}$.[]{data-label="fig:myPGM"}](FigPGMPoggio.jpg)

![An extension of the PGM by , cfr. Fig. \[fig:poggio\], adopted in [@BocCOGN2014] and [@napboc_TIP2015]. In this case $\mathcal{W}$ explicitly depends on the current gaze position $\mathbf{r}_{F}$ and goal $\mathcal{G}$.[]{data-label="fig:myPGM"}](FigPGMmy.jpg)

\[Fig:models\]

In the case the prior $P(\mathbf{L})$ is assumed to be uniform (no spatial bias, no preferred locations), then $ P(\mathbf{L} = 1 \mid \mathbf{F}) \simeq P(  \mathbf{F} \mid  \mathbf{L} = 1) $. The likelihood function $P(  \mathbf{F} \mid  \mathbf{L} = 1)$ can be determined in many ways; e.g., nonparametric kernel density estimation has been addressed by , who use center / surround local regression kernels for computing $\mathbf{F}$.

More generally, taking into account the ratio $f(\mathbf{L})=\frac{P(\mathbf{L} = 1 \mid \mathbf{F})}{P(\mathbf{L} = 0 \mid \mathbf{F})}$ (or, commonly, the log-ratio) casts the saliency detection problem in a classification problem, in particular a discriminative one [@murphy2012machine], for which a variety of learning techniques are readily available. pioneered this approach by learning the saliency discriminant function $f(\mathbf{L})$ directly from human eye tracking data using a support vector machine (SVM). Their approach has paved the way to a relevant number of works from [@judd2009learning] – who trained a linear SVM from human fixation data using a set of low, middle and high-level features to define salient locations–, to most recent ones that wholeheartedly endorse machine learning trends. Henceforth, methods have been proposed relying on sparse representation of “feature words” (atoms) encoded in salient and non-salient dictionaries; these are either learned from local image patches [@yan2010visual; @lang2012saliency] or from eye tracking data of training images [@jiang2015image]. Graph-based learning is one other trend, from the seminal work of to (see the latter, for a brief review of this field). Crucially, for the research practice, data-driven learning methods allow to contend with large scale dynamic datasets. in the vein of and use SVM, but they remarkably exploit state-of-the art computer vision datasets (Hollywood-2 and UCF Sports) annotated with human eye movements collected under the ecological constraints of a visual action recognition task.

As a general comment on (discriminative) machine learning-based methods, on the one hand it is embraceable the criticism by , who surmise that these techniques make “models data-dependent, thus influencing fair model comparison, slow, and to some extent, black-box.” But on the other hand, one important lesson of these approaches lies in that they provides a data-driven way of deriving the most relevant visual features as optimal predictors. The learned patterns can shape receptive fields (filters) that have equivalent or superior predictive power when compared against hand-crafted (and sometimes more complicated) models [@kienzle2009center]. Certainly, this lesson is at the base of the current exponentially growth of methods based on deep learning techniques [@lecun2015deep], in particular Convolutional Neural Networks (CNN, cfr. for a focused review), where the computed features seem to outperform, at least from an engineering perspective, most of, if not all, the state-of-the art features conceived in computer vision.

Again, CNNs, as commonly exploited in the current practice, bring no significant conceptual novelty as to the use of Eq. \[eq:models\]: fixation prediction is formulated as a supervised binary classification problem (in some case, regression is addressed, ). For example, use a linear SVM for learning the saliency discriminant function $f(\mathbf{L})$ after a large-scale search for optimal features $\mathbf{F}$. Similarly, detect salient region via linear SVM fed with features computed from multi-layer sparse network model. [@lin2014saliency] use the simple normalization step [@IttiKoch98] to approximate $P(\mathbf{L} = 1 \mid \mathbf{F})$, where [@kruthiventi2015deepfix] use the last $1 \times 1$ convolutional layer of a fully convolutional net. Cogent here is the outstanding performance of CNN in learning and representing features that correlate well with eye fixations, like objects, faces, context.

Clearly, one problem is the enormous amount of training data necessary to train these networks, and the engineering expertise required, which makes them difficult to apply for predicting saliency. However, by exploiting the well known network from [@AlexNetNIPS2012] as starting point, have given evidence that deep CNN trained on computer vision tasks like object detection boost saliency prediction. The network by has been optimized for object recognition using a massive dataset consisting of more than one million images, and results reported by on static pictures are impressive when compared to state-of-the-art methods, even to previous CNN-based proposals [@vig2014large].

Apart from the straightforward implementation via popular machine-learning algorithms, the “light” model described by Eq. \[eq:models\] is further amenable to a minimal model, which, surprisingly enough, is however capable of accounting for a large number of approaches. This can be easily appreciated by setting $P(\mathbf{F} \mid \mathbf{L}) = const.,  P(\mathbf{L})=const.$ so that Eq. \[eq:models\] reduces to $$P(\mathbf{L} \mid \mathbf{F}) \propto  \frac{1}{P(\mathbf{F})}.
\label{eq:modelItti}$$ Eq. \[eq:modelItti\] states that the probability of fixating a spatial location $\mathbf{L}= (x,y)$ is higher when “unlikely” features (unlikeliness $ \approx \frac{1}  {P(\mathbf{F})}$) occur at that location. In a natural scene, it is typically the case of high contrast regions (with respect to either luminance, color, texture or motion). This is nothing but the salience-based component of the most prominent model in the literature [@IttiKoch98], which Eq. \[eq:modelItti\] re-phrases in probabilistic terms.

A thorough reading of the review by is sufficient to gain the understanding that a great deal of computational models so far proposed (47 over 63 models) are much or less variations of this theme (albeit experimenting with different features, different weights for combining them, etc.) even when sophisticated probabilistic techniques are adopted to shape the distribution $P(\mathbf{F})$ (e.g., nonparametric Bayes techniques, ). Clearly, there are works that have tried to avoid weaknesses related to such a light-modelling of the perceptual input, and have tried to climb up the levels of the representation hierarchy [@schutz2011eye]. Some examples are summarized at a glance in Figure \[Fig:models\] (but see ).

Nevertheless, in spite of its simplicity, Eq. \[eq:modelItti\] is apt to pave the way to interesting frameworks. For instance, by noting that $\log \frac{1}  {P(\mathbf{F})}$ is nothing but Shannon’s Self- Information, information theoretic approaches become available at the algorithmic level. These approaches set computational constraints under the general assumption that saliency computation serves to maximize information sampled from the environment [@bruce2009saliency].

Keeping on with the information theory framework, and going back to Eq. \[eq:models\], a simple manipulation, $$\log P(\mathbf{L} \mid \mathbf{F}) - \log P(\mathbf{L}) =  \log \frac{P(\mathbf{F} \mid \mathbf{L})} {P(\mathbf{F})}  
\label{eq:modelsKL}$$ sets the focus on the discrepancy, or dissimilarity, $\log P(\mathbf{L} \mid \mathbf{F}) - \log P(\mathbf{L}) = \log \frac{P(\mathbf{L} \mid \mathbf{F})}{P(\mathbf{L})}$ between the log-posterior and the log-prior. A (non-commutative) measure, formalizing this notion of dissimilarity is readily available in information theory, namely the Kullback-Leibler (K-L) divergence between two distributions $P(X)$ and $Q(X)$ [@Mackay]: $$D_{KL}(P(X) || Q(X))= \int_X  \log \frac{P(x)} {Q(x)} P(x) dx  
\label{eq:KL}$$ Measuring differences between posterior and prior beliefs of the observers is however a general concept applicable across different levels of abstraction. For instance, one might consider the object-based model [@torralba2006contextual] in Fig. \[fig:torralba\], which can be used for inferring the joint posterior of gazing at certain kinds of objects $\mathbf{O}$ at location $\mathbf{L}$ of a viewed scene, namely, $P(\mathbf{O},\mathbf{L} \mid \mathbf{F}) \propto P(\mathbf{F} \mid \mathbf{O},\mathbf{L}) P(\mathbf{O},\mathbf{L})$. Then, $D_{KL}(P(\mathbf{O},\mathbf{L} \mid \mathbf{F}) || P(\mathbf{O},\mathbf{L}))$ is the average of the log-odd ratio, measuring the divergence between observer’s prior belief distribution on $(\mathbf{O},\mathbf{L})$ and his posterior belief distributions after perceptual data $\mathbf{F}$ have been gathered. Indeed, this is a statement that can be generalized to any model $\mathbf{M}$ in a model space $\mathcal{M}$ and new data observation $\mathbf{D}$ so to define the Bayesian surprise [@balditti2010] $\mathcal{S}(\mathbf{D},\mathbf{M})$: $\mathbf{D}$ is surprising if the posterior distribution resulting from observing $\mathbf{D}$ significantly differs from the prior distribution, i.e., $S(\mathbf{D},\mathbf{M}) = D_{KL}(P(\mathbf{M} \mid \mathbf{D} ) || P(\mathbf{M}))$. have shown that Bayesian surprise attracts human attention in dynamic natural scenes. To recap, Bayesian surprise is a measure of salience based on the K–L divergence.

Eventually, note that the K-L divergence (\[eq:KL\]) is a flexible tool and can be used for different purposes. For instance, when dealing with models of perceptual evaluation such as those specified in Figs \[fig:torralba\], \[fig:poggio\], and \[fig:myPGM\], once the model has been detailed at the computational theory level via its PGM, then using the latter for learning inference and prediction brings in the algorithmic level. Indeed, for any Bayesian generative model other than trivial ones, such steps are usually performed in approximate form [@murphy2012machine]. Stochastic approximation resorting to algorithms such as Markov-chain Monte Carlo (MCMC) and Particle Filtering (PF) is one possible choice; the alternative choice is represented by deterministic optimization algorithms [@murphy2012machine] such as variational Bayes (VB) or belief propagation (BP, a message passing scheme exchanging beliefs between PGM nodes). For example, the model by , following the work of , relies upon BP message passing for inferential steps. Interestingly enough, has argued for a plausible neural implementation of BP. VB algorithms, on the other hand, are based on Eq. \[eq:KL\], where $P$ usually stands for a complete distribution and $Q$ is the approximating distribution; then, parameter (or model) learning is accomplished by minimizing the K-L divergence (as an example, the well known Expectation-Maximization algorithm, EM, can be considered a specific case of the VB algorithm, ). In Section \[sec:action\] we will also touch on a deeper interpretation of the K-L minimization / VB algorithm. But at this point a simple question arises: where have the eye movements gone?

Making a step forward: back to the beginning of active vision {#sec:action}
=============================================================

Visual perception coupled with gaze shifts should be considered the *Drosophila* of perception-action loops. Among the variety of active behaviors the organism can fluently engage to purposively act upon and perceive the world (e.g, moving the body, turning the head, manipulating objects), oculomotor behavior is the minimal, least energy, unit. To perform $3$-$4$ saccades per second, the organism roughly spends $300$ msecs to close the loop ($200$ msecs for motor preparation and execution, $100$ msecs left for perception).

![The perception-action loop unfolded in time as a dynamic PGM. $\mathcal{A}(t)$ denotes the ensemble of time-varying RVs defining the oculomotor action setting; $\mathcal{W}(t)$ stands for the ensemble of time-varying RVs characterising the scene as actively perceived by the observer; $\mathcal{G}$ summarizes the given goal(s), To simplify the graphics, conditional dependencies $\mathcal{G} \rightarrow \mathcal{A}(t+1)$ and $\mathcal{G} \rightarrow \mathcal{W}(t+1)$ have been omitted.[]{data-label="Fig:loop"}](FigLoop.jpg)

One way to make justice of this forgotten link is going back to first principles by re-shaping the problem as the action-perception loop, which is presented in Figure \[Fig:loop\] in the form of a dynamic PGM. The model relies upon the following assumptions:

-   The scene that will be perceived at time $t+1$, namely $\mathcal{W}(t+1)$ is inferred from the raw data $\mathbf{I}$, gazed at $\mathbf{r}_{F}(t+1)$, under the goal $\mathcal{G}$ assigned to the observer, and is conditionally dependent on current perception $\mathcal{W}(t)$. Thus, the perceptual inference problem is summarised by the conditional distribution $P( \mathcal{W}(t+1)|\mathcal{W}(t), \mathbf{r}_{F}(t+1), \mathbf{I},\mathcal{G})$;

-   The external goal $\mathcal{G}$ being assigned, the oculomotor action setting at time $t+1$, $\mathcal{A}(t+1)$, is drawn conditionally on current action setting $\mathcal{A}(t)$ and the perceived scene $\mathcal{W}(t+1)$ under gaze position $\mathbf{r}_{F}(t+1)$; thus, its evolution in time is inferred according to the conditional distribution $P(\mathcal{A}(t+1) | \mathcal{A}(t), \mathcal{W}(t+1), \mathbf{r}_{F}(t+1), \mathcal{G})$.

Note that the action setting dynamics $\mathcal{A}(t) \rightarrow \mathcal{A}(t+1)$ and the scene perception dynamics $\mathcal{W}(t) \rightarrow \mathcal{W}(t+1)$ are intertwined with one another by means of the gaze shift process $\mathbf{r}_{F}(t) \rightarrow \mathbf{r}_{F}(t+1)$: on the one hand next gaze position $\mathbf{r}_{F}(t+1)$ is used to define a distribution on $\mathcal{W}(t+1)$ and $\mathcal{A}(t+1)$; meanwhile, the probability distribution of $\mathbf{r}_{F}(t+1)$ is conditioned on current gaze position, $\mathcal{W}(t)$ and $\mathcal{A}(t)$, namely $P(\mathbf{r}_{F}(t+1)| \mathcal{A}(t), \mathcal{W}(t), \mathbf{r}_{F}(t))$.

We have previously discussed the perceptual evaluation component $\mathcal{W}(t)$. A general way of defining the oculomotor executive control component $\mathcal{A}(t)$ is through the following ensemble of RVs:

-   $\{\mathbf{V}(t), \mathbf{R}(t)\}$: $\mathbf{V}(t)$ is a spatially defined RV used to provide a suitable probabilistic representation of value; $\mathbf{R}(t)$ is a binary RV defining whether or not a payoff (either positive or negative) is returned;

-   $\{\pi(t), z(t),   \xi(t) \}$: an *oculomotor state representation* as defined via the multinomial RV $z(t)$, occurring with probability $\pi(t)$, and determining the choice of motor parameters $\xi(z,t)$ guiding the actual gaze relocation (e.g., lenght and direction of a saccade as opposed to those driving a smooth pursuit) ;

-   $\mathcal{D}(t)$: a set of state-dependent statistical decision rules to be applied on a set of candidate new gaze locations $\mathbf{r}_{new}(t+1)$ distributed according to the posterior pdf of $\mathbf{r}_{F}(t+1)$.

In the end, the actual shift can be summarised as the statistical decision of selecting a particular gaze location $\mathbf{r}^{\star}_{F}(t+1)$ on the basis of $P(\mathbf{r}_{F}(t+1)| \mathcal{A}(t), \mathcal{W}(t), \mathbf{r}_{F}(t))$ so to maximize the expected payoff under the current goal $\mathcal{G}$, and the action/perception cycle boils down to the iteration of the following steps:

1.  Sampling the gaze-dependent current perception: $$\mathcal{W}^{*}(t) \sim P( \mathcal{W}(t)|\mathbf{r}_{F}(t),\mathbf{F}(t), \mathbf{I}(t),\mathcal{G});
    \label{eq:step1}$$

2.  Sampling the appropriate motor behavior (e.g., fixation or saccade): $$\mathcal{A}(t)^{*} \sim P(\mathcal{A}(t) | \mathcal{A}(t-1), \mathcal{W}^{*}(t),\mathcal{G});
    \label{eq:step2}$$

3.  Sampling where to look next: $$\mathbf{r}_{F}(t+1) \sim P(\mathbf{r}_{F}(t+1)| \mathcal{A}(t)^{*} , \mathcal{W}^{*}(t), \mathbf{r}_{F}(t)).
    \label{eq:step3}$$

It is worth noticing that we have chosen to describe the observer’s action / perception cycle in terms of stochastic sampling based on the probabilistic model in Figure \[Fig:loop\]. However, one can recast the inferential problems in terms of deterministic optimization: in brief, optimising the probabilistic model $\mathcal{W}$ of how sensations are caused, so that the resulting predictions can select the optimal $\mathcal{A}$ to guide active sampling (gaze shift) of sensory data. One such approach, which is well known in theoretical neuroscience but, surprisingly, hitherto unconsidered in computer vision, relies on the free-energy principle [@friston2010free; @feldman2010attention; @friston2013anatomy]. Free-energy $\mathcal{F}$ is a quantity from statistical physics and information theory [@Mackay] that bounds the negative log-evidence of sensory data. Under simplifying assumptions, it boils down to the amount of prediction error of sensory data under a model. In such context, the action / perception cycle is the result of a dual minimization process: i) action reduces $\mathcal{F}$ by changing sensory input, namely by sampling (via gaze shifts) what one expects consistent with perceptual inferences; ii) perception reduces $\mathcal{F}$ by making inferences about the causes of sampled sensory signals and changing predictions. Friston defines this process “active inference”. To make a connection with Section \[sec:like\], by minimizing the free-energy, Bayesian surprise is maximised; indeed, in Bayesian learning, free energy minimization is a common rationale behind many optimisation techniques such as VB and BP [@Mackay; @murphy2012machine]

The sampling scheme proposed is a general one and can be instantiated in different ways. For instance, in [@BocFerSMCB2013] the sampling step of Eq. \[eq:step3\] is performed through a generalization of the Langevin SDE equation used in [@BocFerAnnals2012]. Biases and variability are accounted for by the stochastic component of the equation. Since dealing with image sequences, the SDE provides operates in different dynamic modes: pursuit needs to be taken also into account in addition to saccades and fixational movements. Each mode is governed by a specific set of parameters of the $\alpha$-stable distribution estimated from eye tracking data. The choice among modes is accomplished by generalizing the method proposed in [@BocFerSMCB2013], using a Multinoulli distribution on $z(t)$ and with parameters $\pi(t)$ sampled from the conjugate prior, the Dirichlet distribution. In addition, the sampling step is used to accomplish an internal simulation step, where a number of candidates shifts is proposed and the most convenient is selected according to a decision rule. Also, the gaze dependent perception $\mathcal{W}$ can be modeled at any level of complexity (cfr. example in Figure \[Fig:levels\], Section \[sec:emo\]). In it is based on proto-objects sampled from time-varying saliency[^2]. Figure \[Fig:monica\] shows an excerpt of typical results of the model simulation, which compares human variability in gazing (top row) with that of two “simulated observers” (bottom rows) while viewing the `monica03` clip from the CRCNS eye-1 dataset.

![Top row: gaze positions (centers of the enhanced colored circles ) recorded from different observers on different frames of the `monica03` clip from the CRCNS eye-1 dataset, University of South California (freely available online). Middle and bottom rows show, on the same frames, the fovea position of two “observers” simulated by the method described in [@BocFerSMCB2013].[]{data-label="Fig:monica"}](Figmonica03.jpg)

In and the perceptual evaluation component is extended to handle objects and task (external goal) levels by expanding on (cfr. Figure \[Fig:models\]), and the decision rule $\mathcal{D}(t)$ concerning the selection of the gaze is based on the expected reward according to the given goal $\mathcal{G}$ (see Section \[sec:emo\]).

Meanwhile, nothing prevents to conceive more general perceptual evaluation and executive control components, by considering perceptual and action modalities other than the visual ones in the vein of , where eye movements and hand actions have been coupled with the goal of performing a drawing task.

But most important, the action-perception cycle is by and large conceived in the foraging framework (see , for a thorough introduction), which at the most general level is summarized in Table \[metaphor\]. Visual foraging corresponds to the time-varying overt deployment of visual attention achieved through oculomotor actions, namely, gaze shifts. The forager feeds on patchily distributed preys or resources, spends its time traveling between patches or searching and handling food within patches. While searching, it gradually depletes the food, hence, the benefit of staying in the patch is likely to gradually diminish with time. Moment to moment, striving to maximize its foraging efficiency and energy intake, the forager should make decisions: Which is the best patch to search? Which prey, if any, should be chased within the patch? When to leave the current patch for a richer one?

The spatial behavioral patterns exhibited by foraging animals (but also those detected in human mobility data) are remarkably close to those generated by gaze shifts [@viswanathan2011physics]. Figure \[Fig:spider\] presents an intriguing example in this respect.

![Monkey or human: can you tell the difference? The center image has been obtained by superimposing a typical trajectory of spider monkeys foraging in the forest of the Mexican Yucatan, as derived from [@ramos2004levy], on the “party picture” (left image) used in [@brockgeis]. The right image is an actual human scan path (modified after ) []{data-label="Fig:spider"}](FigParty.jpg)

The fact that the physics underlying foraging overlaps with that of several other kinds of complex random searches and stochastic optimization problems [@viswanathan2011physics], and notably with that of visual exploration via gaze shifts [@viswanathan2011physics; @marlow2015temporal; @brockgeis], makes available a variety of analytical tools beyond classic metrics exploited in computer vision or psychology. For instance, in Figure \[Fig:ccdf\], it is shown how the gaze shift amplitude modes from human observers can be compared with those generated via simulation by using the complementary Cumulative Distribution Function (CCDF), which provides a precise description of the distribution of the gaze shift by considering its upper tail behavior. This can be defined as $\overline{F}(x)=P(| \mathbf{r}|>x)=1 - F(x)$, where $F$ is the cumulative distribution function (CDF) of amplitudes. Consideration of the upper tail, i.e. the CCDF of jump lengths is a standard convention in foraging, human mobility, and anomalous diffusion research [@viswanathan2011physics].

To sum up, foraging offers a novel perspective for formulating models and related evaluations of visual attention and oculomotor behavior [@wolfe2013time]. Unifying hypotheses such as the oculomotor continuum from exploration to fixation by can be reconsidered in the light of fundamental theorems of statistical mechanics [@weron2010generalization].

\[metaphor\]

Interestingly enough, the reformulation of visual attention in terms of foraging theory is not simply an informing metaphor. It has been argued that what was once foraging for tangible resources in a physical space became, over evolutionary time, foraging in cognitive space for information related to those resources [@hills2006animal], and such adaptations play a fundamental role in goal-directed deployment of visual attention [@wolfe2013time].

Bringing value into the game: a doorway to affective modulation {#sec:emo}
===============================================================

The introduction of a goal level, either exogenous (originating from outside the observer’s organism) or endogenous (internal) is not an innocent shift.

From a classical cognitive perspective, the assignment of a task to the observer implicitly defines a value for every point of the space, in the sense that information in some points is more relevant than in others for the completion of the task; the shifting of the gaze on a particular point, in turn, determines the payoff that can be gained.

There is a number of psychological and neurobiological studies showing the availability of value maps and loci of reward influencing the final gaze shift [@platt1999neural; @leon1999effect; @ikeda2003reward; @Hikosaka2006]. The payoff is nothing else that the value, with respect to the completion of the task, obtained by moving the fovea in a given position. Thus points associated with high values produce, when fixated, high payoffs since these fixations bring the observer closer to her/his goal. For instance, in [@BocCOGN2014] and in [@napboc_TIP2015], reward was introduced to make a choice among the candidate gaze shifts stochastically sampled according to Eq. \[eq:step3\], in terms of expected reward (e.g., tuned by the probability of finding the task-assigned object).

Figure \[Fig:levels\] presents one example of the different scan paths obtained by progressively reducing the levels of representation in the perceptual evaluation component presented in Figure \[fig:myPGM\] [@BocCOGN2014; @napboc_TIP2015].

![Different scan paths originated from the progressive reduction of representation levels in the perceptual evaluation component presented in Figure \[fig:myPGM\].[]{data-label="Fig:levels"}](FigLevels.jpg)

Value set by the given goal can weight differently the objects within the scene, thus purposively biasing the scan path. Gaze is still uniformly deployed to relevant items within the scene (people, text) when object-based representation is exploited. When salience alone is used, the generated scan path fails in accounting for the relevant items and bears no relation with the semantics which can be attributed to the scene.

The use of value and reward endow attentive models with the capability of handling complex task. For instance, considered the goal of text spotting in unconstrained urban environments, and its validation encompassed data gathered from a mobile eye tracking device [@clav2014]. extended the foraging framework to cope with the difficult problem of attentive monitoring of multiple video streams in a videosurveillance setting.

Yet, developing eye guidance models based on reward is a difficult endeavour and computational models that use reward and uncertainty as central components are still in their infancy (but see the discussion by ). In this respect the remarkable work by Ballard and collegues counters the stream. Whilst salience, proto-objects and objects are representations that have been largely addressed in the context of human eye movements, albeit with different emphasis, in contrast, value has been neglected until recently [@schutz2011eye]. One reason is that in the real world there is seldom direct payoff (no orange juice for a primary reward) for making good eye movements or punishment for bad ones.

However, the high attentional priority of ecologically pertinent stimuli can also be explained by mechanisms that do not implicate learning value through repeated pairings with reward. For example, a bias to attend to socially relevant stimuli is evident from infancy [@anderson2013value]. More generally, the selection of stimuli by attention has important implications for the survival and wellbeing of an organism, and attentional priority reflects the overall value of such selection (see for a discussion). Indeed, engaging attention with potentially harmful and beneficial stimuli guarantees that the relevant ones are selected early so to gauge the exact nature of the potential threat or opportunity and to readily initiate defensive or approach behavior.

Under these circumstances, has proposed a broad definition of reward, which includes “not only the immediate primary rewards, but also other factors: the preference for a novel location or stimulus, the satisfaction of performing well or the desire to complete a given task." Such definition is consistent with the different psychological facets of reward [@berridge2003parsing]: i) learning (including explicit and implicit knowledge produced by associative conditioning and cognitive processes); ii) affect or emotion (implicit “liking” and conscious pleasure); iii) motivation (implicit incentive salience “wanting” and cognitive incentive goals). Thus, value representation level is central to both goal-driven affective and cognitive engagement with stimuli in the outside world.

In this broader perspective, the effort to put value and reward into the game shows his inner worth in that, by accounting for the many aspects of “biological value" - salience, significance, unpredictability, affective content - , it paves the way to a wider dimension of information processing, as most recent results on the affective modulation of the visual processing stream advocate [@pessoa2008relationship; @pessoa2010emotion], and to the effective exploitation of computational attention models in the emerging domain of social signal processing [@vinciarelli2009social].

A number of important studies in the psychological literature (see, for a discussion, ) have addressed the relationship between overt attention behavior and emotional content of pictures. Many of them investigate specific issues related to individuals such as trait anxiety, social anxiety, spider phobia, and exploit restricted sets of stimuli such as emotional faces or spiders. In turn, the study by has exploited natural images and normal subjects, demonstrating an emotional bias both in attentional orienting and engagement among normal participants and using a wider range of emotional pictures. Results might be summarised as follows: i) emotionally pleasant and unpleasant pictures capture attention more readily than neutral pictures; ii) the emotional bias can be observed early in initial orienting and subsequent engagement of attention; iii) the early stimulus-driven attentional capture by emotional stimuli can be counteracted by goal-driven control in later stages of picture processing.

Peculiarly relevant to our case, have shown that visual saliency does influence eye movements, but the effect is reliably reduced when an emotional object is present. Pictures containing negative objects were recognized more accurately and recalled in greater detail, and participants fixated more on negative objects than positive or neutral ones. Initial fixations were more likely to be on emotional objects than more visually salient neutral ones. Consistently with , the overall result suggest that the processing of emotional features occurs at a very early stage of perception.

As a matter of fact, emotional factors are completely neglected in the realm of computational models of attention and gaze shifts. Some efforts have been spent in the field of social robotics, where motivational drives have an indirect influence on attention by influencing the behavioral context, which, in turn, is used to directly manipulate the gains of the attention system (e.g. by tuning the gains of different bottom-up saliency map, ). Yet, beyond these broadly related attempts, taking into account the specific influence of affect on eye-behaviour is not a central concern within this field (for a wide review, see ). Recent works in the image processing and pattern recognition community use eye tracking data for the inverse problem of the recognition of emotional content of images (e.g.,) or implicit tagging of videos [@soleymani2012multimodal]. Thus, they do not address the generative problem of how emotional factors contribute to the generation of gaze shifts in visual tasks.

By contrast, neuroscience has shown that, crucially, cognitive and emotional contributions cannot be separated, as outlined in Figure \[Fig:neuro\].

![Circuits for the processing of visual information and for the executive control [@pessoa2008relationship] show that action and perception components are inextricably linked through the mediation of the amygdala and the posterior orbitofrontal cortex (OFC). These structures are considered to be the neural substrate of emotional valence and arousal [@salzman2010emotion]. The ventral tegmental area (vTA) and the basal forebrain are responsible for diffuse neuromodulatory effects related to reward and attention. Anterior cingulate cortex (ACC) is likely to be involved in conflict detection and/or error monitoring but also in in computing the benefits and costs of acting by encoding the probability of reward. Nucleus accumbens is usually related to motivation.The lateral prefrontal cortex (LPFC) plays a central role at the cognitive level in maintaining and manipulating information, but also integrates this content with both affective and motivational information. Line thickness depicts approximate connection strength[]{data-label="Fig:neuro"}](FigNeuro.jpg)

The scheme presented, which summarises an ongoing debate [@pessoa2008relationship], shows that responses from early and late visual cortex reflecting stimulus significance will be a result of simultaneous top-down modulation from fronto-parietal attentional regions (LPFC) and emotional modulation from the amygdala [@Mohanty2013]. On the one hand, stimulus’ affective value appears to drive attention and enhance the processing of emotionally modulated information. On the other hand, exogenously driven attention influences the outcome of affectively significant stimuli [@pessoa2008relationship]. As a prominent result, the cognitive or affective origin of the modulation is lost and stimulus’ effect on behaviour is both cognitive and emotional. At the same time, the cognitive control system (LPFC, ACC) guides behaviour while maintaining and manipulating goal-related information; however strategies for action dynamically incorporate value through the mediation of the nucleus accumbens, the amygdala, and the OFC. Eventually, basal forebrain cholinergic neurons provide regulation of arousal and attention [@goard2009basal], while dopamine neurons located in the vTA modulate the prediction and expectation of future rewards [@pessoa2008relationship].

It is to be noted in Figure \[Fig:neuro\] the central role of the amygdala and the OFC. It has been argued [@salzman2010emotion] that their tight interaction provides a suitable ground for representing, at the psychological level the core affect dimensions [@russell2003core] of valence (pleasure–displeasure conveyed by the visual stimuli) and arousal (activation–deactivation). From a computational standpoint, the observer’s core affect can in principle be modelled as a dynamic latent space [@vitale2014affective], which we surmise might be readily embedded within the loop as proposed in Fig. \[Fig:newloop\]. This way, gaze shifts would benefit from the crucial emotional mediation between the action control and perceptual evaluation components.

![The perception-action loop unfolded in time as a dynamic PGM as in Figure \[Fig:loop\], but integrated with the core affect latent dimension abstracted as the RV $\mathcal{E}(t)$. $\mathcal{C}(t)$ stands for a higher cognitive control RV. To simplify the graphics, conditional dependencies $\mathcal{G} \rightarrow \mathcal{A}(t+1)$ and $\mathcal{G} \rightarrow \mathcal{W}(t+1)$ have been omitted.[]{data-label="Fig:newloop"}](FigAffectLoop2.jpg)

Conclusion
==========

Time is ripe to abandon the marshland of mass production and evaluation of bottom-up saliency techniques. Such boundless effort is partially based on a fatally flawed assumption [@santini2008context]: that visual data have a meaning *per se*, which can be derived as a function of a certain representation of the data themselves. Meaning is an outcome of an interpretative process rather than a property of the viewed scene. It is the act of perceiving, contextual and situated, that gives a scene its meaning [@wittgenstein2010philosophical].

The course of modelling can be more fruitfully directed not only to climb the hierarchy of representation levels and to cope with overlooked aspects of eye guidance, but to eventually reappraise the observer within his natural setting: an active observer compelled to purposively exploit visual attention for accomplishing real-world tasks [@maurice1945phenomenologie].

[^1]: Matlab simulation is available for download at <http://www.mathworks.com/matlabcentral/fileexchange/38512-visual-scanpaths-via-constrained-levy-exploration-of-a-saliency-landscape>

[^2]: Matlab simulation is available for download at <https://www.researchgate.net/publication/290816849_Ecological_sampling_of_gaze_shifts_Matlab_code>
