The Interactive Self – A Review on Simulating Social Interactions to Understand the Mechanisms of Social Agency
---
paper_title: Discrepancy between explicit judgement of agency and implicit feeling of agency: Implications for sense of agency and its disorders

paper_content:

Abstract The sense of agency refers to the feeling of authorship that “I am the one who is controlling external events through my own action”. A distinction between explicit judgement of agency and implicit feeling of agency has been proposed theoretically. However, there has not been sufficient experimental evidence to support this distinction. We have assessed separate explicit and implicit agency measures in the same population and investigated their relationships. Intentional binding task was employed as an implicit measure and self-other attribution task as an explicit measure, which are known to reflect clinical symptoms of disorders in the sense of agency. The results of the implicit measure and explicit measure were not correlated, suggesting dissociation of the explicit judgement of agency and the implicit feeling of agency.
---
paper_title: The “sense of agency” and its underlying cognitive and neural mechanisms

paper_content:

Abstract The sense of agency is a central aspect of human self-consciousness and refers to the experience of oneself as the agent of one’s own actions. Several different cognitive theories on the sense of agency have been proposed implying divergent empirical approaches and results, especially with respect to neural correlates. A multifactorial and multilevel model of the sense of agency may provide the most constructive framework for integrating divergent theories and findings, meeting the complex nature of this intriguing phenomenon.
---
paper_title: Doing good or bad: How interactions between action and emotion expectations shape the sense of agency

paper_content:

The emotional consequences of our own and others’ actions can influence our agentive self-awareness in social contexts. Positive outcomes are usually linked to the self and used for self-enhancement, whereas negative outcomes are more often attributed to others. In most situations, these causal attribution tendencies seem to be immediately present instead of involving reflective interpretations of the action experience. To address the question at which level of the cognitive hierarchy emotions and action perception interact, we adopted a social reward anticipation paradigm. Here, participants or their interaction partner received positive or negative action outcomes and performed speeded attribution choices regarding causation of the action outcome. Event-Related Potential (ERP) results showed that the emotional value of an outcome already influenced the classical N1 self-attenuation effect, with reduced embodied agentive self-awareness for negative outcomes at initial sensorimotor stages. At the level of...
---
paper_title: Voluntary action and conscious awareness

paper_content:

Humans have the conscious experience of 'free will': we feel we can generate our actions, and thus affect our environment. Here we used the perceived time of intentional actions and of their sensory consequences as a means to study consciousness of action. These perceived times were attracted together in conscious awareness, so that subjects perceived voluntary movements as occurring later and their sensory consequences as occurring earlier than they actually did. Comparable involuntary movements caused by magnetic brain stimulation reversed this attraction effect. We conclude that the CNS applies a specific neural mechanism to produce intentional binding of actions and their effects in conscious awareness.
---
paper_title: Philosophical conceptions of the self: implications for cognitive science

paper_content:

Abstract Several recently developed philosophical approaches to the self promise to enhance the exchange of ideas between the philosophy of the mind and the other cognitive sciences. This review examines two important concepts of self: the ‘minimal self', a self devoid of temporal extension, and the ‘narrative self', which involves personal identity and continuity across time. The notion of a minimal self is first clarified by drawing a distinction between the sense of self-agency and the sense of self-ownership for actions. This distinction is then explored within the neurological domain with specific reference to schizophrenia, in which the sense of self-agency may be disrupted. The convergence between the philosophical debate and empirical study is extended in a discussion of more primitive aspects of self and how these relate to neonatal experience and robotics. The second concept of self, the narrative self, is discussed in the light of Gazzaniga's left-hemisphere ‘interpreter' and episodic memory. Extensions of the idea of a narrative self that are consistent with neurological models are then considered. The review illustrates how the philosophical approach can inform cognitive science and suggests that a two-way collaboration may lead to a more fully developed account of the self.
---
paper_title: The experience of agency: an interplay between prediction and postdiction

paper_content:

The experience of agency, i.e., the registration that I am the initiator of my actions, is a basic and constant underpinning of our interaction with the world. Whereas several accounts have underlined predictive processes as the central mechanism (e.g. the comparator model by C.Frith), others emphasized postdictive inferences (e.g. post hoc inference account by D. Wegner). Based on increasing evidence that both predictive and postdictive processes contribute to the experience of agency, we here present a unifying but at the same time parsimonious approach that reconciles these accounts: predictive and postdictive processes are both integrated by the brain according to the principles of optimal cue integration. According to this framework, predictive and postdictive processes each serve as authorship cues that are continuously integrated and weighted depending on their availability and reliability in a given situation. Both sensorimotor and cognitive signals can serve as predictive cues (e.g. internal predictions based on an efferency copy of the motor command, or cognitive anticipations based on priming). Similarly, other sensorimotor and cognitive cues can each serve as post-hoc cues (e.g. visual feedback of the action or the affective valence of the action outcome). Integration and weighting of these cues might not only differ between contexts and individuals, but also between different subject and disease groups. For example, schizophrenia patients with delusions of influence seem to rely less on (probably imprecise) predictive motor signals of the action and more on post-hoc action cues like e.g. visual feedback and, possibly, the affective valence of the action outcome. Thus, the framework of optimal cue integration offers a promising approach that directly stimulates a wide range of experimentally testable hypotheses on agency processing in different subject groups.
---
paper_title: I move , therefore I am : A new theoretical framework to investigate agency and ownership

paper_content:

Abstract The neurocognitive structure of the acting self has recently been widely studied, yet is still perplexing and remains an often confounded issue in cognitive neuroscience, psychopathology and philosophy. We provide a new systematic account of two of its main features, the sense of agency and the sense of ownership, demonstrating that although both features appear as phenomenally uniform, they each in fact are complex crossmodal phenomena of largely heterogeneous functional and (self-)representational levels. These levels can be arranged within a gradually evolving, onto- and phylogenetically plausible framework which proceeds from basic non-conceptual sensorimotor processes to more complex conceptual and meta-representational processes of agency and ownership, respectively. In particular, three fundamental levels of agency and ownership processing have to be distinguished: The level of feeling, thinking and social interaction. This naturalistic account will not only allow to “ground the self in action”, but also provide an empirically testable taxonomy for cognitive neuroscience and a new tool for disentangling agency and ownership disturbances in psychopathology (e.g. alien hand, anarchic hand, anosognosia for one’s own hemiparesis).
---
paper_title: Discrepancy between explicit judgement of agency and implicit feeling of agency: Implications for sense of agency and its disorders

paper_content:

Abstract The sense of agency refers to the feeling of authorship that “I am the one who is controlling external events through my own action”. A distinction between explicit judgement of agency and implicit feeling of agency has been proposed theoretically. However, there has not been sufficient experimental evidence to support this distinction. We have assessed separate explicit and implicit agency measures in the same population and investigated their relationships. Intentional binding task was employed as an implicit measure and self-other attribution task as an explicit measure, which are known to reflect clinical symptoms of disorders in the sense of agency. The results of the implicit measure and explicit measure were not correlated, suggesting dissociation of the explicit judgement of agency and the implicit feeling of agency.
---
paper_title: Plural self-awareness

paper_content:

It has been claimed in the literature that collective intentionality and group attitudes presuppose some “sense of ‘us’” among the participants (other labels sometimes used are “sense of community,” “communal awareness,” “shared point of view,” or “we-perspective”). While this seems plausible enough on an intuitive level, little attention has been paid so far to the question of what the nature and role of this mysterious “sense of ‘us’” might be. This paper states (and argues for) the following five claims: (1) it is neither the case that the sense in question has the community (or “us”) in its content or as its object nor does the attitude in question presuppose a preexistent community (or “us”) as its subject. (2) The “sense of ‘us’” is plural pre-reflective self-awareness. (3) Plural pre-reflective self-awareness plays the same role in the constitution of a common mind that singular pre-reflective self-awareness plays in the individual mind. (4) The most important conceptions of plural subjects, collective persons, or group agents in the current literature fail to recognize the nature and role of plural self-awareness, and therefore fall short in important respects. (5) In spite of the striking similarities between the plural and the singular mind, there are important differences to consider. The authority of the singular first person point of view has no equivalent in the plural case.
---
paper_title: The development of the social brain in human infancy

paper_content:

Much research has focused on how the adult human brain processes the social world, yet until recently little was known about the early development of these abilities. Developmental studies inform debates about the specificity of social functions in the adult cortex. This review highlights recent work, mainly based on electroencephalography/event-related potential methods, examining the precursors of the human social brain network during infancy in several domains such as face and eye gaze processing, the perception of emotions, decoding biological motion, perceiving human actions and joint attention. The findings illustrate that the human brain is fundamentally adapted to develop within a social context, and that this context contributes to many of the specializations seen in the adult cortex.
---
paper_title: ‘I Interact Therefore I Am’: The Self as a Historical Product of Dialectical Attunement

paper_content:

In this article, moving from being to becoming, we construe the ‘self’ as a dynamic process rather than as a static entity. To this end we draw on dialectics and Bayesian accounts of cognition. The former allows us to holistically consider the ‘self’ as the interplay between internalization and externalization and the latter to operationalize our suggestion formally. Internalization is considered here as the co-construction of bodily hierarchical models of the (social) world and the organism, while externalization is taken as the collective transformation of the world. We do not consider these processes as sequentially linked, but rather as a dialectic between the collective and the individual. This leads us to the suggestion of the self as a historical product of dialectical attunement across multiple time scales, from species evolution and culture to individual development and everyday learning. Subsequently, we describe concrete means for empirically testing our proposal in the form of two-person psychophysiology and multi-level analyses of intersubjectivity. Taken together, we suggest that a fine-grained analysis of social interaction might allow us to reconsider the ‘self’ beyond the static individual, i.e. how it emerges and manifests itself in social relations. Such an approach, we believe, could be relevant in multiple fields, from ethics and psychiatry to pedagogy and artificial intelligence.
---
paper_title: Exploring implicit and explicit aspects of sense of agency

paper_content:

Sense of agency refers to the sense of initiating and controlling actions in order to influence events in the outside world. Recently, a distinction between implicit and explicit aspects of sense of agency has been proposed, analogous to distinctions found in other areas of cognition, notably learning. However, there is yet no strong evidence supporting separable implicit and explicit components of sense of agency. The so-called 'Perruchet paradigm' offers one of the few convincing demonstrations of separable implicit and explicit learning systems. We adopted this approach to evaluate the implicit-explicit distinction in the context of a simple task in which outcomes were probabilistically caused by actions. In line with our initial predictions, we found evidence of a dissociation. We discuss the implications of this result for theories of sense of agency.
---
paper_title: Human agency beliefs influence behaviour during virtual social interactions

paper_content:

In recent years, with the emergence of relatively inexpensive and accessible virtual reality technologies, it is now possible to deliver compelling and realistic simulations of human-to-human interaction. Neuroimaging studies have shown that, when participants believe they are interacting via a virtual interface with another human agent, they show different patterns of brain activity compared to when they know that their virtual partner is computer-controlled. The suggestion is that users adopt an "intentional stance" by attributing mental states to their virtual partner. However, it remains unclear how beliefs in the agency of a virtual partner influence participants' behaviour and subjective experience of the interaction. We investigated this issue in the context of a cooperative "joint attention" game in which participants interacted via an eye tracker with a virtual onscreen partner, directing each other's eye gaze to different screen locations. Half of the participants were correctly informed that their partner was controlled by a computer algorithm ("Computer" condition). The other half were misled into believing that the virtual character was controlled by a second participant in another room ("Human" condition). Those in the "Human" condition were slower to make eye contact with their partner and more likely to try and guide their partner before they had established mutual eye contact than participants in the "Computer" condition. They also responded more rapidly when their partner was guiding them, although the same effect was also found for a control condition in which they responded to an arrow cue. Results confirm the influence of human agency beliefs on behaviour in this virtual social interaction context. They further suggest that researchers and developers attempting to simulate social interactions should consider the impact of agency beliefs on user experience in other social contexts, and their effect on the achievement of the application's goals.
---
paper_title: Eyes on the Mind: Investigating the Influence of Gaze Dynamics on the Perception of Others in Real-Time Social Interaction

paper_content:

Social gaze provides a window into the interests and intentions of others and allows us to actively point out our own. It enables us to engage in triadic interactions involving human actors and physical objects and to build an indispensable basis for coordinated action and collaborative efforts. The object-related aspect of gaze in combination with the fact that any motor act of looking encompasses both input and output of the minds involved makes this non-verbal cue system particularly interesting for research in embodied social cognition. Social gaze comprises several core components, such as gaze-following or gaze aversion. Gaze-following can result in situations of either "joint attention" or "shared attention." The former describes situations in which the gaze-follower is aware of sharing a joint visual focus with the gazer. The latter refers to a situation in which gazer and gaze-follower focus on the same object and both are aware of their reciprocal awareness of this joint focus. Here, a novel interactive eye-tracking paradigm suited for studying triadic interactions was used to explore two aspects of social gaze. Experiments 1a and 1b assessed how the latency of another person's gaze reactions (i.e., gaze-following or gaze version) affected participants' sense of agency, which was measured by their experience of relatedness of these reactions. Results demonstrate that both timing and congruency of a gaze reaction as well as the other's action options influence the sense of agency. Experiment 2 explored differences in gaze dynamics when participants were asked to establish either joint or shared attention. Findings indicate that establishing shared attention takes longer and requires a larger number of gaze shifts as compared to joint attention, which more closely seems to resemble simple visual detection. Taken together, novel insights into the sense of agency and the awareness of others in gaze-based interaction are provided.
---
paper_title: Gazing at me: the importance of social meaning in understanding direct-gaze cues.

paper_content:

Direct gaze is an engaging and important social cue, but the meaning of direct gaze depends heavily on the surrounding context. This paper reviews some recent studies of direct gaze, to understand more about what neural and cognitive systems are engaged by this social cue and why. The data show that gaze can act as an arousal cue and can modulate actions, and can activate brain regions linked to theory of mind and self-related processing. However, all these results are strongly modulated by the social meaning of a gaze cue and by whether participants believe that another person is really watching them. The implications of these contextual effects and audience effects for our theories of gaze are considered.
---
paper_title: Doing good or bad: How interactions between action and emotion expectations shape the sense of agency

paper_content:

The emotional consequences of our own and others’ actions can influence our agentive self-awareness in social contexts. Positive outcomes are usually linked to the self and used for self-enhancement, whereas negative outcomes are more often attributed to others. In most situations, these causal attribution tendencies seem to be immediately present instead of involving reflective interpretations of the action experience. To address the question at which level of the cognitive hierarchy emotions and action perception interact, we adopted a social reward anticipation paradigm. Here, participants or their interaction partner received positive or negative action outcomes and performed speeded attribution choices regarding causation of the action outcome. Event-Related Potential (ERP) results showed that the emotional value of an outcome already influenced the classical N1 self-attenuation effect, with reduced embodied agentive self-awareness for negative outcomes at initial sensorimotor stages. At the level of...
---
paper_title: Why we interact: On the functional role of the striatum in the subjective experience of social interaction

paper_content:

Abstract There is ample evidence that human primates strive for social contact and experience interactions with conspecifics as intrinsically rewarding. Focusing on gaze behavior as a crucial means of human interaction, this study employed a unique combination of neuroimaging, eye-tracking, and computer-animated virtual agents to assess the neural mechanisms underlying this component of behavior. In the interaction task, participants believed that during each interaction the agent's gaze behavior could either be controlled by another participant or by a computer program. Their task was to indicate whether they experienced a given interaction as an interaction with another human participant or the computer program based on the agent's reaction. Unbeknownst to them, the agent was always controlled by a computer to enable a systematic manipulation of gaze reactions by varying the degree to which the agent engaged in joint attention. This allowed creating a tool to distinguish neural activity underlying the subjective experience of being engaged in social and non-social interaction. In contrast to previous research, this allows measuring neural activity while participants experience active engagement in real-time social interactions. Results demonstrate that gaze-based interactions with a perceived human partner are associated with activity in the ventral striatum, a core component of reward-related neurocircuitry. In contrast, interactions with a computer-driven agent activate attention networks. Comparisons of neural activity during interaction with behaviorally naive and explicitly cooperative partners demonstrate different temporal dynamics of the reward system and indicate that the mere experience of engagement in social interaction is sufficient to recruit this system.
---
paper_title: A unifying computational framework for motor control and social interaction

paper_content:

Recent empirical studies have implicated the use of the motor system during action observation, imitation and social interaction. In this paper, we explore the computational parallels between the processes that occur in motor control and in action observation, imitation, social interaction and theory of mind. In particular, we examine the extent to which motor commands acting on the body can be equated with communicative signals acting on other people and suggest that computational solutions for motor control may have been extended to the domain of social interaction.
---
paper_title: Beyond Autism: Introducing the Dialectical Misattunement Hypothesis and a Bayesian Account of Intersubjectivity

paper_content:

Drawing on sociocultural theories and Bayesian accounts of brain function, in this article we construe psychiatric conditions as disorders of social interaction to fully account for their complexity and dynamicity across levels of description and temporal scales. After an introduction of the theoretical underpinnings of our integrative approach, we take autism spectrum conditions (ASC) as a paradigm example and discuss how neurocognitive hypotheses can be translated into a Bayesian formulation, i.e., in terms of predictive processing and active inference. We then argue that consideration of individuals (even within a Bayesian framework) will not be enough for a comprehensive understanding of psychiatric conditions and consequently put forward the dialectical misattunement hypothesis, which views psychopathology not merely as disordered function within single brains but also as a dynamic interpersonal mismatch that encompasses various levels of description. Moving from a mere comparison of groups, i.e., “healthy” persons versus “patients,” to a fine-grained analysis of social interactions within dyads and groups of individuals will open new avenues and may allow to avoid an overly neurocentric scope in psychiatric research as well as help to reduce social exclusion.
---
paper_title: Duration matters: Dissociating neural correlates of detection and evaluation of social gaze

paper_content:

Abstract The interpretation of interpersonal gaze behavior requires the use of complex cognitive processes and guides social interactions. Among a variety of different gaze characteristics, gaze direction and gaze duration modulate crucially the meaning of the “social gaze”. Nevertheless, prior neuroimaging studies disregarded the relevance of gaze duration by focusing on gaze direction only. The present functional magnetic resonance imaging (fMRI) study focused on the differentiation of these two gaze parameters. Therefore direct gaze displayed by virtual characters was contrasted with averted gaze and, additionally, systematically varied with respect to gaze duration (i.e., 1, 2.5 or 4 s). Consistent with prior findings, behavioral data showed that likeability was higher for direct than for averted gaze and increased linearly with increasing direct gaze duration. On the neural level, distinct brain regions were associated with the processing of gaze direction and gaze duration: (i) the comparison between direct and averted gaze revealed activations in bilateral occipito-temporal regions including the posterior superior temporal sulcus (pSTS); (ii) whereas increasing duration of direct gaze evoked differential neural responses in the medial prefrontal cortex (MPFC) including orbitofrontal and paracingulate regions. The results suggest two complementary cognitive processes related to different gaze parameters. On the one hand, the recruitment of multimodal sensory regions in the pSTS indicates detection of gaze direction via complex visual analysis. On the other hand, the involvement of the MPFC associated with outcome monitoring and mentalizing indicates higher-order social cognitive processes related to evaluation of the ongoing communicational input conveyed by direct gaze duration.
---
paper_title: Toward a second-person neuroscience.

paper_content:

In spite of the remarkable progress made in the burgeoning field of social neuroscience, the neural mechanisms that underlie social encounters are only beginning to be studied and could - paradoxically - be seen as representing the "dark matter" of social neuroscience. Recent conceptual and empirical developments consistently indicate the need for investigations that allow the study of real-time social encounters in a truly interactive manner. This suggestion is based on the premise that social cognition is fundamentally different when we are in interaction with others rather than merely observing them. In this article, we outline the theoretical conception of a second-person approach to other minds and review evidence from neuroimaging, psychophysiological studies, and related fields to argue for the development of a second-person neuroscience, which will help neuroscience to really "go social"; this may also be relevant for our understanding of psychiatric disorders construed as disorders of social cognition.
---
paper_title: Eyes that bind us: Gaze leading induces an implicit sense of agency

paper_content:

Abstract Humans feel a sense of agency over the effects their motor system causes. This is the case for manual actions such as pushing buttons, kicking footballs, and all acts that affect the physical environment. We ask whether initiating joint attention – causing another person to follow our eye movement – can elicit an implicit sense of agency over this congruent gaze response. Eye movements themselves cannot directly affect the physical environment, but joint attention is an example of how eye movements can indirectly cause social outcomes. Here we show that leading the gaze of an on-screen face induces an underestimation of the temporal gap between action and consequence (Experiments 1 and 2). This underestimation effect, named ‘temporal binding,’ is thought to be a measure of an implicit sense of agency. Experiment 3 asked whether merely making an eye movement in a non-agentic, non-social context might also affect temporal estimation, and no reliable effects were detected, implying that inconsequential oculomotor acts do not reliably affect temporal estimations under these conditions. Together, these findings suggest that an implicit sense of agency is generated when initiating joint attention interactions. This is important for understanding how humans can efficiently detect and understand the social consequences of their actions.
---
paper_title: Subjectivity and Selfhood: Investigating the First-Person Perspective

paper_content:

What is a self? Does it exist in reality or is it a mere social construct -- or is it perhaps a neurologically induced illusion? The legitimacy of the concept of the self has been questioned by both neuroscientists and philosophers in recent years. Countering this, in Subjectivity and Selfhood, Dan Zahavi argues that the notion of self is crucial for a proper understanding of consciousness. He investigates the interrelationships of experience, self-awareness, and selfhood, proposing that none of these three notions can be understood in isolation. Any investigation of the self, Zahavi argues, must take the first-person perspective seriously and focus on the experiential givenness of the self. Subjectivity and Selfhood explores a number of phenomenological analyses pertaining to the nature of consciousness, self, and self-experience in light of contemporary discussions in consciousness research. Philosophical phenomenology -- as developed by Husserl, Heidegger, Sartre, Merleau-Ponty, and others -- not only addresses crucial issues often absent from current debates over consciousness but also provides a conceptual framework for understanding subjectivity. Zahavi fills the need -- given the recent upsurge in theoretical and empirical interest in subjectivity -- for an account of the subjective or phenomenal dimension of consciousness that is accessible to researchers and students from a variety of disciplines. His aim is to use phenomenological analyses to clarify issues of central importance to philosophy of mind, cognitive science, developmental psychology, and psychiatry. By engaging in a dialogue with other philosophical and empirical positions, says Zahavi, phenomenology can demonstrate its vitality and contemporary relevance.
---
paper_title: Sense of agency in joint action: influence of human and computer co-actors

paper_content:

Intentional binding is the perceived shortening of the time between a voluntary action and its consequent effect and has been suggested as an implicit measure of agency. This shortening has been linked to processes underlying action preparation and is also affected by post-movement feedback. Intentional binding has been demonstrated in joint action tasks involving two humans, but it is unknown whether it occurs for tasks involving a human working alongside a non-human partner. This experiment investigated the influence of high-level feedback on the experience of agency and whether binding occurs in human–computer joint action settings. Participants were involved in two versions an action task involving another ‘agent’. In one version, two participants (a genuine participant and a confederate) sat side by side, separated by a curtain that prevented vision of the other person. In baseline conditions, both participants were instructed to make a self-paced action and judge the time of the action by reporting the position of a rotating clock-hand on a computer screen. In other baseline conditions, participants judged the time of an auditory tone. In operant conditions, participants made actions and the genuine participant’s action was followed 200 ms later by a tone on every trial. To examine the effect of post-movement information on binding and explicit agency judgments, a colour cue was presented on each trial informing participants about which person’s action caused the tone. In another version of the task, participants were paired with a computer instead of a human co-actor. Post-movement information affected the genuine participant’s explicit agency judgments but had no effect on intentional binding, which always occurred. In the human–computer version, participants never showed binding, even when they explicitly judged that their action had caused the tone. We suggest that human–human partnerships result in the formation of a new ‘we’ agentic identity, but that human–computer partnerships lead to inhibition of the processes that mediate the pre-reflective sense of agency.
---
paper_title: Higher order thoughts in action: consciousness as an unconscious re-description process

paper_content:

Metacognition is usually construed as a conscious, intentional process whereby people reflect upon their own mental activity. Here, we instead suggest that metacognition is but an instance of a larger class of representational re-description processes that we assume occur unconsciously and automatically. From this perspective, the brain continuously and unconsciously learns to anticipate the consequences of action or activity on itself, on the world and on other people through three predictive loops: an inner loop, a perception–action loop and a self–other (social cognition) loop, which together form a tangled hierarchy. We ask what kinds of mechanisms may subtend this form of enactive metacognition. We extend previous neural network simulations and compare the model with signal detection theory, highlighting that while the latter approach assumes that both type I (objective) and type II (subjective, metacognition-based) decisions tap into the same signal at different hierarchical levels, our approach is closer to dual-route models in which it is assumed that the re-descriptions made possible by the emergence of meta-representations occur independently and outside of the first-order causal chain. We close by reviewing relevant neurological evidence for the idea that awareness, self-awareness and social cognition involve the same mechanisms.
---
paper_title: Minds Made for Sharing: Initiating Joint Attention Recruits Reward-related Neurocircuitry

paper_content:

The ability and motivation to share attention is a unique aspect of human cognition. Despite its significance, the neural basis remains elusive. To investigate the neural correlates of joint attention, we developed a novel, interactive research paradigm in which participants' gaze behavior-as measured by an eye tracking device-was used to contingently control the gaze of a computer-animated character. Instructed that the character on screen was controlled by a real person outside the scanner, 21 participants interacted with the virtual other while undergoing fMRI. Experimental variations focused on leading versus following the gaze of the character when fixating one of three objects also shown on the screen. In concordance with our hypotheses, results demonstrate, firstly, that following someone else's gaze to engage in joint attention resulted in activation of anterior portion of medial prefrontal cortex (MPFC) known to be involved in the supramodal coordination of perceptual and cognitive processes. Secondly, directing someone else's gaze toward an object activated the ventral striatum which-in light of ratings obtained from participants-appears to underlie the hedonic aspects of sharing attention. The data, therefore, support the idea that other-initiated joint attention relies upon recruitment of MPFC previously related to the "meeting of minds." In contrast, self-initiated joint attention leads to a differential increase of neural activity in reward-related brain areas, which might contribute to the uniquely human motivation to engage in the sharing of experiences.
---
paper_title: Open Minds: The Social Making of Agency and Intentionality

paper_content:

In Open Minds, Wolfgang Prinz offers the novel claim that agency and intentionality are first perceived and understood in others, and that it is only through practices and discourses of social mirroring that individuals come to apply these features to themselves and to shape their architectures for volition and cognition accordingly. Developing a (social science) constructive approach within a (cognitive science) representational framework, Prinz argues that the architectures for agency (volition) and intentionality (cognition) arise from particular kinds of social interaction and communication. Rather than working as closed, individual systems, our minds operate in ways that are fundamentally open to other minds. Prinz describes mirror systems and mirror games, particular kinds of representational mechanisms and social games that provide tools for aligning closed individual minds with other minds. He maps the formation of an architecture for volition, addressing issues of agency and intention-based top-down control, then outlines the ways the same basic ideas can be applied to an architecture for cognition, helping to solve basic issues of subjectivity and intentionality. Addressing the reality and efficacy of such social artifacts as autonomy and free will, Prinz contends that our beliefs about minds are not just beliefs about their workings but powerful tools for making them work as we believe. It is through our beliefs that our minds work in a particular way that we actually make them work in that way.
---
paper_title: The “social” and “interpersonal” body in spatial cognition. The role of agency and interagency

paper_content:

In order to interact effectively, we need to represent our action as produced by human beings. According to direct access theories, the first steps of visual information processing offer us an informed direct grasp of the situation, especially when social and interpersonal components are implicated. Biological system detection may be the gateway of such smart processes and then may influence initial stages of perception fostering adaptive social behaviour. To investigate early neural correlates of human agency detection in ecological situations with more high or low social impact, we compared scenes showing a human versus artificial agent interacting with a human agent. Twenty volunteers participated in the study. They were asked to observe dynamic visual stimuli showing realistic interactions. ERP (event-related potentials) were recorded. Each stimulus depicted an arm executing a gesture addressed to a human agent. Visual features of the arm were manipulated: in half of trials, it was real; in other trials, it was deprived of some details and transformed in a statue-like arm. EEG morphological analysis revealed an early negative deflection peaking at about 155 ms. Peak amplitude data have been statistically analysed by repeated-measures ANOVAs. It was found that the peak was ampler in the left inferior posterior region when the gesturing arm was human. The early negative deflection, N150, which we found to be different between the human and artificial conditions, is presumably associated with human agency detection in high interpersonal context.
---
paper_title: Beyond self-serving bias: diffusion of responsibility reduces sense of agency and outcome monitoring

paper_content:

Diffusion of responsibility across agents has been proposed to underlie decreased helping and increased aggression in group behaviour. However, few studies have directly investigated effects of the presence of other people on how we experience the consequences of our actions. This EEG study investigated whether diffusion of responsibility simply reflects a post-hoc self-serving bias, or rather has direct effects on how we process the outcomes of our actions, and our experience of agency over them. Participants made voluntary actions whose outcomes were more or less negative. Presence of another potential agent reduced participants' sense of agency over those outcomes, even though it was always obvious who caused each outcome. Further, presence of another agent reduced the amplitude of feedback-related negativity evoked by outcome stimuli, suggesting reduced outcome monitoring. The presence of other agents may lead to diffusion of responsibility by weakening the neural linkage between one's actions and their outcomes.
---
paper_title: A Non-Verbal Turing Test: Differentiating Mind from Machine in Gaze-Based Social Interaction

paper_content:

In social interaction, gaze behavior provides important signals that have a significant impact on our perception of others. Previous investigations, however, have relied on paradigms in which participants are passive observers of other persons’ gazes and do not adjust their gaze behavior as is the case in real-life social encounters. We used an interactive eye-tracking paradigm that allows participants to interact with an anthropomorphic virtual character whose gaze behavior is responsive to where the participant looks on the stimulus screen in real time. The character’s gaze reactions were systematically varied along a continuum from a maximal probability of gaze aversion to a maximal probability of gaze-following during brief interactions, thereby varying contingency and congruency of the reactions. We investigated how these variations influenced whether participants believed that the character was controlled by another person (i.e., a confederate) or a computer program. In a series of experiments, the human confederate was either introduced as naive to the task, cooperative, or competitive. Results demonstrate that the ascription of humanness increases with higher congruency of gaze reactions when participants are interacting with a naive partner. In contrast, humanness ascription is driven by the degree of contingency irrespective of congruency when the confederate was introduced as cooperative. Conversely, during interaction with a competitive confederate, judgments were neither based on congruency nor on contingency. These results offer important insights into what renders the experience of an interaction truly social: Humans appear to have a default expectation of reciprocation that can be influenced drastically by the presumed disposition of the interactor to either cooperate or compete.
---
paper_title: Losing Control in Social Situations: How the Presence of Others Affects Neural Processes Related to Sense of Agency

paper_content:

Social contexts substantially influence individual behavior, but little is known about how they affect cognitive processes related to voluntary action. Previously, it has been shown that social context reduces participants’ sense of agency over the outcomes of their actions and outcome monitoring. In this fMRI study on human volunteers, we investigated the neural mechanisms by which social context alters sense of agency. Participants made costly actions to stop inflating a balloon before it burst. On “social” trials, another player could act in their stead, but we analyzed only trials in which the other player remained passive. We hypothesized that mentalizing processes during social trials would affect decision-making fluency and lead to a decreased sense of agency. In line with this hypothesis, we found increased activity in the bilateral temporo-parietal junction (TPJ), precuneus, and middle frontal gyrus during social trials compared with nonsocial trials. Activity in the precuneus was, in turn, negatively related to sense of agency at a single-trial level. We further found a double dissociation between TPJ and angular gyrus (AG): activity in the left AG was not sensitive to social context but was negatively related to sense of agency. In contrast, activity in the TPJ was modulated by social context but was not sensitive to sense of agency.
---
paper_title: What makes human cognition unique? from individual to shared to collective intentionality

paper_content:

It is widely believed that what distinguishes the social cognition of humans from that of other animals is the belief–desire psychology of four–year–old children and adults (so–called theory of mind). We argue here that this is actually the second ontogenetic step in uniquely human social cognition. The first step is one year old children's understanding of persons as intentional agents, which enables skills of cultural learning and shared intentionality. This initial step is ‘the real thing’ in the sense that it enables young children to participate in cultural activities using shared, perspectival symbols with a conventional/normative/reflective dimension—for example, linguistic communication and pretend play—thus inaugurating children's understanding of things mental. Understanding beliefs and participating in collective intentionality at four years of age—enabling the comprehension of such things as money and marriage—results from several years of engagement with other persons in perspective–shifting and reflective discourse containing propositional attitude constructions.
---
paper_title: Beliefs about human agency influence the neural processing of gaze during joint attention

paper_content:

ABSTRACTThe current study measured adults’ P350 and N170 ERPs while they interacted with a character in a virtual reality paradigm. Some participants believed the character was controlled by a human (“avatar” condition, n = 19); others believed it was controlled by a computer program (“agent” condition, n = 19). In each trial, participants initiated joint attention in order to direct the character’s gaze toward a target. In 50% of trials, the character gazed toward the target (congruent responses), and in 50% of trials the character gazed to a different location (incongruent response). In the avatar condition, the character’s incongruent gaze responses generated significantly larger P350 peaks at centro-parietal sites than congruent gaze responses. In the agent condition, the P350 effect was strikingly absent. Left occipitotemporal N170 responses were significantly smaller in the agent condition compared to the avatar condition for both congruent and incongruent gaze shifts. These data suggest that belief...
---
paper_title: Virtual reality in neuroscience research and therapy

paper_content:

The compatibility of virtual reality systems with brain imaging techniques and their use for animal research have aided the widespread adoption of virtual reality environments in both experimental and therapeutic domains. Here the authors review advances in virtual reality technology and its applications.
---
paper_title: Vision and Action in Virtual Environments: Modern Psychophysics in Spatial Cognition Research

paper_content:

The classical psychophysical approach to human perception has been to study isolated aspects of perception using well-controlled and strongly simplified laboratory stimuli. This so-called cue reduction technique has successfully led to the identification of numerous perceptual mechanisms, and has in many cases guided the uncoverage of neural correlates (see chapters elsewhere in this volume). Its limitations, however, lie in the almost complete ignorance of the intimate relationship among action, perception, and the environment in which we live. Real world situations are so different from the stimuli used in classical psychophysics and the context in which they are presented that applying laboratory results to daily life situations often becomes impractical, if not impossible. At the Max-Planck- Institute for Biological Cybernetics in Tubingen, we pursue a behavioral approach to human action and perception that proves especially well-suited for studying more complex cognitive functions, such as object recognition and spatial cognition. The recent availability of high-fidelity "virtual reality" environments enables us to provide subjects a level of sensory realism and dynamic sensory feedback that approaches their experiences in the real world. At the same time, we can keep the ultimate control over all stimulus aspects that are required by the rules of psychophysics. In this chapter, we take a closer look at these developments in spatial cognition research and present results from several different experimental studies that we have conducted using this approach.
---
paper_title: Virtual Gaze. A Pilot Study on the Effects of Computer Simulated Gaze in Avatar-Based Conversations

paper_content:

The paper introduces a novel methodology for the computer simulation of gaze behavior in net-based interactions. 38 female participants interacted using a special avatar platform, allowing for the real-time transmission of nonverbal behavior (head and body movement, gestures, gaze) as captured by motion trackers, eye tracking devices and data gloves. During interaction eye movement and gaze direction of one partner was substituted by computer simulated data. Simulated duration of directed gaze (looking into the face of the visa-vis) was varied lasting 2 seconds in one condition and 4 seconds in the other. Mutual person perception (impression ratings) and individual experience of social presence (short questionnaire) were measured as dependent variables. The results underline the validity of the computer animation approach. Consistent with the literature the longer gaze duration was found to cause significantly better evaluations of the interaction partners and higher levels of co-presence.
---
paper_title: Cardiovascular and cortisol reactivity and habituation to a virtual reality version of the Trier Social Stress Test: A pilot study

paper_content:

Summary The Trier Social Stress Test (TSST) is a widely used protocol to induce stress in laboratory settings. Briefly, in the TSST, the test participant is asked to hold a speech and to do an arithmetic task in front of an audience. In the present pilot study, we examined endocrine and autonomic reactivity and habituation to repeated stress provocations using a virtual reality (VR) version of TSST. The VR system was a CAVE™ system with three rear projected walls (4 m× 3 m), and one floor projection. The system also included a head tracking system and passive stereoscopy. The virtual audience consisted of one woman, and two men. Ten healthy men, mean age 28.3 years (24–38 years), were confronted with the test twice (1 week between sessions), during which salivary cortisol, heart rate (HR), high frequency heart rate variability (HF-HRV, parasympathetic activity), and T-wave amplitude (TWA, suggested to be related to sympathetic influence on myocardial performance) were assessed. Cortisol secretion showed a marked increase (88% vs. baseline) during the first stress provocation, but habituated in the second session. The magnitude of HR and TWA reactivity during stress provocation was approximately the same at both sessions, implying a stable increase in sympathetic activity. Heart rate showed a maximum increase of 40% at the first session, and 32% at the second. TWA showed a maximum decrease of 42% at the first session, and 39% at the second. The results resemble those obtained in prior studies using the real-life TSST. If these results can be replicated with larger samples, VR technology may be used as a simple and standardized tool for social stress induction in experimental settings.
---
paper_title: Toward a second-person neuroscience.

paper_content:

In spite of the remarkable progress made in the burgeoning field of social neuroscience, the neural mechanisms that underlie social encounters are only beginning to be studied and could - paradoxically - be seen as representing the "dark matter" of social neuroscience. Recent conceptual and empirical developments consistently indicate the need for investigations that allow the study of real-time social encounters in a truly interactive manner. This suggestion is based on the premise that social cognition is fundamentally different when we are in interaction with others rather than merely observing them. In this article, we outline the theoretical conception of a second-person approach to other minds and review evidence from neuroimaging, psychophysiological studies, and related fields to argue for the development of a second-person neuroscience, which will help neuroscience to really "go social"; this may also be relevant for our understanding of psychiatric disorders construed as disorders of social cognition.
---
paper_title: The human dynamic clamp as a paradigm for social interaction

paper_content:

Social neuroscience has called for new experimental paradigms aimed toward real-time interactions. A distinctive feature of interactions is mutual information exchange: One member of a pair changes in response to the other while simultaneously producing actions that alter the other. Combining mathematical and neurophysiological methods, we introduce a paradigm called the human dynamic clamp (HDC), to directly manipulate the interaction or coupling between a human and a surrogate constructed to behave like a human. Inspired by the dynamic clamp used so productively in cellular neuroscience, the HDC allows a person to interact in real time with a virtual partner itself driven by well-established models of coordination dynamics. People coordinate hand movements with the visually observed movements of a virtual hand, the parameters of which depend on input from the subject’s own movements. We demonstrate that HDC can be extended to cover a broad repertoire of human behavior, including rhythmic and discrete movements, adaptation to changes of pacing, and behavioral skill learning as specified by a virtual “teacher.” We propose HDC as a general paradigm, best implemented when empirically verified theoretical or mathematical models have been developed in a particular scientific field. The HDC paradigm is powerful because it provides an opportunity to explore parameter ranges and perturbations that are not easily accessible in ordinary human interactions. The HDC not only enables to test the veracity of theoretical models, it also illuminates features that are not always apparent in real-time human social interactions and the brain correlates thereof.
---
paper_title: Immersive Virtual Environment Technology as a Methodological Tool for Social Psychology

paper_content:

Historically, at least 3 methodological problems have dogged experimental social psychology: the experimental control-mundane realism trade-off, lack of replication, and unrepresentative sampling. We argue that immersive virtual environment technology (IVET) can help ameliorate, if not solve, these methodological problems and, thus, holds promise as a new social psychological research tool. In this article, we first present an overview of IVET and review IVET-based research within psychology and other fields. Next, we propose a general model of social influence within immersive virtual environments and present some preliminary findings regarding its utility for social psychology. Finally, we present a new paradigm for experimental social psychology that may enable researchers to unravel the very fabric of social interaction.
---
paper_title: Why and how to use virtual reality to study human social interaction: The challenges of exploring a new research landscape

paper_content:

As virtual reality (VR) technology and systems become more commercially available and accessible, more and more psychologists are starting to integrate VR as part of their methods. This approach offers major advantages in experimental control, reproducibility, and ecological validity, but also has limitations and hidden pitfalls which may distract the novice user. This study aimed to guide the psychologist into the novel world of VR, reviewing available instrumentation and mapping the landscape of possible systems. We use examples of state-of-the-art research to describe challenges which research is now solving, including embodiment, uncanny valley, simulation sickness, presence, ethics, and experimental design. Finally, we propose that the biggest challenge for the field would be to build a fully interactive virtual human who can pass a VR Turing test - and that this could only be achieved if psychologists, VR technologists, and AI researchers work together.
---
paper_title: Engagement with artificial intelligence through natural interaction models

paper_content:

As Artificial Intelligence (AI) systems become more ubiquitous, what user experience design paradigms will be used by humans to impart their needs and intents to an AI system, in order to engage in a more social interaction? In our work, we look mainly at expression and creativity based systems, where the AI both attempts to model or understand/assist in processes of human expression and creativity. We therefore have designed and implemented a prototype system with more natural interaction modes for engagement with AI as well as other human computer interaction (HCI) where a more open natural communication stream is beneficial. Our proposed conversational agent system makes use of the affective signals from the gestural behaviour of the user and the semantic information from the speech input in order to generate a personalised, human-like conversation that is expressed in the visual and conversational output of the 3D virtual avatar system. We describe our system and two application spaces we are using it in - a care advisor / assistant for the elderly and an interactive creative assistant for uses to produce art forms.
---
paper_title: Automatic imitation in a rich social context with virtual characters

paper_content:

It has been well established that people respond faster when they perform an action that is congruent with an observed action than when they respond with an incongruent action. Here we propose a new method of using interactive Virtual Characters (VCs) to test if social congruency effects can be obtained in a richer social context with sequential hand-arm actions. Two separate experiments were conducted, exploring if it is feasible to measure spatial congruency (Experiment 1) and anatomical congruency (Experiment 2) in response to a VC, compared to the same action sequence indicated by three virtual balls. In Experiment 1, we found a robust spatial congruency effect for both VC and virtual balls, modulated by a social facilitation effect for participants who felt the VC was human. In Experiment 2 which allowed for anatomical congruency, a form by congruency interaction provided evidence that participants automatically imitate the actions of the VC but do not imitate the balls. Our method and results build a bridge between studies using minimal stimuli in automatic interaction and studies of mimicry in a rich social interaction, and open new research venue for future research in the area of automatic imitation with a more ecologically valid social interaction.
---
paper_title: Pupil dilation as an index of preferred mutual gaze duration

paper_content:

Most animals look at each other to signal threat or interest. In humans, this social interaction is usually punctuated with brief periods of mutual eye contact. Deviations from this pattern of gazing behaviour generally make us feel uncomfortable and are a defining characteristic of clinical conditions such as autism or schizophrenia, yet it is unclear what constitutes normal eye contact. Here, we measured, across a wide range of ages, cultures and personality types, the period of direct gaze that feels comfortable and examined whether autonomic factors linked to arousal were indicative of people's preferred amount of eye contact. Surprisingly, we find that preferred period of gaze duration is not dependent on fundamental characteristics such as gender, personality traits or attractiveness. However, we do find that subtle pupillary changes, indicative of physiological arousal, correlate with the amount of eye contact people find comfortable. Specifically, people preferring longer durations of eye contact display faster increases in pupil size when viewing another person than those preferring shorter durations. These results reveal that a person's preferred duration of eye contact is signalled by physiological indices (pupil dilation) beyond volitional control that may play a modulatory role in gaze behaviour.
---
paper_title: Simulating social interactions for the experimental investigation of joint attention

paper_content:

Abstract Social interactions are, by their nature, dynamic and reciprocal − your behaviour affects my behaviour, which affects your behaviour in return. However, until recently, the field of social cognitive neuroscience has been dominated by paradigms in which participants passively observe social stimuli from a detached “third person” perspective. Here we consider the unique conceptual and methodological challenges involved in adopting a “second person” approach whereby social cognitive mechanisms and their neural correlates are investigated within social interactions ( Schilbach et al., 2013 ). The key question for researchers is how to distil a complex, intentional interaction between two individuals into a tightly controlled and replicable experimental paradigm. We explore these issues within the context of recent investigations of joint attention − the ability to coordinate a common focus of attention with another person. We review pioneering neurophysiology and eye-tracking studies that have begun to address these issues; offer recommendations for the optimal design and implementation of interactive tasks, and discuss the broader implications of interactive approaches for social cognitive neuroscience.
---
paper_title: Human agency beliefs influence behaviour during virtual social interactions

paper_content:

In recent years, with the emergence of relatively inexpensive and accessible virtual reality technologies, it is now possible to deliver compelling and realistic simulations of human-to-human interaction. Neuroimaging studies have shown that, when participants believe they are interacting via a virtual interface with another human agent, they show different patterns of brain activity compared to when they know that their virtual partner is computer-controlled. The suggestion is that users adopt an "intentional stance" by attributing mental states to their virtual partner. However, it remains unclear how beliefs in the agency of a virtual partner influence participants' behaviour and subjective experience of the interaction. We investigated this issue in the context of a cooperative "joint attention" game in which participants interacted via an eye tracker with a virtual onscreen partner, directing each other's eye gaze to different screen locations. Half of the participants were correctly informed that their partner was controlled by a computer algorithm ("Computer" condition). The other half were misled into believing that the virtual character was controlled by a second participant in another room ("Human" condition). Those in the "Human" condition were slower to make eye contact with their partner and more likely to try and guide their partner before they had established mutual eye contact than participants in the "Computer" condition. They also responded more rapidly when their partner was guiding them, although the same effect was also found for a control condition in which they responded to an arrow cue. Results confirm the influence of human agency beliefs on behaviour in this virtual social interaction context. They further suggest that researchers and developers attempting to simulate social interactions should consider the impact of agency beliefs on user experience in other social contexts, and their effect on the achievement of the application's goals.
---
paper_title: Eyes on the Mind: Investigating the Influence of Gaze Dynamics on the Perception of Others in Real-Time Social Interaction

paper_content:

Social gaze provides a window into the interests and intentions of others and allows us to actively point out our own. It enables us to engage in triadic interactions involving human actors and physical objects and to build an indispensable basis for coordinated action and collaborative efforts. The object-related aspect of gaze in combination with the fact that any motor act of looking encompasses both input and output of the minds involved makes this non-verbal cue system particularly interesting for research in embodied social cognition. Social gaze comprises several core components, such as gaze-following or gaze aversion. Gaze-following can result in situations of either "joint attention" or "shared attention." The former describes situations in which the gaze-follower is aware of sharing a joint visual focus with the gazer. The latter refers to a situation in which gazer and gaze-follower focus on the same object and both are aware of their reciprocal awareness of this joint focus. Here, a novel interactive eye-tracking paradigm suited for studying triadic interactions was used to explore two aspects of social gaze. Experiments 1a and 1b assessed how the latency of another person's gaze reactions (i.e., gaze-following or gaze version) affected participants' sense of agency, which was measured by their experience of relatedness of these reactions. Results demonstrate that both timing and congruency of a gaze reaction as well as the other's action options influence the sense of agency. Experiment 2 explored differences in gaze dynamics when participants were asked to establish either joint or shared attention. Findings indicate that establishing shared attention takes longer and requires a larger number of gaze shifts as compared to joint attention, which more closely seems to resemble simple visual detection. Taken together, novel insights into the sense of agency and the awareness of others in gaze-based interaction are provided.
---
paper_title: Toward a second-person neuroscience.

paper_content:

In spite of the remarkable progress made in the burgeoning field of social neuroscience, the neural mechanisms that underlie social encounters are only beginning to be studied and could - paradoxically - be seen as representing the "dark matter" of social neuroscience. Recent conceptual and empirical developments consistently indicate the need for investigations that allow the study of real-time social encounters in a truly interactive manner. This suggestion is based on the premise that social cognition is fundamentally different when we are in interaction with others rather than merely observing them. In this article, we outline the theoretical conception of a second-person approach to other minds and review evidence from neuroimaging, psychophysiological studies, and related fields to argue for the development of a second-person neuroscience, which will help neuroscience to really "go social"; this may also be relevant for our understanding of psychiatric disorders construed as disorders of social cognition.
---
paper_title: The Use of Virtual Characters to Assess and Train Non-Verbal Communication in High-Functioning Autism

paper_content:

High-functioning autism (HFA) is a neurodevelopmental disorder, which is characterized by life-long socio-communicative impairments on the one hand and preserved verbal and general learning and memory abilities on the other. One of the areas where particular difficulties are observable is the understanding of non-verbal communication cues.Thus, investigating the underlying psychological processes and neural mechanisms of non-verbal communication in HFA allows a better understanding of this disorder, and potentially enables the development of more efficient forms of psychotherapy and trainings. However, the research on non-verbal information processing in HFA faces several methodological challenges. The use of virtual characters (VCs) helps to overcome such challenges by enabling an ecologically valid experience of social presence, and by providing an experimental platform that can be systematically and fully controlled. To make this field of research accessible to a broader audience, we elaborate in the first part of the review the validity of using VCs in non-verbal behavior research on HFA, and we review current relevant paradigms and findings from social-cognitive neuroscience. In the second part, we argue for the use of VCs as either agents or avatars in the context of “transformed social interactions.” This allows for the implementation of real-time social interaction in virtual experimental settings, which represents a more sensitive measure of socio-communicative impairments in HFA. Finally, we argue that VCs and environments are a valuable assistive, educational and therapeutic tool for HFA.
---
paper_title: Minds Made for Sharing: Initiating Joint Attention Recruits Reward-related Neurocircuitry

paper_content:

The ability and motivation to share attention is a unique aspect of human cognition. Despite its significance, the neural basis remains elusive. To investigate the neural correlates of joint attention, we developed a novel, interactive research paradigm in which participants' gaze behavior-as measured by an eye tracking device-was used to contingently control the gaze of a computer-animated character. Instructed that the character on screen was controlled by a real person outside the scanner, 21 participants interacted with the virtual other while undergoing fMRI. Experimental variations focused on leading versus following the gaze of the character when fixating one of three objects also shown on the screen. In concordance with our hypotheses, results demonstrate, firstly, that following someone else's gaze to engage in joint attention resulted in activation of anterior portion of medial prefrontal cortex (MPFC) known to be involved in the supramodal coordination of perceptual and cognitive processes. Secondly, directing someone else's gaze toward an object activated the ventral striatum which-in light of ratings obtained from participants-appears to underlie the hedonic aspects of sharing attention. The data, therefore, support the idea that other-initiated joint attention relies upon recruitment of MPFC previously related to the "meeting of minds." In contrast, self-initiated joint attention leads to a differential increase of neural activity in reward-related brain areas, which might contribute to the uniquely human motivation to engage in the sharing of experiences.
---
paper_title: It’s in your eyes—using gaze-contingent stimuli to create truly interactive paradigms for social cognitive and affective neuroscience

paper_content:

The field of social neuroscience has made remarkable progress in elucidating the neural mechanisms of social cognition. More recently, the need for new experimental approaches has been highlighted that allow studying social encounters in a truly interactive manner by establishing ’online’ reciprocity in social interaction. In this article, we present a newly developed adaptation of a method which uses eyetracking data obtained from participants in real time to control visual stimulation during functional magnetic resonance imaging, thus, providing an innovative tool to generate gaze-contingent stimuli in spite of the constraints of this experimental setting. We review results of two paradigms employing this technique and demonstrate how gaze data can be used to animate a virtual character whose behavior becomes ’responsive’ to being looked at allowing the participant to engage in ’online’ interaction with this virtual other in real-time. Possible applications of this setup are discussed highlighting the potential of this development as a new ’tool of the trade’ in social cognitive and affective neuroscience.
---
paper_title: A Non-Verbal Turing Test: Differentiating Mind from Machine in Gaze-Based Social Interaction

paper_content:

In social interaction, gaze behavior provides important signals that have a significant impact on our perception of others. Previous investigations, however, have relied on paradigms in which participants are passive observers of other persons’ gazes and do not adjust their gaze behavior as is the case in real-life social encounters. We used an interactive eye-tracking paradigm that allows participants to interact with an anthropomorphic virtual character whose gaze behavior is responsive to where the participant looks on the stimulus screen in real time. The character’s gaze reactions were systematically varied along a continuum from a maximal probability of gaze aversion to a maximal probability of gaze-following during brief interactions, thereby varying contingency and congruency of the reactions. We investigated how these variations influenced whether participants believed that the character was controlled by another person (i.e., a confederate) or a computer program. In a series of experiments, the human confederate was either introduced as naive to the task, cooperative, or competitive. Results demonstrate that the ascription of humanness increases with higher congruency of gaze reactions when participants are interacting with a naive partner. In contrast, humanness ascription is driven by the degree of contingency irrespective of congruency when the confederate was introduced as cooperative. Conversely, during interaction with a competitive confederate, judgments were neither based on congruency nor on contingency. These results offer important insights into what renders the experience of an interaction truly social: Humans appear to have a default expectation of reciprocation that can be influenced drastically by the presumed disposition of the interactor to either cooperate or compete.
---
paper_title: Using dual eye tracking to uncover personal gaze patterns during social interaction

paper_content:

We report the personal eye gaze patterns of people engaged in face-to-face getting acquainted conversation. Considerable differences between individuals are underscored by a stability of eye gaze patterns within individuals. Results suggest the existence of an eye-mouth gaze continuum. This continuum includes some people showing a strong preference for eye gaze, some with a strong preference for mouth gaze, and others distributing their gaze between the eyes and mouth to varying extents. Additionally, we found evidence of within-participant consistency not just for location preference but also for the duration of fixations upon the eye and mouth regions. We also estimate that during a 4-minute getting acquainted conversation mutual face gaze constitutes about 60% of conversation that occurs via typically brief instances of 2.2 seconds. Mutual eye contact ranged from 0–45% of conversation, via very brief instances. This was despite participants subjectively perceiving eye contact occurring for about 70% of conversation. We argue that the subjective perception of eye contact is a product of mutual face gaze instead of actual mutual eye contact. We also outline the fast activity of gaze movements upon various locations both on and off face during a typical face-to-face conversation.
---
paper_title: What Can Virtual Reality Teach Us About Prosocial Tendencies in Real and Virtual Environments

paper_content:

As virtual environments (VEs) become increasingly central to people's lives (Terry, 2002), understanding reactions to VEs may be as important as understanding behavior in the real world (Yee, Bailenson, Urbanek, Chang, & Merget, 2007). Immersive Virtual Environment Technology (IVET), which is now being used in psychological research (Blascovich et al., 2002), can provide greater experimental control, more precise measurement, ease of replication across participants, and high ecological validity, making it attractive for researchers. It also can create links between researchers who study basic social psychological processes and those who study new media. In two studies we examined people's reactions as they navigated through a virtual world and interacted with virtual people, some of whom needed help. Participants' compassion and tendency to experience personal distress predicted emotional reactions (concern) and proxemic behavior (gaze orientation and degree of interpersonal distance) to a virtual person ...
---
paper_title: How Our Personality Shapes Our Interactions with Virtual Characters - Implications for Research and Development

paper_content:

There is a general lack of awareness for the influence of users' personality traits on human-agent-interaction (HAI). Numerous studies do not even consider explanatory variables like age and gender although they are easily accessible. The present study focuses on explaining the occurrence of social effects in HAI. Apart from the original manipulation of the study we assessed the users' personality traits. Results show that participants' personality traits influenced their subjective feeling after the interaction, as well as their evaluation of the virtual character and their actual behavior. From the various personality traits those traits which relate to persistent behavioral patterns in social contact (agreeableness, extraversion, approach avoidance, self-efficacy in monitoring others, shyness, public self-consciousness) were found to be predictive, whereas other personality traits and gender and age did not affect the evaluation. Results suggest that personality traits are better predictors for the evaluation outcome than the actual behavior of the agent as it has been manipulated in the experiment. Implications for research on and development of virtual agents are discussed.
---
paper_title: Avatar-Mediated Networking: Increasing Social Presence and Interpersonal Trust in Net-Based Collaborations

paper_content:

This study analyzes the influence of avatars on social presence, interpersonal trust, perceived communication quality, nonverbal behavior, and visual attention in Net-based collaborations using a comparative approach. A real-time communication window including a special avatar interface was integrated into a shared collaborative workspace. Communication modes under investigation were text chat, audio, audio-video, and avatar. Significant differences were found between text chat and all other communication modalities in perceived intimateness, co-presence, and emotionally-based trust. Microanalyses of nonverbal activity and visual attention point to similarities between video and avatar modes, both showing higher levels of exposure to the virtual other and visual attention, in particular in the initial phase of interaction as compared to text and audio.
---
paper_title: How social is social responses to computers? The function of the degree of anthropomorphism in computer representations

paper_content:

Testing the assumption that more anthropomorphic (human-like) computer representations elicit more social responses from people, a between-participants experiment (N=168) manipulated 12 computer agents to represent four levels of anthropomorphism: low, medium, high, and real human images. Social responses were assessed with users' social judgment and homophily perception of the agents, conformity in a choice dilemma task, and competency and trustworthiness ratings of the agents. Linear polynomial trend analyses revealed significant linear trends for almost all the measures. As the agent became more anthropomorphic to being human, it received more social responses from users.
---
paper_title: Virtual Gaze. A Pilot Study on the Effects of Computer Simulated Gaze in Avatar-Based Conversations

paper_content:

The paper introduces a novel methodology for the computer simulation of gaze behavior in net-based interactions. 38 female participants interacted using a special avatar platform, allowing for the real-time transmission of nonverbal behavior (head and body movement, gestures, gaze) as captured by motion trackers, eye tracking devices and data gloves. During interaction eye movement and gaze direction of one partner was substituted by computer simulated data. Simulated duration of directed gaze (looking into the face of the visa-vis) was varied lasting 2 seconds in one condition and 4 seconds in the other. Mutual person perception (impression ratings) and individual experience of social presence (short questionnaire) were measured as dependent variables. The results underline the validity of the computer animation approach. Consistent with the literature the longer gaze duration was found to cause significantly better evaluations of the interaction partners and higher levels of co-presence.
---
paper_title: Discerning intentions in dynamic human action

paper_content:

Abstract When we observe others in motion, we usually care little about the surface behaviors they exhibit. What matters are their underlying intentions. Judgments about intentions and intentionality dictate how we understand and remember others' actions, how we respond, and what we predict about their future action. A generative knowledge system underlies our skill at discerning intentions, enabling us to comprehend intentions even when action is novel and unfolds in complex ways over time. Recent work spanning many disciplines illuminates some of the processes involved in intention detection. We review these developments and articulate a set of questions cutting across current theoretical dividing lines.
---
paper_title: Dimensions of Mind Perception

paper_content:

Participants compared the mental capacities of various human and nonhuman characters via online surveys. Factor analysis revealed two dimensions of mind perception, Experience (for example, capacity for hunger) and Agency (for example, capacity for self-control). The dimensions predicted different moral judgments but were both related to valuing of mind.
---
paper_title: Social Facilitation Effects of Virtual Humans

paper_content:

OBJECTIVE ::: To investigate whether virtual humans produce social facilitation effects. ::: ::: ::: BACKGROUND ::: When people do an easy task and another person is nearby, they tend to do that task better than when they are alone. Conversely, when people do a hard task and another person is nearby, they tend to do that task less well than when they are alone. This phenomenon is referred to in the social psychology literature as social facilitation. The present study investigated whether virtual humans can evoke a social facilitation response. ::: ::: ::: METHOD ::: Participants were given different tasks to do that varied in difficulty. The tasks involved anagrams, mazes, and modular arithmetic. They did the tasks alone, in the company of another person, or in the company of a virtual human on a computer screen. ::: ::: ::: RESULTS ::: For easy tasks, performance in the virtual human condition was better than in the alone condition, and for difficult tasks, performance in the virtual human condition was worse than in the alone condition. ::: ::: ::: CONCLUSION ::: As with a human, virtual humans can produce social facilitation. ::: ::: ::: APPLICATION ::: The results suggest that designers of virtual humans should be mindful about the social nature of virtual humans; a design decision as to when and how to present a virtual human should be a deliberate and informed decision. An ever-present virtual human might make learning and performance difficult for challenging tasks.
---
paper_title: Believable Agents: Building Interactive Personalities

paper_content:

Abstract : In the traditional character-based arts of film, books, animation, or theater, the audience is presented with vivid, powerful, personality-rich characters such as Rick from Casablanca or the Genie or Flying Carpet from Walt Disney's Aladdin. Unless one can afford a troop of improvisational actors (and not even then for characters such as the Flying Carpet), one is only able to watch these rich characters. Believable agents allow people to not just watch, but also interact with such powerful, personality-rich characters. Believable agents are a combination of autonomous agents and believable characters from the traditional character-based arts, such as animation, film or literature. They are accurately described both as autonomous agents with the same powerful properties as characters from the arts, and as computer-based, interactive versions of personality-rich characters. Believable agents could be powerfully used for art, entertainment, as part of interactive story systems, social user interfaces, or teaching systems.
---
paper_title: It’s no accident: Our bias for intentional explanations

paper_content:

Abstract Three studies tested the idea that our analyses of human behavior are guided by an “intentionality bias,” an implicit bias where all actions are judged to be intentional by default. In Study 1 participants read a series of sentences describing actions that can be done either on purpose or by accident (e.g., “He set the house on fire”) and had to decide which interpretation best characterized the action. To tap people’s initial interpretation, half the participants made their judgments under speeded conditions; this group judged significantly more sentences to be intentional. Study 2 found that when asked for spontaneous descriptions of the ambiguous actions used in Study 1 (and thus not explicitly reminded of the accidental interpretation), participants provided significantly more intentional interpretations, even with prototypically accidental actions (e.g., “She broke the vase”). Study 3 examined whether more processing is involved in deciding that something is unintentional (and thus overriding an initial intentional interpretation) than in deciding that something is unpleasant (where there is presumably no initial “pleasant” interpretation). Participants were asked to judge a series of 12 sentences on one of two dimensions: intentional/unintentional (experimental group) or pleasant/unpleasant (control group). People in the experimental group remembered more unintentional sentences than people in the control group. Findings across the three studies suggest that adults have an implicit bias to infer intention in all behavior. This research has important implications both in terms of theory (e.g., dual-process model for intentional reasoning), and practice (e.g., treating aggression, legal judgments).
---
paper_title: I See What You Mean: How Attentional Selection Is Shaped by Ascribing Intentions to Others

paper_content:

The ability to understand and predict others’ behavior is essential for successful interactions. When making predictions about what other humans will do, we treat them as intentional systems and adopt the intentional stance, i.e., refer to their mental states such as desires and intentions. In the present experiments, we investigated whether the mere belief that the observed agent is an intentional system influences basic social attention mechanisms. We presented pictures of a human and a robot face in a gaze cuing paradigm and manipulated the likelihood of adopting the intentional stance by instruction: in some conditions, participants were told that they were observing a human or a robot, in others, that they were observing a human-like mannequin or a robot whose eyes were controlled by a human. In conditions in which participants were made to believe they were observing human behavior (intentional stance likely) gaze cuing effects were significantly larger as compared to conditions when adopting the intentional stance was less likely. This effect was independent of whether a human or a robot face was presented. Therefore, we conclude that adopting the intentional stance when observing others’ behavior fundamentally influences basic mechanisms of social attention. The present results provide striking evidence that high-level cognitive processes, such as beliefs, modulate bottom-up mechanisms of attentional selection in a top-down manner.
---
paper_title: The Effect of the Agency and Anthropomorphism on Users' Sense of Telepresence, Copresence, and Social Presence in Virtual Environments

paper_content:

We report on an experiment that examined the influence of anthropomorphism and perceived agency on presence, copresence, and social presence in a virtual environment. The experiment varied the level of anthropomorphism of the image of interactants: high anthropomorphism, low anthropomorphism, or no mage. Perceived agency was manipulated by telling the participants that the image was either an avatar controlled by a human, or an agent controlled by a computer. The results support the prediction that people respond socially to both human and computer-controlled entities, and that the existence of a virtual image increases telepresence. Participants interacting with the less-anthropomorphic image reported more copresence and social presence than those interacting with partners represented by either no mage at all or by a highly anthropomorphic image of the other, indicating that the more anthropomorphic images set up higher expectations that lead to reduced presence when these expectations were not met.
---
paper_title: The overlap between false belief and spatial reorientation in the temporo-parietal junction: The role of input modality and task

paper_content:

ABSTRACTNeuroimaging research has demonstrated that the temporo-parietal junction (TPJ) is activated when unexpected stimuli appear in spatial reorientation tasks as well as during thinking about the beliefs of other people triggered by verbal scenarios. While the role of potential common component processes subserved by the TPJ has been extensively studied to explain this common activation, the potential confounding role of input modality (spatial vs. verbal) has been largely ignored. To investigate the role of input modality apart from task processes, we developed a novel spatial false belief task based on moving shapes. We explored the overlap in TPJ activation across this novel task and traditional tasks of spatial reorientation (Posner) and verbal belief (False Belief vs. Photo stories). The results show substantial overlap across the same spatial input modality (both reorientation and false belief) as well as across the common task process (verbal and spatial belief), but no triple overlap. This sug...
---
paper_title: Beliefs about the Minds of Others Influence How We Process Sensory Information

paper_content:

Attending where others gaze is one of the most fundamental mechanisms of social cognition. The present study is the first to examine the impact of the attribution of mind to others on gaze-guided attentional orienting and its ERP correlates. Using a paradigm in which attention was guided to a location by the gaze of a centrally presented face, we manipulated participants' beliefs about the gazer: gaze behavior was believed to result either from operations of a mind or from a machine. In Experiment 1, beliefs were manipulated by cue identity (human or robot), while in Experiment 2, cue identity (robot) remained identical across conditions and beliefs were manipulated solely via instruction, which was irrelevant to the task. ERP results and behavior showed that participants' attention was guided by gaze only when gaze was believed to be controlled by a human. Specifically, the P1 was more enhanced for validly, relative to invalidly, cued targets only when participants believed the gaze behavior was the result of a mind, rather than of a machine. This shows that sensory gain control can be influenced by higher-order (task-irrelevant) beliefs about the observed scene. We propose a new interdisciplinary model of social attention, which integrates ideas from cognitive and social neuroscience, as well as philosophy in order to provide a framework for understanding a crucial aspect of how humans' beliefs about the observed scene influence sensory processing.
---
paper_title: Is someone watching me? - consideration of social facilitation effects in human-robot interaction experiments

paper_content:

This discussion paper considers the implications of the psychological phenomena of social facilitation effects for human-robot interaction (HRI) studies. Research studies in HRI have significantly increased over the past few years. Such studies investigate, for example, robot appearance and behaviour, and the responses of subjects to robots. However, the possible effects of the experimental context on results from human-robot interaction studies have attracted little attention. In this paper, we provide an overview of robot trials with children and adults, as part of the Cogniron project. Observations from video footage are reported, with particular consideration for the influence of the social context and social facilitation effects, including task complexity, evaluation context and type of presence on outcomes of human-robot interaction studies. Lessons learnt and future design implications for human-robot interaction studies are provided.
---
paper_title: From Mind Perception to Mental Connection: Synchrony as a Mechanism for Social Understanding

paper_content:

Connecting deeply with another mind is as enigmatic as it is fulfilling. Why people ‘‘click’’ with some people but not others is one of the great unsolved mysteries of science. However, researchers from psychology and neuroscience are converging on a likely physiological basis for connection ‐ neural synchrony (entrainment). Here, we review research on the necessary precursors for interpersonal synchrony: the ability to detect a mind and resonate with its outputs. Further, We describe potential mechanisms for the development of synchrony between two minds. We then consider recent neuroimaging and behavioral evidence for the adaptive benefits of synchrony, including neural efficiency and the release of a reward signal that promotes future social interaction. In nature, neural synchrony yields behavioral synchrony. Humans use behavioral synchrony to promote neural synchrony, and thus, social bonding. This reverse-engineering of social connection is an important innovation likely underlying this distinctively human capacity to create large-scale social coordination and cohesion.
---
paper_title: Transformational and Transactional Leadership in Virtual and Physical Environments

paper_content:

Transformational and transactional leadership in both physical (i.e., face-to-face) and virtual settings were examined in a laboratory experiment. Leadership style (transformational or transactiona...
---
paper_title: How we experience immersive virtual environments: the concept of presence and its measurement *

paper_content:

This paper reviews the concept of presence in immersive virtual environments, the sense of being there signalled by people acting and responding realistically to virtual situations and events. We argue that presence is a unique phenomenon that must be distinguished from the degree of engagement, involvement in the portrayed environment. We argue that there are three necessary conditions for presence: the (a) consistent low latency sensorimotor loop between sensory data and proprioception; (b) statistical plausibility: images must be statistically plausible in relation to the probability distribution of images over natural scenes. A constraint on this plausibility is the level of immersion; (c) behaviour-response correlations: Presence may be enhanced and maintained over time by appropriate correlations between the state and behaviour of participants and responses within the environment, correlations that show appropriate responses to the activity of the participants. We conclude with a discussion of methods for assessing whether presence occurs, and in particular recommend the approach of comparison with ground truth and give some examples of this.
---
paper_title: At the Heart of It All: The Concept of Presence

paper_content:

A number of emerging technologies including virtual reality, simulation rides, video conferencing, home theater, and high definition television are designed to provide media users with an illusion that a mediated experience is not mediated, a perception defined here as presence. Traditional media such as the telephone, radio, television, film, and many others offer a lesser degree of presence as well. This article examines the key concept of presence. It begins by noting practical and theoretical reasons for studying this concept. Six conceptualizations of presence found in a diverse set of literatures are identified and a detailed explication of the concept that incorporates these conceptualizations is presented. Existing research and speculation about the factors that encourage or discourage a sense of presence in media users as well as the physiological and psychological effects of presence are then outlined. Finally, suggestions concerning future systematic research about presence are presented.
---
paper_title: Measuring Presence in Virtual Environments: A Presence Questionnaire

paper_content:

The effectiveness of virtual environments (VEs) has often been linked to the sense of presence reported by users of those VEs. (Presence is defined as the subjective experience of being in one place or environment, even when one is physically situated in another.) We believe that presence is a normal awareness phenomenon that requires directed attention and is based in the interaction between sensory stimulation, environmental factors that encourage involvement and enable immersion, and internal tendencies to become involved. Factors believed to underlie presence were described in the premier issue of Presence: Teleoperators and Virtual Environments. We used these factors and others as the basis for a presence questionnaire (PQ) to measure presence in VEs. In addition we developed an immersive tendencies questionnaire (ITQ) to measure differences in the tendencies of individuals to experience presence. These questionnaires are being used to evaluate relationships among reported presence and other research variables. Combined results from four experiments lead to the following conclusions: ::: the PQ and ITQ are internally consistent measures with high reliability; ::: there is a weak but consistent positive relation between presence and task performance in VEs; ::: individual tendencies as measured by the ITQ predict presence as measured by the PQ; and ::: individuals who report more simulator sickness symptoms in VE report less presence than those who report fewer symptoms.
---
paper_title: Virtual reality in neuroscience research and therapy

paper_content:

The compatibility of virtual reality systems with brain imaging techniques and their use for animal research have aided the widespread adoption of virtual reality environments in both experimental and therapeutic domains. Here the authors review advances in virtual reality technology and its applications.
---
paper_title: Toward a More Robust Theory and Measure of Social Presence: Review and Suggested Criteria

paper_content:

At a time of increased social usage of net and collaborative applications, a robust and detailed theory of social presence could contribute to our understanding of social behavior in mediated environments, allow researchers to predict and measure differences among media interfaces, and guide the design of new social environments and interfaces. A broader theory of social presence can guide more valid and reliable measures. The article reviews, classifies, and critiques existing theories and measures of social presence. A set of criteria and scope conditions is proposed to help remedy limitations in past theories and measures and to provide a contribution to a more robust theory and measure of social presence.
---
paper_title: Virtual encounters. Creating social presence in net-based collaborations

paper_content:

The paper discusses the experience of "social presence" as a relevant effect dimension of avatarmediated net-communication. Special attention is paid to measurement issues combining subjective verbal reports of social presence with objective behavioral data relating nonverbal activity, visual attention and person perception. Data will be reported from a study comparing the effects of different real time communication modes (text, audio, video and avatar) in a shared collaborative workspace. Results point to a significant difference between text and all other communication modes, indicating that audio, video and avatar systems worked similarly well in creating an experience of social presence. Analyses of behavioral data yield similar levels of visual attention for both video and avatar conferencing modes, which is however decreasing over time. The data raises critical questions about the added value of avatar systems and the specific requirements those systems have to meet to prove superior to mere audio/video transmissions.
---
paper_title: The social psychology of telecommunications

paper_content:

A maritime assembly incorporating a foundation raft for founding, or when founded, on a sea bed, and having venting means responsive to the movement of waves whereby pore pressure at at least one point beneath the raft is reduced with respect to the hydro-static pressure appropriate to the external depth of water below the mean level of water in which the raft is (to be) founded.
---
paper_title: The Effect of the Agency and Anthropomorphism on Users' Sense of Telepresence, Copresence, and Social Presence in Virtual Environments

paper_content:

We report on an experiment that examined the influence of anthropomorphism and perceived agency on presence, copresence, and social presence in a virtual environment. The experiment varied the level of anthropomorphism of the image of interactants: high anthropomorphism, low anthropomorphism, or no mage. Perceived agency was manipulated by telling the participants that the image was either an avatar controlled by a human, or an agent controlled by a computer. The results support the prediction that people respond socially to both human and computer-controlled entities, and that the existence of a virtual image increases telepresence. Participants interacting with the less-anthropomorphic image reported more copresence and social presence than those interacting with partners represented by either no mage at all or by a highly anthropomorphic image of the other, indicating that the more anthropomorphic images set up higher expectations that lead to reduced presence when these expectations were not met.
---
paper_title: From presence to consciousness through virtual reality

paper_content:

Immersive virtual environments can break the deep, everyday connection between where our senses tell us we are and where we are actually located and whom we are with. The concept of 'presence' refers to the phenomenon of behaving and feeling as if we are in the virtual world created by computer displays. In this article, we argue that presence is worthy of study by neuroscientists, and that it might aid the study of perception and consciousness.
---
paper_title: Engagement with artificial intelligence through natural interaction models

paper_content:

As Artificial Intelligence (AI) systems become more ubiquitous, what user experience design paradigms will be used by humans to impart their needs and intents to an AI system, in order to engage in a more social interaction? In our work, we look mainly at expression and creativity based systems, where the AI both attempts to model or understand/assist in processes of human expression and creativity. We therefore have designed and implemented a prototype system with more natural interaction modes for engagement with AI as well as other human computer interaction (HCI) where a more open natural communication stream is beneficial. Our proposed conversational agent system makes use of the affective signals from the gestural behaviour of the user and the semantic information from the speech input in order to generate a personalised, human-like conversation that is expressed in the visual and conversational output of the 3D virtual avatar system. We describe our system and two application spaces we are using it in - a care advisor / assistant for the elderly and an interactive creative assistant for uses to produce art forms.
---
paper_title: Toward a Taxonomy of Copresence

paper_content:

This paper contributes to the presence literature by explicating the meanings and subtypes of copresence. Copresence is defined here as consisting of two dimensions: copresence as mode of being with others, and copresence as sense of being with others. Mode of copresence refers to the physical conditions that structure human interaction. Six such conditions are delineated. Sense of copresence, on the other hand, refers to the subjective experience of being with others that an individual acquires in interaction. The main argument of this paper is that mode of copresence affects sense of copresence, and knowledge of now the former affects the latter will benefit copresence design.
---
paper_title: A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments

paper_content:

This paper reviews the concepts of immersion and presence in virtual environments VEs. We propose that the degree of immersion can be objectively assessed as the characteristics of a technology, and has dimensions such as the extent to which a display system can deliver an inclusive, extensive, surrounding, and vivid illusion of virtual environment to a participant. Other dimensions of immersion are concerned with the extent of body matching, and the extent to which there is a self-contained plot in which the participant can act and in which there is an autonomous response. Presence is a state of consciousness that may be concomitant with immersion, and is related to a sense of being in a place. Presence governs aspects of autonomie responses and higher-level behaviors of a participant in a VE. The paper considers single and multiparticipant shared environments, and draws on the experience of ComputerSupported Cooperative Working CSCW research as a guide to understanding presence in shared environments. The paper finally outlines the aims of the FIVE Working Group, and the 1995 FIVE Conference in London, UK.
---
paper_title: I move , therefore I am : A new theoretical framework to investigate agency and ownership

paper_content:

Abstract The neurocognitive structure of the acting self has recently been widely studied, yet is still perplexing and remains an often confounded issue in cognitive neuroscience, psychopathology and philosophy. We provide a new systematic account of two of its main features, the sense of agency and the sense of ownership, demonstrating that although both features appear as phenomenally uniform, they each in fact are complex crossmodal phenomena of largely heterogeneous functional and (self-)representational levels. These levels can be arranged within a gradually evolving, onto- and phylogenetically plausible framework which proceeds from basic non-conceptual sensorimotor processes to more complex conceptual and meta-representational processes of agency and ownership, respectively. In particular, three fundamental levels of agency and ownership processing have to be distinguished: The level of feeling, thinking and social interaction. This naturalistic account will not only allow to “ground the self in action”, but also provide an empirically testable taxonomy for cognitive neuroscience and a new tool for disentangling agency and ownership disturbances in psychopathology (e.g. alien hand, anarchic hand, anosognosia for one’s own hemiparesis).
---
paper_title: ‘I Interact Therefore I Am’: The Self as a Historical Product of Dialectical Attunement

paper_content:

In this article, moving from being to becoming, we construe the ‘self’ as a dynamic process rather than as a static entity. To this end we draw on dialectics and Bayesian accounts of cognition. The former allows us to holistically consider the ‘self’ as the interplay between internalization and externalization and the latter to operationalize our suggestion formally. Internalization is considered here as the co-construction of bodily hierarchical models of the (social) world and the organism, while externalization is taken as the collective transformation of the world. We do not consider these processes as sequentially linked, but rather as a dialectic between the collective and the individual. This leads us to the suggestion of the self as a historical product of dialectical attunement across multiple time scales, from species evolution and culture to individual development and everyday learning. Subsequently, we describe concrete means for empirically testing our proposal in the form of two-person psychophysiology and multi-level analyses of intersubjectivity. Taken together, we suggest that a fine-grained analysis of social interaction might allow us to reconsider the ‘self’ beyond the static individual, i.e. how it emerges and manifests itself in social relations. Such an approach, we believe, could be relevant in multiple fields, from ethics and psychiatry to pedagogy and artificial intelligence.
---
paper_title: Life as we know it

paper_content:

This paper presents a heuristic proof (and simulations of a primordial soup) suggesting that life—or biological self-organization—is an inevitable and emergent property of any (ergodic) random dynamical system that possesses a Markov blanket. This conclusion is based on the following arguments: if the coupling among an ensemble of dynamical systems is mediated by short-range forces, then the states of remote systems must be conditionally independent. These independencies induce a Markov blanket that separates internal and external states in a statistical sense. The existence of a Markov blanket means that internal states will appear to minimize a free energy functional of the states of their Markov blanket. Crucially, this is the same quantity that is optimized in Bayesian inference. Therefore, the internal states (and their blanket) will appear to engage in active Bayesian inference. In other words, they will appear to model—and act on—their world to preserve their functional and structural integrity, leading to homoeostasis and a simple form of autopoiesis.
---
paper_title: The cerebellum contributes to somatosensory cortical activity during self-produced tactile stimulation. NeuroImage

paper_content:

Abstract We used fMRI to examine neural responses when subjects experienced a tactile stimulus that was either self-produced or externally produced. The somatosensory cortex showed increased levels of activity when the stimulus was externally produced. In the cerebellum there was less activity associated with a movement that generated a tactile stimulus than with a movement that did not. This difference suggests that the cerebellum is involved in predicting the specific sensory consequences of movements and providing the signal that is used to attenuate the sensory response to self-generated stimulation. In this paper, we use regression analyses to test this hypothesis explicitly. Specifically, we predicted that activity in the cerebellum contributes to the decrease in somatosensory cortex activity during self-produced tactile stimulation. Evidence in favor of this hypothesis was obtained by demonstrating that activity in the thalamus and primary and secondary somatosensory cortices significantly regressed on activity in the cerebellum when tactile stimuli were self-produced but not when they were externally produced. This supports the proposal that the cerebellum is involved in predicting the sensory consequences of movements. In the present study, this prediction is accurate when tactile stimuli are self-produced relative to when they are externally produced, and is therefore used to attenuate the somatosensory response to the former type of tactile stimulation but not the latter.
---
paper_title: The free-energy self: A predictive coding account of self-recognition

paper_content:

Abstract Recognising and representing one's self as distinct from others is a fundamental component of self-awareness. However, current theories of self-recognition are not embedded within global theories of cortical function and therefore fail to provide a compelling explanation of how the self is processed. We present a theoretical account of the neural and computational basis of self-recognition that is embedded within the free-energy account of cortical function. In this account one's body is processed in a Bayesian manner as the most likely to be “me”. Such probabilistic representation arises through the integration of information from hierarchically organised unimodal systems in higher-level multimodal areas. This information takes the form of bottom-up “surprise” signals from unimodal sensory systems that are explained away by top-down processes that minimise the level of surprise across the brain. We present evidence that this theoretical perspective may account for the findings of psychological and neuroimaging investigations into self-recognition and particularly evidence that representations of the self are malleable, rather than fixed as previous accounts of self-recognition might suggest.
---
paper_title: The “sense of agency” and its underlying cognitive and neural mechanisms

paper_content:

Abstract The sense of agency is a central aspect of human self-consciousness and refers to the experience of oneself as the agent of one’s own actions. Several different cognitive theories on the sense of agency have been proposed implying divergent empirical approaches and results, especially with respect to neural correlates. A multifactorial and multilevel model of the sense of agency may provide the most constructive framework for integrating divergent theories and findings, meeting the complex nature of this intriguing phenomenon.
---
paper_title: Sense of agency in the human brain

paper_content:

The experience of controlling our own actions is an important feature of human mental life. The processes giving rise to this experience are thought to be disrupted in some psychiatric disorders. In this article, Haggard describes recent developments in our understanding of the cognitive processes and neural mechanisms underlying the sense of agency.
---
paper_title: The free-energy principle: a unified brain theory?

paper_content:

A free-energy principle has been proposed recently that accounts for action, perception and learning. This Review looks at some key brain theories in the biological (for example, neural Darwinism) and physical (for example, information theory and optimal control theory) sciences from the free-energy perspective. Crucially, one key theme runs through each of these theories — optimization. Furthermore, if we look closely at what is optimized, the same quantity keeps emerging, namely value (expected reward, expected utility) or its complement, surprise (prediction error, expected cost). This is the quantity that is optimized under the free-energy principle, which suggests that several global brain theories might be unified within a free-energy framework.
---
paper_title: Explaining delusions of control: The comparator model 20years on

paper_content:

Abstract Over the last 20 years the comparator model for delusions of control has received considerable support in terms of empirical studies. However, the original version clearly needs to be replaced by a model with a much greater degree of sophistication and specificity. Future developments are likely to involve the specification of the role of dopamine in the model and a generalisation of its explanatory power to the whole range of positive symptoms. However, we will still need to explain why symptoms can be so variable and we still do not understand the origin of the most mysterious symptom of all: thought insertion.
---
paper_title: Abnormalities in the Awareness and Control of Action

paper_content:

Much of the functioning of the motor system occurs without awareness. Nevertheless, we are aware of some aspects of the current state of the system and we can prepare and make movements in the imagination. These mental representations of the actual and possible states of the system are based on two sources: sensory signals from skin and muscles, and the stream of motor commands that have been issued to the system. Damage to the neural substrates of the motor system can lead to abnormalities in the awareness of action as well as defects in the control of action. We provide a framework for understanding how these various abnormalities of awareness can arise. Patients with phantom limbs or with anosognosia experience the illusion that they can move their limbs. We suggest that these representations of movement are based on streams of motor commands rather than sensory signals. Patients with utilization behaviour or with delusions of control can no longer properly link their intentions to their actions. In these cases the impairment lies in the representation of intended movements. The location of the neural damage associated with these disorders suggests that representations of the current and predicted state of the motor system are in parietal cortex, while representations of intended actions are found in prefrontal and premotor cortex.
---
paper_title: Somatosensory activations during the observation of touch and a case of vision-touch synesthesia

paper_content:

In this study, we describe a new form of synaesthesia in which visual perception of touch elicits conscious tactile experiences in the perceiver. We describe a female subject (C) for whom the observation of another person being touched is experienced as tactile stimulation on the equivalent part of C’s own body. Apart from this clearly abnormal synesthetic experience, C is healthy and normal in every other way. In this study, we investigate whether C’s ‘mirrored touch’ synesthetic experience is caused by overactivity in the neural system that responds to the observation of touch. A functional MRI experiment was designed to investigate the neural system involved in the perception of touch in a group of 12 non-synesthetic control subjects and in C. We investigated neural activity to the observation of touch to a human face or neck compared with the observation of touch to equivalent regions on an object. Furthermore, to investigate the somatosensory topography of the activations during observation of touch, we compared activations when observing a human face or neck being touched with activations when the subjects themselves were touched on their own face or neck. The results demonstrated that the somatosensory cortex was activated in the non-synesthetic subjects by the mere observation of touch and that this activation was somatotopically organized such that observation of touch to the face activated the head area of primary somatosensory cortex, whereas observation of touch to the neck did not. Moreover, in non-synesthetic subjects, the brain’s mirror system—comprising premotor cortex, superior temporal sulcus and parietal cortex—was activated by the observation of touch to another human more than to an object. C’s activation patterns differed in three ways from those of the non-synesthetic controls. First, activations in the somatosensory cortex were significantly higher in C when she observed touch. Secondly, an area in left premotor cortex was activated in C to a greater extent than in the non-synesthetic group. Thirdly, the anterior insula cortex bilaterally was activated in C, but there was no evidence of such activation in the non-synesthetic group. The results suggest that, in C, the mirror system for touch is overactive, above the threshold for conscious tactile perception.
---
paper_title: Together, Slowly but Surely: The Role of Social Interaction and Feedback on the Build-Up of Benefit in Collective Decision-Making

paper_content:

That objective reference is necessary for formation of reliable beliefs about the external world is almost axiomatic. However, Condorcet (1785) suggested that purely subjective information—if shared and combined via social interaction—is enough for accurate understanding of the external world. We asked if social interaction and objective reference contribute differently to the formation and build-up of collective perceptual beliefs. In three experiments, dyads made individual and collective perceptual decisions in a two-interval, forced-choice, visual search task. In Experiment 1, participants negotiated their collective decisions with each other verbally and received feedback about accuracy at the end of each trial. In Experiment 2, feedback was not given. In Experiment 3, communication was not allowed but feedback was provided. Social interaction (Experiments 1 and 2 vs. 3) resulted in a significant collective benefit in perceptual decisions. When feedback was not available a collective benefit was not initially obtained but emerged through practice to the extent that in the second half of the experiments, collective benefits obtained with (Experiment 1) and without (Experiment 2) feedback were robust and statistically indistinguishable. Taken together, this work demonstrates that social interaction was necessary for build-up of reliable collaborative benefit, whereas objective reference only accelerated the process but—given enough opportunity for practice—was not necessary for building up successful cooperation.
---
paper_title: Misattributions of agency in schizophrenia are based on imprecise predictions about the sensory consequences of one's actions

paper_content:

The experience of being the initiator of one's own actions seems to be infallible at first glance. Misattributions of agency of one's actions in certain neurological or psychiatric patients reveal, however, that the central mechanisms underlying this experience can go astray. In particular, delusions of influence in schizophrenia might result from deficits in an inferential mechanism that allows distinguishing whether or not a sensory event has been self-produced. This distinction is made by comparing the actual sensory information with the consequences of one's action as predicted on the basis of internal action-related signals such as efference copies. If this internal prediction matches the actual sensory event, an action is registered as self-caused; in case of a mismatch, the difference is interpreted as externally produced. We tested the hypothesis that delusions of influence are based on deficits in this comparator mechanism. In particular, we tested whether patients’ impairments in action attribution tasks are caused by imprecise predictions about the sensory consequences of self-action. Schizophrenia patients and matched controls performed pointing movements in a virtual-reality setup in which the visual consequences of movements could be rotated with respect to the actual movement. Experiment 1 revealed higher thresholds for detecting experimental feedback rotations in the patient group. The size of these thresholds correlated positively with patients’ delusions of influence. Experiment 2 required subjects to estimate their direction of pointing visually in the presence of constantly rotated visual feedback. When compared to controls, patients’ estimates were significantly better adapted to the feedback rotation and exhibited an increased variability. In interleaved trials without visual feedback, i.e. when pointing estimates relied solely on internal action-related signals, this variability was likewise increased and correlated with both delusions of influence and the size of patients’ detection thresholds as assessed in the first experiment. These findings support the notion that delusions of influence are based on imprecise internal predictions about the sensory consequences of one's actions. Moreover, we suggest that such imprecise predictions prompt patients to rely more strongly on (and thus adapt to) external agency cues, in this case vision. Such context-dependent weighted integration of imprecise internal predictions and alternative agency cues might thus reflect the common basis for the various misattributions of agency in schizophrenia patients.
---
paper_title: Why can't you tickle yourself?

paper_content:

One way that is guaranteed to cheer up a tired, cross child or to reduce an adult to a simpering fit of giggles is a few light strokes on the sole of the foot or the armpit. Some people are more tickly than others, and ticklishness varies from situation to situation.1 But 1 question has baffled minds for years: why can't you tickle yourself? Go on—try it. The sensation produced will never rival that of an external stimulus. ::: ::: The sensation of tickle is perceived by the part of the brain called the somatosensory cortex (also called S1). Studies using functional magnetic resonance imaging, a technique that shows changes in brain activity in different situations according to cerebral blood flow, have shown increased S1 activity in response to an externally produced tactile stimulus. However, the same activity is not produced after an identical, self-produced stimulus.2 ::: ::: We need to turn our attention away from the cortical areas of the brain to the cerebellum, a structure attached to its base, to understand why this is the case. The cerebellum has a role in coordinating movement and receives both sensory input from the body (through the spinal cord) and motor input from the higher cortical areas that command movement. In other words, 1 of its roles is the comparison of the expected response to a stimulus (relying on descending commands from the motor cortex of the brain) with the actual response (relying on sensory feedback from the receptors in the skin). If the 2 responses match, the cerebellum kicks in a mechanism to suppress the effects of the stimulation, and the end result is diminished activity in the somatosensory cortex and with it, diminished perception of the tickle. This cancellation of the sensory consequences of a motor command is a phenomenon known as reafference.3
---
paper_title: A theory of cortical responses

paper_content:

This article concerns the nature of evoked brain responses and the principles underlying their generation. We start with the premise that the sensory brain has evolved to represent or infer the causes of changes in its sensory inputs. The problem of inference is well formulated in statistical terms. The statistical fundaments of inference may therefore afford important constraints on neuronal implementation. By formulating the original ideas of Helmholtz on perception, in terms of modern-day statistical theories, one arrives at a model of perceptual inference and learning that can explain a remarkable range of neurobiological facts. It turns out that the problems of inferring the causes of sensory input (perceptual inference) and learning the relationship between input and cause (perceptual learning) can be resolved using exactly the same principle. Specifically, both inference and learning rest on minimizing the brain’s free energy, as defined in statistical physics. Furthermore, inference and learning can proceed in a biologically plausible fashion. Cortical responses can be seen as the brain’s attempt to minimize the free energy induced by a stimulus and thereby encode the most likely cause of that stimulus. Similarly, learning emerges from changes in synaptic efficacy that minimize the free energy, averaged over all stimuli encountered. The underlying scheme rests on empirical Bayes and hierarchical models of how sensory input is caused. The use of hierarchical models enables the brain to construct prior expectations in a dynamic and context-sensitive fashion. This scheme provides a principled way to understand many aspects of cortical organization and responses. The aim of this article is to encompass many apparently unrelated anatomical, physiological and psychophysical attributes of the brain within a single theoretical perspective. In terms of cortical architectures, the theoretical treatment predicts that sensory cortex should be arranged hierarchically, that connections should be reciprocal and that forward and backward connections should show a functional asymmetry (forward connections are driving, whereas backward connections are both driving and modulatory). In terms of synaptic physiology, it predicts associative plasticity and, for dynamic models, spike-timing-dependent plasticity. In terms of electrophysiology, it accounts for classical and extra classical receptive field effects and long-latency or endogenous components of evoked cortical responses. It predicts the attenuation of responses encoding prediction error with perceptual learning and explains many phenomena such as repetition suppression, mismatch negativity (MMN) and the P300 in electroencephalography. In psychophysical terms, it accounts for the behavioural correlates of these physiological phenomena, for example, priming and global precedence. The final focus of this article is on perceptual learning as measured with the MMN and the implications for empirical studies of coupling among cortical areas using evoked sensory responses.
---
paper_title: An Interoceptive Predictive Coding Model of Conscious Presence

paper_content:

We describe a theoretical model of the neurocognitive mechanisms underlying conscious presence and its disturbances. The model is based on interoceptive prediction error and is informed by predictive models of agency, general models of hierarchical predictive coding and dopaminergic signaling in cortex, the role of the anterior insular cortex (AIC) in interoception and emotion, and cognitive neuroscience evidence from studies of virtual reality and of psychiatric disorders of presence, specifically depersonalization/derealization disorder. The model associates presence with successful suppression by top-down predictions of informative interoceptive signals evoked by autonomic control signals and, indirectly, by visceral responses to afferent sensory signals. The model connects presence to agency by allowing that predicted interoceptive signals will depend on whether afferent sensory signals are determined, by a parallel predictive-coding mechanism, to be self-generated or externally caused. Anatomically, we identify the AIC as the likely locus of key neural comparator mechanisms. Our model integrates a broad range of previously disparate evidence, makes predictions for conjoint manipulations of agency and presence, offers a new view of emotion as interoceptive inference, and represents a step toward a mechanistic account of a fundamental phenomenological property of consciousness.
---
paper_title: ‘I Interact Therefore I Am’: The Self as a Historical Product of Dialectical Attunement

paper_content:

In this article, moving from being to becoming, we construe the ‘self’ as a dynamic process rather than as a static entity. To this end we draw on dialectics and Bayesian accounts of cognition. The former allows us to holistically consider the ‘self’ as the interplay between internalization and externalization and the latter to operationalize our suggestion formally. Internalization is considered here as the co-construction of bodily hierarchical models of the (social) world and the organism, while externalization is taken as the collective transformation of the world. We do not consider these processes as sequentially linked, but rather as a dialectic between the collective and the individual. This leads us to the suggestion of the self as a historical product of dialectical attunement across multiple time scales, from species evolution and culture to individual development and everyday learning. Subsequently, we describe concrete means for empirically testing our proposal in the form of two-person psychophysiology and multi-level analyses of intersubjectivity. Taken together, we suggest that a fine-grained analysis of social interaction might allow us to reconsider the ‘self’ beyond the static individual, i.e. how it emerges and manifests itself in social relations. Such an approach, we believe, could be relevant in multiple fields, from ethics and psychiatry to pedagogy and artificial intelligence.
---
paper_title: The Predictive Mind

paper_content:

Preface Introduction PART I: THE MECHANISM 1. Perception as causal inference 2. Prediction error minimisation 3. Prediction error, context, and precision 4. Action and expected experience PART II: THE WORLD 5. Binding is inference 6. Is predicting seeing? 7. Precarious prediction 8. Surprise and misrepresentation PART III: THE MIND 9. Precision, attention, and consciousness 10. Perceptual unity in action 11. The fragile mirror of nature 12. Into the predictive mind Concluding remarks: The mind in prediction Acknowledgements Bibliography Index
---
paper_title: Beyond Autism: Introducing the Dialectical Misattunement Hypothesis and a Bayesian Account of Intersubjectivity

paper_content:

Drawing on sociocultural theories and Bayesian accounts of brain function, in this article we construe psychiatric conditions as disorders of social interaction to fully account for their complexity and dynamicity across levels of description and temporal scales. After an introduction of the theoretical underpinnings of our integrative approach, we take autism spectrum conditions (ASC) as a paradigm example and discuss how neurocognitive hypotheses can be translated into a Bayesian formulation, i.e., in terms of predictive processing and active inference. We then argue that consideration of individuals (even within a Bayesian framework) will not be enough for a comprehensive understanding of psychiatric conditions and consequently put forward the dialectical misattunement hypothesis, which views psychopathology not merely as disordered function within single brains but also as a dynamic interpersonal mismatch that encompasses various levels of description. Moving from a mere comparison of groups, i.e., “healthy” persons versus “patients,” to a fine-grained analysis of social interactions within dyads and groups of individuals will open new avenues and may allow to avoid an overly neurocentric scope in psychiatric research as well as help to reduce social exclusion.
---
paper_title: Philosophical conceptions of the self: implications for cognitive science

paper_content:

Abstract Several recently developed philosophical approaches to the self promise to enhance the exchange of ideas between the philosophy of the mind and the other cognitive sciences. This review examines two important concepts of self: the ‘minimal self', a self devoid of temporal extension, and the ‘narrative self', which involves personal identity and continuity across time. The notion of a minimal self is first clarified by drawing a distinction between the sense of self-agency and the sense of self-ownership for actions. This distinction is then explored within the neurological domain with specific reference to schizophrenia, in which the sense of self-agency may be disrupted. The convergence between the philosophical debate and empirical study is extended in a discussion of more primitive aspects of self and how these relate to neonatal experience and robotics. The second concept of self, the narrative self, is discussed in the light of Gazzaniga's left-hemisphere ‘interpreter' and episodic memory. Extensions of the idea of a narrative self that are consistent with neurological models are then considered. The review illustrates how the philosophical approach can inform cognitive science and suggests that a two-way collaboration may lead to a more fully developed account of the self.
---
paper_title: Bayesian model selection for group studies — Revisited

paper_content:

Abstract In this paper, we revisit the problem of Bayesian model selection (BMS) at the group level. We originally addressed this issue in Stephan et al. (2009) , where models are treated as random effects that could differ between subjects, with an unknown population distribution. Here, we extend this work, by (i) introducing the Bayesian omnibus risk (BOR) as a measure of the statistical risk incurred when performing group BMS, (ii) highlighting the difference between random effects BMS and classical random effects analyses of parameter estimates, and (iii) addressing the problem of between group or condition model comparisons. We address the first issue by quantifying the chance likelihood of apparent differences in model frequencies. This leads to the notion of protected exceedance probabilities. The second issue arises when people want to ask “whether a model parameter is zero or not” at the group level. Here, we provide guidance as to whether to use a classical second-level analysis of parameter estimates, or random effects BMS. The third issue rests on the evidence for a difference in model labels or frequencies across groups or conditions. Overall, we hope that the material presented in this paper finesses the problems of group-level BMS in the analysis of neuroimaging and behavioural data.
---
paper_title: Observing the Observer (I): Meta-Bayesian Models of Learning and Decision-Making

paper_content:

In this paper, we present a generic approach that can be used to infer how subjects make optimal decisions under uncertainty. This approach induces a distinction between a subject's perceptual model, which underlies the representation of a hidden "state of affairs" and a response model, which predicts the ensuing behavioural (or neurophysiological) responses to those inputs. We start with the premise that subjects continuously update a probabilistic representation of the causes of their sensory inputs to optimise their behaviour. In addition, subjects have preferences or goals that guide decisions about actions given the above uncertain representation of these hidden causes or state of affairs. From a Bayesian decision theoretic perspective, uncertain representations are so-called "posterior" beliefs, which are influenced by subjective "prior" beliefs. Preferences and goals are encoded through a "loss" (or "utility") function, which measures the cost incurred by making any admissible decision for any given (hidden) state of affair. By assuming that subjects make optimal decisions on the basis of updated (posterior) beliefs and utility (loss) functions, one can evaluate the likelihood of observed behaviour. Critically, this enables one to "observe the observer", i.e. identify (context- or subject-dependent) prior beliefs and utility-functions using psychophysical or neurophysiological measures. In this paper, we describe the main theoretical components of this meta-Bayesian approach (i.e. a Bayesian treatment of Bayesian decision theoretic predictions). In a companion paper ('Observing the observer (II): deciding when to decide'), we describe a concrete implementation of it and demonstrate its utility by applying it to simulated and real reaction time data from an associative learning task.
---
paper_title: Precise minds in uncertain worlds: predictive coding in autism.

paper_content:

There have been numerous attempts to explain the enigma of autism, but existing neurocognitive theories often provide merely a refined description of 1 cluster of symptoms. Here we argue that deficits in executive functioning, theory of mind, and central coherence can all be understood as the consequence of a core deficit in the flexibility with which people with autism spectrum disorder can process violations to their expectations. More formally we argue that the human mind processes information by making and testing predictions and that the errors resulting from violations to these predictions are given a uniform, inflexibly high weight in autism spectrum disorder. The complex, fluctuating nature of regularities in the world and the stochastic and noisy biological system through which people experience it require that, in the real world, people not only learn from their errors but also need to (meta-)learn to sometimes ignore errors. Especially when situations (e.g., social) or stimuli (e.g., faces) become too complex or dynamic, people need to tolerate a certain degree of error in order to develop a more abstract level of representation. Starting from an inability to flexibly process prediction errors, a number of seemingly core deficits become logically secondary symptoms. Moreover, an insistence on sameness or the acting out of stereotyped and repetitive behaviors can be understood as attempts to provide a reassuring sense of predictive success in a world otherwise filled with error.
---
paper_title: Inferring cognitive traits of individual subjects through gaze controlled video games

paper_content:

Conventional paradigms can successfully capture many cognitive processes, unveiling several (patho-)physiological mechanisms. However, often they come at the cost of tedious tasks for the participants, which can impose critical restrictions especially when it comes to certain neurocognitive conditions. Immersive videogames could provide an entertaining alternative of higher ecological validity. Eye or other body movements could then be modeled for unveiling cognition, for example by modeling frameworks based on predictive coding theories for brain function. In this project, we present an approach that combines video games, eye tracking and Bayesian modeling (i.e. HGF, [1]) for studying learning and decision-making. To that end, first, an interactive videogame was developed. The player’s goal was to maximize a score by efficiently exploiting regularities in the environment, i.e. the game’s structure. Second, we conducted a proof-of-concept study with 10 healthy volunteers, considering different probabilistic conditions across two spatial dimensions. Behavior was monitored by recording eye-gaze and/or mouse-movements. Third, the behavioral responses were modeled with a two-dimensional Bayesian framework using hierarchical-Gaussian-filtering, as well as classic reinforcement learning, yielding insights about both learning and decision-making processes. Bayesian model comparison was used to infer on the cognitive strategy applied by the participants. According to the winning model, subjects appeared to use a volatility dependent learning rate. Their response variability was found to be highly influenced by the precision (i.e. confidence) of predictions and responses appeared to be spatially correlated. Finally, spillover effects between the learning and the response model were identified across dimensions for certain subjects. The 2D computational framework implemented in this study provides a first step towards interactive paradigms that allow for a detailed investigation of multimodal prediction precision. As such it could be well suited to study spectrum conditions such as autism or schizophrenia for which aberrant precision has been hypothesized. 1. Mathys C, Daunizeau J, Friston KJ, & Stephan KE. (2011). A Bayesian foundation for individual learning under uncertainty. Frontiers in human neuroscience, 5:39.
---
paper_title: A Bayesian Foundation for Individual Learning Under Uncertainty

paper_content:

Computational learning models are critical for understanding mechanisms of adaptive behavior. However, the two major current frameworks, reinforcement learning (RL) and Bayesian learning, both have certain limitations. For example, many Bayesian models are agnostic of inter-individual variability and involve complicated integrals, making online learning difficult. Here, we introduce a generic hierarchical Bayesian framework for individual learning under multiple forms of uncertainty (e.g., environmental volatility and perceptual uncertainty). The model assumes Gaussian random walks of states at all but the first level, with the step size determined by the next higher level. The coupling between levels is controlled by parameters that shape the influence of uncertainty on learning in a subject-specific fashion. Using variational Bayes under a mean field approximation and a novel approximation to the posterior energy function, we derive trial-by-trial update equations which (i) are analytical and extremely efficient, enabling real-time learning, (ii) have a natural interpretation in terms of RL, and (iii) contain parameters representing processes which play a key role in current theories of learning, e.g., precision-weighting of prediction error. These parameters allow for the expression of individual differences in learning and may relate to specific neuromodulatory mechanisms in the brain. Our model is very general: it can deal with both discrete and continuous states and equally accounts for deterministic and probabilistic relations between environmental events and perceptual states (i.e., situations with and without perceptual uncertainty). These properties are illustrated by simulations and analyses of empirical time series. Overall, our framework provides a novel foundation for understanding normal and pathological learning that contextualizes RL within a generic Bayesian scheme and thus connects it to principles of optimality from probability theory.
---
paper_title: The crisis of minimal self-awareness in schizophrenia: A meta-analytic review

paper_content:

Abstract Disturbances of the minimal self, characterized by abnormal sense of the body, body ownership and agency have been proposed as the phenomenological phenotype of schizophrenia. However, self-disturbances have not been extensively investigated, in part, due to the subjective nature of such processes, and the associated difficulty of studying these phenomena using empirical methodology. Of 115 potential studies on self-awareness in schizophrenia, a total of 25 studies met the inclusion criteria for the meta-analysis comprising 690 patients with schizophrenia and 979 healthy controls. We calculated Hedge's g to obtain a better estimate for the standardized mean difference in small samples. We identified significant basic self-disturbance in schizophrenia, as compared with healthy controls (25 studies, effect size = 0.51). Additional comparison of three sub-categories of the sense of body ownership (4 studies, effect size = 0.91), the sense of agency (15 studies, effect size = 0.49), and self-reported subjective experiences (6 studies, effect size = 0.57) also confirmed group differences. The complete set of 25 studies, and the studies in the sub-categories showed the statistical homogeneity of the characteristics. After a correction for potential publication bias using the trim-and-fill method, the main findings for all studies combined remained significant. Overall, patients with schizophrenia showed deficits in the sense of the minimal self, driven by abnormal sense of body ownership and sense of agency. Interestingly, the disturbed sense of agency in schizophrenia suggests an exaggerated self-consciousness rather than a diminished sense of self. Further research that utilizes sophisticated study designs is needed to examine the nature of self-disturbances in schizophrenia.
---
paper_title: Investigating social gaze as an action-perception online performance

paper_content:

In interpersonal interactions, linguistic information is complemented by non-linguistic information originating largely from facial expressions. The study of online face-to-face social interaction thus entails investigating the multimodal simultaneous processing of oral and visual percepts. Moreover, gaze in and of itself functions as a powerful communicative channel. In this respect, gaze should not be examined as a purely perceptive process but also as an active social performance. We designed a task involving multimodal deciphering of social information based on virtual characters, embedded in naturalistic backgrounds, who directly address the participant with non-literal speech and meaningful facial expressions. Eighteen adult participants were to interpret an equivocal sentence which could be disambiguated by examining the emotional expressions of the character speaking to them face-to-face. To examine self-control and self-awareness of gaze in this context, visual feedback is provided to the participant by a real-time gaze-contingent viewing window centered on the focal point, while the rest of the display is blurred. Eye-tracking data showed that the viewing window induced changes in gaze behaviour, notably longer visual fixations. Notwithstanding, only half the participants ascribed the window displacements to their eye movements. These results highlight the dissociation between non volitional gaze adaptation and self-ascription of agency. Such dissociation provides support for a two-step account of the sense of agency composed of pre-noetic monitoring mechanisms and reflexive processes. We comment upon these results, which illustrate the relevance of our method for studying online social cognition, especially concerning Autism Spectrum Disorders (ASD) where poor pragmatic understanding of oral speech are considered linked to visual peculiarities that impede face exploration.
---
paper_title: Neural correlates of “social gaze” processing in high-functioning autism under systematic variation of gaze duration

paper_content:

Direct gaze is a salient nonverbal signal for social interest and the intention to communicate. In particular, the duration of another's direct gaze can modulate our perception of the social meaning of gaze cues. However, both poor eye contact and deficits in social cognitive processing of gaze are specific diagnostic features of autism. Therefore, investigating neural mechanisms of gaze may provide key insights into the neural mechanisms related to autistic symptoms. Employing functional magnetic resonance imaging (fMRI) and a parametric design, we investigated the neural correlates of the influence of gaze direction and gaze duration on person perception in individuals with high-functioning autism (HFA) and a matched control group. For this purpose, dynamically animated faces of virtual characters, displaying averted or direct gaze of different durations (1 s, 2.5 s and 4 s) were evaluated on a four-point likeability scale. Behavioral results revealed that HFA participants showed no significant difference in likeability ratings depending on gaze duration, while the control group rated the virtual characters as increasingly likeable with increasing gaze duration. On the neural level, direct gaze and increasing direct gaze duration recruit regions of the social neural network (SNN) in control participants, indicating the processing of social salience and a perceived communicative intent. In participants with HFA however, regions of the social neural network were more engaged by averted and decreasing amounts of gaze, while the neural response for processing direct gaze in HFA was not suggestive of any social information processing.
---
paper_title: The perception of self-produced sensory stimuli in patients with auditory hallucinations and passivity experiences: evidence for a breakdown in self-monitoring

paper_content:

Background. To test the hypothesis that certain psychotic symptomatology is due to a defect in selfmonitoring, we investigated the ability of groups of psychiatric patients to dierentiate perceptually between self-produced and externally produced tactile stimuli. Methods. Responses to tactile stimulation were assessed in three groups of subjects: schizophrenic patients; patients with bipolar aective disorder or depression; and normal control subjects. Within the psychiatric groups subjects were divided on the basis of the presence or absence of auditory hallucinations and}or passivity experiences. The subjects were asked to rate the perception of a tactile sensation on the palm of their left hand. The tactile stimulation was either self-produced by movement of the subject’s right hand or externally produced by the experimenter. Results. Normal control subjects and those psychiatric patients with neither auditory hallucinations nor passivity phenomena experienced self-produced stimuli as less intense, tickly and pleasant than identical, externally produced tactile stimuli. In contrast, psychiatric patients with these symptoms did not show a decrease in their perceptual ratings for tactile stimuli produced by themselves as compared with those produced by the experimenter. This failure to show a dierence in perception between self-produced and externally produced stimuli appears to relate to the presence of auditory hallucinations and}or passivity experiences rather than to the diagnosis of schizophrenia. Conclusions. We propose that auditory hallucinations and passivity experiences are associated with an abnormality in the self-monitoring mechanism that normally allows us to distinguish selfproduced from externally produced sensations.
---
paper_title: Virtual reality in neuroscience research and therapy

paper_content:

The compatibility of virtual reality systems with brain imaging techniques and their use for animal research have aided the widespread adoption of virtual reality environments in both experimental and therapeutic domains. Here the authors review advances in virtual reality technology and its applications.
---
paper_title: The sense of agency in autism spectrum disorders: a dissociation between prospective and retrospective mechanisms?

paper_content:

While a large number of studies have reported impairments in social and interpersonal abilities in individuals with autism spectrum disorder (ASD), relatively few studies have focused on self-related knowledge in this population. One of the processes implicated in the physical dimension of the Self is the sense of agency (SoA), i.e., the experience of initiating and controlling one’s own actions and producing desired changes in the world via these actions. So far, the few studies investigating SoA in ASD have reported contrasting results, with some showing spared, others impaired SoA. Here, we review the existing literature and suggest that the distinction between prospective and retrospective mechanisms of the SoA might help reconcile the existing findings. In the light of a multi-componential model of SoA, we propose the view that a specific impairment at the level of prospective mechanisms acting on internal agency signals (i.e., the intention, action selection, or command produced to achieve the goal) may be responsible for the reduced SoA in ASD, along with spared retrospective mechanisms. Future research should shed light on the impact of abnormal SoA on social and self-related dysfunctions in ASD.
---
paper_title: Towards a second-person neuropsychiatry

paper_content:

Psychiatric disorders can affect our ability to successfully and enjoyably interact with others. Conversely, having difficulties in social relations is known to increase the risk of developing a psychiatric disorder. In this article, the assumption that psychiatric disorders can be construed as disorders of social interaction is reviewed from a clinical point of view. Furthermore, it is argued that a psychiatrically motivated focus on the dynamics of social interaction may help to provide new perspectives for the field of social neuroscience. Such progress may be crucial to realize social neuroscience's translational potential and to advance the transdiagnostic investigation of the neurobiology of psychiatric disorders.
---
paper_title: The “sense of agency” and its underlying cognitive and neural mechanisms

paper_content:

Abstract The sense of agency is a central aspect of human self-consciousness and refers to the experience of oneself as the agent of one’s own actions. Several different cognitive theories on the sense of agency have been proposed implying divergent empirical approaches and results, especially with respect to neural correlates. A multifactorial and multilevel model of the sense of agency may provide the most constructive framework for integrating divergent theories and findings, meeting the complex nature of this intriguing phenomenon.
---
paper_title: Doing good or bad: How interactions between action and emotion expectations shape the sense of agency

paper_content:

The emotional consequences of our own and others’ actions can influence our agentive self-awareness in social contexts. Positive outcomes are usually linked to the self and used for self-enhancement, whereas negative outcomes are more often attributed to others. In most situations, these causal attribution tendencies seem to be immediately present instead of involving reflective interpretations of the action experience. To address the question at which level of the cognitive hierarchy emotions and action perception interact, we adopted a social reward anticipation paradigm. Here, participants or their interaction partner received positive or negative action outcomes and performed speeded attribution choices regarding causation of the action outcome. Event-Related Potential (ERP) results showed that the emotional value of an outcome already influenced the classical N1 self-attenuation effect, with reduced embodied agentive self-awareness for negative outcomes at initial sensorimotor stages. At the level of...
---
paper_title: Treatment for chronic depression using Cognitive Behavioral Analysis System of Psychotherapy (CBASP)

paper_content:

The article presents an overview of the chronic depression construct in the Diagnostic and Statistical Manual of Mental Disorders (DSMs) since 1980, including developmental etiology and psychopathological patterns of the chronically depressed adult. Then the Cognitive Behavioral Analysis System of Psychotherapy (CBASP), an empirically supported psychotherapy and the only therapy developed specifically to treat chronic depression, is described. CBASP addresses the idiosyncratic problems of the chronically depressed adult using Situational Analysis, the Interpersonal Discrimination Exercise, and Behavioral Skill Training/Rehearsal techniques. The first two techniques are illustrated using a verbatim case example. © 2003 Wiley Periodicals, Inc. J Clin Psychol/In Session.
---
paper_title: Gaze perception in social anxiety and social anxiety disorder

paper_content:

Clinical observations suggest abnormal gaze perception to be an important indicator of social anxiety disorder (SAD). Experimental research has yet paid relatively little attention to the study of gaze perception in SAD. In this article we first discuss gaze perception in healthy human beings before reviewing self-referential and threat-related biases of gaze perception in clinical and non-clinical socially anxious samples. Relative to controls, socially anxious individuals exhibit an enhanced self-directed perception of gaze directions and demonstrate a pronounced fear of direct eye contact, though findings are less consistent regarding the avoidance of mutual gaze in SAD. Prospects for future research and clinical implications are discussed.
---
paper_title: Altered Pre-reflective Sense of Agency in Autism Spectrum Disorders as Revealed by Reduced Intentional Binding

paper_content:

Autism spectrum disorders (ASDs) are neurodevelopmental conditions that severely affect social interaction, communication and several behavioural and cognitive functions, such as planning and monitoring motor actions. A renewed interest in intrapersonal cognition has recently emerged suggesting a putative dissociation between impaired declarative processes, such as autobiographical memory, and spared implicit processes, such as the sense of agency (SoA) in ASDs. However, so far only a few studies have investigated the integrity of SoA using tasks exclusively tapping reflective mechanisms. Since pre-reflective processes of SoA are based on the same predictive internal models that are involved in planning and monitoring actions, we hypothesized that pre-reflective aspects of SoA, as measured by the intentional binding effect, would be altered in adults with high functioning autism spectrum disorders, relative to volunteers with typical development. Here, in accordance with our hypothesis, we report reduced IB in participants with ASDs.
---
paper_title: Disrupted prediction errors index social deficits in autism spectrum disorder

paper_content:

Social deficits are a core symptom of autism spectrum disorder; however, the perturbed neural mechanisms underpinning these deficits remain unclear. It has been suggested that social prediction errors—coding discrepancies between the predicted and actual outcome of another’s decisions—might play a crucial role in processing social information. While the gyral surface of the anterior cingulate cortex signalled social prediction errors in typically developing individuals, this crucial social signal was altered in individuals with autism spectrum disorder. Importantly, the degree to which social prediction error signalling was aberrant correlated with diagnostic measures of social deficits. Effective connectivity analyses further revealed that, in typically developing individuals but not in autism spectrum disorder, the magnitude of social prediction errors was driven by input from the ventromedial prefrontal cortex. These data provide a novel insight into the neural substrates underlying autism spectrum disorder social symptom severity, and further research into the gyral surface of the anterior cingulate cortex and ventromedial prefrontal cortex could provide more targeted therapies to help ameliorate social deficits in autism spectrum disorder. ::: ::: * Abbreviations ::: : ACCg/s ::: : anterior cingulate cortex gyrus/sulcus ::: ASD ::: : autism spectrum disorder ::: BOLD ::: : blood oxygen level-dependent ::: TD ::: : typically developing ::: vmPFC ::: : ventromedial prefrontal cortex
---
paper_title: Beyond Autism: Introducing the Dialectical Misattunement Hypothesis and a Bayesian Account of Intersubjectivity

paper_content:

Drawing on sociocultural theories and Bayesian accounts of brain function, in this article we construe psychiatric conditions as disorders of social interaction to fully account for their complexity and dynamicity across levels of description and temporal scales. After an introduction of the theoretical underpinnings of our integrative approach, we take autism spectrum conditions (ASC) as a paradigm example and discuss how neurocognitive hypotheses can be translated into a Bayesian formulation, i.e., in terms of predictive processing and active inference. We then argue that consideration of individuals (even within a Bayesian framework) will not be enough for a comprehensive understanding of psychiatric conditions and consequently put forward the dialectical misattunement hypothesis, which views psychopathology not merely as disordered function within single brains but also as a dynamic interpersonal mismatch that encompasses various levels of description. Moving from a mere comparison of groups, i.e., “healthy” persons versus “patients,” to a fine-grained analysis of social interactions within dyads and groups of individuals will open new avenues and may allow to avoid an overly neurocentric scope in psychiatric research as well as help to reduce social exclusion.
---
paper_title: Self–other integration and distinction in schizophrenia: A theoretical analysis and a review of the evidence

paper_content:

Difficulties in self-other processing lie at the core of schizophrenia and pose a problem for patients' daily social functioning. In the present selective review, we provide a framework for understanding self-other integration and distinction, and impairments herein in schizophrenia. For this purpose, we discuss classic motor prediction models in relation to mirror neuron functioning, theory of mind, mimicry, self-awareness, and self-agency phenomena. Importantly, we also discuss the role of more recent cognitive expectation models in these phenomena, and argue that these cognitive models form an essential contribution to our understanding of self-other integration and distinction. In doing so, we bring together different lines of research and connect findings from social psychology, affective neuropsychology, and psychiatry to further our understanding of when and how people integrate versus distinguish self and other, and how this goes wrong in schizophrenia patients.
---
paper_title: Sense of agency in health and disease: A review of cue integration approaches

paper_content:

Sense of agency (SoA) is a compelling but fragile experience that is augmented or attenuated by internal signals and by external cues. A disruption in SoA may characterise individual symptoms of mental illness such as delusions of control. Indeed, it has been argued that generic SoA disturbances may lie at the heart of delusions and hallucinations that characterise schizophrenia. A clearer understanding of how sensorimotor, perceptual and environmental cues complement, or compete with, each other in engendering SoA may prove valuable in deepening our understanding the agency disruptions that characterise certain focal neurological disorders and mental illnesses. Here we examine the integration of SoA cues in health and illness, describing a simple framework of this integration based on Bayesian principles. We extend this to consider how alterations in cue integration may lead to aberrant experiences of agency.
---
paper_title: Forgetting the best when predicting the worst: Preliminary observations on neural circuit function in adolescent social anxiety

paper_content:

a b s t r a c t Social anxiety disorder typically begins in adolescence, a sensitive period for brain development, when increased complexity and salience of peer relationships requires novel forms of social learning. Dis- ordered social learning in adolescence may explain how brain dysfunction promotes social anxiety. Socially anxious adolescents (n = 15) and adults (n = 19) and non-anxious adolescents (n = 24) and adults (n = 32) predicted, then received, social feedback from high and low-value peers while undergoing func- tional magnetic resonance imaging (fMRI). A surprise recall task assessed memory biases for feedback. Neural correlates of social evaluation prediction errors (PEs) were assessed by comparing engagement to expected and unexpected positive and negative feedback. For socially anxious adolescents, but not adults or healthy participants of either age group, PEs elicited heightened striatal activity and negative fronto-striatal functional connectivity. This occurred selectively to unexpected positive feedback from high-value peers and corresponded with impaired memory for social feedback. While impaired memory also occurred in socially-anxious adults, this impairment was unrelated to brain-based PE activity. Thus, social anxiety in adolescence may relate to altered neural correlates of PEs that contribute to impaired learning about social feedback. Small samples necessitate replication. Nevertheless, results suggest that the relationship between learning and fronto-striatal function may attenuate as development progresses. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://
---
paper_title: Intact unconscious processing of eye contact in schizophrenia

paper_content:

Abstract The perception of eye gaze is crucial for social interaction, providing essential information about another person’s goals, intentions, and focus of attention. People with schizophrenia suffer a wide range of social cognitive deficits, including abnormalities in eye gaze perception. For instance, patients have shown an increased bias to misjudge averted gaze as being directed toward them. In this study we probed early unconscious mechanisms of gaze processing in schizophrenia using a technique known as continuous flash suppression. Previous research using this technique to render faces with direct and averted gaze initially invisible reveals that direct eye contact gains privileged access to conscious awareness in healthy adults. We found that patients, as with healthy control subjects, showed the same effect: faces with direct eye gaze became visible significantly faster than faces with averted gaze. This suggests that early unconscious processing of eye gaze is intact in schizophrenia and implies that any misjudgments of gaze direction must manifest at a later conscious stage of gaze processing where deficits and/or biases in attributing mental states to gaze and/or beliefs about being watched may play a role.
---
paper_title: Virtual-reality-based cognitive behavioural therapy versus waiting list control for paranoid ideation and social avoidance in patients with psychotic disorders: a single-blind randomised controlled trial

paper_content:

Summary Background Many patients with psychotic disorders have persistent paranoid ideation and avoid social situations because of suspiciousness and anxiety. We investigated the effects of virtual-reality-based cognitive behavioural therapy (VR-CBT) on paranoid thoughts and social participation. Methods In this randomised controlled trial at seven Dutch mental health centres, outpatients aged 18–65 years with a DSM-IV-diagnosed psychotic disorder and paranoid ideation in the past month were randomly assigned (1:1) via block randomisation to VR-CBT (in addition to treatment as usual) or the waiting list control group (treatment as usual). VR-CBT consisted of 16 individual therapy sessions (each 1 h long). Assessments were done at baseline, after treatment (ie, 3 months from baseline), and at a 6 month follow-up visit. The primary outcome was social participation, which we operationalised as the amount of time spent with other people, momentary paranoia, perceived social threat, and momentary anxiety. Analysis was by intention to treat. This trial was retrospectively registered with ISRCTN, number 12929657. Findings Between April 1, 2014, and Dec 31, 2015, 116 patients with a psychotic disorder were randomly assigned, 58 to the VR-CBT group and 58 to the waiting list control group. Compared with the control, VR-CBT did not significantly increase the amount of time spent with other people at the post-treatment assessment. Momentary paranoid ideation (b=–0·331 [95% CI −0·432 to −0·230], p Interpretation Our results suggest that the addition of VR-CBT to standard treatment can reduce paranoid ideation and momentary anxiety in patients with a psychotic disorder. Funding Fonds NutsOhra, Stichting tot Steun VCVGZ.
---
paper_title: Pre-Conceptual Aspects of Self-Awareness in Autism Spectrum Disorder: The Case of Action-Monitoring

paper_content:

Two experiments were conducted to explore the extent to which individuals with autism experience difficulties in monitoring their own actions, both online and in memory. Participants with autism performed similarly in terms of levels and, importantly, patterns of performance to IQ-matched comparison participants. Each group found it easier to monitor their own actions/agency than to monitor the agency of the experimenter in a computerized task requiring individuals to distinguish person-caused from computer-caused changes in phenomenology. Both groups also showed a typical ‘self-reference effect’, recalling their own actions better than those of the experimenter. Both tasks appear to be reliable markers of underlying action monitoring ability, performance on the ‘Self’ conditions of each task being significantly associated, independent of verbal ability.
---
paper_title: The Use of Virtual Characters to Assess and Train Non-Verbal Communication in High-Functioning Autism

paper_content:

High-functioning autism (HFA) is a neurodevelopmental disorder, which is characterized by life-long socio-communicative impairments on the one hand and preserved verbal and general learning and memory abilities on the other. One of the areas where particular difficulties are observable is the understanding of non-verbal communication cues.Thus, investigating the underlying psychological processes and neural mechanisms of non-verbal communication in HFA allows a better understanding of this disorder, and potentially enables the development of more efficient forms of psychotherapy and trainings. However, the research on non-verbal information processing in HFA faces several methodological challenges. The use of virtual characters (VCs) helps to overcome such challenges by enabling an ecologically valid experience of social presence, and by providing an experimental platform that can be systematically and fully controlled. To make this field of research accessible to a broader audience, we elaborate in the first part of the review the validity of using VCs in non-verbal behavior research on HFA, and we review current relevant paradigms and findings from social-cognitive neuroscience. In the second part, we argue for the use of VCs as either agents or avatars in the context of “transformed social interactions.” This allows for the implementation of real-time social interaction in virtual experimental settings, which represents a more sensitive measure of socio-communicative impairments in HFA. Finally, we argue that VCs and environments are a valuable assistive, educational and therapeutic tool for HFA.
---
paper_title: Virtual Reality Objectifies the Diagnosis of Psychiatric Disorders: A Literature Review

paper_content:

Background To date, a diagnosis in psychiatry is largely based on a clinical interview and questionnaires. The retrospective and subjective nature of these methods leads to recall and interviewer biases. Therefore there is a clear need for more objective and standardized assessment methods to support the diagnostic process. The introduction of Virtual Reality (VR) creates the possibility to simultaneously provoke and measure psychiatric symptoms. Therefore, VR could contribute to the objectivity and reliability in the assessment of psychiatric disorders. Objective In this literature review we will evaluate the assessment of psychiatric disorders by means of VR environments. First, we investigate if these VR environments are capable of simultaneously provoking and measuring psychiatric symptoms. Next, we compare these measures with traditional diagnostic measures. Methods We performed a systematic search using Pubmed, Embase and Psycinfo; references of selected articles were checked for eligibility. We identified studies from 1990-2016 on VR used in the assessment of psychiatric disorders. Studies were excluded if VR was used for therapeutic purposes, if a different technique was used or in case of limitation to a non-clinical sample. Results A total of 39 studies were included for further analysis. The disorders most frequently studied included schizophrenia (n=15), developmental disorders (n=12), eating disorders (n=3) and anxiety disorders (n=6). In ADHD the most comprehensive measurement was used including several key symptoms of the disorder. Most of the studies however concerned the use of VR to assess a single aspect of a psychiatric disorder. Discussion In general, nearly all VR environments studied were able to simultaneously provoke and measure psychiatric symptoms. Furthermore, in 14 studies significant correlations were found between virtual reality measures and traditional diagnostic measures. Relatively small clinical sample sizes were used, impeding definite conclusions. Based on this review, the innovative technique of virtual reality shows potential to contribute to objectivity and reliability in the psychiatric diagnostic process.
---
paper_title: What Can Psychiatric Disorders Tell Us about Neural Processing of the Self?

paper_content:

Many psychiatric disorders are associated with abnormal self-processing. While these disorders also have a wide-range of complex, and often heterogeneous sets of symptoms involving different cognitive, emotional and motor domains, an impaired sense of self can contribute to many of these. Research investigating self-processing in healthy subjects has facilitated identification of changes in specific neural circuits which may cause altered self-processing in psychiatric disorders. While there is evidence for altered self-processing in many psychiatric disorders, here we will focus on four of the most studied ones, schizophrenia, autism spectrum disorder (ASD), unipolar depression and borderline personality disorder (BPD). We review evidence for dysfunction in two different neural systems implicated in self-processing, namely the cortical midline system (CMS) and the mirror-neuron system (MNS), as well as contributions from altered inter-hemispheric communication (IHC). We conclude that while abnormalities in frontal-parietal activity and/or connectivity in the CMS are common to all four disorders there is more disruption of integration between frontal and parietal regions resulting in a shift towards parietal control in schizophrenia and ASD which may contribute to the greater severity and delusional aspects of their symptoms. Abnormalities in the MNS and in IHC are also particularly evident in schizophrenia and ASD and may lead to disturbances in sense of agency and the physical self in these two disorders. A better future understanding of how changes in the neural systems sub-serving self-processing contribute to different aspects of symptom abnormality in psychiatric disorders will require that more studies carry out detailed individual assessments of altered self-processing in conjunction with measurements of neural functioning.
---
paper_title: Attentional bias away from positive social information mediates the link between social anxiety and anxiety vulnerability to a social stressor.

paper_content:

Accumulating evidence suggests that social anxiety is associated with biased processing of positive social information. However, it remains to be determined whether those biases are simply correlates of, or play a role in maintaining social anxiety. The current study examined whether diminished attentional allocation for positive social cues mediates the link between social anxiety and anxiety reactivity to a social-evaluative task. Forty-three undergraduate students ranging in severity of social anxiety symptoms completed a baseline measure of attentional bias for positive social cues (i.e., modified probe detection task) and subsequently delivered an impromptu videotaped speech. Mediation analyses revealed that the tendency to allocate attention away from positive social stimuli mediated the effect of social anxiety on change in state anxiety in response to the stressor. The current findings add to a nascent empirical literature suggesting that aberrant processing of positive social information may contribute to the persistence of excessive social anxiety.
---
paper_title: Is the Sense of Agency in Schizophrenia Influenced by Resting-State Variation in Self-Referential Regions of the Brain?

paper_content:

Schizophrenia is a disturbance of the self, of which the attribution of agency is a major component. In this article, we review current theories of the Sense of Agency, their relevance to schizophrenia, and propose a novel framework for future research. We explore some of the models of agency, in which both bottom-up and top-down processes are implicated in the genesis of agency. We further this line of inquiry by suggesting that ongoing neurological activity (the brain's resting state) in self-referential regions of the brain can provide a deeper level of influence beyond what the current models capture. Based on neuroimaging studies, we suggest that aberrant activity in regions such as the default mode network of individuals with schizophrenia can lead to a misattribution of internally/externally generated stimuli. This can result in symptoms such as thought insertion and delusions of control. Consequently, neuroimaging can contribute to a more comprehensive conceptualization and measurement of agency and potential treatment implications.
---
paper_title: Only connect? Communication, identification, and autism

paper_content:

Abstract In this paper, we elaborate a theoretical position and report an empirical study on a specific form of interpersonal engagement: the propensity to identify with the subjective orientation of another person. On the basis of a hypothesis that individuals with autism have a relative lack of this form of intersubjective connectedness (Hobson, 1993, 2002), we predicted that children and adolescents with autism would contrast with matched participants without autism (n=12 per group) in specific aspects of communication when someone requested them to “Get Pete to do this” and demonstrated actions in Pete's absence. As predicted, on blind ratings of videotapes of participants’ communication, those with autism achieved lower scores on four indices of identification that were selected a priori: emotional engagement, sharing experience in joint attention, communication of style, and shifting in communicative role. The two groups were almost completely separate on a composite measure of identification. We co...
---
paper_title: Automatic attentional orienting to other people’s gaze in schizophrenia

paper_content:

ABSTRACTExplicit tests of social cognition have revealed pervasive deficits in schizophrenia. Less is known of automatic social cognition in schizophrenia. We used a spatial orienting task to investigate automatic shifts of attention cued by another person’s eye gaze in 29 patients and 28 controls. Central photographic images of a face with eyes shifted left or right, or looking straight ahead, preceded targets that appeared left or right of the cue. To examine automatic effects, cue direction was non-predictive of target location. Cue–target intervals were 100, 300, and 800 ms. In non-social control trials, arrows replaced eye-gaze cues. Both groups showed automatic attentional orienting indexed by faster reaction times (RTs) when arrows were congruent with target location across all cue–target intervals. Similar congruency effects were seen for eye-shift cues at 300 and 800 ms intervals, but patients showed significantly larger congruency effects at 800 ms, which were driven by delayed responses to inco...
---
paper_title: Schizophrenia as a self-disorder due to perceptual incoherence

paper_content:

Abstract The aim of this review is to describe the potential relationship between multisensory disintegration and self-disorders in schizophrenia spectrum disorders. Sensory processing impairments affecting multisensory integration have been demonstrated in schizophrenia. From a developmental perspective multisensory integration is considered to be crucial for normal self-experience. An impairment of multisensory integration is called ‘perceptual incoherence’. We theorize that perceptual incoherence may evoke incoherent self-experiences including depersonalization, ambivalence, diminished sense of agency, and ‘loosening of associations’ between thoughts, feelings and actions that lie within the framework of ‘self-disorders’ as described by Sass and Parnas (2003) . We postulate that subconscious attempts to restore perceptual coherence may induce hallucinations and delusions. Increased insight into mechanisms underlying ‘self-disorders’ may enhance our understanding of schizophrenia, improve recognition of early psychosis, and extend the range of therapeutic possibilities.
---
paper_title: Getting a Grip on Social Gaze: Control over Others’ Gaze Helps Gaze Detection in High-Functioning Autism

paper_content:

We investigated the influence of control over a social stimulus on the ability to detect direct gaze in high-functioning autism (HFA). In a pilot study, 19 participants with and 19 without HFA were compared on a gaze detection and a gaze setting task. Participants with HFA were less accurate in detecting direct gaze in the detection task, but did not differ in their ability to establish direct gaze in the setting task. In the main experiment, the results of the pilot study were replicated with 37 participants with and 39 without HFA, suggesting that individuals with HFA have a specific deficit in the passive perception of social cues as opposed to the active control, which seems to be intact.
---
paper_title: Valence and agency influence striatal response to feedback in patients with major depressive disorder

paper_content:

BACKGROUND: ::: ::: Reduced sensitivity to positive feedback is common in patients with major depressive disorder (MDD). However, findings regarding negative feedback are ambiguous, with both exaggerated and blunted responses being reported. The ventral striatum (VS) plays a major role in processing valenced feedback, and previous imaging studies have shown that the locus of controls (self agency v. external agency) over the outcome influences VS response to feedback. We investigated whether attributing the outcome to one's own action or to an external agent influences feedback processing in patients with MDD. We hypothesized that depressed participants would be less sensitive to the feedback attribution reflected by an altered VS response to self-attributed gains and losses. ::: METHODS: ::: ::: Using functional MRI and a motion prediction task, we investigated the neural responses to self-attributed (SA) and externally attributed (EA) monetary gains and losses in unmedicated patients with MDD and healthy controls. ::: RESULTS: ::: ::: We included 21 patients and 25 controls in our study. Consistent with our prediction, healthy controls showed a VS response influenced by feedback valence and attribution, whereas in depressed patients striatal activity was modulated by valence but was insensitive to attribution. This attribution insensitivity led to an altered ventral putamen response for SA - EA losses in patients with MDD compared with healthy controls. ::: LIMITATIONS: ::: ::: Depressed patients with comorbid anxiety disorder were included. ::: CONCLUSION: ::: ::: These results suggest an altered assignment of motivational salience to SA losses in patients with MDD. Altered striatal response to SA negative events may reinforce the belief of not being in control of negative outcomes contributing to a cycle of learned helplessness.
---
paper_title: Observing and participating in social interactions: Action perception and action control across the autistic spectrum

paper_content:

Autism is a developmental condition, characterized by difficulties of social interaction and communication, as well as restricted interests and repetitive behaviors. Although several important conceptions have shed light on specific facets, there is still no consensus about a universal yet specific theory in terms of its underlying mechanisms. While some theories have exclusively focused on sensory aspects, others have emphasized social difficulties. However, sensory and social processes in autism might be interconnected to a higher degree than what has been traditionally thought. We propose that a mismatch in sensory abilities across individuals can lead to difficulties on a social, i.e. interpersonal level and vice versa. In this article, we, therefore, selectively review evidence indicating an interrelationship between perceptual and social difficulties in autism. Additionally, we link this body of research with studies, which investigate the mechanisms of action control in social contexts. By doing so, we highlight that autistic traits are also crucially related to differences in integration, anticipation and automatic responding to social cues, rather than a mere inability to register and learn from social cues. Importantly, such differences may only manifest themselves in sufficiently complex situations, such as real-life social interactions, where such processes are inextricably linked.
---
paper_title: Neural correlates of anticipation and processing of performance feedback in social anxiety

paper_content:

Fear of negative evaluation, such as negative social performance feedback, is the core symptom of social anxiety. The present study investigated the neural correlates of anticipation and perception of social performance feedback in social anxiety. High (HSA) and low (LSA) socially anxious individuals were asked to give a speech on a personally relevant topic and received standardized but appropriate expert performance feedback in a succeeding experimental session in which neural activity was measured during anticipation and presentation of negative and positive performance feedback concerning the speech performance, or a neutral feedback-unrelated control condition. HSA compared to LSA subjects reported greater anxiety during anticipation of negative feedback. Functional magnetic resonance imaging results showed deactivation of medial prefrontal brain areas during anticipation of negative feedback relative to the control and the positive condition, and medial prefrontal and insular hyperactivation during presentation of negative as well as positive feedback in HSA compared to LSA subjects. The results indicate distinct processes underlying feedback processing during anticipation and presentation of feedback in HSA as compared to LSA individuals. In line with the role of the medial prefrontal cortex in self-referential information processing and the insula in interoception, social anxiety seems to be associated with lower self-monitoring during feedback anticipation, and an increased self-focus and interoception during feedback presentation, regardless of feedback valence. Hum Brain Mapp 35:6023–6031, 2014. © 2014 Wiley Periodicals, Inc.
---
paper_title: You looking at me?: Interpreting social cues in schizophrenia

paper_content:

Background. Deficits in the perception of social cues are common in schizophrenia and predict functional outcome. While effective communication depends on deciphering both verbal and non-verbal features, work on non-verbal communication in the disorder is scarce. Method. This behavioural study of 29 individuals with schizophrenia and 25 demographically matched controls used silent video-clips to examine gestural identification, its contextual modulation and related metacognitive representations. Results. In accord with our principal hypothesis, we observed that individuals with schizophrenia exhibited a preserved ability to identify archetypal gestures and did not differentially infer communicative intent from incidental movements. However, patients were more likely than controls to perceive gestures as self-referential when confirmatory evidence was ambiguous. Furthermore, the severity of their current hallucinatory experience inversely predicted their confidence ratings associated with these self-referential judgements. Conclusions. These findings suggest a deficit in the contextual refinement of social-cue processing in schizophrenia that is potentially attributable to impaired monitoring of a mirror mechanism underlying intentional judgements, or to an incomplete semantic representation of gestural actions. Non-verbal communication may be improved in patients through psychotherapeutic interventions that include performance and perception of gestures in group interactions.
---
paper_title: Implicit and Explicit Processes in Social Cognition

paper_content:

In this review we consider research on social cognition in which implicit processes can be compared and contrasted with explicit, conscious processes. In each case, their function is distinct, sometimes complementary and sometimes oppositional. We argue that implicit processes in social interaction are automatic and are often opposed to conscious strategies. While we are aware of explicit processes in social interaction, we cannot always use them to override implicit processes. Many studies show that implicit processes facilitate the sharing of knowledge, feelings, and actions, and hence, perhaps surprisingly, serve altruism rather than selfishness. On the other hand, higher-level conscious processes are as likely to be selfish as prosocial.
---

Title: The Interactive Self – A Review on Simulating Social Interactions to Understand the Mechanisms of Social Agency

Section 1: The Interactive Self and the Sense of Social Agency
Description 1: Write about the fundamental feeling of self-awareness, body ownership, and the sense of agency within motor actions and social interactions.

Section 2: The Minimal Self and Agency
Description 2: Describe the most basic experience of selfhood related to internal predictive sensorimotor cues and research within action and motor control.

Section 3: The Concept of Social Agency
Description 3: Explain the transfer of agency into the social domain and the relevance of sensory cues, social context, and social norms in experienced social agency.

Section 4: Simulating Social Interactions in Virtual Reality
Description 4: Discuss the development and benefits of virtual reality for behavioral research, especially in simulating social interactions with ecological validity and experimental control.

Section 5: Gaze-Contingent Paradigms
Description 5: Illustrate the use of responsive anthropomorphic virtual characters and eye-tracking methods to simulate and analyse social interactions and social agency.

Section 6: Interacting with Virtual Agents
Description 6: Analyze the perception of interactions with virtual agents versus human-controlled avatars, including the aspects critical for successful simulations.

Section 7: The Experience of Presence in Simulated Social Interactions
Description 7: Define the experience of presence and social presence in virtual environments and its relation to the study of social agency.

Section 8: Describing Social Agency with Predictive Coding
Description 8: Outline the predictive coding framework as a model for understanding the neural mechanisms underlying social agency.

Section 9: A Computational Model Describing the Experience of Social Agency
Description 9: Present a formal hierarchical Bayesian model describing the specific mechanisms of social agency and its relevance to different social interactions.

Section 10: Social Agency to Investigate Transdiagnostic Social Impairments Across Psychiatric Disorders
Description 10: Explore social agency in clinical populations to understand transdiagnostic social impairments and potential for research and treatment options.

Section 11: Conclusion
Description 11: Summarize the importance of virtual realities in studying self-experiences and social agency and suggest future research directions.