10.54103/ai/15443



In this paper, I will work on the relationship between perception and imagination in Virtual Reality, claiming that “hallucination” is the ordinary motor also for online perception and not only a deviant form of it. First, I will deal with the problem of perception from the point of view of cognitive semiotics and I will try to underline the crucial role of imagination, claiming that perception is a form of “controlled hallucination”. Later, I will focus on the relationship between perception, hallucination, and memory in Virtual Reality. On the one hand, I will claim that Virtual Reality expresses the transition from actual perception to imagination, memory or dream through another actual perception. On the other hand, I will claim that it can express it without any problems through the old techniques coming from cinema and other audiovisual languages, since they partially share the very same formal apparatus of enunciation. I will demonstrate all of this by analyzing Resident Evil 7: Biohazard. 



application/pdf
2785-7433
15443
I
en
Claudio Paolucci
AN-ICON. Studies in Environmental Images [ISSN 2785-7433]
Resident Evil
Perception, hallucination, virtual reality. From controlled hallucination to Resident Evil 7: Biohazard
Text.Serial.Journal
https://riviste.unimi.it/index.php/anicon
1
Thematic Section
2022-05-16
2021-04-14
2022-05-16
2022-11-28
AN-ICON
Claudio Paolucci
2022/05/16
10.54103/ai/15443
University of Bologna 
2785-7433
I
Resident Evil
en
Perception, hallucination, virtual reality. From controlled hallucination to Resident Evil 7: Biohazard
https://riviste.unimi.it/index.php/anicon/article/download/15443/15711
1
              Perception,
              hallucination,
virtual reality. From controlled
hallucination to Resident Evil 7:
Biohazard Perception
by Claudio Paolucci
                      Imagination
                      Hallucination
                      Enunciation
                      Resident Evil




                                      Issue №1 Year 2022
                                      → Altered states
                                      Edited by Giancarlo Grossi
                                      and Andrea Pinotti
Perception, hallucination,
virtual reality. From controlled
hallucination to Resident Evil 7:
Biohazard
by Claudio Paolucci

Abstract                        In this paper, I will work on the relationship be-
                  tween perception and imagination in Virtual Reality, claiming
                  that “hallucination” is the ordinary motor also for online
                  perception and not only a deviant form of it. First, I will deal
                  with the problem of perception from the point of view of
                  cognitive semiotics and I will try to underline the crucial role
                  of imagination, claiming that perception is a form of “con-
                  trolled hallucination”. Later, I will focus on the relationship
                  between perception, hallucination, and memory in Virtual
                  Reality. On the one hand, I will claim that Virtual Reality
                  expresses the transition from actual perception to imagina-
                  tion, memory or dream through another actual perception.
                  On the other hand, I will claim that it can express it with-
                  out any problems through the old techniques coming from
                  cinema and other audiovisual languages, since they par-
                  tially share the very same formal apparatus of enunciation.
                  I will demonstrate all of this by analyzing Resident Evil 7:
                  Biohazard.




Perception Imagination                    Hallucination Enunciation Resident Evil

To quote this essay: C. Paolucci, “Perception, hallucination, virtual reality. From controlled hallucination
to Resident Evil 7: Biohazard”, AN-ICON. Studies in Environmental Images, no. 1 (2022): 112-128

CLAUDIO PAOLUCCI                                    112                                            AN-ICON
                                 Perception, imagination,
                                 and the control of the reality

                             First, I will deal with the problem of percep-
               tion from the point of view of cognitive semiotics.1 I will try
               to underline the crucial role of imagination, claiming that
               perception is a form of “controlled hallucination”,2 where,
               by “controlled hallucination”, I mean the product of the
               imagination controlled by the world. The main idea is that
              “hallucination” is the model of perception and not a devi-
               ant form of it. With “hallucination”, as defined in percep-
               tion studies and in the neurogeometry of vision,3 I mean
               the morphological activity of the production of forms by
               the imagination, which remains crucial both when it is not
               controlled by the world – as in the case of hallucination,
               imagination, or dream – and when it is controlled by the
               world, as in the case of online perception. In Virtual Reality,
               the world that controls perception is substituted by a tech-
               nology, a prosthesis capable of creating a strong effect of
               reality, a simulacrum with an effect of presence that no oth-
               er audiovisual has ever been able to build.4 In Augmented
               Reality, on the other side, the technology adds elements
               to the world that controls perception, without a full-blown
               substitution, as it happens with Virtual Reality.
                             Maybe, the word “hallucination” can be mis-
               leading, since perceptual phenomena under the aegis of
               hallucination may seem to lose the concreteness that I
               want to characterize them as having. It is possible that

      1 See C. Paolucci, Cognitive Semiotics. Integrating signs, minds, menaing and cognition
      (Berlin-New York: Springer, 2021).
      2 See A. Clark, Surfing Uncertainty: Prediction, Action, and the Embodied Mind (Oxford:
      Oxford University Press, 2016); J. Koenderink, “Vision and information”, in L. Albertazzi et al.,
      eds., Perception Beyond Inference: The Information Content of Visual Processes (Cambridge,
      MA: MIT Press, 2010): 27-57.
      3    See J. Koenderink, “Vision and information”; A. Sarti et al., “The symplectic structure of
      the primary visual cortex”, Biological Cybernetics 98 (2008): 33-48.
      4 See C. Paolucci, “Una percezione macchinica: realtà virtuale e realtà aumentata tra
      simulacri e protesi dell’enunciazione”, in F. Biggio et al., eds., Meaning–Making in Extended
      Reality. Senso e Virtualità (Rome: Aracne, I Saggi di Lexia, 2020): 43-62.

CLAUDIO PAOLUCCI                                113                                            AN-ICON
             “figuration” would fit better with the ideas I will develop here,
              since no Sartrean “derealization” is involved.5 However,
              since our brains try to guess what is out there, and to the
              extent that that guess matches the evolving sensory data,
              we perceive the world, in Surfing Uncertainty, Clark recalls
              the slogan coined by the vision scientist Ramesh Jain that
              perception is “controlled hallucination”. This is the direc-
              tion I am going to take. But, precisely because “this view
              of perception puts us in genuine cognitive contact with the
              salient aspects of our environment”, Clark suggests we
              consider hallucination as a form of “uncontrolled percep-
              tion”.6 However, Clark’s view – if it is put like that – is the
              classical one that thinks of perception as grounding both
              hallucination and imagination, which are supposed to be
             “deviant” or “uncontrolled” forms of perception. Since I want
              to claim the opposite, I will continue to use “hallucination”,
              and since there is a well-established tradition regarding this
              concept in the field of perception studies, I will do so with
              the caveat that “hallucination” does not imply any kind of
             “derealization” of perceptual phenomena from a phenom-
              enological point of view.7
                           Indeed, with “hallucination”, or “figuration”, I
              indicate a process of microgenesis8 that continuously pro-
              duces the next thread of perceptual experience while the
              current one fades, and does so without voluntary control.9
              I claim that meaning guides this microgenesis, or hallucina-
              tion, and that imagination is the engine of this microgenesis
              guided by meaning.



      5 See G. Matteucci, Estetica e natura umana. La mente estesa tra percezione, emozione ed
      espressione (Rome: Carocci, 2019).
      6 A. Clark, Surfing Uncertainty: 326. See also paragraph 6.10.
      7 See S. Gallagher, D. Zahavi, “Primal impression and enactive perception”, in V. Arstila, D.
      Lloyd, eds., Subjective Time: The Philosophy, Psychology, and Neuroscience of Temporality
      (Cambridge: MIT Press, 2014): 83-99.
      8 See J.W. Brown, Self–Embodying Mind: Process, Brain Dynamics and the Conscious
      Present (Barytown, NY: Barytown Ltd, 2020)
      9 See J. Koenderink, “Vision and information”.

CLAUDIO PAOLUCCI                               114                                         AN-ICON
                                                                            Fig. 1. Kanizsa’s triangle.

                             For instance, in Kanizsa’s triangle (Fig. 1), the
               triangle is totally a product of our hallucination, since it is
               the only thing that is not present in the stimuli. It is not by
               chance that, as Reddy and colleagues show, not only are
               online perception and imagination closely related in the
               brain, but perception, as it occurs in creatures like us, is
               co-emergent with imagination.10 What we perceive is literal-
               ly (not metaphorically) the future, not the present, because
               perception is the anticipation of the next thread of sensory
               information through previous knowledge. Indeed, in percep-
               tion we build through imagination the world that we expect.
                            There is a beautiful experiment by Adams and
               colleagues that shows that, in some circumstances, we
               hear the presence of the absence, that is, we hallucinate
               something that is not there, but we expect to be there.11
               When silence arrives, we literally do not hear it as we
               are supposed to do (i.e. as nothing playing). In its place,
               we hear the presence of the absent sound that we were

      10 L. Reddy et al., “Reading the mindʼs eye: decoding category information during mental
      imagery”, NeuroImage 50, no. 2, (2010): 818-825; G. Ganis et al. “Brain areas underlying visual
      mental imagery and visual perception: An fMRI study”, in Cognitive Brain Research 20, no. 2
      (2004): 226-241.
      11 R.A. Adams et al., “Predictions not commands: active inference in the motor system”,
      Brain Structure and Function 218, no. 3 (2013): 611-643.

CLAUDIO PAOLUCCI                               115                                          AN-ICON
              expecting. Adams and colleagues’ experiment runs as fol-
              lows. They used a simple computer simulation of birdsong.
              A multi-layer prediction machine processes sequences of
              simulated bird-chirps. The simulations were then repeated
              but omitting the last three chirps of the original signal. At
              the first missing chirp, the network responds with a strong
              precise moment where the first missing chirp should have
              occurred, the system generated a brief, transient illusory
              percept. This hallucinated percept was not strong, but the
              timing was correct with respect to the missing chirp. Thus,
              our perceptive system first dimly “perceived” (hallucinated)
              the missing chirp, before responding with a strong error
              signal when the actual absence of the anticipated sensory
              evidence became apparent. Of course, Kanizsa’s triangle
              is a visual correspondent of Adams and colleagues’ ex-
              periment. This is what I call perception as a “controlled
              hallucination”, which is the general functioning of our rich,
              world-revealing perception at any level, since it concerns
              an organism structurally coupled with its environment trying
              to minimize disorder and surprise.12

                                The Goethean account of perception

                          I refer here to what Jan Koenderink, a cognitive
             scientist and mathematician who works on the connection
             between theory of singularities and perception, used to call
             the “Marrian” and the “Goethean” accounts of perception.
      ■      According to the Marrian account: perception is the result
      of standard computations on optical data.
      ■      According to the Goethean account: perception is con-
      trolled hallucination, or “controlled figuration”.
                         The mainstream view in cognitive science and
             neuroscience, which is often also the commonsense view,

      12 S. Gallagher, Enactivist Interventions: Rethinking the Mind (Oxford: Oxford University
      Press, 2017).

CLAUDIO PAOLUCCI                               116                                         AN-ICON
               is that perception is all about a kind of passive imprinting
               of the world upon the sense organs and the brain. As Eg-
               ner and colleagues say, “on traditional accounts the visual
               system was seen as a passive analyzer of bottom-up sen-
               sory information”.13 This is a view of the perceiving brain
               as highly stimulus-driven, taking energetic inputs from the
               senses and turning them into a coherent percept by a kind
               of inwards flowing stream.
                             The Predictive Processing account of percep-
               tion takes a different direction and includes a top-down
               predictive aspect in its account.14 However, Predictive Pro-
               cessing thinks of perception as a kind of new schematism
               between aesthetics (the sensory data) and the concepts
               (the priors). According to this view, our brains are proac-
               tive: they are constantly buzzing as they try to predict the
               sensory signals arriving across all modalities. When such
               proactive brains “match” the incoming sensory signal, we
               perceive the world, understand it, and are immediately posi-
               tioned to imagine it so as to act in it too. However, “halluci-
               nation” is different from the mainstream notion of “prior”. A
               prior – as used in Bayesian inference – is a generic, usually
               statistical, property.15 For example,

 light comes from above is such a prior (if put in suitable format).
 It applies, on the average, for terrestrial animals that live in open
 spaces. Such priors package ‘frozen’ prior experience as it were.
‘Hallucinations’ differ by not being frozen, applying to the actual
 situation. Hallucinations can be regarded as specific, necessar-
 ily tentative, instantiations of the observer’s present “situational
 awareness” instead of its average past.16

       13 T. Egner et al., “Expectation and surprise determine neural population responses in the
       ventral visual stream”, Journal of Neuroscience 30, no. 49 (2010): 16601-16608.
       14 See A. Clark, Surfing Uncertainty.
       15 D. Purves, “Why we see things the way we do: evidence for a wholly empirical strategy
       of vision”, Philosophical Transactions of the Royal Society of London. Series B, Biological
       Sciences 356 (2001): 285-297.
       16 J. Koenderink, “Vision and information”: 32.

CLAUDIO PAOLUCCI                               117                                         AN-ICON
                            This is important, because we do not always
               update our perceptions according to our past experience
               and according to the changes in our priors. This is very
               well seen in the popular Muller-Layer illusion, where even
               when we learn that the two lines have the same length,
               we still keep on seeing them in the previous way. In per-
               ception, “data” are constructed through an attunement of
               the organism and the world, where the organism looks for
               elements that are worth for him to look for. For instance, as
               in Kanizsa’s triangle, we look for edges and we perceive
               edges even if edges are the only thing that are not present
               in the stimulus. This is because, given evolutionary pressure
               and experience driven plasticity,

we are wired to the environment in order to produce states that track
edges when exposed to discontinuities. The system is physically at-
tuned to such things, ‘set up to be set off’ by such visual discontinuities.17
                            Exploring the world, the organism casts his
               questions to the environment through imagination and pre-
               dicts its answers until he encounters resistance. This very
               action turns optical or sound structure into “data”, which
               are not sent from the world to the organism through senses
               but are the actual product of the autopoietic structure of
               the system of perception. Perception occurs when the top
               down activity of the imagination succeeds at generating
               the sensory data for itself, building a coherent story that
               paves the way for efficacious action. When it can gener-
               ate the future sensory data, the agent perceives the world.
               When it cannot, encountering a resistance, it tries a new
               attunement or changes the world through action. Therefore,
               data are built up because we produce them in looking for
               what we need for action, in order to minimize our work in
               the environment.

       17   S. Gallagher, Enactivist Interventions: 120.

CLAUDIO PAOLUCCI                                 118                             AN-ICON
                                Moving towards virtual
                                and augmented reality

                            The difference between the mainstream view,
              where data are sent by the environment and processed
              through perception, and the view where they are the
              product of what we look for in a coupled environment, can
              be operationalized as follows: perception is sensorily guid-
              ed potential behavior.18
                            Potential is key here. Perception is grounded
              on imagination, as it concerns the potential behavior con-
              nected to a coherent “story” we are building in order to act
              in the world and minimize disorder.19
                            Perception as sensorily guided potential be-
              havior is meant to reveal a world of salient, meaningful, in-
              teracting causes selected in the light of human needs and
              possibilities, which is exactly what pragmatists had in mind
              when they were telling us that the meaning of something
              consists in its conceivable practical bearings. This also
              marks the difference between pragmatism and behaviorism,
              since perception is neither action nor behavior. This prag-
              matist idea is consistent with the Affordance Competition
              Hypothesis originally introduced by Cisek,20 which is also
              a key hypothesis for the Predictive Processing by Andy
              Clark.21 The brain processes sensory information to specify,
              in parallel, several potential actions that are currently avail-
              able. These potential actions compete against each other




      18 J. Koenderink, “Vision and information”: 32.
      19 C. Paolucci, “Social cognition, mindreading and narratives. A cognitive semiotics
      perspective on narrative practices from early mindreading to autism spectrum disorders”,
      Phenomenology and the Cognitive Sciences 18, no. 2 (2019): 375-400; and Cognitive
      Semiotics. Integrating Signs, Minds, Meaning and Cognition (Berlin, Heidelberg, New York:
      Springer, 2021).
      20 P. Cisek, “Cortical mechanisms of action selection: The affordance competition
      hypothesis”, Philosophical Transactions of the Royal Society B 362 (2007): 1585-1599.
      21 A. Clark, Surfing Uncertainty: Chapter 6.

CLAUDIO PAOLUCCI                              119                                         AN-ICON
                for further processing, while information is collected to bias
                this competition until a single response is selected.22
                              If we go beyond Cisek’s terminology of “pro-
                cessing information”, like I think we have to do in order
                to focus on the main problem of perception as controlled
                hallucination, we see that, at the moment, we are see-
                ing a good deal of confirmation regarding this “pragma-
                tist” approach from the neurosciences.23 The classic dis-
                tinction between perception, cognition and action simply
                fails to reflect not only the phenomenology of our expe-
                rience, but also the global functional architecture of the
                brain, where, for instance, motor systems are active and
                play a huge part also in perception, decision making,
                social cognition and problem solving.24

Increasing and highly suggestive evidence challenges the view
of core cognitive capacities (such as planning and deciding) as
neurophysiologically distinct from the circuitry of sensorimotor con-
trol. For example, decisions concerning eye movements and the
execution of eye movements recruit highly overlapping circuits
in lateral intraparietal area (LIP), frontal eye fields (FEF), and the
superior colliculus [...]. In the same vein, a perceptual decision task
(one in which the decision is reported by an arm movement) re-
vealed marked responses within premotor cortex corresponding
to the process of deciding upon a response (Romo et al. 2004).
Quite generally, wherever a decision is to be reported by (or oth-
erwise invokes) some motor action, there looks to be an entwining
       22 P. Cisek, “Cortical mechanisms of action selection”: 1585.
       23 See C. Paolucci, “Per una concezione strutturale della cognizione: semiotica e scienze
       cognitive tra embodiment ed estensione della mente”, in M. Graziano, C. Luverà, eds.,
       Bioestetica, bioetica, biopolitica. I linguaggi delle scienze cognitive (Messina: Corisco Edizioni,
       2012): 245-276; V. Cuccio, F. Caruana, “Il corpo come icona. Abduzione, strumenti ed
       Embodied Simulation”, VS-Quaderni di Studi Semiotici 120 (2015): 93-103.
       24 See V. Gallese, “Mirror neurons and the neural exploitation hypothesis: from embodied
       simulation to social cognition”, in J.A. Pineda, ed., Mirror Neuron Systems (New York: Humana
       Press, 2009): 163-190; A.M. Borghi, F. Caruana, “Embodied Cognition, una nuova psicologia”,
       Giornale Italiano di Psicologia 35, no. 1 (2013): 23-48; V. Gallese et al., “A unifying view of the
       basis of social cognition”, Trends in Cognitive Sciences 8, no. 9 (2004): 396-403; G. Rizzolatti,
       C. Sinigaglia, “Mirror neurons and motor intentionality”, Functional Neurology 22, no. 4 (2007):
       205-221.

CLAUDIO PAOLUCCI                                  120                                           AN-ICON
of perceptuo-motor processing and decision-making, leading Cis-
ek and Kalaska to suggest that ‘decisions, at least those reported
through actions, are made within the same sensorimotor circuits
that are responsible for planning and executing the associated
action’ (Cisek and Kalaska 2011: 274). In cortical associative regions
such as posterior parietal cortex (PPC), Cisek and Kalaska go on
to argue, activity does not seem in any way to respect the tradi-
tional divisions between perception, cognition, and action. Instead
we find neuronal populations that trade in shifting and context-re-
sponsive combinations of perceiving, deciding, and acting, and in
which even single cells may participate in many such functions
(Andersen and Buneo 2003).25
                             If perception is supposed to work as a process
               that is continuous with action, the Marrian casual chain is
               inverted. Instead of “data” arriving at the eye, being pro-
               cessed, being further processed and finally resulting in a
              “representation” of the scene in front of us, the agent ex-
               plores the world in any conceivable direction, casting his
               questions and producing data in relation to what he needs
               for action, until it encounters resistance.
                             This is why imagination is the real engine for
               online perception. Since imagination is a faculty that allows
               us to move our consciousness from a proximal to a distal
               place, which can be in the past (memory), future (pros-
               pects) or in an invented reality, this “looking for the future”
               in online perception, trying to anticipate the next thread of
               the world, is grounded exactly on imagination.
                             This places Virtual Reality and Augmented Real-
               ity (VAR) in radical continuity with online perception. Indeed,
               if, in phenomenological terms, hallucination is a “percep-
               tion in the absence of the external object”, and if the ob-
               jects that appear in virtual reality are artificial simulacra of
               presence that we perceive without them being anchored in

       25   A. Clark, Surfing Uncertainty: 178.

CLAUDIO PAOLUCCI                                  121                    AN-ICON
              our experience of the physical world, we shall understand
              immersive virtual experience as a form of voluntary and
              deliberate hallucination. But this does not imply any kind of
              derealization: it simply implies the substitution of the control
              of the world with the control of a technology.
                           This is why VAR promises important scientific
              applications, which, in a few years, will radically change
              many of our laboratories of psychology, neuroscience and
              cognitive sciences. Indeed, on the one hand, we want a
              world in which reality is not so “real” as to modify the ex-
              perimental results and influence them, and VAR provides us
              with only a virtual reality, which can be controlled in detail
              and put into brackets at will. On the other hand, we also
              want a world that is not so unreal as to be indistinguish-
              able from the normal conditions of a laboratory, which at
              full capacity lives by cutting off every variable that cannot
              be controlled and ends up producing data that have the
              purpose of explaining our experience in a condition that
              we know to be a radical impoverishment of this very same
              experience. VARs allows us to create a very strong effect
              of presence within a world-environment,26 capable of sim-
              ulating a reality that remains only virtual and can therefore
              be controlled in its different parameters. At the same time,
              VAR allows us to increase the “gradient of reality” inside a
              laboratory, integrating those experiences and variables that
              a laboratory usually tends to cut away, in the name of the
              robustness of its measurements. Thus, the conditions of
              the laboratory are “augmented” with a reconstructed and
              simulated reality, which we can see and experience only
              thanks to the prostheses of VAR. In this respect, VAR is a
              prosthetic technology that is in its own way unprecedented,
              capable of generating a controllable world without losing at
              the same time the phenomenological richness of the world.


      26   A. Pinotti, “Self–negating images: towards an-iconology”, Proceedings 2017, I, 856 (2017).

CLAUDIO PAOLUCCI                               122                                          AN-ICON
                             Precisely because of its ability to make the
               world present through perception, without the world being
               in any way unamendable – since it can be controlled – it is
               completely normal that people who set up environments
               in VAR exploit this possibility, preferring online perception
               to other expressive possibilities and using online percep-
               tion in order to express dreams, hallucinations or memo-
               ries. However, this choice does not in any way reside in
               a technological, semiotic or enunciative limit of VAR and
               its language. On the contrary, it is only a “stylistic” choice,
               which can be suspended. As we will see, many of these
               suspensions, which result in the use of old cinematographic
               techniques within Virtual Reality, can be seen for example
               in an extraordinary game for Play Station VR such as Res-
               ident Evil 7: Biohazard.
                             Elsewehere,27 I tried to show how, from a semi-
               otic point of view,28 VAR works by holding together a formal
               apparatus of the prosthetic type of enunciation, typical
               of audiovisuals, and a formal apparatus of enunciation of
               the simulacral type, which is instead typical of other se-
               miotic systems, such as verbal language.29 If this is true,
               as I believe it is, Virtual Reality and Augmented Reality are
               not incapable of effectively representing the modifications
               of perception that cinema has always expressed through
               its representative strategies, such as dissolve, shift from

      27 C. Paolucci, “Una percezione macchinica”.
      28 Some semiotic approaches to VR can be found in G. Bettetini, L’Ulisse semiotico e le
      sirene informatiche (Milano: Bompiani, 2006); and B.R. Barricelli et al., “Semiotic Framework
      for Virtual Reality Usability and UX Evaluation: a Pilot Study”, in M. De Marsico et al., eds.,
      Games-Human Interaction (2018): 1-6. A nice take on the illusory cancellation of the distance
      that occurs during an immersive experience and the establishment of a critical distance in the
      user, linked to the emergence of a meta experiential competence, can be found in F. Biggio,
     “Semiotics of distances in virtual and augmented environments”, Img Journal 2, no. 3 (2020):
      82-103. Some important remarks on virtuality and subjectivity can be found in a nice book
      by Ruggero Eugeni, La condizione postmediale (Brescia: La scuola, 2015) and in the reader
      edited by F. Biggio et al., Meaning-Making in Extended Reality. Senso e Virtualità (Rome:
      Aracne, 2020). However, the most comprehensive and original semiotic work on this topic
      is the PhD thesis by Gianmarco Giuliana, Meaningfulness and Experience in Virtual Realities.
      Semiotics of a Digital Pla(y)typus, University of Turin (2021).
      29 See C. Paolucci, “Prothèses de la subjectivité. L’appareil formel de l’énonciation dans
      l’audiovisuel”, in M.G. Dondero et al., eds., Les plis du visuel. Réflexivité et énonciation dans
      l’image (Limoges: Éditions Lambert-Lucas, 2017): 53-68.

CLAUDIO PAOLUCCI                                123                                           AN-ICON
             color to black-and-white, flou or blur, used to indicate, in a
             point-of-view shot, the transition from actual perception to
             memory, dream, daydream, fantasy, or hallucination. VAR
             can use all these techniques without any problems at all,
             since it shares with cinema and audiovisual languages the
             very same formal apparatus of enunciation. We can have
             a dissolve or a shift from color to black-and-white also in
             VAR. If this is not done, or it is not done so often, it is only
             because of a choice from the “authors” or the VAR, who
             clearly opt for a “pan-perceptive way” for stylistic or “aes-
             thetical” reasons.
                           But this does not mean that VAR cannot ex-
             press non-perceptive conditions such as memory, dream,
             daydream, fantasy, or hallucination. On the contrary, it does
             that through perception, perfectly expressing the transi-
             tion from actual perception to memory, dream, daydream,
             fantasy, or hallucination in a “pan-perceptive” way. Indeed,
             Virtual Reality expresses the transition from the actual per-
             ception through the actual perception. And sometimes it in-
             corporates the old audiovisual techniques, embodying them
             inside this transition from perception through perception.

                         A conclusion in the form of a case study.
                         Resident Evil 7: Biohazard

                         As a case study, I will work here on Resident
             Evil 7: Biohazard, a survival horror game for PS4 devel-
             oped by Capcom. The game, which can be played through
             Play Station VR, is full of shifts from actual perception to
             memories, dreams, and hallucinations. All these shifts are
             expressed through pure perception, while incorporating
             from time to time some techniques originating from cinema
             or audiovisual languages.
                         Resident Evil 7: Biohazard is set somewhere in
             the southern United States, after the murder of a three-man

CLAUDIO PAOLUCCI                    124                              AN-ICON
              TV crew by the infected Baker family members Jack and Mia.
              Ethan Winters is the protagonist, who is searching for his
              missing wife, Mia, which leads him to a derelict plantation
              mansion, home of the Baker family. At the very beginning
              of the story, the player, in the role of Ethan, finds a video-
              tape containing a short video shot by the television crew
              and, in an adjacent room, he also finds a VCR and a TV
              where he can see it. Notice how the videotape is a “tech-
              nological quotation”: it is through an old audiovisual that
              we come to know what happened in the past,30 now that
              we are in VR. The player has some expectations at this
              point:31 we know that a TV crew has been there before
              us, because we have previously explored their abandoned
              van. However, the narration of this flashback, which also
              gives the player important information on the topography of
              the house, takes place by giving the player control of one
              or three crew members, not surprisingly the cameraman,
              who must film what happened previously and return it to
              the eyes of the player, who is controlling him through his
              joypad. Here the flashback and the memory, in a form that
              clearly identifies that they are flashbacks and memories,
              are obtained through the perception of a “machinic” eye
              that was there, watching for us. And who now identifies
              with us and our avatar.32
                           The very same thing happens when the player
              controls Mia Winters, in the central part of the plot. Mia
              hallucinates (we will understand later that she has been
              infected and her mind is controlled by Eveline) and sees
              a 10-year-old girl who tells her to watch another video-
              tape, “so they can be a family”. The videotape introduces

      30 According to Kirkland, in the Resident Evil saga “old media technologies contribute a
      sense of the real perceived as lacking in digital media, yet central to a generically-significant
      impression of embodiment”. See E. Kirkland, “Resident Evil’s typewriter: horror videogames
      and their media”, Games and Culture 4, no. 2 (2009).
      31 On the way the Resident Evil saga handles expectations of its players, see C. Reed,
     “Resident Evil’s rhetoric: the communication of corruption in survival horror video games”,
      Games and Culture 11, no. 6 (2016).
      32 https://www.youtube.com/watch?v=SBsOqYpx2ng&t=1550s

CLAUDIO PAOLUCCI                                 125                                            AN-ICON
             a memory through which we learn that Mia was a scientist,
             taking care of a bioweapon under the form of a little girl,
             but something went wrong. Once again, it is through actual
             perception that memories and hallucinations are performed.
             It is through perception that Virtual Reality expresses the
             transition from actual perception. Controlling Mia Winters,
             playing through her and perceiving through her eyes and
             ears, we come to know the truth about the story that we
             are playing as Ethan Winters, to the point that one of our
             aims is to send Ethan (ourselves) the very same message
             we receive when Resident Evil 7: Biohazard starts.33
                           However, the main moments connected to the
             relationship between perception, memories and halluci-
             nation in Virtual Reality are still to come in Resident Evil
             7. At a certain point of the plot, Ethan meets Jack Baker’s
             daughter, Zoe, who also wants to escape Baker’s house.
             Zoe tells Ethan that his wife Mia is still alive, even if Ethan
             (us) has already killed her, since Mia has been given the very
             same infection the Bakers have, and this infection gives
             her body powerful regenerative abilities and extreme mu-
             tations. Ethan is told that she and Mia would need to have
             their infections cured by a special serum first, before leav-
             ing Baker’s house. Ethan heads out to an old house to find
             the ingredients for the serum, where he is forced to battle
             and kill Marguerite Baker. Once he retrieves the ingredient,
             Ethan begins to have strange visions of a young girl. From
             this point on, hallucinations, memories and perceptions
             coexist and alternate in all the Resident Evil 7 gameplay
             and VAR has no problems at expressing their development
             throughout the whole story at all, using the very same tech-
             niques that audiovisual languages used to employ.
                           For instance, in the final boss fight, Ethan per-
             ceives the world through a grayish film and sees Eveline

      33 See a complete walkthrough of this part of the game here: https://www.youtube.com/
      watch?v=Gh3CkPI0UpA

CLAUDIO PAOLUCCI                            126                                      AN-ICON
              in her 10-year old girl form, that we know being nothing
              else but a hallucination, since we are told thanks to the
              Nexbas document found in the salt mines that

almost immediately after the infection, the subject begins to see
images of Eveline (though she is not in fact there) and even hear
her voice (which is inaudible to anyone else). Auditions with infected
subjects throughout the stages of infection reveal that at first, the
phantom Eveline appears to be a normal young girl, sometimes
desiring companionship or assistance.
                            However, after being able to approach her and
              inject the toxin that we have previously synthetized in the
              neck, we see an explosion of light flood the screen and
              then dissolve, indicating the transition from actual percep-
              tion, that is hallucination, to real online perception. Indeed,
              when dissolve and blur fade, the girl we are holding in our
              arms reveals herself to be an old monstrous lady whom we
              have already met (Baker’s “grandmother” in the wheelchair),
              that melts into the ground in a colorful scenario (not grey-
              ish as before), telling us “why does everybody hate me? I
              just wanted a family”.34 In her actual form she attacks the
              player, giving birth to the final boss fight.
                            Memories make no exception and are ex-
              pressed perfectly fine (like hallucinations are) in Virtual
              Reality. For instance, immediately before, the player is told
              the true story through the reliving of a painful memory by
              Mia while he is exploring the house. While we are perceiv-
              ing the actual scene in the house and after we have found
              a doll on the ground close to a wheelchair (foreshadowing
              Eveline’s true identity), we hear a buzz sound that indicates
              a clear discontinuity and Mia shows up in our perceptive
              field without interacting with us nor seeing that we are there.

       34 The whole sequence can be seen here: https://www.youtube.com/
       watch?v=Rs8bkVhDuA0

CLAUDIO PAOLUCCI                           127                            AN-ICON
              What we see is a memory by Mia, while she was taking care
              of Eveline, the E-Series Biohazard weapon (E-001) built by
              the company she was working for (a common appearance
              was selected for the bioweapons; that of a roughly 10-year-
              old girl, to ensure ease blending in with urban population). It
              is through these memories perceived through Virtual Reality
              that we come to know that Eveline infected Mia because
              she wanted Mia to be her mother. Immediately later, we are
              inside that memory, we can interact with it and we are part
              of its compound: indeed, Eveline asks us to be her father
              (“and if he does not want to be my father, he can die”) and
              we ask ourselves “why am I seeing this?”.35
                            As we see, VAR has no problems at all at in us-
              ing all the classic audiovisual techniques in order to express
              the transition from online perception to memory, dream or
              hallucination. It simply prefers doing that the majority of
              the time through its pan-perceptive model. However, not
              only can VAR utilise the old audiovisual techniques, but
              it looks like that also the old audiovisual languages used
              to express hallucinations, memories and dreams through
              perception: they simply did not give the observer the sen-
              sation of “being there”. How could cinema express memory,
              dream or hallucination through dissolve, blur or flou if not
              through perception? Since both cinema and VAR share
              the very same prosthetic structure of their formal appara-
              tus of enunciation, they both use a prosthesis (a screen, a
              mounted display etc.) in order to make us see things that
              we could not have seen without the text.36
                            In this way, from a semiotic point of view, VAR
              confirms its mixed nature, keeping together a simulacral
              and a prosthetic structure of its language.



      35 See the whole sequence at https://www.youtube.com/watch?v=p7TZk2iYM-k
      36 C. Paolucci, Persona. Soggettività nel linguaggio e semiotica dell’enunciazione (Milano:
      Bompiani, 2020): Chapter 6.

CLAUDIO PAOLUCCI                              128                                         AN-ICON
AN-ICON has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme. Grant agreement No. 834033 AN-ICON.
The project is hosted by the Department of Philosophy “Piero Martinetti” – Department of Excellence at the State University of Milan.



                                                                                           1 / I                                                                              AN-ICON
