10.54103/ai/19765
This paper focuses on the concept of acting with and in images in the context of recent AR and VR artworks. The thesis is that the (virtual) hand plays a significant role in an immersive experience. Referring to Doris Kolesch’s relational concept of immersion as one that considers not only the status of being evolved but also the process of getting inside, three forms of actions in and with images are discussed: the hand as stage, the hand as a symbiotic contact zone, and the hand as a designing hand. With artworks by Jeremy Bailey, Aristarkh Chernyshev, Rachel Rossin, and Florian Meisenberg, this contribution aims to contour the forms of action in which the (virtual) hand, in particular, allows an immersive experience by interaction with the virtual sphere and knows how to combine distance with nearness.
application/pdf
2785-7433
19765
II
en
Julia Reich
https://creativecommons.org/licenses/by/4.0
AN-ICON. Studies in Environmental Images [ISSN 2785-7433]
Contemporary art
What to do in /with images? The (virtual) hand in augmented and virtual reality.
Text.Serial.Journal
https://riviste.unimi.it/index.php/anicon
2
Thematic Section
2023-12-29
2023-01-26
2023-12-29
2024-02-15
AN-ICON
Julia Reich
2023/12/29
10.54103/ai/19765
Ruhr-University Bochum
2785-7433
II
Contemporary art
en
Voigt-Antons, J. N., et al., “Influence of Hand Tracking as a Way of Interaction in Virtual Reality on User Experience,” Twelfth International Conference on Quality of Multimedia Experience (2020): 1-4, https://doi.org/10.1109/QoMEX48832.2020.9123085.
What to do in /with images? The (virtual) hand in augmented and virtual reality.
https://riviste.unimi.it/index.php/anicon/article/download/19765/20014
2
                What to do in/with
                images? The (virtual)
hand in augmented
and virtual
by Julia Reich
              real   it y
        Immersive experience
                 Virtual reality
                 Augmented reality
                 Virtual hand
                 Contemporary art




                       Issue №2            Year 2023
                       → Immersions and Dives:
                       From the Environment
                       to Virtual Reality
                       Edited by Roberto P. Malaspina,
                       Elisabetta Modena, and Sofia Pirandello
What to do in /with images?
The (virtual) hand in augmented
and virtual reality.
JULIA REICH, Ruhr-Universität Bochum, https://orcid.org/0000-0003-3516-3558
https://doi.org/10.54103/ai/19765



Abstract                      This paper focuses on the concept of acting with
                 and in images in the context of recent AR and VR artworks.
                 The thesis is that the (virtual) hand plays a significant role
                 in an immersive experience. Referring to Doris Kolesch’s
                 relational concept of immersion as one that considers not
                 only the status of being evolved but also the process of
                 getting inside, three forms of actions in and with images
                 are discussed: the hand as stage, the hand as a symbio-
                 tic contact zone, and the hand as a designing hand. With
                 artworks by Jeremy Bailey, Aristarkh Chernyshev, Rachel
                 Rossin, and Florian Meisenberg, this contribution aims to
                 contour the forms of action in which the (virtual) hand, in
                 particular, allows an immersive experience by interaction
                 with the virtual sphere and knows how to combine distance
                 with nearness.




Keywords         Immersive experience Virtual reality                      Augmented reality
                 Virtual hand         Contemporary art
To quote this essay: J. Reich, “What to Do in/with Images? The (Virtual) Hand in Augmented and Virtual
Reality,” AN-ICON. Studies in Environmental Images [ISSN 2785-7433] 2, no. 2 (2023): 127-143,
https://doi.org/10.54103/ai/19765.

JULIA REICH                                      127                                        AN-ICON
                           “Viewing is already an activity, and distance from
               the work is its necessary condition. Immersion? Not neces-
               sary. We are already in the picture.”1 With these words, Pe-
               ter Geimer ends his polemical assessment of the immersion
               trend in contemporary art and in exhibitions. His critique,
               published almost five years ago, is mainly directed at the
               seemingly obstructive distance between the visitor and the
               artwork, as well as at the artistic and curatorial unreflected
               affirmation of an immersion-based paradigm of experience.
               What was considered hype at the time, however, as Oliver
               Grau’s art historical genealogy of virtual art impressively
               unfolds,2 turns out to be neither new nor based purely on
               media technology. Rather, the increased emergence from
               a temporal distance suggests itself as the advance of a
               second virtuality boom in the art and cultural landscape,
               which has become a matter of course today, and which
               was additionally fueled by the Covid-19 pandemic. While
               the pandemic caused the entire global society to practice
               social distancing, immersive technologies, such as virtual
               reality (VR) and augmented reality (AR), that challenge the
               polarity of close and far, not only became increasingly pre-
               valent in the field of art but also obtained an impact in our
               everyday lives. Whether in regard to medical applications
               or in the context of commercial instruments, these immer-
               sive scenarios permeate our life worlds.3 In contrast to
               the first wave of virtuality in the 1990s, whose discursive
               tenor tended to emphasize the otherworldly and specta-
               cular, AR and VR technologies have recently been used by
               artists to highlight the fragility and permeability between



       1 P. Geimer, “Kunst und Immersion: Der Trend zum Bildersturm,” FAZ (July 23, 2018), https://
       www.faz.net/aktuell/feuilleton/kunst-und-architektur/was-soll-der-trend-zur-immersion-in-der-
       kunst-15701142.html, accessed April 23, 2023 [my translation].
       2 The author begins his genealogical analysis of virtual reality in art with the example of the
       pre-Christian Pompeian Villa dei Misteri and includes analog as well as digital simulation
       spaces. Cfr. O. Grau, Virtual Art: From Illusion to Immersion (Cambridge MA: MIT Press, 2003).
       3 Cfr. S. Rieger, A. Schäfer, A. Tuschling, eds., Virtuelle Lebenswelten: Körper – Räume –
       Affekte (Bielefeld: Transcript, 2022). At this point, reference should also be made to the
       Collaborative Research Center 1567 Virtual Lifeworlds [Virtuelle Lebenswelten] at the Ruhr-
       University Bochum, which is dedicated to the interdisciplinary study of various manifestations
       of virtuality.

JULIA REICH                                     128                                          AN-ICON
              “art, mediating technologies, and daily life”4 through im-
               mersive experiences. According to Doris Kolesch, immer-
               sion is foremost an experience of a “threshold and transi-
               tion,” a “dynamic of oscillating between embeddedness
               and distance, of submersion and surfacing,” and less a
               complete absorption by the artwork.5 While AR immerses
               virtual objects in the physical environment, to which users
               can relate in the hybrid zone of the display, VR offers the
               possibility of entering a completely designed visual world,
               that is accessible via head-mounted displays (HMD). VR
               and AR, with their environmental images6 or hybrid image
               spaces, tend to locate themselves in zones of permeability
               and transience between the virtual and physical sphere.
                             Particularly in the context of virtual art forms, an
               artistic interest emerges in testing those threshold experien-
               ces and making them reflectible via a technically achieved
               nearness. And here the (virtual) hand, through its activity
               in immersive art forms, reduces the distance that makes
               a reflexive reception possible in the first place. Not only
               immersive-virtual works create a perceived loss of distan-
               ce, but also their viewers, who enter a relationship with
               and into images. Immersion is thus not only defined as a
               media-technical being enveloped but is also understood
               as a “relational concept”7 and, therefore, equally bound
               to oneself actively getting inside.8 As multifaceted as the
               concept of immersion is, it derives from the physical process,
       4 D. Kolesch, “Immersion and Spectatorship at the Interface of Theatre, Media Tech and
       Daily Life: An Introduction,” in D. Kolesch, T. Schütz, S. Nikoleit eds., Staging Spectators
       in Immersive Performances: Commit Yourself! (Oxon/New York: Routledge, 2019): 1-17, 9,
       https://doi.org/10.4324/9780429198274.
       5 The author distinguishes between two variants of immersive experience, mental-
       psychological and perceptual-psychic situatedness. While the former primarily addresses the
       cognitive level, where the recipient’s attention is directed, such as when reading a book, the
       latter promises a whole-body experience that involves the recipient as an active and essential
       entity. Contrary to the reproach of an unreflected appropriation, which Peter Geimer also
       addresses, Kolesch sees in immersive situations a potential of an “interruption of aesthetic
       illusion.” D. Kolesch, “Immersion and Spectatorship”: 8.
       6 Image worlds in VR in particular illustrate the tendency of the image to become an
       environment, as they are characterized by an unframedness, presentness and immediatness,
       and in this way, make their own image status precarious. A. Pinotti, “Towards An-Iconology:
       The Image as Environment,” Screen 61, no. 4 (2020): 594-603, 602, https://doi.org/10.1093/
       screen/hjaa060.
       7 D. Kolesch, “Immersion and Spectatorship”: 4.
       8 T. Hochscherf, H. Kjär, P. Rupert-Kruse, “Phänomene und Medien der Immersion,” in
       Jahrbuch immersiver Medien: Immersion: Abgrenzung, Annäherung, Erkundung (Kiel: Schüren,
       2011): 9-18, 14, http://dx.doi.org/10.25969/mediarep/18222.

JULIA REICH                                     129                                         AN-ICON
                which is first and foremost a directed movement that is fol-
                lowed by the topos of surroundedness.
                             And, as quite a few works of film and art hi-
                story demonstrate, the first – often exploratory and reco-
                gnizing – immersive movement into unknown terrain is led
                by the hand9. By contrast, the 21st century resembles an
                era of the hands’ oblivion [Handvergessenheit], as Jochen
                Hörisch notes.10 According to him, it is precisely cogni-
                tively abstract, immaterial processes and values that are
                displacing the dimensions of handiness and craftsmanship
                in the (post)digital age, even though they are based on
                them.11 Yet it is these aspects that seem to be emphasi-
                zed in AR artworks, when one’s own hand literally moves
                forward into the screen-viewed hybrid sphere, or when the
                hand in VR works takes on the function of a tool by means
                of hand tracking and starts to interact with virtually found
                objects. While so-called data gloves were already used in
                the early VR art of the 1990s to navigate from one space
                to another, the possibilities for action have multiplied con-
                siderably.12 If one considers immersion in this sense as a
                bodily movement that creates a simultaneity of being here
                and there, of which the recipients are quite aware, then the
                stretching forward and pulling back of the hand seems to
                be paradigmatic for a perception of difference, from which
                a self-reflexive quality can emerge.13
                             Based on this observation, this paper focuses
                on the significance of the (virtual) hand and its forms of
                action in AR and VR art. While the concept of image act(ion)
                [Bildhandlungen] is applied to different image types and

       9 In their introduction, Burcu Dogramaci and Fabienne Liptay discuss an immersion
       conceptualized particular in film and in the process name various film scenes in which the
       sense of sight is usually doubted and therefore a reassurance by hand takes place. For
       example, in the case of Neo, the Matrix protagonist, who recognizes his own reality as a
       dream by touching a billowing mirror. Cfr. F. Liptay, B. Dogramaci, eds., Immersion in the Visual
       Arts and Media (Leiden-Boston: Brill, 2016): 1-17.
       10 J. Hörisch, Hände: Eine Kulturgeschichte (München: Hanser, 2020): 22.
       11 Ibid.
       12 An early example in which data gloves were used to provide orientation and navigation
       in the virtual space with hand movements is Monika Fleischmann’s and Wolfgang Strauss’
       installation Home of the Brain (1989-1992).
       13 According to Doris Kolesch, immersion include not only the process of diving in, but also
       that one of surfacing, which in the case of the hand can be understood as a stretching forward
       and pulling back. D. Kolesch, “Immersion and spectatorship”: 9.

JULIA REICH                                      130                                          AN-ICON
                widely discussed in the discourse of image studies and
                art philosophy, it can only be meaningfully related to the
                interactive image, according to Silvia Seja. Because only
                the interactive image allows an action with things, ima-
                ges, spaces, and bodies that are merely virtually in the
                image and thus present and manageable.14 Referring to
                simulated scenarios, Inge Hinterwaldner points out that
                interactive images not only allow but also significantly
                shape and influence actions.15 Users both intervene in
                and are influenced by the iconic configuration, as it de-
                termines the way in which they can interact with it, as is
                the case in AR and VR artworks.16 Accordingly, iconicity
                and interactivity are reciprocal.17
                             Along with current works by Jeremy Bailey, Ari-
                starkh Chernyshev, Rachel Rossin, and Florian Meisenberg,
                this contribution aims to contour the forms of action in
                which the (virtual) hand, in particular, allows an immersive
                experience in interaction with the virtual sphere and knows
                how to combine distance with nearness. In this context, the
                actions in and with images are further developed on the
                basis of three perspectives: the hand as a stage, the hand
                as a symbiotic contact zone, and the hand as a designing
                hand. Prior to examining these artistic works in detail, it
                may be useful to first explain the technological background
                and development, determining the importance of the hand
                in relation to virtual sceneries.

                                  The (virtual) hand

                            The constant progress of media technology de-
                velopments in the field of hand recognition seems to be
                something of a paradox when one considers the hands’
                oblivion in the 21st century, as identified by Hörisch. In

       14 S. Seja, “Der Handlungsbegriff in der Bild- und Kunstphilosophie,” in I. Reichle, S. Siegel, A.
       Spelten, eds. Verwandte Bilder: Die Fragen der Bildwissenschaft (Kadmos: Berlin, 2007): 97-112, 111.
       15 I. Hinterwaldner, The Systemic Image: A New Theory of Interactive Real-Time Simulations
       (2010), trans. E. Tucker (Cambridge/London: MIT Press, 2017): 229.
       16 Ibid. The author emphasizes two qualities of the systematic image focusing on interaction:
       the presentational and the operative aspect, and therefore, highlights the image as its own
       interface.
       17 Ibid., especially the chapter “Iconicity and Interactivity”: 215-271.

JULIA REICH                                       131                                            AN-ICON
               February 2023, Mark Zuckerberg published a short demo
               video of the now-available Direct Touch feature for the VR
               headsets Meta Quest 2 and Quest Pro. The feature pro-
               mises a more intuitive operation in the VR and mixed rea-
               lity view,18 via gesture control, manual scrolling, and tap-
               ping, for example, in a superimposed browser page or a
               basketball game. Thereby, the user’s hands are tracked with
               external headset cameras and appear in the user’s view as
               grayish virtual hands. What can be traced in this current
               example is the technological genesis of the (virtual) hand,
               which seems far from complete. After the first prototypes
               in the 1970s, the first commercial data glove developed by
               VPL Research was launched in 1987 and already featured
               gesture recognition and tactile feedback.19 In addition to
               the further development of wearables and external periphe-
               rals, such as handheld controllers, vision-based tracking
               experiments with gesture recognition started parallel in the
               1980s.20 Dependence on previously complex calibrations
               and external power sources was no longer necessary with
               the 2013 launch of Leap Motion Technology. Although now
               taken for granted, for example, in VR gaming, it marked
               an important step towards free-hand interaction with the
               desktop screen and later within a VR environment. With this
               technology, small infrared sensors and cameras track the
               hands motions and visualize them in VR or desktop view.
               While Leap Motion Tracking is now mostly implemented
               in VR headsets, there have also been efforts to combine
               this with AR applications on private devices to provide
               more natural interaction with mobile AR objects.21 In the
               early years, AR interaction was mainly based on physical
               objects with markers. More recent applications, however,

       18 In this context, the mixed reality mode is understood as an interweaving of real
       environment and virtual elements, which clearly comes close to the passthrough mode
       mentioned later, but also makes the separation to AR questionable. For this aspect, Cfr. A.
       Urban, J. Reich, M. van der Veen, “Passthrough: Von Portalen, Durchblicken und Übergängen
       zwischen den (virtuellen) Welten,” Kunstforum International 290 (2023): 86-95.
       19 Cfr. P. Premaratne, Human Computer Interaction Using Hand Gestures (Singapur: Springer,
       2014): 5-12.
       20 Ibid: 12f.
       21 Cfr. M. Kim, J. Y. Lee, “Touch and Hand Gesture-based Interactions for Directly
       Manipulating 3D Virtual Objects in Mobile Augmented Reality,” Multimed Tools Appl 75 (2016):
       16529-16550, https://doi.org/10.1007/s11042-016-3355-9

JULIA REICH                                    132                                        AN-ICON
               increasingly use hand tracking. What this very brief outline
               illustrates is the increasing desire for the most device-free
               and intuitive handling possible in and with virtual environ-
               ments, which is reflected in these technological innova-
               tions. Even if hand tracking is not an essential feature for
               VR applications, and 360° VR films, for example, usually
               manage without it, numerous research studies point to an
               increased sense of immersion and presence in the virtual
               environment with visualized hands.22 While technological
               advancements have made it possible to simulate manual
               activity and seem to have brought the hand out of oblivion,
               the hand movements required in virtual environments are
               often different from those needed in daily life. For instan-
               ce, simply flicking your index finger won’t be enough to
               put a basketball into a basket, as Digital Touch simulates.
               Thus, while these endeavors may bring the hand out of
               oblivion, they still fall short of replicating true-to-life expe-
               riences. Rather, the hand seems to adapt to the existing
               motion patterns of the virtual hand. As will become clear
               in the following, the desire to hold one’s own hand in the
               virtual world does not first arise from hand tracking but
               starts with image configurations that presuppose much
               less interaction.

                                 The hand as stage

                            In the context of the AR Biennal (Aug. 22nd,
               2021-Apr. 24th, 2022), initiated by the NRW Forum, visitors
               were able to explore and marvel at AR sculptures in the
               public spaces of Düsseldorf, Cologne, and Essen using a
               specially developed app on their devices. Regular strollers
               in the Düsseldorf Hofgarten became accustomed to people
               performing crazy movements with their smartphones held

       22 Cfr. G. Buckingham, “Hand Tracking for Immersive Virtual Reality: Opportunities and
       Challenges,” Frontiers in Virtual Real 2 (2021), https://doi.org/10.3389/frvir.2021.728461; J.
       N. Voigt-Antons et al., “Influence of Hand Tracking as a Way of Interaction in Virtual Reality
       on User Experience,” Twelfth International Conference on Quality of Multimedia Experience
       (2020): 1-4, https://doi.org/10.1109/QoMEX48832.2020.9123085. For the complex discussion
       of immersion and presence sensations in virtual space, cfr. M. I. Berkman, E. Akan, “Presence
       and Immersion in Virtual Reality,” in N. Lee, ed., Encyclopedia of Computer Graphics and
       Games (Cham: Springer, 2019): 1-10, https://doi.org/10.1007/978-3-319-08234-9_162-1.

JULIA REICH                                     133                                         AN-ICON
               high or staging themselves for a photo as if they were inte-
               racting with invisible objects. In short, these were physical
               actions that addressed not so much the real-life environ-
               ment but rather the hybrid space of the display view. These
               aimed to enter the picture by anticipating the hybrid zone
               of the AR work, as is the case with Jeremy Bailey’s oversi-
               zed steel bean YOUar, stainless steel ellipsoidal arc (2021)
               (Fig.1). As expected, AR works tempt us to document that
               individual moment of hybrid interaction in the image spa-
               ce via screenshot, simply because of their genuine form
               of appearance in the display. This need, trained by social
               networks, is additionally motivated by the app’s own recor-
               ding function, which enables uncomplicated, one-handed
               screen recording while the other hand can enter into a rela-
               tionship with the virtual objects.23 Like illusionistic vacation
               snapshots, where different distance ratios enable one’s
               own fingers to hold, for example, the top of the Eiffel Tower,
               there are numerous screenshots from the AR Biennal in
               which the palm acts as a stage for the augmented objects.
                            Unlike in photography, here the hand becomes
               the ground for the figure, making it part of the environment




                                                             Fig. 1. Jeremy Bailey, YOUar, stainless
                                                             steel ellipsoidal arc, 2021, Augmented
                                                             Reality App, at Düsseldorf AR-Biennal,
                                                         2021, photograph by Katja Illner, courtesy
                                                          of the Artist and NRW-Forum Düsseldorf.


               or replacing it in interaction. Therefore, these image actions
               can be described as an anticipation of one’s own bodily pla-
               cement in the image and result in pictorial relations between

       23 With common devices, a screen shot requires two hands, since two remotely located keys
       have to be pressed simultaneously.

JULIA REICH                                  134                                         AN-ICON
               body, virtual object, and space. In doing so, they stand out
               as self-evident, playful explorations of a boundary sphere
               and thus emphasize the close connection between a sense
               of immersion and the user’s movements.
                             Symptomatic of these not directly intended
               image actions, our hands prove to be a central interface
               to the (physical and virtual) world in the digital age as well,
               in which manual grasping is still intertwined with cognitive
               comprehension.24 While AR figures can generally be pla-
               ced anywhere, the palm of the hand seems to offer itself
               as a particularly appealing stage. Surreal proportions are
               emphasized in the image; a physical nearness to the virtual
               figure is suggested; and one’s own body, moving forward
               into the virtual sphere, is immersed in it. Conversely, the
               hand has no influence on the movements of the figure and
               cannot change anything in the AR, but rather adapts to it
               and thus, as a stage-like presentation site, resets itself in
               its actual ability to act and create.
                             While these movements are individual amuse-
               ments of the users, in Jeremy Bailey’s video work Nail Art
               Museum (2014) (Fig. 2) the hand becomes a very concrete
               stage of an AR exhibition. In the exaggerated manner of a
               DIY YouTube video, Bailey, who remains anonymous, notes
               a renewed shift in the artistic paradigm of creation. If artists
               moved into organizational-curatorial roles as early as the
               1960s, the turn away from manual creation seems to have
               been amplified by the digital and transformed into the crea-
               tion of entire worlds. Bailey’s proposal is an AR application
               that allows everyone to independently curate exhibitions,
               appropriate existing works, and literally present them on
               their own fingertips. Through AR, company logos, palm
               trees, and iconic artworks of every era – from the ancient
               Venus de Milo to Ai Weiwei’s Neolithic vases to Jeff Koon’s
               Balloon Dog – can be assembled on one’s own hand. The
               artworks, themselves in thrall to a consumer culture, are
               perched on finger-bound museum pedestals. In the role of
               his extravagant alter ego – the self-proclaimed “famous new

       24 J. Hörisch, Hände: 18.

JULIA REICH                           135                              AN-ICON
              media artist,” – Bailey satirizes the self-staging practices of
              social media, addressing museums and artists who in turn
              use these networks as exhibition spaces. In doing so, he
              touches upon the sensitive tension between the topos of
              a democratization of art via its mediatized (over)availability
              and the question of artistic-creative innovation in times of
              its medial (re)producibility. Bailey further exacerbates this
              relationship with the aforementioned oversized AR mir-
              ror bean: its unmistakable model is Anish Kapoor’s steel
              sculpture Cloud Gate (2004-2006), with which countless
              tourists pose daily for the perfect snapshot. Its social me-
              dia usability continues to be effective in Bailey’s AR and is
              even facilitated since the sculpture can even be placed on
              one’s own hand with a click.




                                                                  Fig. 2. Jeremy Bailey,
                                                      Nail Art Museum, 2014, video
                                                      Performance and Augmented
                                                    Reality, still from video, courtesy
                                                                            of the Artist.




                          The hand as symbiotic contact zone

                          In contrast to the preceding image acts, in whi-
              ch the hand becomes dissimilar to itself because it fun-
              ctions more as a stage or exhibition space, the AR appli-
              cation Personal Information Organism. PiO 1.1. (2019) by
              Aristarkh Chernyshev, and Rachel Rossin’s mixed reality
              theater The Maw Of (2022) focus on the hand in its physical
              genuineness, namely as a contact zone between humans
              and technology. At the interface between science fiction,
JULIA REICH                          136                                      AN-ICON
               biotechnology, and speculative art, both works allude to
               so-called anthropophilic media25 for which their unobtru-
               siveness and cuddliness toward the body and its every-
               day routines are particularly characteristic. Affect-sensitive
               wearables, such as smartwatches, are examples of this.
               These rely less on the user’s activity at the interface but
               rather measure, collect, and utilize personal body data and
               mental states in the mode of passivity, such as the oxygen
               saturation in the blood or an incipient feeling of frustration.26
                            With the AR PiO 1.1. (Fig. 3), which can be
               accessed via QR code on social channels of Instagram or
               Snapchat, Chernyshev imagines a digital hybrid organism
               consisting of a genetically modified leech and a smartpho-
               ne whose natural habitat is the human body. The creature,
               which nestles tenderly around the wrist, lives on the blood
               of its user but, in return, takes care of his or her health. It
               does this by continuously monitoring the user’s body, even
               releasing insulin in the case of a rise in blood sugar. It also
               proves to be a practical tool for Zoom conferencing. While
               such symbioses are still speculative, the direct link between
               our brain and the machine has recently become real. The
               controversial media mogul Elon Musk and his neurotech
               company, for example, announced recently that they would
               be conducting clinical studies on humans with so-called
               brain-machine interfaces.27 The fact that Chernyshev’s PiO
               1.1. so far only gets under the skin in its conception allows
               users to experience a futuristic interaction with a wearable
               assistance creature that intuitively adapts to the movement
               of one’s own hand. When used, the wrist becomes the
               contact zone of an imagined symbiosis, transforming at
               the same time into a control surface with various display

       25 Cfr. M. Andreas, D. Kasprowicz, S. Rieger, eds., Unterwachen und Schlafen: Anthropophile
       Medien nach dem Interface (Lüneburg: Meson Press, 2018).
       26 For Michael Andreas, Dawid Kasprowicz and Stefan Rieger, this mode of passivity is a
       central marker for the definition of “antropophilic media,” which, in contrast to actively used
       tools, provoke a reduction of distance, since they operate in a new physical, social, and
       semantic nearness. Of particular interest is the underlying thesis of a shift from technical-
       medial surveillance to a surveillance that increasingly eludes perception as such and outwits
       the users. Cfr. M. Andreas, Unterwachen: 19.
       27 R. Levy, “Elon Musk Expects Neuralink’s Brain Chip to Begin Human Trials in 6 Months,”
       Reuters (December 1, 2022), https://www.reuters.com/technology/elon-musk-says-expects-
       neuralink-begin-human-trials-six-months-2022-12-01/, accessed May 4, 2023.

JULIA REICH                                     137                                          AN-ICON
                    Fig. 3. Aristarkh Chernyshev, Personal
                     Information Organism. PiO 1.1., 2019,
                       screenshot from Augmented Reality
                                 App, courtesy of the Artist.




                         Fig. 4. Rachel Rossin, The Maw of,
                       2022, view of the exhibition “KW on
                      location: Rachel Rossin The Maw of”
                        at Tieranatomisches Theater Berlin,
                     2022, photograph by Frank Sperling,
                    courtesy of the Artist and KW Institute
                                      for Contemporary Art.

JULIA REICH   138                                 AN-ICON
              modalities and thus suggesting self-control. This collides
              with the underlying consent of permanent information uti-
              lization and must, therefore, be a hollow promise. The qui-
              ckly transferred consent of a foreign control in assistance
              systems such as Alexa is closely related to the design of
              socially compatible counterparts, which is why PiO’s 1.1.
              animal-like movements also generate familiarity but thus
              prompt a self-reflection of the quickly conceded acceptan-
              ce in dealing with the artwork.
                             Rachel Rossin’s multimedia setting The Maw
              Of (Fig. 4) not only combines various media formats (instal-
              lation, VR and AR, video, and net art) that blur the boun-
              daries between the virtual and physical worlds as well as
              technological and organic systems, but also the bodies that
              inhabit them. Rossin’s work is decidedly based on recent
              research experiments that fuse body, mind, and techno-
              logy. These experiments are no longer about developing
              prosthetic extensions of the human body but rather about
              an invasive fusion of hardware and the nervous system,
              by means of which our thought center can act beyond the
              body. The central storyline is a narrative interwoven through
              the media formats and accompanied by a manga figure,
              in which the visitors themselves are conceived, as agents
              of a larger techno-organic network. They follow the figure
              as a machine spirit through a widely ramified network that
              embodies the human nervous system. When visiting the
              work at the Tieranatomisches Theater in Berlin (Sept.14th–
              Oct. 22nd 2022), the application on an HMD enabled a view
              (Fig. 5) into the symbiotic sphere Rossin transmedially desi-
              gned. In the midst of a lush grassy hill environment, which
              is revealed by a superimposed progression diagram as
              genuinely calculated and instantaneously processed, two
              bluish transparent hands appear. They directly implement
              the hand’s own movements and gestures in the virtual en-
              vironment by means of Leap Motion. The media-reflexive
              and at the same time instructive text field, “you are looking
              for your hands,” brings one’s own hands into the field of
              vision. Since they appear uniquely in both spheres, they
              are a contact zone: in the palms of the hands, text codes
JULIA REICH                         139                            AN-ICON
                       Fig. 5. Rachel Rossin, The
                    Maw of, 2022, screenshot from
                                HMD-Experience,
                             courtesy of the Artist.




                       Fig. 6. Rachel Rossin, The
                    Maw of, 2022, screenshot from
                                HMD-Experience,
                             courtesy of the Artist.

JULIA REICH   140                          AN-ICON
              and symbols alternate with wondrous beings, such as a
              blazing flame spirit or a human-shaped nervous system.
              This idiosyncratic interplay continues even as the virtual
              environment recedes and users find themselves in their
              physical, but colorless, world with the same virtual figures
              (Fig. 6). In this superimposed mode of perception, also
              called “pass-through,” the user’s own hands intersect with
              the represented hands, blurring the boundaries between an
              action in physical and virtual space – for example, when a
              virtual moth settled on the hand can be shaken off – and
              paradoxically leading to a disembodiment as well as a sen-
              sitization of the user’s own corporeality.

                          The designing hand

                            While the focus so far has been on the active
              hand, which has been integrated into the art works as a
              stage or contact zone, the perspective of the designing
              hand will be examined in conclusion using the example of
              Florian Meisenberg’s VR installation Pre-Alpha Courtyard
              Games (raindrops on my cheek) (2017). As a collaborative
              project between Meisenberg and programmer Jan Ahrens,
              Pre-Alpha connects installable, sculptural, and painterly
              elements with VR, video, and design processes. In the
              exhibition, visitors are greeted by a carpet drawn up in
              the manner of an infinity cove used for photography. On
              its left side, a vertical second-screen projection gives out-
              siders a glimpse into the intimate VR sphere. By putting
              on the HMD in the midst of the virtual environment with
              its rudimentary cosmic world reference, users can model
              their own virtual objects with pattern-like hand represen-
              tations, almost like God-like creators (Fig 7). For this pur-
              pose, a grid shape shoots up from the underground onto
              the image surface, which goes back to the basic geometric
              shapes of 3D programs, so-called graphic primitives, with
              which illusionistic VR worlds are “built.” Even though the
              hands do not feel any resistance in reality, the shape can
              be bent and distorted in all directions by lightly touching it
              in accordance with physical laws, thus referring to artistic
JULIA REICH                          141                            AN-ICON
              modeling processes. In the next step, the naked grid can
              be clothed with texturing material. This derives from the
              artist’s own image archive, from which individual images
              with a textile texture randomly rise up, fluttering in front of
              the user’s hands. In addition to Meisenberg’s physically
              existent paintings, this archive contains all kinds of image
              material – from antique portrait busts to net-genuine me-
              mes to online head texture maps – that are made available
              to the users for designing the grid surface. Quite literally,
              an action with images is invoked in this way. The specific
              gesture of two palms raised in front of the HMD causes
              the appearance of those double-sided images that can be




                                                 Fig. 7. Florian Meisenberg, Pre-Alpha
                                                   Courtyard Games (raindrops on my
                                                  cheek), 2017, screenshot from HMD-
                                                     Experience, courtesy of the Artist.


              manually applied to the grid shape.
                           While such creation processes delegated to
              museum visitors may have a special visual value for visi-
              tors outside the VR, this process is withdrawn in Pre-Alpha.
              While the second screen usually provides a voyeuristic live
              insight into the processes within VR, in this case it merely
              shows the pantomime-like hand movements of the immer-
              sed user around an empty center. The VR-internally desi-
              gned virtual object remains intimate, eludes a view, and
              meanwhile shifts the focus to the manual performance of
              the (non-) creating hands of the immersed user. In this way,
              the user on the stage-like carpet becomes an exposed



JULIA REICH                          142                                     AN-ICON
              performer and twists the exhibition logic inherent in the
              exhibition space.
                           Along the provisional spectrum of the three for-
              ms of action in and with images in AR and VR artworks pre-
              sented here, the aim was to clarify the extent to which a loss
              of distance achieved by hand does not primarily subscribe
              to a technological euphoria or an affirmative immersive
              experience, but rather offers the recipient the opportunity
              for reflection in the sounding out of those border zones
              between the physical and virtual spheres, one’s own body
              and other bodies. With the focus on the hand, it becomes
              apparent to what extent immersion, in the sense of getting
              inside a direct contact or a design, grants the potential
              of becoming aware of and critically sensitizing oneself to
              those technologies that permeate our lives. Immersion as
              a productive extension does not exclude emergence – as
              exemplified by the hand. For whoever puts on the VR gog-
              gles must also take them off again, willy-nilly.




JULIA REICH                          143                            AN-ICON
AN-ICON has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme. Grant agreement No. 834033 AN-ICON.
The project is hosted by the Department of Philosophy “Piero Martinetti” – Department of Excellence at the State University of Milan.



                                                                                           1 / I                                                                              AN-ICON
