This article offers a close reading and a critique of Hasson et al.’s Neurocinematics, focusing on its treatment of the notion of control, meaning a predictable neural and cognitive activation triggered by film stimuli. In the first part of the article I suggest that the use of control in neurocinematics on the one hand relies on a similarly problematic – but still more nuanced – use of the notion in cognitive film theory, and on the other hand reflects a unidirectional model of communication which brackets out noisy cases that diverge from predictable behavior. In the second part, I argue that these “noisy” cases are exactly the ones that pertain the most to a complex and dynamic view of brain activity and film-mind communication. The dialogue between film studies and neuroscience can become more complex too, escaping from a problematic definition of film effectiveness with regards to predictable viewer reactions.
application/pdf
2036-461X
16337
22-23
en
https://creativecommons.org/licenses/by/4.0
Cinéma & Cie. Film and Media Studies Journal
Neurocinematics and the Discourse of Control: Towards a Critical Neurofilmology
Text.Serial.Journal
https://riviste.unimi.it/index.php/cinemaetcie
14
Thematic issue / Section thématique
2014-10-01
2021-09-10
2014-10-01
2024-02-14
Cinéma & Cie
Maria Poulaki
2014/10/01
University of Surrey
2036-461X
22-23
en
Neurocinematics and the Discourse of Control: Towards a Critical Neurofilmology
https://riviste.unimi.it/index.php/cinemaetcie/article/download/16337/17018
14
Neurocinematics and the Discourse of Control:
Towards a Critical Neurofilmology
Maria Poulaki, University of Surrey




Abstract

   This article offers a close reading and a critique of Hasson et al.’s Neurocinematics,
   focusing on its treatment of the notion of control, meaning a predictable neural
   and cognitive activation triggered by film stimuli. In the first part of the article I
   suggest that the use of control in neurocinematics on the one hand relies on a
   similarly problematic – but still more nuanced – use of the notion in cognitive
   film theory, and on the other hand reflects a unidirectional model of communica-
   tion which brackets out noisy cases that diverge from predictable behavior. In
   the second part, I argue that these “noisy” cases are exactly the ones that pertain
   the most to a complex and dynamic view of brain activity and film-mind com-
   munication. The dialogue between film studies and neuroscience can become
   more complex too, escaping from a problematic definition of film effectiveness
   with regards to predictable viewer reactions.


   Neurocinematics, a term proposed by a research group in the Psychology De-
partment of New York University to synthesize cognitive neuroscience and film
studies, is the study of films through the use of functional magnetic resonance
imaging during film watching under experimental conditions. The fMRI tech-
nology produces a time-series of 3D images (very much like a digital “film”) pic-
turing brain activity in specific brain regions of the subjects/spectators. Higher
neural activity in particular brain regions, manifest from the increased blood
flow, results in a change in the image intensity of the fMRI. Since film viewing
is hard to control under experimental conditions due to the complexity of film
stimuli, which approximate natural vision, neurocinematics researchers have ap-
plied the method of inter-subject correlation (ISC). ISC helps to assess the simi-
larities/differences in brain activity across viewers, looking at common patterns
of response time courses in different brain regions.




                                           Cinéma & Cie, vol. XIV, no. 22/23, Spring/Fall 2014
                                          Maria Poulaki


Neurocinematics, control and attention

   In 2008 a research group led by psychologist Uri Hasson published a sum-
mary of neurocognitive cinema research so far, led by Hasson and colleagues
in the journal Projections. The article was written in an accessible way to appeal
to humanities/film scholars as well as to the wider public.1 In this article the
authors make the assumption that some films can control the viewers’ neural
responses, in the sense that “the sequence of neural states evoked by the movie is
reliable and predictable.”2 This “brain control” that some movies can effectuate
is also, according to the authors, a form of “mind control,” since neuroscience
presupposes that there is a direct link between neural states and mental states
(defined as “percepts, emotions, thoughts, attitudes, etc.”).3 In the same paper
Hasson et al. frequently talk about film “effectiveness,” relating this property
with increased ISC response to a film and therefore increased control of neural
and mental states across viewers. The logical conclusion drawn from the above
premises is that the most effective film is the one with the strongest control over
the mind of the viewer. A number of experiments they conducted within the last
decade permitted Hasson et al. to comprise a tentative “hierarchy of effective-
ness” among the films they used as stimuli. Directorial style seems to be for them
the most important factor contributing to mind control, since Alfred Hitchcock,
for instance, brought the highest case of ISC with his Alfred Hitchcock Presents
TV series episode Bang! You’re Dead (1961). At the bottom of the hierarchy,
the least effective testing material proved to be an “unstructured segment of
reality,”4 i.e. raw footage from a camera placed at a random spot of a public
space, capturing in a static frame random occurrences in front of its lens.
   The emerging field of neurocinematics tends to connect in a causal relation-
ship brain activation with the filmmakers’ skills of directing viewers’ attention,
which results in control of their minds, evident in the orchestrated activation of a
number of different brains as shown in fMRI scans. Informed by older and more
recent debates in film studies, the authors of the Neurocinematics article, mainly
comprised by psychologists and neuroscientists, joined by one Cinema Studies
researcher, Ohad Landesman, show a special interest in cognitive film theory, and
draw inspiration from some of its own assumptions – especially those of scholars
David Bordwell and Kristin Thompson, Noël Carroll, and Ira Konigsberg,5 who
have written on techniques of directing and guiding attention in films.


1
  Uri Hasson, Ohad Landesman, Barbara Knappmeyer, Ignacio Vallines, Nava Rubin, David J.
Heeger, “Neurocinematics: The Neuroscience of Film,” in Projections. The Journal for Movies and
Mind, no. 1, 2008, pp. 1-26.
2
  Ibidem.
3
  Ibidem.
4
  Ibidem.
5
  Among the sources Hasson et al. cite are: David Bordwell, Narration in the Fiction Film, The Uni-
versity of Wisconsin Press, Madison 1985; David Bordwell, Noël Carroll, Post-Theory: Reconstructing
Film Studies, The University of Wisconsin Press, Madison 1996; David Bordwell, Kristin Thompson,

40
                          Neurocinematics and the Discourse of Control


   Hasson et al. equate what cognitive film theory describes as control of atten-
tion with their neurological view of “mind control,” and assert that the mesmer-
izing power of movies lies in their ability to take control of viewers’ minds. More-
over, they point out that “viewers often seek and enjoy such control because it
allows them to become deeply absorbed (and mentally engaged) in the movie.”6
To support this claim the authors quote Konigsberg: “Part of the pleasure of
viewing a film is having our attention guided in an immediate and controlled
manner, seeming to have the camera do the looking for us.”7
   Here Konigsberg refers to something more than attention: to the taking over
of purposeful intentional perception by another agency – and this is an issue that
other strands of film theory have tackled before cognitivism. However, neurocin-
ematics does not at all thematize this dimension of film control. Rather, Hasson
et al. point out two characteristics of films that evoke controlled responses.
   On the one hand, it is of course attention, measured by tracking the viewers’
eye movements (in both silent and sound films), that allows researchers to follow
what exactly the subjects see and in which part of the frame they focus each time.
Substantial work on eye tracking as a vector of attention to audiovisual stimuli
has also been done by psychologist Tim Smith.8 Smith has shown how film direc-
tors and editors, intuitively taking advantage of certain “flaws” of natural percep-
tion, such as change blindness, use editing in such a way so as to aid the specta-
tor’s construction of a fluent and believable diegetic space. However, when it
comes to neurologically manifest “control,” eye tracking does not seem sufficient
to reach any concluding statement. Hasson et al. too are careful to clarify that it
is not just the immediate following of the action assessed by the eye position that
leads to high ISC (controlled brain responses across viewers). Equal amounts of
attention, as in the case of a backward played clip, can lead to low ISC because
of lower intelligibility. Attention is therefore a necessary but not sufficient condi-
tion for a high ISC.9 Intelligibility and comprehension according to the intentions
of the storyteller-director is the factor fulfilling the second role. This is something
that their later work demonstrates further.10


Film Art: An Introduction, The University of Wisconsin Press, Madison 2008; Noël Carroll, Theoriz-
ing the Moving Image, Cambridge University Press, New York 1996; Ira Konigsberg, “Film Studies
and the New Science,” in Projections. The Journal for Movies and Mind, no. 1, 2007, pp. 1-24.
6
  Uri Hasson et al., “Neurocinematics: The Neuroscience of Film,” cit.
7
  Ira Konigsberg, “Film Studies and the New Science,” cit.
8
   Tim Smith, “The Attentional Theory of Cinematic Continuity,” in Projections. The Journal for
Movies and Mind, no. 1, 2011, pp. 1-27.
9
  It is remarkable that attention is here discussed as an externally manipulated variable, rather than
an internally controlling factor, as in the “attention driven regulation” according to which the brain
“controls” where to place emphasis in a film (e.g., what area to look at, or what dimension, such as
color, motion, orientation). In this view, the brain does not “just” react to the stream of stimuli that
a film provides but can control and streamline its reaction. This dimension of attention is discussed
by Joseph Magliano, Jeffrey Zacks, “The Impact of Continuity Editing in Narrative Film on Event
Segmentation,” in Cognitive Science, no. 8, 2011, pp. 1489-1517.
10
   See Greg J. Stephens, Lauren J. Silbert, Uri Hasson, “Speaker–Listener Neural Coupling Under-

                                                                                                     41
                                           Maria Poulaki


   Even though they consider high ISC as a vector of engagement in the movie,
towards the end of their paper Hasson et al. question the direct link between the
two variables. As they say, films with low ISC might still produce a deep engage-
ment, which, however, for unknown reasons can vary between individuals. The
effectiveness of movies, however, is still defined in relation to the high ISC, to
the directed joined attention and intelligibility of many viewers, which surpasses
individual variability and produces similar patterns of neural activation in differ-
ent areas of their brains.


Nuances of film control in cognitive film theory

   Apart from questioning the self-evidence of the link between directorial guid-
ance of attention as approached by cognitive film theory and “mind control” as
defined in neurocinematics, I want to focus on the notion of control itself and the
use of this term (in connection with film viewing) that traverses neuroscientific
and cognitivist approaches to film.
   The emphasis that neurocinematics places on control resonates with some well-
established observations of cognitive film theory and with the use of the same term
in this discipline. Control here also seems to be in the hands (and minds) of the
filmmakers, rather than those of the spectators. For example, David Bordwell and
Kristin Thompson in Film Art associate film directorship with control of cinema-
tography, mise en scène, sound and other film dimensions. By controlling these
aspects, especially in fiction films,11 the filmmakers can in turn control the viewers’
experience, what they see and understand12 – therefore, both attention and intel-
ligibility. Moreover, films can also control what viewers know in the long run, as
their narration can be more or less restrictive, and at times, even “omniscient.”13
   In his discussion of cinema in Theorizing the Moving Image, Carroll uses the term
referring to control of attention by movies, and to the way editing controls the percep-
tual responses of viewers.14 Comparing movies to theater, Carroll observes that the
former exert a much hightened degree of “control over the spectator’s attention.”15
He also observes that films assure “effortless” comprehension – through the use of
various camera techniques – and thus are cognitively “perspicuous.”
   Another cognitivist, Ed Tan, expands on this “effortless” dimension of film


lies Successful Communication,” in Proceedings of the National Academy of Sciences, no. 32, 2010,
pp. 14425-14430.
11
   David Bordwell, Kristin Thompson, Film Art: An Introduction, cit., p. 29.
12
   Ibidem. Quoting from the book: “The frame’s control of the scale of the event has also controlled
our understanding of the event itself” (p. 182), “the duration and speed of the mobile frame can
significantly control our perception of the shot over time” (p. 201), “by controlling editing rhythm,
the filmmaker controls the amount of time we have to grasp and reflect on what we see” (p. 227).
13
   Ivi, p. 89.
14
   Noël Carroll, Theorizing the Moving Image, cit., p. 13.
15
   Ivi, p. 84.

42
                         Neurocinematics and the Discourse of Control


spectatorship, and underlines how film narration exerts control upon the view-
ers’ percepts and emotions.

     The viewers are […] given the strong impression that their movement and sight is
     being controlled. It is the film, or more precisely, the film’s narration that determines
     what the viewers see, when they see it and how. As viewers, we adopt a variety of
     points of observation in space, but the selection and timing of them are completely
     beyond our control. It is the film that imposes them upon us. The viewers are aware
     to some extent of the selection being made by some instance controlling their view.
     That sense of being controlled may add to the feeling that the fictional world exists
     independently of the viewers’ world.16

   Interestingly, the reverse feeling of lack of control on behalf of the viewers,
is considered to be lying at the core of the film-viewing pleasure, being the pre-
requisite, according to Tan, for the state of “intense observation” that manifests
viewers’ emotional engagement.17
   Certain remarks by Tan, such as that “the film’s control over what the viewers
see, and how and when they see it, effectively leads them into an elaborate fan-
tasy from which there is little or no escape,” to some sort of “controlled invisible
witness illusions,”18 echo past theoretical approaches to film as a “dream fac-
tory,” such as those of Jean Baudry and Christian Metz.19 These and other post-
structuralist film theorists criticized the ideological functioning of the cinema
institution and the forms of spectatorship and subjectivity it creates. However
in Tan’s cognitivist account, as well as largely in the cognitivist strands of film
theory, the observations on the illusionary function of cinema are stripped from
the psychoanalytical and critical connotations of poststructuralism and instead
credited with some up-to-date scientific “objectivity,” as well as with a claim that
viewers are active, and by choosing to cooperate, they gain maximum reward.20
   As already mentioned, Konigsberg, stressing once more the fact that cin-
ema images are illusions in comparison to real life perception, describes how
the source of viewing pleasure lies in the viewer’s controlled experience in the
theater, where the film utilizing techniques of focus “does the work” for the
eye. The feeling of being captive into the fictional world is for Tan, as well as for
Konigsberg, one of the most important sources of viewing pleasure within the
cinematic situation, creating what Noel Burch earlier called the “diegetic effect.”


16
   Ed Tan, “Film-induced Affect as a Witness Emotion,” in Poetics, vol. 23, 1994, pp. 7-32.
17
   Ibidem.
18
   Ibidem.
19
   See Jean Baudry, The Apparatus: Metapsychological Approaches to the Impression of Reality in
Cinema, in Philip Rosen (ed.), Narrative, Apparatus, Ideology: A film theory reader, Columbia Uni-
versity Press, New York 1986, pp. 299-318; Christian Metz, “The Fiction Film and its Spectator,”
in New Literary History, no. 1, 1976, pp. 75-105.
20
   “Perhaps it is most accurate to say that the viewer has willingly delegated control to the narra-
tion, expecting in return certain gains, such as being entertained” (Ed Tan, “Film-induced Affect
as a Witness Emotion,” cit.).

                                                                                                 43
                                           Maria Poulaki


   Even though, as already mentioned, neurocinematics does not touch upon the
arresting of agency that films effectuate in order to take control of our percepts
and emotions, and does not “control” the effect of this variable upon ISC, it is
certainly no coincidence that neurocinematics picks up on this emphasis on con-
trol by cognitive film theory and amplifies it. One just has to think of the fact that
controlled viewing meets perhaps its ultimate realization in the viewing conditions
of the neurocinematic experiments, where viewers lie (almost) still inside an fMRI
scanner, which is at least kinaesthetically much more restrictive that the traditional
movie theater. At the same time, and a bit ironically, this type of control is more
and more dissolved in contemporary society, where portable or urban screens
change our film viewing habits and increasingly distract our attention.


Control and effective communication

   The implications of the use of the term control go unquestioned in both
cognitive film theory and neurocinematics, and the underlying assumption
that film is by definition skillful and successful only if it manages to control
the viewer seems interdisciplinarily contagious. Attempting an ideological cri-
tique to the notion in a poststructuralist way would be more than plausible,
but it would also be useful to add another perspective and approach the con-
trol problematic from the aspect of communication studies. Neurocinematics
associates control of mind-brain responses with effective communication be-
tween film and viewer – marked by attention and intelligibility. However, the
emphasis on “effects” that films have on brains can be seen as outdated from
the point of view of media and communication sciences, as it resonates with an
“old-media” paradigm of one-way communication, according to which, an ac-
tion (message sent through media) leads to a reaction (in this case, brain-mind
activation), expected and predicted by the action. According to the classical
Shannon-Weaver mathematical model of communication, input and output of
a message/signal can be matched provided that noise is omitted from the mes-
sage.21 Hasson et al.’s emphasis on predictability in their definition of control
in their neurocinematics research, but also in later research on communication
between speaker and listener,22 somehow brackets out noisy cases (low ISC) as
being the non-standard ones. This happens even when steps are taken towards
the study of variability, as in the 2009 study of Hasson with an international
group of psychologists and psychiatrists, who investigated the non-standard or


21
   See Claude Shannon, Warren Weaver, A Mathematical Theory of Communication, University of
Illinois Press, Champaign 1963.
22
   As Stephens et al. remark, “the speaker’s activity is spatially and temporally coupled with the lis-
tener’s activity. This coupling vanishes when participants fail to communicate” (Greg J. Stephens,
Lauren J. Silbert, Uri Hasson, “Speaker–Listener Neural Coupling Underlies Successful Com-
munication,” cit.).

44
                        Neurocinematics and the Discourse of Control


atypical cases of individuals with autism. The patterns of neural activation of
these subjects during film watching differed from those of the “typical” (not
autistic) subjects as well as from those of other autistic individuals, because they
dispayed low ISC; however the fMRI scans of the autistic subjects still showed
increased intra-subject correlation, that is, reliability and predictability (con-
trolled responses) within subjects on repeated viewings. Moreover, omitting
noise, certain patterns of common activation were revealed.23Although here
variability proves to be an important concern for neurocinematics, it still ap-
pears as a problem to be solved by uncovering underlying similarities and using
demonstrated “highly effective” films to do so. Interestingly, the variability of
“typical” subjects in response to less effective films does not seem as appealing
to neurocinematics as that of atypical groups, which could potentially demon-
strate a certain – even aberrant – reliability in their processing patterns.
   As far as effectiveness is concerned, neurocinematics suggests that the “inef-
fective” film cases of low ISC (which demonstrate variable brain activity across
viewers, and therefore, low control), can be either due to a less engaged process-
ing of the incoming information (e.g., as in a state of day-dreaming) or to an in-
tensely engaged but variable (across individuals) processing of a movie sequence.
Even though these “typical atypical cases” are not studied further, they nonethe-
less could be the potential case studies of Semir Zeki’s proposal that variability
of brain activation (triggered by the same visual response) will be the next “giant
step” in neuroaesthetic research.24
   In what appears as a self-reflexive meta-comment, in their Neurocinematics pa-
per Hasson et al. question their own hierarchy of film effectiveness, according
to which Hitchcock and Leone seem to be particularly mesmerizing directors.
On the one hand, they claim that neurocinematics offers empirical evidence for
“the long-lasting distinction in film theory between films that remain faithful as
much as possible to reality and those that seek to control or distort it” and on the
other hand slightly criticize the films that their own research proves most effec-
tive, as belonging to a tradition of controlled aesthetics and message manipulation
through highly structured editing (e.g., Hollywood, Michael Moore’s documen-
taries). This tradition is opposed to the tradition of “democratic ambiguity of the
image” – represented by films with loose editing (e.g., European Art Cinema,
Italian neo-Realism, direct cinema documentaries). For this meta-comment the
authors draw on classical film theory and particularly André Bazin, who argued in
What is Cinema that highly edited and carefully staged films exert more control,
on the one hand over the external world (manipulating its reality/truth) and on



23
   See Uri Hasson, Galia Avidan, Hagar Gelbard, Ignacio Vallines, Michal Harel, Nancy Minshew,
Marlene Behrmann, “Shared and Idiosyncratic Cortical Activation Patterns in Autism Revealed
Under Continuous Real-life Viewing Conditions,” in Autism Research, no. 4, 2009, pp. 220-231.
24
   Semir Zeki, “Statement on Neuroaesthetics,” The Institute of Neuroaesthetics website, http://
neuroesthetics.org/statement-on-neuroesthetics.php, last visit 19 January 2014.

                                                                                             45
                                          Maria Poulaki


the other hand over the viewer’s experience of the world.25 It is quite contradic-
tory though that Hasson and his colleagues criticize the notion of control that
themselves prioritize as most important in film effectiveness and the way they de-
fine it. Even though they acknowledge that film effectiveness can be ideologically
problematic, they still establish it on the same grounds of controlled aesthetics.


Marketing neurocinematics: Personalizing control

   Defending a just-as-much degree of control by movies on minds, one that
would make the film effective in attracting attention and mesmerizing viewers
but not too trivial (as in the case of “maximal control”),26 neurocinematics rises
as a field potentially profitable in its market applications. It should be no surprise
that the results of early neurocinematics research were embraced fast and with
enthusiasm by the industry and particularly by (neuro)marketing, the field from
which the term neurocinema is said to be coming from.27 The incentive is given
already by Hasson et al.: among the potential applications of neurocinematics,
they refer to the way ISC and its pattern of development over time can offer “a
new neuro-editing tool for assessing the moment-to-moment impact of a given
film.”28 Apart from the benefit this can have for the filmmaker in terms of edit-
ing to maximize audience engagement in particular film scenes (and correct for
the lack of it, in cases when ISC falls), the researchers keep an eye on potential
marketing applications. As they explain,

     the ISC analysis of brain activity can also serve as a measurement of systematic differ-
     ences in how various groups of individuals (defined by age, gender, sexual preference,
     ethnicity, cultural background, etc.) respond to the same film. Measuring the ISC for
     different cultural groups may allow us to study the underlying neuronal substrates that
     correlate with inter-cultural differences. Moreover, it would allow us to assess the impact
     of a given film on different target groups.29

   The overall emphasis that neurocinematics places on effects, coupled with the
control discourse, becomes particularly problematic in its real-world implica-
tions, and the way that the discourse of neurocinema reaches the wider public.
For instance, through film neuromarketing, companies promise to guide Hol-
lywood producers and directors on how to make their movies more influential


25
   See André Bazin, The Evolution of the Language of Cinema in Hugh Gray (ed.), What is Cinema,
vol. 1, University of California Press, Berkeley 1967, pp. 23-40.
26
   Ibidem.
27
   According to Khalid Hammou’s, Hasan Galib’s and Jihane Melloul’s article “The Contributions
of Neuromarketing in Marketing Research,” in Journal of Management Research, no. 4, 2013, pp.
20-33), the term was coined in the inaugural address of marketing professor Ale Smidts in 2002.
28
   Uri Hasson et al., “Neurocinematics: The Neuroscience of Film,” cit.
29
   Ibidem.

46
                          Neurocinematics and the Discourse of Control


upon audiences, enhancing the level of control they can exert upon brains. In
this respect, the words of Peter Katz, producer and assistant at one of the first
neuromarketing companies, Mindsign Neuromarketing, echo the suggestions of
Hasson and his colleagues.

     Movies could easily become more effective at fulfilling the expectations of their particular
     genre. Theatrical directors can go far beyond the current limitations of market research to
     gain access into their audience’s subconscious mind. The filmmakers will be able to track
     precisely which sequences/scenes excite, emotionally engage or lose the viewer’s inter-
     est based on what regions of the brain are activated. From that info a director can edit,
     re-shoot an actor’s bad performance, adjust a score, pump up visual effects and apply any
     other changes to improve or replace the least compelling scenes. Studios will create trail-
     ers that will [be] more effective at winning over their intended demographic. Marketing
     executives will know in a TV spot whether or not to push the romance- or action-genre
     angle because, for example, a scene featuring the leads kissing at a coffee shop could sub-
     consciously engage the focus group more than a scene featuring a helicopter exploding.30

   Even more interesting than this neuro-enhanced effectiveness anticipated by
Katz is a reverse tendency which sets off from a different, interactive or “new
media” perspective and developments in film personalization, and expects the
brains of viewers to “take over” and guide the projection of a film. In experimen-
tal settings (like that of Pia Tikka’s “enactive cinema”)31 but also in commercial
applications, spectators may be able to give feedback to the projection system
by means of physiological data, and then in turn “fed back” with scenes or story
versions that their individual brain seems to be wanting to see. Here, it is not the
common, orchestrated reaction of brains that is of interest, but rather, the indi-
vidualized, variable and even marginal reactions and how they can be included
in and predicted by the system. Former NeuroFocus CEO A.K. Pradeep explains
(predicting convergence between games and neurocinema):

     Multiple if not infinite versions of one film with myriad story twists and endings will be
     produced and consumed. Netflix and Facebook will play a big part in film “personaliza-
     tion.” “Real-time instant consumer brain response-based personalization will create true
     dynamic modifications of the same movie and afford endless delight to consumers.”32

   This direction of neuroaesthetic film research and its marketing application
is remarkable because it takes a perspective different than that of Hasson et al.


30
   Peter Katz, “Neurocinema Aims to Change the Way Movies are Made,” interview by Curtis Sil-
ver, in Wired, 23 September 2009, http://www.wired.com/geekdad/2009/09/neurocinema-aims-
to-change-the-way-movies-are-made, last visit 19 January 2014.
31
   See Pia Tikka, Enactive Cinema: Simulatorium Eisensteinense, PhD dissertation, University of
Art and Design Publication Series, Helsinki 2008.
32
   See Kevin Randall, “Rise of Neurocinema: How Hollywood Studios Harness your Brainwaves to
Win Oscars,” in Fast Company, 2011, http://www.fastcompany.com/1731055/rise-neurocinema-
how-hollywood-studios-harness-your-brainwaves-win-oscars, last visit 19 January 2014.

                                                                                              47
                                          Maria Poulaki


In informational terms, instead of bracketing out noise it seeks to classify it and
model its different realizations within a system of alternative film/clip versions.
Far from abandoning control, this approach seeks to control for what in Has-
son et al.’s methodology would be a low ISC, in other words, for how minds can
wonder in different trajectories not directly triggered by the film-stimulus.


The complexity of film-mind

   The popularization of the control-effectiveness discourse is problematic in
the sense that it undermines the complexity of the film-mind system and cre-
ates a closed film-viewer loop of action-reaction, both when the film controls
the viewer’s mind and when the latter controls the film and is in turn controlled
in a personalized loop. However, as it is known since the 1970s when biologists
Humberto Maturana and Francisco Varela published their work on autopoie-
sis and self-organizing systems,33 the reaction of brains to any kind of stimuli
(including films) is never a linear process and depends more on the internal
organization of the brain than on the external stimuli – an organization that,
we can add, is to a significant extent also socially shaped. When considered as
autonomous self-organizing systems, film system and viewer’s cognitive system
each have their own internal organization and their own temporality. Moreo-
ver, taking one more step to the direction of complex systems theory, the reac-
tion of the brain to film stimuli is not instant but cumulative and emergent, just
like the cognitive organization itself.
   It is worth taking a closer look at the Neurocinematics article, as well as
Hasson’s later research, as it can shed more light on the temporality of the
brain and how it shapes its reaction to films. Using silent films as stimuli, Has-
son et al. identified a hierarchy of “temporal receptive windows” in the brain,
suggesting that different cortical regions respond to stimuli in different time-
scales.34 In the paper Hasson co-authored with Lerner and colleagues in 2011
the temporal brain structure was tested using as stimulus an orally narrated
story.35 It was found that frontal areas are the ones that respond only after
listening to whole paragraphs, rather than individual words or sentences. An
earlier version of this experiment was held by Hasson et al. in 2008 using film
stimuli with parts of the same silent movie clip shown to subjects shuffled in
bigger, medium or smaller chunks. These experiments, apart from showing the
brain as a self-organizing system with its own temporality, also demonstrate

33
   Francisco J. Varela, Humberto R. Maturana, Ricardo Uribe, “Autopoiesis: The organization of
living systems, its characterization and a model,” in Biosystems, no. 4, 1974, pp. 187-196.
34
   Uri Hasson, Eunice Yang, Ignacio Vallines, David J. Heeger, Nava Rubin, “A Hierarchy of Temporal
Receptive Windows in Human Cortex,” in The Journal of Neuroscience, no. 10, 2008, pp. 2539-2550.
35
   Julia Lerner, Christopher Honey, Lauren Silbert, Uri Hasson, “Topographic Mapping of a Hi-
erarchy of Temporal Receptive Windows Using a Narrated Story,” in The Journal of Neuroscience,
no. 8, 2011, pp. 2906-2915.

48
                        Neurocinematics and the Discourse of Control


the lack of a one-to-one relationship between stimulus and response, as certain
brain areas react to an accumulation of audiovisual information, for instance to
whole sequences instead of individual shots.
   The brain areas responding only to large chunks of information (thus only af-
fected by the long-time scale shuffling) are those cortical regions responsible for
higher order narrative processing and plot understanding. This differential tem-
poral functioning was found similar across subjects, through the ISC method.
However, as the writers suggest, even though ISC is an indicator of response
reliability (demonstrated by control and predictability of activation patterns), it
is not one of response amplitude. The latter has a low ISC, therefore an increased
variability across viewers, even though it is an indicator of “incessant processing,
presumably aimed to extract meaningful information from the stimuli.”36 Here
we come again to the problem of engagement that we mentioned earlier. The
viewers might be heavily engaged in their communication with the film, however
in a variable way from one another, and perhaps even in a way that was not in-
tended by the sender (filmmaker). The communication in this sense might not
fulfill the criteria of neurocinematic effectiveness set by Hasson et al., but can
still have a more complex, indirect, and perhaps long-term impact.
   Making a parallelism between their neurocinematic findings and 1960s op
art and optical illusions (such as the Akiyoshi Kitaoka 2003 Rotating Snakes),
which are not perceived in the same way by all subjects, Hasson et al. in the
Hierarchy of Temporal Receptive Windows note that “In all of these cases, vis-
ual neurons presumably respond with large amplitudes while processing the
stimuli, but the responses are unreliable, leading to a failure to ‘lock in’ to a
consistent and stable perceptual organization.”37
   This “failure to ‘lock in’ to a consistent and stable perceptual organization” is
exactly what neuroscientists such as Francisco Varela and Scott Keslo find a vital
characteristic of the brain as a complex dynamic system. According to Kelso, this
incessant instability is not unique to the visual cortex; it is rather a generic fea-
ture of the brain’s working as a complex dynamic network, and affects not only
early percepts but also complicated thoughts, such as narrative understanding.
At every level of processing, the brain is characterized by “nonstationary dynam-
ics,” a prerequisite for pattern formation when encountered with a meaningful
task.38 Referring to the dynamics of perception, Varela used as examples cases of
multistability, like the man/woman figure,39 where two forms of the same object
(two “pictorial attractors”)40 are at the same time present in one single object/


36
   Uri Hasson et al., “A Hierarchy of Temporal Receptive Windows in Human Cortex,” cit.
37
   Ibidem.
38
   Scott Kelso, Dynamic Patterns: The Self-organization of Brain and Behavior, MIT Press, Cam-
bridge (MA) 1995, pp. 283-284.
39
   Gerald H. Fisher, “Measuring Ambiguity,” in The American Journal of Psychology, no. 4, 1967,
pp. 541-557.
40
   See also Robert Gregson, “Transitions Between Two Pictorial Attractors,” in Nonlinear Dynamics,
Psychology and Life Sciences, no. 1, 2004, pp. 41-63.

                                                                                               49
                                         Maria Poulaki


image.41 According to Varela’s neurophenomenology, optical illusions like this
demonstrate the mind’s functioning as a dynamical system, the neural circuits of
which are always found in a state a cognitive drift/flow, in which

     the geometry of phase space needs to be characterized by an infinity of unstable regions,
     and the system flows between them spontaneously even in the absence of external driv-
     ing forces. There are no attractor regions in phase space, but rather ongoing sequences of
     transient visits in a complex pattern of motion, modulated only by external coupling.42

   In this view, the low ISC can be considered an indicator of the phase space of
brain activity without major attractors, while the high ISC is the manifestation
of the presence of an attractor which gives a similar reliable pattern of response
across brains. Causal determination of the phase space and creation of attractors
is what leads to the predictability of response patterns, within and across brains.
   Within the complex systems paradigm, even the notion of directorial control
itself can be revised. There are authors who have already attempted to approach
filmmaking through this lens. For instance Jan Simons in his book Playing the
Waves,43 drawing on complex systems simulation methods, analyzes the direc-
torial style of Lars von Trier, a director often discussed for his obsession with
control. He explains how the filmmaker’s techniques set parameters which caus-
ally determine the phase space of the film’s narrative, and by extension, of the
viewer’s cognitive response. The phase space in physics is a term referring to a
representation of all possible states that a system might take. Editing together
multiple takes of the same scene, von Trier provides the viewers with a phase
space of multiple narrative trajectories instead of a single one, as it happens in
most films. A similar observation is made by Stephen Shaviro about Nick Hook-
er’s technique of shooting in his early music videos.44
   The notion of control as setting parameters and determining the phase space
indicates that even the stimulus can be multiple, not only the interpretation.
This way the latter is still controlled but in a different way than the one praised
by Carroll in films that promote a unique understanding, having the spectator
“always looking where he or she should be looking, always attending to the right
details and thereby comprehending, nearly effortlessly, the ongoing action pre-


41
   Hasson has also investigated this phenomenon from a different perspective in Uri Hasson,
Talma Hendler, Dafna Ben Bashat, Rafael Malach, “Vase or Face? A Neural Correlate of Shape-
Selective Grouping Processes in the Human Brain,” in Journal of Cognitive Neuroscience, no. 6,
2001, pp. 744-753.
42
   Francisco J. Varela, “The Specious Present: A Neurophenomenology of Time Consciousness,” in
Jean Petitot, Francisco J. Varela, Bernard Pachoud, Jean-Michel Roy (eds.), Naturalizing Phenom-
enology: Issues in Contemporary Phenomenology and Cognitive Science, Stanford University Press,
Stanford 1999, pp. 266-314.
43
   Jan Simons, Playing the Waves: Lars von Trier’s Game Cinema, Amsterdam University Press,
Amsterdam 2007.
44
   Steven Shaviro, “Post-Cinematic Affect: On Grace Jones, Boarding Gate and Southland Tales,”
in Film-Philosophy, no. 1, 2010, pp. 1-102.

50
                        Neurocinematics and the Discourse of Control


cisely in the way it is meant to be understood.”45 The complex systems perspec-
tive helps to move away from a simple and one to one stimulus-response idea of
film effectiveness, either in time, or in space.
   In this line of thinking, the process of “structural coupling”46 would offer a
good alternative for the description of the relationship between film and viewer
as autonomous systems. Thus, a film’s textual system and the brain of the viewer
can be engaged in a nonlinear and open communication process, contributing
to a complex and dynamical cinematic experience. Then we can say that these
structurally-coupled systems “will have an interlocked history of structural trans-
formations, selecting each other’s trajectories.”47
   This article, rather than presenting new research results, suggests a pause to
reflect on already conducted research. Due to space limitations the main focus has
been on Hasson et al.’s seminal – concerning the impact on film studies and the
non-academic public – article, with only brief references to other important contri-
butions to neuro-cinema research. Pause, distance and focus are not only necessary
tools for film analysis; they are also crucial first steps for a critical neurofilmology.




45
   Noël Carroll, Theorizing the Moving Image, cit., p. 84.
46
   See Humberto R. Maturana, Francisco J. Varela, The Tree of Knowledge: The Biological Roots of
Human Understanding, Shambhala Publications, Boston 1987.
47
   Francisco J. Varela, Principles of Biological Autonomy, Elsevier, New York 1979, pp. 48-49.

                                                                                             51
