I fumetti nella ricerca sulle funzioni cerebrali
10.13125/2039-6597/3455
The important role of Comics, in communication related to popular contexts and with people of all ages is undoubted. The meaning of comics goes beyond the pure representation of an image with a text, and this is one of the reasons why Comics have been used, in some instances, to study brain functions related to emotions, humourappreciation, image comprehension and visual language.In the past 20 years, the availability of instruments that could measure the activity of the brain in vivohas allowed us to develop experimental paradigms aimed at exploring the neural changes related not only to simple behaviours but also to increasingly complex functions such as emotions and consciousness [...]
application/pdf
2039-6597
3455
15
en
Micaela Morelli
Between
Comics and studies on brain functions
Text.Serial.Journal
https://ojs.unica.it/index.php/between
8
Essay
2018-05-30
2018-09-19
2018-07-10
2018-09-19
1
Micaela Morelli
2018/05/30
10.13125/2039-6597/3455
University of Cagliari
2039-6597
15
en
Comics and studies on brain functions
https://ojs.unica.it/index.php/between/article/download/3455/3021
8
                    Comics and studies
                    on brain functions
                              Micaela Morelli

     The important role of Comics, in communication related to
popular contexts and with people of all ages is undoubted. The
meaning of comics goes beyond the pure representation of an image
with a text, and this is one of the reasons why Comics have been used,
in some instances, to study brain functions related to emotions,
humour appreciation, image comprehension and visual language.
     In the past 20 years, the availability of instruments that could
measure the activity of the brain in vivo has allowed us to develop
experimental paradigms aimed at exploring the neural changes related
not only to simple behaviours but also to increasingly complex
functions such as emotions and consciousness. Modern neuroimaging
includes functional Magnetic Resonance Image (fMRI), which is the
most widely used brain imaging method to study human brain
function. fMRI can be used to evaluate regional Cerebral Blood Flow
(rCBF) and Blood Oxygen Level Dependent (BOLD) neural responses.
This technique is based on the manipulation of hydrogen nuclei
contained in water using a magnetic feld that generates a signal
transformed in an image. The image produced, in turn, depends on
tissue properties as the brain engages in different activities. These
methodologies, together with more classical electroencephalography
methods (EEG) that measure electrical signalling in the brain and the
electrophysiological event-related brain potential (ERP), provide
important tools in the feld of Neuroscience to discover the human
brain. EEG and ERP provide excellent temporal resolution, allowing
for the detection of brain events at the millisecond timescale, whereas
the spatial localisation of these signals is not precise. In contrast, the
temporal resolution of fMRI is measured in seconds, but the spatial
localisation of fMRI signals is measured in millimetres.
     The application of these tools to studies performed in the feld of
Comics have evidenced how this form of communication, although
representing a new way of communication, touches some native



Between, vol. VIII, n. 15 (Maggio/May 2018)
Micaela Morelli, Comics and Studies on Brain Functions



qualities owned by humans, thus adding great interest to this form of
representation and communication.
      In this brief article, I shall describe some of the studies that have
utilised Comics and delineate their importance for understanding the
human mind. I will also describe the link between studies on
Neuroscience and the different methods of representation that Comics
utilise.

             How the brain processes visual narrative
      The use of verbal language for communication is a specifc
attribute of humans. However, verbal language is not always the only
and best means to transmit information in interpersonal interaction.
People can in fact share their understanding of the world using non-
verbal cues such as facial expressions, gestures and postures.
      Interestingly, the understanding of non-verbal messages is
supported by the existence of mirror neurons contained in premotor
and motor cortex of the brain. A mirror neuron is a neuron that is
activated both when an animal acts and when the animal observes the
same action performed by another. These neurons are important for
understanding the intention and the actions performed by other people
and for learning new skills by imitation (Rizzolatti - Craighero 2004).
Mirror neurons have been studied in several contexts, including
autism.
      Analogous to the manner by which words are combined in verbal
language, individual images can be combined to form larger elements
that enable complex visual narratives. Constituent structures for verbal
language or visual narrative differ both in their basic units (words or
images) and in the rules by which they are combined. However, the
brain, as demonstrated by Cohn et al. (2014), engages similar
neurocognitive mechanisms to build a narrative structure to
comprehend language or visual narrative sequences as in Comics. This
makes our brain capable of perceiving Comics, as other abilities, like
language.
      ERP studies on N400 and N600, two event-related brain potential
responses linked to meaning processing that can be utilised for
examining language processing, showed that perception, attention,
memory and language jointly participate in the neural events
responsible for meaning processes. Moreover, the results suggest that
linguistic and non-linguistic information rely on similar semantic
memory networks (Kutas - Federmeie 2011; Cohn - Kutas 2015).




2
                                       Between, vol. VIII, n. 15 (Maggio/May 2018)



      In a study using Comics in which ERPs were recorded, Manfredi
et al. (2017) replaced visual narrative sequence with phonetically
imitate action sounds (e.g. Pow!), or words that describe an action (e.g.
Punch!), that were congruent or incongruent within their sequence
contexts. The results showed that integration of word/letter/symbol
strings within visual narratives elicited ERP patterns typically
observed for written sentence processing, suggesting that the brain
employs similar mechanisms for analysing the two different ways of
expression. ‘Narrative grammar’, as in Comics, organises the semantics
of event structures in sequential images as much as syntax organises
meaning in sentences, and manipulations of this narrative grammar
elicit ERP similar to manipulations of linguistic syntax (Cohn et al.
2014, Cohn –Kutas 2015).

                        Importance of faces
     Attractiveness judgements using faces, studied by fMRI, have
demonstrated that brain areas belonging to the socio-emotional
structures participate in face judgement, evidencing how faces are the
most important means to recognise other people intentions (Haist -
Anzures 2017).
     Faces are essential for the development of interaction among
people. Faces attract the viewer’s attention as demonstrated by studies
on eye-tracking map, where they attract the gaze of the viewer’s even
when the viewer is asked to attend to different stimuli (Hari - Kujala
2009) (Fig. 1). The attractiveness of faces has a great social and
commercial impact. Besides beauty and mood, the face communicates
health, gaudiness, personality and ethnic background (Hari - Kujala
2009). The expression of face refects the person’s feelings. Through
vision, we may therefore recognise basic sentiments such as fear,
disgust and rage, which are processed by the ‘limbic/primitive brain’,
the amygdala, the thalamus and the hypothalamus; however, one may
also recognise face expressions that need cognitive processes, such as
happiness and sadness, that are elaborated by brain structures
classifed as ‘superior’ and belonging to the neocortex such as the
frontal cortex (FC), the cingulate cortex and the anterior insula (Fig.2).
     This distinctive trait develops very early in human life as shown
by newborn infants who recognise faces better than other objects
(Grossmann - Johnson 2007). However, in line with the fact that the
human brain reaches full maturation in terms of growth in size,
connectivity, density of neurons and synapse during the frst 20 years
(Dekaban 1978; Innocenti - Price 2005), the development of face-



                                                                                3
Micaela Morelli, Comics and Studies on Brain Functions




                                              Fig. 1




4
  Between, vol. VIII, n. 15 (Maggio/May 2018)




Fig. 2




                                           5
Micaela Morelli, Comics and Studies on Brain Functions



processing ability matures slowly over the frst two decades of life
(Haist - Anzures 2017).
     Therefore, the extensive representation of faces in Comics touches
instinctive qualities, and Comics have been instrumental in studies
aimed at recognising brain areas involved in the perception of pleasant
or negative emotions (Hari - Kujala 2009).

                    Gesture, posture and movement
     A typical representation in Comics are the motion lines that
appear in graphic representations to depict the path of a moving object.
A remarkable study on motion lines was performed by Cohn and
Maher (2015) through ERPs in a group of experienced comic readers
among university students. Interestingly, the motion lines that
delineate the direction of an action in Comics were not perceived as
basic biological aspects of the visual system, but they were recognised
as vocabulary of the visual language. These results suggest that motion
lines are not tied to aspects of the visual system but are components of
the ‘vocabulary’ of the visual language.

                Reward system and humour Comics
     Several preclinical and clinical studies have revealed that one of
the most important neurotransmitters, dopamine, which is enriched in
brain areas that constitute the reward system (mesolimbic system,
ventral striatum/nucleus accumbens, ventral tegmental area), is
activated by pleasant food assumption, sex and drug abuse (Di Chiara
and Bassareo 2007).
     In this frame, pleasant feelings such as those produced by humour
Comics have been the object of several interesting investigations,
which have demonstrated how comical Cartoons activate the same
brain areas activated by rewarding stimuli, providing the experimental
evidence of a neurobiological link between reward and humour
(Mobbs et al. 2003; Berns 2004; Samson et al. 2008).
     Osaka and Osaka (2005) successfully demonstrated the hypothesis
of reward components of laughter by fMRI and showed that
visualisation of mimic words and emotional facial expression words,
highly suggestive of laughter, signifcantly activate striatal reward
centres, including the putamen/caudate/nucleus accumbens, the
prefrontal cortices (PFCs), the dorsal anterior cingulate cortex and the
supplementary motor area (SMA), whereas non-mimic words under the



6
                                        Between, vol. VIII, n. 15 (Maggio/May 2018)



same task that did not imply laughter did not activate these areas in
humans.
      Several studies have applied Comics to provide a neurobiological
link between theories of humour and hedonic processes. Comics are, in
fact, one of the most common humour medium by showing pictures
containing incongruous elements that have to be detected and resolved
to understand the punch line.
      Funny Comics have been used by several fMRI studies to examine
the association between humour intensity and the network of cortical
and subcortical regions involved in the degree of reward elicited by the
stimulus. The results of those studies revealed an association between
humour intensity and brain activation in cortical areas important for
cognitive processes, such as the temporo-occipital junction (TPJ) and
FC, and subcortical areas such as the ventral striatum, the nucleus
accumbens, the anterior thalamus, the ventral tegmentum, the
hypothalamus and the amygdala (Berns 2004; Samson et al. 2008).
These studies identifed, through Comics, the brain ‘humour
appreciation network’. Supported by the study of Mobbs et al. (2003), it
is reasonable to conclude from these studies that activation of the
nucleus accumbens refects the hedonic feeling that accompanies
humour, whereas the activation of amygdala is of clinical interest as
this region has been implicated in the pathological features of several
affective disorders, and diminished dopaminergic tone in the
amygdala has been implicated in the emotional memory dysfunction
and anhedonia observed in depression. In addition, TPJ and FC and
the temporal cortex have been claimed to be involved in the
incongruity resolution process that allows the comprehension of the
scene or generally in humour perception.
      In addition, these studies, by showing activation by funny
Cartoons and, as mentioned below, by funny flms, of the SMA suggest
the presence of a strong motor component and propose that laughter is
an important component of this hedonic process. At the same time,
activation of the left lateral inferior frontal lobe, including Broca’s area,
would possibly refect the language-based decoding of the stimuli.
      Iwase et al. (2002) investigated the rCBF by positron emission
tomography (PET) during laughter or smile induced by visual material
in funny flms and the facial muscle contraction evaluated by
electromyogram (EMG) and demonstrated a signifcant correlation of
the two in the SMA and the putamen, two brain areas important for
movement, but not in the primary motor area (M1), which is also
involved in motor performance. In contrast, with voluntary facial
movement, a signifcant correlation between rCBF and EMG was found
in both M1 and the SMA. In addition, laughter or smile, as opposed to


                                                                                 7
Micaela Morelli, Comics and Studies on Brain Functions



voluntary movement, activated visual association areas, the temporal
cortex and the orbitofrontal and medial PFCs, whereas voluntary facial
movement did not. These results allow the demonstration of specifc
cognitive neural substrates of facial expression during pleasant
emotion as opposed to voluntary, not emotional, facial movement.
     The fMRI study conducted by Samson et al. (2008) focused in
more detail on cognitive humour processing. These researchers used
Comics representing three different conditions based on visual
resemblance, pure semantic relationships, or Cartoons that required
additional mentalising abilities. The results revealed a network of TPJ,
inferior frontal gyrus and ventromedian PFC involved in pure
semantic relationships, whereas visual resemblance showed activation
in the extrastriate cortex, and Comics that required additional
mentalising abilities activated median PFC, TPJ, the posterior superior
temporal sulcus and the posterior middle temporal gyrus. Their study
on examination of different Comics revealed that processing of
different logical mechanisms underlying cognitive humour processing
depends on separate neural networks (Samson et al. 2008).
     In an ad-hoc study conducted by Osaka et al. (2014), fMRI was
used to evaluate humour appreciation. A four-frame sequence comic,
Manga that included introduction (frst scene), development (second
scene), turn (third scene) and conclusion (fourth scene, punch line),
was utilised. The most amusing scene appeared in the last scene and
was essential for understanding the humour of the Manga. Beginning
with the second frame (development scene), activation of TPJ was
observed, followed by activations of the temporal and PFC during
viewing of the third frame (turn scene). During the fourth frame
(punch line), strongly increased activations were detected in the
median PFC and the cerebellum. The study revealed that activation of
the left TPJ initiates the early stages of the comprehension process of
the humour story line. This activation then reached the superior
temporal sulcus and the median PFC, which produced humour
appreciation for the punch line and the cerebellum. Thus, expectation
before presentation of the punch line plays an important role in the
appreciation of humour. Activation of brain areas by funny and non-
funny Manga revealed no differences in brain activation for the frst,
second or third frame. However, a signifcant increase in activation of
the cerebellum in both hemispheres for the funny Manga in the fourth
frame was observed. Interestingly, distinguishable differences in
activation of the cerebellum between funny and non-funny conditions
were also found for the fourth frame (Osaka et al. 2014). Activation of
the left cerebellum in response to humour has been reported by
previous studies showing that affective components of humour are


8
                                      Between, vol. VIII, n. 15 (Maggio/May 2018)



related to activation of the left cerebellum (Bartolo et al. 2006).
Interestingly, the cerebellum, which is critical in social cognition, has
reciprocal connections with the PFC via the thalamus.
     One key fnding is that comprehension accuracy levels correlated
with humour-comprehension responses in the left TPJ, as measured by
BOLD neural responses (Campbel et al. 2015). This fnding represents a
novel and precise neural linkage to humour comprehension. A second
key fnding is that the superior frontal gyrus was uniquely associated
with humour appreciation, suggesting that a complex cognitive
processing underlies humour appreciation. Therefore, Campbel et al.
(2015) provided an operational distinction between humour
comprehension and appreciation.

                      Psychiatric disorders
      Theory of mind (ability to attribute mental states to others) and
empathy (ability to infer emotional experiences) are both important
abilities to understand disorders characterised by impairments of social
cognition. Vollm et al. (2006) performed an fMRI study using different
Cartoons to understand disorders characterised by impairments of
social cognition, such as autism and psychopathy. The results showed
that theory of mind and empathy are associated with common areas of
activation such as the medial PFC, the TPJ and the temporal poles. In
addition, theory of mind revealed increased activations of the lateral
orbitofrontal cortex, the middle frontal gyrus, the cuneus and the
superior temporal gyrus. On the other hand, empathy was associated
with enhanced activations of the paracingulate, the anterior and
posterior cingulate Cortex and the amygdala. These results
demonstrated that empathy requires the additional recruitment of
networks involved in emotional processing (Vollm et al. 2006),
underlining how the use of Cartoons can be instrumental in
understanding both brain function and dysfunction.

                            Conclusions
     One of the central challenges of the 21st century science is to
understand the human mind in biological terms. The human brain is
continuously hit by millions of information and continuously flters
them unconsciously to make quick choices and judgement all the time.
Neuroscience is important not only because it gives us a deeper
understanding of what makes us what we are but also because it
allows a signifcant series of dialogues between brain science and other


                                                                               9
Micaela Morelli, Comics and Studies on Brain Functions



areas of knowledge to help in understanding this spectacular part of
our body.
     Comics are apparently far away from classical investigations used
to understand brain functions. However, thanks to their schematic and
reproducible structure and, at the same time, the rich high emotional
impact, they allow to easily measure complex brain responses that can
be categorised in a more precise manner compared with more
sophisticated tools of analysis.




10
                                      Between, vol. VIII, n. 15 (Maggio/May 2018)




                            Bibliografia
    Bartolo, A. - Benuzzi, F., Nocetti, L., Baraldi, P., Nichelli, P.,
“Humor comprehension and appreciation: an FMRI study”, J Cogn
Neurosci, 11 (2006): 1789-98.
    Berns, G.S., “Something funny happened to reward”, TRENDS in
Cognitive Sciences, 8.5 (2004): 193-194.
    Campbell, D.W. - Wallace, M. G. - Modirrousta, M. - Polimeni, J.O.,
McKeen, N.A. - Reiss, J.P., “The neural basis of humour comprehension
and humour appreciation: The roles of the temporoparietal junction
and superior frontal gyrus”, Neuropsychologia, 79(Pt A) (2015): 10-20.
    Cohn, N. - Jackendoff, R. - Holcomb, P.J. - Kuperberg, G.R., “The
grammar of visual narrative: Neural evidence for constituent structure
in sequential image comprehension”, Neuropsychologia, 64 (2014): 63–70.
    Cohn, N. - Kutas, M., “Getting a cue before getting a clue: Event-
related potentials to inference in visual narrative comprehension”,
Neuropsychologia 77 (2015): 267–278.
    Cohn, N. - Maher, S., “The notion of the motion: The
neurocognition of motion lines in visual narratives”, Brain Res, 19.1601
(2015): 73–84.
    Di Chiara, G. - Bassareo V., “Reward system and addiction: what
dopamine does and doesn’t do”, Curr Opin Pharmacol, 7.1 (2007):69-76.
    Dekaban, A.S., “Changes in brain weights during the span of
human life: relation of brain weights to body heights and body
weights”, Ann Neurol, 4 (1978): 345–356.
    Grossmann, T. - Johnson, M.H., “The development of the social
brain in human infancy”, Eur J Neurosci, 25 (2007): 909–919.
    Haist, F. - Anzures, G., “Functional development of the brain’s
face-processing system”, Rev Cogn Sci, 8.1-2 (2017): e1423.
    Hari, M. - Kujala, M.V., “Brain Basis of Human Social Interaction:
From Concepts to Brain Imaging”, Physiol Rev, 89 (2009): 453–479.
    Kutas, M. - Federmeie, K.D., “Thirty years and counting: Finding
meaning in the N400 component of the event related brain potential
(ERP)”, Annu Rev Psychol, 62 (2011): 621–647.
    Innocenti, G.M. - Price, D.J., “Exuberance in the development of
cortical networks”, Nat Rev Neurosci, 6 (2005): 955–965.
    Iwase, M - Ouchi, Y. - Okada, H. - Yokoyama, C. - Nobezawa, S. -
Yoshikawa, E. - Tsukada, H. - Takeda, M. - Yamashita, K. - Takeda, M.
- Yamaguti, K. - Kuratsune, H. - Shimizu, A. - Watanabe, Y., “Neural



                                                                              11
Micaela Morelli, Comics and Studies on Brain Functions



substrates of human facial expression of pleasant emotion induced by
comic flms: a PET Study”, Neuroimage, 17.2 (2002): 758-68.
     Manfredi, M. - Cohn, N. - Kuta, M., “When a hit sounds like a kiss:
an electrophysiological exploration of semantic processing in visual
narrative”, Brain Lan, 169 (2017): 28–38.
     Mobbs, D. - Greicius, M.D. - Abdel-Azim, E. - Menon, V. - Reiss,
A.L., “Humor modulates the mesolimbic reward centers”, Neuron, 40
(2003): 1041–1048.
     Osaka, M. - Yaoi, K. - Minamoto, T. - Osaka, N., “Serial changes of
humor comprehension for four-frame comic Manga: an fMRI study”,
Scientific Reports, 4 (2014): 5828.
     Osaka, N. - Osaka, M., “Striatal reward areas activated by implicit
laughter induced by mimic words in humans: a functional magnetic
resonance imaging study, Neuroreport, 16.15 (2005):1621-4.
     Rizzolatti, G. - Craighero, L., The mirror-neuron system; Annual
Review of Neuroscience, 27.1 (2004): 169–192.
     Samson, A.C. . Zysset, S. - Huber, O., “Cognitive humor
processing: different logical mechanisms in nonverbal cartoons-an
fMRI study”, Soc Neurosci 3.2 (2008): 125-40.
     Volm, B.A. - Taylor, A.N.W. - Richardson, P.A. - Corcoran, R. -
Stirling, J. - McKie, S. - Deakin, J.F.W. - Elliott, R., “Neuronal correlates
of theory of mind and empathy: A functional magnetic resonance
imaging study in a nonverbal task”, NeuroImage, 29 (2006): 90 – 98.



                                     The author
Micaela Morelli

       Professor of Pharmacology, Micaela Morelli, has a long
experience in research on drugs active in the central nervous system
such as antipsychotics, drugs of abuse and anti-Parkinson drugs. Her
preclinical research has focused on the study of motor dysfunctions
that characterize Parkinson's disease and their biochemical and
molecular correlates. She has also studied the neurotoxic effects of
psychostimulants such as 3,4-methylenedioxymethamphetamine
(ecstasy). She is the author of about 210 publications in international
journals with impact factor. She performed some of her research at the
University of Arizona (Tucson). She is pro-rector of the research of the
University of Cagliari and president of the Italian Society of
Neurosciences.



12
                                     Between, vol. VIII, n. 15 (Maggio/May 2018)



      Email: morelli@unica.it



                             The paper
Date sent: 15/03//2018
Date accepted: 30/04/2018
Date published: 30/05/2018



                    How to quote this paper
Morelli, Micaela, “Comics and Studies on Brain Functions”, Spazi tra le
nuvole. Lo spazio nel fumetto, Eds. G.V. Distefano, M. Guglielmi, L.
Quaquarelli, Between, VIII.15 (2018), http://www.betweenjournal.it




                                                                             13
