Wordless: Interpreting Visual Sequence as Storytelling - De ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Angelo Piepoli Wordless: Interpreting Visual Sequence as Storytelling Towards a Semiotic-Textological Approach to Comics and Pictorial Narrative without Verbal Components Abstract: This article describes how visual sequences become storytelling. The most recent research on comics language has insisted that sequentiality, rather than the juxtaposition of words and images, is the main feature of comics. Comics made up only of drawings, for example, owe to sequentiality the possibility of con- ferring a narrative sense on the juxtaposition of those drawings. Turning to János Sándor Petőfi’s semiotic textology, this paper argues that, within the sequence, the satisfaction of textuality requirements – proper to verbal-text linguistics – such as connexity, cohesion, constringency, and coherence is what facilitates the interpretation of comic books, or parts of them, that do not include a verbal com- ponent, and make it possible to accept them as multimedia texts in a broad sense. The paper also attempts to highlight the active role of the reader/interpreter/pro- ducer in conferring narrative sense on a sequence of pictorial elements. Keywords: closure, coherence, cohesion, comics, comprehensive vehiculum, con- nexity, constringency, coreferentiality, creative exercises, explicative interpreta- tion, formatio, FS architectonics, graphic narrative, interpretation, János Sándor Petőfi, multimedial communicate, relatum, relatum imago, Scott McCloud, semi- otic textology, sensus, textuality, vehiculum, vehiculum imago, wordless comics, wordless narratives 1 Pictorial sequences, languages, and texts A generalized conception considers comics as literary products produced through the use of a language based on two distinct systems of signs, words and images, whose elements are combined into a seemingly indissoluble interconnection. 1 1 I owe the completion of this contribution to many people. János Sándor Petőfi is no longer among us, but he continues to deeply influence all of my writing: with him I contracted a human and scientific debt that I will not be able to repay. My deepest gratitude goes to the students who, during my experience as a teaching assistant at the University of Macerata, offered me a treas- Open Access. © 2021 Angelo Piepoli, published by De Gruyter. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. https://doi.org/10.1515/9783110642056-013
158 Angelo Piepoli
Accordingly, erasing one of the two systems would compromise the sense of the
story or might entirely change it, especially if the verbal component is sacrificed.
Interestingly, Rodolphe Töpffer, widely and debatably regarded as the “inventor”
of comics, wrote in the preface to “Histoire de Monsieur Jabot”:
This little book is of a mixed nature. It is composed of a series of autograph-line drawings.
Each of these drawings is accompanied by one or two lines of text. The drawings, without
this text, would have nothing but an obscure meaning; the text, without the drawings,
would mean nothing. The whole thing together forms a kind of novel all the more original
in that it does not resemble a novel more than any other thing. (Töpffer 1837, 334; my trans-
lation)
Most recent research on the language of comics, following Comics & Sequential
Art by author and artist Will Eisner (1985), insists on sequentiality, rather than
the juxtaposition of words and images, as the main feature of comics. After Eisner,
Scott McCloud proposed his famous definition: “juxtaposed pictorial and other
images in deliberate sequence, intended to convey information and/or to produce
an aesthetic response in the viewer” (McCloud 1994 [1993], 9; my emphasis). Yet
McCloud also found the real core of comics in closure, in the “phenomenon of
observing the parts but perceiving the whole” (63) inherent to the specific nature
of human perception, as well as the engine that enables the elements in his defi-
nition to function. Thus, McCloud considered juxtaposed pictorial elements in a
deliberate sequence without verbal components as comics.
Since the 1960s, similar issues have captivated academics interested in the
study of languages and their applications. Analytic approaches to multimedial
relational linguistic objects such as films, comics, or plays, expanded as schol-
ars sought to apply linguistic methods to domains beyond verbal expression. In
Europe, many have attempted to examine the double articulation in non-verbal
media (Pasolini 1972, 203–204; Deleuze 1984 [1983], 25; Gubern 1975 [1972], 95)
and, in the last two decades of the twentieth century, the concept of text began to
be applied to semiotic relations between signifiers and signifieds entirely lacking
verbal elements. In the current vocabulary of many research sectors, “language,”
ure-trove of discoveries, new ideas, and research cues. I greatly thank Cristina Díaz Baamonde,
whose kind cooperation made it possible to conduct and analyse the exercise with the primary
school students. I thank Scott McCloud and Ivo Milazzo for their kind replies, suggestions, and
support. A big “thank you” goes to Prof. Luciano Vitacolonna for criticism and useful advice
during the drafting of the essay, and Lisa DeTora, whose excellent editing was respectful to the
original content. Finally, I would like to thank Susana Díaz Baamonde for patient support before
and during the writing of this text.Wordless: Interpreting Visual Sequence as Storytelling 159
“linguistic,” and “text” have become widely understood in a broad sense that
extends well beyond the limits formerly accepted by traditional linguistics.
Of course, interpreting texts that are not exclusively verbal or non-verbal, and
verifying their acceptability as texts (textuality, in a broad sense), has been com-
plicated by the lack of a unified theoretical framework that, in addition to other
problems,
– in the case of a multimedial text, accounts for the peculiarities of the different
media and the respective syntactic and semantic (in a broad sense) contribu-
tions of the different components to the supposed logical-conceptual unity of
which they are part; and
– allows verification of the acceptability as a textual unit (in a broad sense) of
texts (in a broad sense) that are multimedial or supposedly unimedial without
a verbal component.
2 S
emiotic textology: Towards a unified model of
signification
My search for a unified framework of signification led to a promising answer devel-
oped by the linguist and semiotician János Sándor Petőfi. An interest in verbal
texts was the starting point of Petőfi’s research. He developed various models to
deal with different aspects of the analysis and interpretation of verbal, predomi-
nantly verbal, and non-verbal objects that could or could not be considered texts,
the latter of which he termed “communicate” (as a noun) to avoid extensional
ambiguity. Semiotic textology was Petőfi’s ultimate development:
I use the term “textology” to avoid other terms, such as, for example, text grammar or text
linguistics, which refer to specific – and restricted – disciplines. I use the term “semiotic”
to express the idea that, in the framework of Semiotic Textology, texts are treated as sign
complexes from a syntactic (or, more generally, formal), a semantic, and a pragmatic point
of view. (Petőfi 2004a, 75; my translation)
Consistent with his mathematical training, and in order to identify univocally the
elements discussed, Petőfi presented his model using symbols, diagrams, and a
thesaurus of Latin words (see fig. 1).
Within Petőfi’s semiotic textology, analytical approaches to multimedial
objects such as comics and graphic narratives must take into due consideration
the signifier – marking a clear difference from approaches, such as Greimasian
semiotics, which focus almost exclusively on the signified – on the basis of a sign
model that makes it possible to integrate manifestations of signs belonging to dif-160 Angelo Piepoli
Fig. 1: Mental model of the perception of signifying
entities (Petőfi 2001, 23).
ferent semiotic media. Petőfi started from a model of the perception of signifying
entities (“En” in fig. 1) understood as non-ingenuous, meaning that “the anal-
ogous mental image of an entity (= EnIm) and its conceptual analysis (= AnCo)
cannot be separated from one another” (Petőfi 2001, 22; my translation). In figure
1, “EI | M | AC” represents, in the strict sense, “a mental model of a ‘Janus Bifrons’
type constituted by a mental image assigned to the entity to be perceived and by
the conceptual analysis congruent to it {EnIm, AnCo}” (Petőfi 2001, 23; my trans-
lation). The light rectangle surrounding the dark rectangle with the symbol “En”
indicates an “immediate context” (23; my translation) (settings) from which the
entities to be perceived may not always be separated discretely.
According to Petőfi, the next step in constructing an integrative sign model is
to double specularly the model of perception in figure 1 (see fig. 2). A vehiculum
(= Ve), whose mental image (= VeIm, vehiculum imago) is divisible into its physi-
cal-semiotic (= F, figura) and linguistic-semiotic (= L, lingua) facies, functions as
the physical manifestation of the signifier within a sign. In this model, the entity
constituting the real referent (within the signified) is called relatum (= Re).
Fig. 2: Model of the constituents of signifier and signified (Petőfi 2001, 24).Wordless: Interpreting Visual Sequence as Storytelling 161
Fig. 3: The semiotic constitutive factors of a sign complex (Petőfi 2004a, 79, 86).
Here, the conceptual analysis (AnCo) related to the Ve is called formatio (= Fo),
while the conceptual analysis related to the Re is called sensus (= Se). As shown
in figure 3, both the formatio and the sensus in turn have two aspects: the systemic
(formatio systemica = Fσ and sensus systemicus = Sσ, respectively the formal and
semantic architectonics assignable to a Ve within the epistemological framework
of the given linguistic system, understood broadly) and the contextual (forma-
tio contextualis = Fc and sensus contextualis = Sc, respectively the formal and
semantic architectonics assignable to a Ve in the given communication situation;
Petőfi 2004a, 78–79). In fact, Petőfi distinguished between contextual architecton-
ics, related to the given context and co-text, and systemic architectonics, which
include all hypothetical entities corresponding to the respective contextual enti-
ties (represented in the diagram by the symbols shown in round brackets). On the
other hand, he distinguished between formal architectonics, namely the signifier
(significans), and semantic architectonics, namely the signified (significatum;
Petőfi 2004a, 79–80).
Petőfi thus identified the six constituent factors of a sign complex with (Ve),
VeIm, Fo, Se, ReIm, and (Re). The elements in parentheses are not necessarily
part of a sign complex, since the two parts of the model are associated with a
not necessarily and/or exclusively perceptive explanation: “in some cases it is
not only unnecessary but also impossible to require the physical presence of Re
[cases of fictional signification]; and we can imagine cases in which this obser-
vation is also valid for Ve [cases of mystic and/or hallucinatory communication]”162 Angelo Piepoli
Fig. 4: The relations between sign components and textuality criteria.
Notes from university classes with Prof. János Sándor Petőfi.
(Petőfi 2001, 25; my translation). The last step in the construction of an integra-
tive sign model
consists in determining the possible types of the mental apparatus Ve | M | Re, or, as it might
better be termed, of the model Ve | M | Re (namely of the relation “Signifier–Signified,” or, if
you like, of “signification”). The sign model [presented so far] can be applied as an “integra-
tive sign model” since it contains no restriction as regards the aforementioned types. This is
an important aspect because if we are dealing, for example, with a dance accompanied by
mimicry, gestures, singing (rhythmic and melodic text) as the vehiculum (= Ve) of a multi
medial text, its media components signify in different ways of “signification,” but in an
organic unit. The existence of this organic unit can only be guaranteed if the different types
of “signification” can be modelled and integrated (“superimposed”) within the theoretical
framework itself. (Petőfi 2001, 25; my translation).
We can see that this sign model can be read starting from the Ve (or its mental
image) to arrive at the Re (or its mental image), in which case we would be des-
cribing the process of reception of a sign complex, or proceeding in the opposite
direction, in which case we would be describing the process of production of a
sign complex (Petőfi 2001, 28).
One of the main objectives of Petőfi’s theoretical framework is the explicative
interpretation of sign complexes, namely the assignment of formal and semantic
architectonics to the constituents of a Ve, which makes it possible to represent the
static relations between the architectonic elements (structural explicative inter-
pretation) and information regarding the logical-temporal construction of such
relations (procedural explicative interpretation; Petőfi 2004a, 80). According to
Petőfi, an explicative interpretation is first-degree when any constitutive element
of the Ve can be interpreted literally, partially figurative when some constitutiveWordless: Interpreting Visual Sequence as Storytelling 163
elements must be interpreted figuratively but others can be interpreted literally,
and second-degree whenever a Ve is assigned symbolic significance, regardless
of possible first-degree or partially figurative interpretations (2004a, 80–81). In
other words, the process of assigning the architectonics “means the creation of a
world of which the text presumably speaks at the level of first [or partially figura-
tive] or second degree” (Petőfi 2004a, 100; my translation, emphasis in original).
3 T
extuality criteria and the productive role of the
interpreter
In semiotic textology, relations between the sign components and the properties
of a text can be defined in reference to the text world assigned to a Ve through its
specific architectonics as follows (see fig. 4).
– Connexity is the central aspect of the formal architectonics. The constituents
of a connex Ve are linked to each other by micro- and macrosyntactic rela-
tions (“syntactic” in a broad sense, intended as formal relations). Connex-
ity “indicates the syntactic continuity of a chain of ‘linguistic elements’ of a
vehiculum to be interpreted” (Petőfi 2004a, 101; my translation).
– Coreferentiality is “the relation between the ‘linguistic elements’ of a vehicu-
lum to be interpreted that refer to the same object and/or state of affairs that
constitutes the [text] world” (101; my translation).
– Cohesion is the central aspect of the sensus (Se) architectonics. It refers to “the
semantic-thematic relations between the constituents of a chain of ‘linguistic
elements’ of a vehiculum to be interpreted; a complete vehiculum is cohesive
in the strict sense in rare cases, but we find ‘isles with internal cohesion’ in
any vehiculum: cohesion is necessarily united with coreferentiality” (101; my
translation).
– Constringency is the central aspect of the relatum (Re) architectonics (the ref-
erential component of semantic architectonics). Constringency “indicates a
continuous and complete network of states of affairs that constitute a world
fragment; considering a world fragment as constringent (a network of states of
affairs as continuous and complete) depends primarily on the knowledge and
the assumptions and expectations of the interpreter as regards the world/s
acceptable as first- and/or second-degree interpretation in a given case, and
not on the ‘linguistic constitution’ of the vehiculum to be interpreted” (Petőfi
2004a, 100; my translation).
– Coherence is the central aspect of the acceptability of a relational semiotic
object as a complete and fulfilled text. A text “can be considered coherent if164 Angelo Piepoli
and only if the interpreter can construct a constringent world fragment as an
interpretation for that given text” (101; my translation, emphasis in original).
Petőfi clarified that “connexity is not a necessary requirement for cohesion; like-
wise, neither connexity nor cohesion are necessary requirements for coherence;
a coreferential network (even if not all the elements of a supposed coreferential
network are expressed explicitly) is, by contrast, a relevant basis for the creation
of a constringent world fragment” (2004a, 101; my translation, emphasis in orig-
inal).
As seen in figure 4, the criteria of textuality are not limited to traditional texts.
Clearly, their fulfilment in a sign complex constructed with verbal and pictorial
components, like comics, is complicated by semiotic variations between these
disparate elements combined in the same Ve, especially if their architectonic
organization occurs under different systems of rules. For example, with reference
to Nelson Goodman’s theory (1968, 130–154), in the Ve of comics texts, elements
of a medium with a notational system are combined with elements of a medium
without such a system. If the analysis of the verbal component makes it possi-
ble to clearly distinguish between formal and semantic architectonics, it is not
equally possible to treat these architectonics separately when analysing a picto-
rial component, even though it is possible to find a formal organization and rele-
vant rules of correspondence with the Se. With reference to the pictorial compo-
nent, in the most recent phase of his research, Petőfi (2004b, 189; see also Piepoli
[2010, 122]) spoke of an architectonica formalis-semantica (FS architectonics: Fo
and Se are closely linked – entangled, I would say – and mutually dependent)
in which the sensus is dominant (fS architectonics). Petőfi pointed out that the
problem of a joint analysis of various media components occurs even when all
such components have Goodman’s notational system, as in musical texts. He also
spoke of FS architectonics in the musical medium, despite the fact that musical
notation respects the requirements of a notational system: in the analysis of the
musical vehiculum, the formal aspect is dominant, yet it is not possible to clearly
distinguish semantic architectonics (Petőfi spoke in this case of Fs architectonics).
However, regardless of the semiotic media included, a sign complex is generally
acceptable as a communicate if, in a given or supposed communication situation,
it is possible to assign to it a constringent world fragment.
To achieve such an objective, a reader cannot avoid considering, in addition
to the elements physically present in the Ve, other elements that are absent but
still assumed to be a part of it. We can consider these elements, which Petőfi
(2004a, 82) called the inferential vehiculum, as resulting from the process of com-
pletion by inference, something similar to what McCloud refers to as closure. In
my opinion, we can state that, from this perspective:Wordless: Interpreting Visual Sequence as Storytelling 165
– the reader/interpreter, as well as being the receiver, is at the same time also
a producer (of the inferential vehiculum and the consequent interpretation of
the textual vehiculum and the comprehensive vehiculum);
– in the light of semiotic textology, the inferential vehiculum is more precisely
an (inferential) VeIm to which the reader/interpreter assigns relevant archi-
tectonics; and
– both the production of the inferential VeIm and the consequent interpretation
of the textual Ve and the comprehensive Ve (textual + inferential) depend on
a reader/interpreter(/producer)’s knowledge, hypotheses, preferences, and
psychophysical dispositions (factors that Petőfi [2004a, 84–86] called the
bases of interpretation, listed here in order of decreasing intersubjectivity),
as well as on their perceptual and cognitive capacities.
It is important to consider the crucial role of the inferential Ve (often less con-
spicuous in verbal texts or musical communicates, but always absolutely evident
in sign complexes such as graphic narratives) in the process of interpretation/
attribution of architectonics to what, in general, we could call a communicative
comprehensive vehiculum.
4 Towards the analysis of wordless narration
The pivotal role of the verbal text in the context (and co-text) of graphic narra-
tives is clearer when trying to detect what happens during interpretation. In the
case of a typical comics story, for example, coreference chains are configured
transversally between two media components, so that, if the verbal component is
removed (and, necessarily, also its pictorial treatment), it would become impossi-
ble to assign a constringent world fragment to the resulting vehiculum. In the case
of comics texts or any pictorial narratives with a verbal component, Petőfi spoke
of equidominantly verbal multimedial texts: regardless of the relative proportion
of the verbal medium, it “works” on a par with the other media of the vehicu-
lum (“equi-”), but its qualitative contribution to sense is prevalent (“dominantly
verbal”) – so much so that sometimes an analyst can decide to treat as a text
a pictorial narrative with only a verbal title. Although Petőfi was in the process
of abandoning this position, it remains understandable that words can strongly
influence chains of coreference between the linguistic elements of the various
media in a multimedial text. Often, the verbal vehiculum also significantly affects
pictorial linearization (sequential mental organization of the linguistic elements
that compose the vehiculum), which is a necessary condition for interpretation.166 Angelo Piepoli
This is valid for the order both of the panels in a sequence and of the pictorial
elements in a single panel.
In the case of graphic narratives without verbal components, the pivotal
role is played entirely by pictorial components. No system of rules for images is
strictly comparable to the grammar of natural languages, yet the study of pictorial
languages suggests that useful systems of rules for the recognition of both their
formal and semantic organization can be traced, making it possible to account in
detail for the acceptability of wordless comics as expressing constringent world
fragments. Thus, the criteria of textuality (pending an alternative term that does
not directly refer to the sole realm of texts) can be considered a condition of
acceptability for what Petőfi would call comics-communicates.
If a canonical system for an unambiguous representation of all relevant
factors of the formal and semantic architectonics of pictorial languages (analo-
gous to the canonical grammar conceived by Petőfi to make possible an intersub-
jective discussion on the interpretations of verbal texts) is still a work in progress,
we can resort to the traditional instrument of paraphrase in natural language. It
is impossible to detail completely all possible analysis of a pictorial communicate
without the verbal medium,2 but we can highlight some important aspects.
In the absence of words, coreferentiality chains may be created by the inter-
pretation of distinct pictorial elements as sharing reference to the same objects/
states of affairs of the relevant world fragment, a fundamental condition of the
latter’s constringency and, consequently, of the completeness and coherence of
the communicate undergoing analysis. A possible system for probing the con-
struction of reference chains by a graphic narrative’s reader emerges through cre-
ative exercises and, once again, the verbal instrument.
5 Creative exercises as a litmus test of textuality
Preparatory creative exercises for explicative interpretations are used in semiotic
textology to teach the interpretation of formal and semantic architectonics. These
exercises are also a useful instrument for investigating the knowledge, expecta-
tions, and preferences of the interpreter that can be associated with the communi-
cate undergoing interpretation. In relation to the latter, they simultaneously offer
intelligible feedback on factors that intervene in the interpretation of the analysed
vehiculum and in the process of accepting or rejecting it as suitable for expressing
a constringent world fragment.
2 Some relevant aspects are mentioned in Piepoli (2010, 136–140).Wordless: Interpreting Visual Sequence as Storytelling 167 Fig. 5: The pictorial component of “Diogenes und die bösen Buben von Korinth” by Wilhelm Busch (1959 [1862]). I modified the vehiculum of “Diogenes und die bösen Buben von Korinth” [Dio- genes and the Bad Boys of Corinth], a communicate of the Bildergeschichte type by Wilhelm Busch (1959 [1862]), by removing the verbal component (fig. 5). The resulting vehiculum was further subdivided for an exercise given to a group of third- and fourth-year primary school students in Spain. The fourteen drawings
168 Angelo Piepoli
were numbered progressively and printed horizontally in one or two rows across
three landscape-oriented sheets: 1–2–3 on the first page, 4–5–6 and 7–8–9 on two
rows on the second, and 10–11–12 and 13–14 on two rows on the third, with enough
blank space under each drawing for a caption. The students were asked to write
few lines under each panel to tell the story told in the sequence, and give their
story a title.
I reproduce two of the completed exercises.3
D. (8 years old)
The Naughty Children
1. He was a man who was vagabond and lived in a barrel.
2. There were two children who were very naughty and annoyed the vagabond man.
3. They kicked him [,] hit the barrel.
4. Then they left laughing.
5. They went to get a hose and threw water on him.
6. And they left insulting the vagabond man.
7. The two children returned and rolled the barrel back and forth.
8. They kept rolling it.
9. They did not stop rolling it
10. The clothes got hooked on the barrel of the two naughty children.
11. The naughty children rolled.
12. In the end the barrel crushed the children.
13. They looked like a carpet.
14. The vagabond man slept for an hour. The end.4
R. (9 years old)
The Barrel with Legs
1. Once upon a time there was a barrel that had legs.
2. One day a girl went to see what was happening but she did not know,
3. she called her sister to help her and then
3 I will reproduce as closely as possible the original text of each exercise in footnotes. I will use
forward slashes to indicate where the students wrapped lines, which in a detailed analysis can
have a structural relevance (considerations that should be made with the original vehiculum to
hand). In some cases, it appears that line-wrap substituted for punctuation (one of them, which
I inferred, can be found in one of the examples shown: it is text 3 by D.).
4 “D[…] (8 años) Los niños traviesos: 1. Era un hombre que/ era vagabundo y/ vivía en un barril.
2. Había dos niños/ que eran muy traviesos/ y molestaban al/ hombre vagabundo. 3. Le daban
patadas/ golpeaban el barril. 4. Después se fueron/ riendo. 5. Fueron por una/ manguera y/ le
echaron agua. 6. Y se fueron insultando al/ hombre vagabundo. 7. Volvieron los dos niños tra-
viesos/ y rodaron/ el barril para adelante y para atrás. 8. Siguieron rodán-/ dolo. 9. No pararon
de/ rodarlo 10. La ropa se enganchó/ en el barril de los dos niños/ traviesos. 11. Rodaron los
niños/ traviesos. 12. Al final los niños/ los aplastó el barril. 13. Parecieron una alfombra. 14. El
hombre vagabundo durmió/ por una hora. Fin.”Wordless: Interpreting Visual Sequence as Storytelling 169
4. And they started to wonder what would be inside and another man came.
5. And they made a hole in the barrel.
6. Then they left and the man asked himself what they were doing
7. And the girls pushed the barrel with legs downhill.
8. They pulled it over in the sand to stop it.
9. And they kept rolling the barrel with legs.
10. And they stopped it again.
11. They stopped it, the man inside was already dizzy.
12. And they crashed in the sand suddenly.
13. And the man who was inside the barrel came out.
14. Since then there has been a new man of the barrel and it will not be known who he is.5
Even a superficial analysis shows that different chains of coreferentiality were
created. Both students identified the object in the centre of drawings 1–12 and 14,
and on the left in drawing 13, as a “barrel,” yet they were not equally unanimous
in linking other elements to the same referents.
According to D., the “man” in texts 1, 2, 6, and 14 would have the same ref-
erent represented in full in drawings 13 and 6 and partially in all the other draw-
ings. Within the context of this interpretation, the feet and legs in drawings 1–5, 7,
and 14, together with other body parts in the other drawings, belong to the same
“man,” who “lived in” the “barrel” (text 1). Furthermore, D. identified the “man”
as “vagabond” (texts 1, 2, 6, and 14) – possibly because the man is barefoot, poorly
dressed, has a long beard and lives in a barrel. Regarding other referents, D. wrote
of “two children” with the same referents represented, partially or fully (except for
one of them, whose pictorial vehiculum is completely inferential in drawing 2), in
drawings 2–13 and also verbally referred to with “they” throughout the text. Here
too, we could infer that D. established a pragmatic relation between the behav-
iour found in the pictorial sequence and the qualification of the two referents as
“naughty.”
In R.’s text, a first-degree interpretation suggests that the “legs” in texts 1,
7, and 9, and the legs drawn throughout the pictorial sequence, belong to the
“barrel.” Only through a partially figurative interpretation are we able to link the
5 “R[…] (9 años) El barril con piernas: 1. Había una vez un/ barril que tenía/ piernas. 2. Un día
una niña/ fue a ver lo que/ pasaba pero no/ lo sabía, 3. llamó a su hermana/ para que le ayudase/
y entonces 4. Y se pusieron/ a pensar qué habría/ dentro y vino otro/ hombre. 5. Y le hicieron/
un hueco al/ barril. 6. Luego se fueron y/ el hombre se/ preguntó qué hacían 7. Y las niñas empu-
jaron/ el barril con/ patas cuesta abajo. 8. Lo pararon en/ la arena para/ detenerlo. 9. Y siguieron
rodando/ el barril con patas. 10. Y lo volvieron/ a parar. 11. Lo pararon,/ el hombre/ de dentro ya
estaba/ mareado. 12. Y se estrellaron/ en la arena de/ repente. 13. Y salió el hombre/ que había
dentro/ del barril. 14. Desde entonces hay un nuevo/ hombre del barril y no/ se sabrá quién es.”170 Angelo Piepoli
verbal and pictorial legs to the referent of the “man” in texts 11 and 13 (which we
can refer to with “/man1/”). Based on what R. wrote, the referent of the “man” in
his text 4 would not be /man1/ but precisely “another man” (text 4): we can refer to
it with “/man2/.” Presumably, the referent of the “man” in text 6 is /man2/, too, and
so is the referent of the man represented as standing in drawing 6. Consequently,
the head, upper body, and arm represented on the left in drawing 4 would refer to
parts of /man2/, while the legs would refer to parts of /man1/. On the other hand,
continuing to proceed by inference, the legs and hand represented in drawing
14 would refer to another referent, /man3/, according to the verbal expression
“a new man” in text 14. R. linked the other subjects represented throughout the
pictorial sequence to the same two referents and thought that they were children,
too. In contrast to D., R. specified that they are two little girls (“una niña” [a girl]
and “su hermana” [her sister] in R.’s original text), perhaps because of their hair
and dresses.
Overall, the students recalled a various range of referents. Similarly to R.,
others considered that the legs on the left in drawing 4 belonged to one referent
(“un niño” [a kid]) and the head, upper body, arm, and hand to another (“otro
niño” [another kid], “el abuelo del niño del tubo” [the grandfather of the child
of the tube], “un hombre feo” [an ugly man]). Overall, the other subjects could
represent a boy and a girl, an elf and a girl, a girl and her brother, two elves, two
boys, two girls, or generically two children; the object they handle in drawings 5
and 6 would represent a hose, a drill, or a bellows; the dark settings would refer
to night time; and so on with other pictorial and verbal elements. I have said that
this kind of creative exercise is also useful for investigating the knowledge of the
interpreters: it is worth mentioning in this respect that none of the students linked
the subject/s in the barrel to Diogenes, while some adults to whom I have shown
the same modified vehiculum recalled the Greek philosopher from Sinope as the
referent of the character in the sequence.
Despite a few problems with connexity and cohesion (possibly attributable
to the developing linguistic competence of the young interpreters/producers), the
verbal texts themselves remain readable/interpretable. This would indicate that
the young students accepted the pictorial sequence as a complete communicate,
which would be further confirmed by the fact that many of them wrote “the end”
as a conclusion to their text. By comparing all the exercises of the group of stu-
dents, it is possible to notice generally that a separation of the two media compo-
nents appears to affect the verbal text more than the pictorial sequence, consider-
ing that the verbal part often lacks some clarifying references that can be found in
the pictorial part, although it is also true that in the text we often find expressions
such that in the drawings we cannot find elements with the same direct referent
(such cases could also include the proper nouns some students assigned the char-Wordless: Interpreting Visual Sequence as Storytelling 171
Fig. 6: Manipulated versions of page 120 from Understanding Comics by Scott McCloud
(1994 [1993]). © Scott McCloud. Reprint and excerpts rights held by HarperCollins.
acters, since, until proven otherwise, there is no pictorial element in the drawing
referring directly to the chosen names).
Of course, not all the sequences of drawings presented in similar exercises
were always automatically considered pictorial communicates for the sole reason
that the students were able to associate to each drawing a portion of a coherent
verbal text. Certainly, in many cases, a narrative property was recognized in a
pictorial sequence only after the completion of the relevant exercise.
In another exercise, I scanned and digitally edited a page from McCloud’s
Understanding Comics, removing a figurative panel and every verbal element from
the page, as shown in figure 6. During a session of creative exercises within a
lesson in the applied semiotics course at the University of Macerata, I gave the
students a print of the page and asked them if they thought it could be considered
as the vehiculum of a communicate. Many replied negatively, some were doubtful,
few answered affirmatively. I asked them to convert the sequence of apparently
non-figurative drawings into a multimedial communicate (multimedial text, in
this case) using captions, filling in the box in each panel, and I allowed them to
choose the vertical position of the page they preferred. They were not required to
give it a title, and they could use the graphic space from which I had removed the
panel to write their name and other data.172 Angelo Piepoli
Two examples are shown below, one for each allowed page position.6
Version A:
“1. There was a tremendous bang.
2. There was blood everywhere …
3. I tried to figure out where those flashes came from.
4. Suddenly everything became quiet.
5. … then something like an electric shock …
6. It was the end of the world!!”7
Version B:
“1. When you’re in the throes of nothingness …
2. when you’re in the grip of confusion …
3. … and apathy …
4. … madness takes over …
5. … a murderous madness …
6. … that makes you lose your mind … then you feel like breaking everything!”8
Except for a few exercises that were not completed, the multimedial texts pro-
duced, when shown to other students, were evaluated as being acceptable texts. I
would like to underline that the creation of chains of coreferentiality in almost all
the exercises was carried out within a partially figurative or a symbolic interpre-
tation (second-degree interpretation). Another interesting result was that, after
the conclusion of the exercise, the ranks of those who believed that the pictorial
sequence alone could be considered a pictorial communicate grew, despite some
of these students making it a condition that the boxes used for the captions be
removed (implicitly inferring other pictorial elements below them).
6 I report two examples from Piepoli (2011, 464–468), albeit with a new, revised translation.
7 “1. C’è stato un botto tremendo 2. Il sangue era ovunque … 3. Cercai di capire da dove arri
vassero quei lampi 4. All’improvviso tutto si calmò 5. … poi come una scossa elettrica … 6. Era la
fine del mondo!!”
8 “1. Quando si è in preda al nulla … 2. quando si è in preda alla confusione … 3. … e all’apatia…
4. … subentra una follia … 5. … una follia omicida … 6. … che fa perdere la testa … allora viene la
voglia di spaccare tutto!”Wordless: Interpreting Visual Sequence as Storytelling 173
6 T
he need for an all-embracing analytical frame-
work and the power of linguistic competence
The use of creative exercises can help determine whether a sequence of images
fulfils a narrative function for a reader, but it cannot be fully conclusive since it
does not allow one to analyse the reception of all the signifying elements and
aspects of the pictorial component. The complexity of contemporary comics lan-
guage, for example, implies the need for an analytical framework that also takes
into account the influences of other languages and the transposition of some of
their relevant rule systems to the target language, as well as a methodology that
provides for the explicit presentation of their linguistic functioning in the ana-
lysed comic communicate.
The current unavailability of a canonical system for images does not prevent
an interpreter from accounting for these elements using a detailed paraphrase in
natural language, albeit with a lower guarantee of clarity and intersubjectivity.
At this point, a professional interpretative approach is taken, since the interpret-
ers are required not to limit themselves to using their own system of knowledge,
but to try to reconstruct the set of knowledge that is considered adequate for the
interpretation of the vehiculum in question (Petőfi 2004a, 85–86, 99–100). This
knowledge includes linguistic competence in the analysed language.
The sequence shown in figure 7 was the subject of a discussion with the stu-
dents who carried out the exercise in figure 6. The wordless page is part of a larger
comic text that, within the framework of the story, could be considered a textual
unit marked by coherence. As such, I presented it as a possible communicate
and asked the students to discuss in groups and see if they could come up with
an acceptable explanation. Apart from the widespread use of a second-degree
explicative interpretation (symbolic interpretation), almost two-thirds of the
groups spent more time telling what “was happening” in the second, fourth, and
(to a lesser extent) sixth vignette than what happened in the others (or what they
represented). All the variants were considered valid interpretations of an accept-
able communicate. After a few lessons on film language and comics language,
they repeated the same activity. This time, they were able to explain the phenom-
enon observed previously, making explicit that the use of panels without borders
by the artist Ivo Milazzo gives a sense of greater vastness of the related scenes
and emphasizes the events represented in them, and that their alternation with
the bordered panels, in combination with the changes in framing, contributes to
giving the story narrative rhythm and dramatic movement. Some students were
also able to observe that the way in which the unbordered panels alternate with
the bordered panels contributes to determining the graphic balance of the page.174 Angelo Piepoli Fig. 7: Page 85 from “Sciopero” by Giancarlo Berardi and Ivo Milazzo (1984). © Giancarlo Berardi, Ivo Milazzo.
Wordless: Interpreting Visual Sequence as Storytelling 175
This example may be useful for understanding how the acquisition of some lin-
guistic competences allowed the students to assign various functions to the alter-
nating technique of panel unbordering, including that of indicating supraseg-
mental traits so as to constitute a sort of pictorial prosody.
In conclusion, if this last example was intended to show how a refined lin-
guistic competence affects the depth and the detail of an interpretation, the exer-
cises carried out with schoolchildren tend at the same time to show that there is
no need to share the same knowledge patrimony of grammatical rules in order
to succeed in finding significance in a sequential pictorial vehiculum. Failure to
comply with some rules of framing or a conventional sequential reading of a series
of images, or even the choice to relegate variable portions of the vehiculum to
the role of vehiculum settings, can be considered secondary when a reader/inter-
preter, young as they might be, succeeds in assigning a coherent interpretation to
a sequential pictorial vehiculum.
Works cited
Berardi, Giancarlo, and Ivo Milazzo. “Sciopero.” Ken Parker #58. Milan: Editoriale CEPIM, 1984.
Busch, Wilhelm. “Diogenes und die bösen Buben von Korinth.” Werke: Historisch-kritische
Gesamtausgabe. By Busch. 1862. Hamburg: Standard-Verlag, 1959. 157–164.
Deleuze, Gilles. Cinema 1: L’immagine-movimento. 1983. Milan: Ubulibri, 1984.
Eisner, Will. Comics & Sequential Art. Tamarac: Poorhouse Press, 1985.
Goodman, Nelson. Languages of Art: An Approach to a Theory of Symbols. Indianapolis, New
York, and Kansas City: Bobbs-Merrill, 1968.
Gubern, Román. Il linguaggio dei comics. 1972. Milan: Milano Libri, 1975.
McCloud, Scott. Understanding Comics: The Invisible Art. 1993. Northampton: Kitchen Sink
Press for HarperPerennial, 1994.
Pasolini, Pier Paolo. Empirismo eretico. Milan: Garzanti, 1972.
Petőfi, János Sándor. “Alcuni aspetti della Testologia Semiotica: Modello segnico – Tipi di
interpretazione.” Sistemi segnici e loro uso nella comunicazione umana. Vol. 5. Ed. Petőfi
and Giuliana Pascucci. Macerata: Quaderni dell’Università di Macerata, 2001. 21–30.
Petőfi, János Sándor. Scrittura e interpretazione: Introduzione alla Testologia Semiotica dei testi
verbali. Rome: Carocci Editore, 2004a.
Petőfi, János Sándor. A szöveg mint komplex jel: Bevezetés a szemiotikai-textológiai szövegsze-
mléletbe. Budapest: Akadémiai Kiadó, 2004b.
Piepoli, Angelo. “Fondamenti di un approccio testologico semiotico al testo fumettistico.”
Prospettive di semiotica del testo. Ed. Luciano Vitacolonna. Lanciano: Carabba, 2010.
111–143.
Piepoli, Angelo. “Sequential Art Between Language and Textuality.” International Journal of
Comic Art 13.2 (2011): 458–472.
Töpffer, Rodolphe. “Histoire de Mr Jabot.” Bibliothèque universelle de Genève 18 (1837):
334–337.176 Angelo Piepoli Dr Angelo Piepoli received his PhD in Information and Communication Theory from the University of Macerata in 2007. He is a scholar of semiotic textology and the history and techniques of comics. He acted as a research collaborator at the former Department of Social Change, Legal Institutions, and Communication, and as a teaching assistant for courses in semiotics and applied semiotics at the former Faculty of Communication, at the University of Macerata, where he collab- orated with the linguist and semiotician János Sándor Petőfi. He has published various contributions on comics studies in journals such as Heteroglossia and IJOCA, as well as in volumes such as Il ’68 Diffuso, Prospettive di Semiotica del Testo, and Imaginaire et Idéologie du Plurilinguisme Littéraire et Numérique. He acted as editor-in-chief, programme director, and writer at Radio Università di Macerata, where he presented Splash Page!, a radio programme on comic art and publishing. He is a founding member of the ICLA Research Committee on Comics Studies and Graphic Narrative.
You can also read
