"Comic Books: An Evolving Multimodal Literacy"

About the Author

Taylor Quimby graduated from Keene State College in 2010 with an individualized major in Aesthetic Studies. He currently works as a board operator and afternoon host for New Hampshire Public Radio.  An avid reader of both comics and philosophy, Taylor also enjoys spending his time playing the ukulele, video games, and writing plays.  Taylor lives in Hooksett, New Hampshire, with his family.

Contents

Introduction

Defining the Comic

The Speech Balloon...

Consider the speech...

Literacy and content

Literacy events    

Other Representations

Conclusion

Works Cited

Consider the speech bubble. 

Aside from the textual content, the size of the text may indicate that dialogue is being spoken at a raised or hushed volume.  The font of the text is sometimes used to indicate accents or vocal qualities.  The color of the text may change in order to indicate emotional state of the speaker.  In addition, all of these qualities can also be conveyed by the structure of the balloon itself.  Todd Klein, who lettered for the visionary series The Sandman, was asked to create unique letter designs for a number of characters.  On his website, he describes how "Neil [the writer] had a specific idea about Delirium's style, that it represent a sort of mad variety, getting louder and softer, like something going in and out of focus. Despair just had a rough balloon edge to denote a ragged, rough voice … Destruction, when we finally met him, had an extra bold border to denote a loud, booming voice" [see Fig 2].

[Fig. 2: Excerpts from Brief Lives, a collection of issues from The Sandman, by Neil Gaiman. The characters featured from left to right are Dream, Destruction, and Delirium]

In an effort to capture sensual qualities that can’t be communicated in a strictly visual sense (and would be cumbersome to describe using language in the limited space available) comics use a multimodal combination of linguistics and visuals to represent other modes of meaning (particularly audio meaning) synaesthetically (McCloud 134).  

Perhaps the most fascinating device of comic multimodality is the panel.  “The panel,” McCloud explains, “acts as a sort of general indicator that time or space is being divided” (99).  Like speech bubbles, panels can come in any number of shapes and sizes, and those variations can effect how a comic is read.  A long, silent panel may indicate more time has passed than a shorter panel with the same content (McCloud 101).  A full-page illustration could indicate a moment of great narrative importance is occurring, influencing the reader to spend more time viewing its contents.  In this way, the panel not only dictates time as it progresses in the narrative, it dictates how much time a reader ought to spend on any given sequence.  Panels spatially change or influence the narrative of a comic.  When we process sets of visual, linguistic, and synaesthetic modes of meaning spatially in context with one another using panels, the narrative of comics is formed. This multimodal relationship between sets of information contained in panels is what Cope and Kalantzis call intertextuality.  “Intertextuality,” they explain, “draws attention to the potentially complex ways in which meanings, such as linguistic meanings, are constituted through relationships to other texts, either real or imaginary, to other text types … to other narratives, and other modes of meanings” (30).  

Even though Cope and Kalantzis are interested in multimodality in regard to pedagogy, their study inadvertently says a great deal about comics.  Describing the importance of multimodality in the context of mass media’s complex landscape, they tell us that

Reading the mass media for their linguistic meanings alone is not enough. Magazines employ vastly different visual grammars according to their social and cultural content.  A script of a sitcom would have none of the qualities of the programme if you didn’t have a "feel" for its unique gestural, audio, and visual meanings. (28)

Cope and Kalantzis argue that the world is being experienced in an increasingly multimodal fashion where meaning-making is essentially a process of integration.  Because comics, more than any other form of print literature, resemble the cultural and technological move towards multimodality, they may have a unique teaching potential.  Scott McCloud, who sees no limit to what the medium can achieve artistically and otherwise, was hired by Google in 2008 to create a comic book explaining their new web browser.   It was so popular in its friendly descriptions of typically boring technical details, George Gene Gustines of the New York Times called the move on Google’s part, “akin to hiring Paul McCartney to write a jingle.”  But if Google is at the forefront of media innovation (as some would argue), then perhaps a future where comic books play a role in education isn’t so far-fetched.