Tuesday, October 29, 2013

Coming soon...

My first printed copy of my upcoming book arrived in the mail today!

The book looks beautiful inside and out, and it makes me even more excited in anticipation for its release very soon: December 5th in the UK and January 30th in the US, though preorders are already available.

As the release date(s) get closer I'll start posting more about the contents and what it covers. Stay tuned!

Sunday, October 27, 2013

New Paper: Beyond Speech Balloons and Thought Bubbles

I'm pleased to announce that my paper, "Beyond Speech Balloons and Thought Bubbles," (PDF) has now been published in the journal Semiotica. The paper dissects the ways in which text and image connect to each other. I argue that speech balloons and thought bubbles are actually two "surface" representations of a deeper structure where the meaning can be varied by just a few primitive variables. The overall discussion then reveals a lot about the elements involved in text/image relations, their connection to panels and other aspects of the visual language of comics.

This paper has actually been through a long journey. The first draft of this paper was actually posted on my website way back in 2003—ten years ago! I then expanded and refined it into its current form in 2009, and after pulling it from the journal where it was originally accepted (and languished for some time without being published), it finally found a published home. So, I'm glad it's now finally out!

The full paper can be accessed directly here (pdf), and is of course available with all my other papers.

Full abstract:
Speech balloons and thought bubbles are among the most recognizable visual signs of the visual language used in comics. These enclosed graphic containers provide a way in which text and image can interface with each other. However, their stereotypical meanings as representing speech or thought betray much deeper semantic richness. This paper uses these graphic signs as a platform for examining the multimodal interfaces between text and image, and details four types of interfaces that characterize the connections between modalities: Inherent, Emergent, Adjoined, and Independent relationships. Each interface facilitates different levels of multimodal integration, tempered by principles of Gestalt grouping and underlying semantic features. This process allows the possibility of creating singular cohesive units of text and image that is on par with other multimodal interfaces, such as between speech and gesture.

Cohn, Neil. 2013. Beyond speech balloons and thought bubbles: The integration of text and image. Semiotica. 2013(197): 35-63.

Tuesday, October 15, 2013

Review: Comics and Language by Hannah Miodrag

Among the many books on "comics theory" published this year, Comics and Language by Hannah Miodrag may be the most relevant for my own interests. It directly analyzes the relationship between comics and language, especially proposals that "comics are a language." There are many complex aspects of this work, and I could easily spend great lengths discussing the minutia in every chapter. I'll instead try to hit upon the biggest points in this review as I see them.

The book is divided into three sections. The first tackles the actual language in comics, often pointing out that it has been under-appreciated in research. The second section looks at claims (like my own) that sequential images constitute linguistic-like structures. The final section analyzes how the component parts of images may or may not be like languages.

Overall, I found it a curious read, being one of the more well-reasoned and better written works on comics (a rare treat), yet at the same time being deeply flawed. On the one hand, she often makes insightful and important clarifications showing a clear understanding of many of the underlying concepts she discusses. For example, she expertly applies Groensteen's theories and my own (older) theories to the analysis of several works—and does so with aptitude. Rarely does it seem that she misunderstands any of the theories of comics she discusses, and several of her critiques could have a profound impact on the field of "comics studies" at large. On the other hand, the work also contains glaringly huge problems that undercut the foundations of many of her broader arguments.

Altogether, the book is potentially an important read for a burgeoning "comics studies," but must be read with the understanding of some caveats.

Disentangling comics and language

The surface statement of the book argues against the idea that "comics are a language"—be it the graphic form itself (what I would call "visual language") or the combination of images and text. She also argues that the use of verbal language in comics is significantly under analyzed, and that analysis of the images could benefit from art historical treatments discussing style.

Like many works arguing against comics being like language (and even some arguing for this comparison), Miodrag does not entirely tease out the issues involved, even if she understands them. For example, she acknowledges and understands my idea of a "visual language" that is used in comics (and, with rare praise on her part, likes the iteration of my grammar she discusses), but does not describe how it changes the argument with regards to the comparisons of comics and language broadly. My argument is not that "comics are a language" or "comics are like a language." Rather, it's that comics are written using two processes: writing and drawing, and that the structures of both of these are similar (that is, the visual language created by drawing is similar in structure to the verbal language created by writing/speaking).

Miodrag persists instead to treat "comics" as a unified object—not a symptom of its parts (writing and drawing) embedded within a certain sociocultural context—and therefore maintains comparing "comics" and "language" throughout, as if the issue were not more complicated than that. I can see two possible reasons for why she maintains this stance. One is that the more nuanced argument is one solely made by myself, and Miodrag would, understandably, rather argue against the shortcomings of the broader field as a whole, thereby sidestepping my claims rather than seeing them as a triangulation of these issues. A second reason though could be that this orientation lies in the desire to discuss comics both as art objects (i.e., works of literature/aesthetics) as well as to discuss how they work (i.e., how the structure works). This relates to a broader point underlying the book, which I'll return to later.

Analyses and insights

The best parts of the book are when Miodrag analyzes actual comics, whether applying particular theoretical concepts to these discussions or not. In particular, she excels in her descriptions of the creative use of verbal language in a variety of comics. For example, her demonstration of Herriman's playing with the sounds and meanings of words and phrases in Krazy Kat is both insightful and compelling.

To this broader issue, Miodrag importantly points out that literary analysis of comics may not emphasize enough the skills of writing in the text of comics. She argues that the notion of "good writing" in comics often overlooks the actual artistry of wordplay in favor of lauding wider notions of storytelling and interesting plot ideas. She aims at Moore's Watchmen in particular, which is lauded as a masterpiece, yet Miodrag convincingly finds it a bit lacking in the contribution of its prose. This broader critique is certainly not made very often (though, apparently was recently... were they listening to her??), yet it is hard to deny the necessity for analysis of text in an artform that uses both verbal and visual languages. This may be my favorite insight and advocacy of scholarship she raises. A parallel argument is also raised about integrating art historical viewpoints into "comics studies," a stance with equal logic.

She also nicely points out that not all comics should be considered as "literary" or "artwork," just as not all novels are considered to reach that level. This is another rare statement of honesty in comic theory, since the defensive "comics are just as good as all that other stuff" viewpoint often casts the net too wide in claiming that "comics are art/literature," without acknowledging the wide variance of quality.

Misunderstanding language

In contrast, Miodrag is most out of water when discussing the structure and properties of language itself. Unfortunately, this orientation permeates the entire book. Here, the problem is one of paradigms: Miodrag's conception of language uses "semiotic" notions of language (structuralist, post-structuralist). Within this paradigm, Miodrag's critiques make sense and are well-reasoned. However, this view has long been rejected from the dominant thinking of how language works since the "cognitive revolution" in the 1960s (and the subsequent advances made since then), making most of her assumptions about linguistic mechanisms outdated and/or wrong.

For example, Miodrag bases her criteria for what constitutes "language" on things like the use of arbitrary and discrete signs which use minimal units. She apparently gets these criteria from a 1986 book by J.T. Mitchell (not a linguist), though these criteria actually come from a well-known paper by the linguist Charles Hockett in 1960 (uncited by Miodrag) where he proposes "design features" of language to distinguish it from the systems of communication used by animals. However, this list of features predated the cognitive revolution by many years, and failed to be updated by the insights that this paradigm shift allowed. Furthermore, many of these criteria are thrown into doubt from the research on sign language, which also appeared long after Hockett's list (and which sign language research often had to fight against).**

Thus, for a book that is centered around disentangling the claims about the structure used in comics being likened to language, it is woefully under-informed about contemporary notions about what language is and how it works. In fact, the only recent works of linguistics research that are cited (such as Pinker's Language Instinct) are surprisingly mentioned in contexts outside framing how language works. Rather, citations pointing to expertise on language rely largely on the semiotics literature (often fairly old) and other sources outside the authoritative fields that study language.

This is a large oversight, especially given that the book otherwise appears to be well-researched, logically organized, and contain genuine insights. If claims are to be made about language with any authority, then it is not unreasonable to expect that contemporary notions of language be consulted and used to address the issue. Without this, many of Miodrag's doubts of the linguistic status of elements in comics fall flat because they are grounded in a view of language that is widely accepted as outdated.

Although, to be fair, most of the works claiming "comics are language" that Miodrag argues against are also outdated and/or under-informed, coming from the same semiotic perspective. Indeed, Miodrag's larger point is to argue against the notion of "comics as language" embedded within this semiotic framework specifically. In this criticism she is correct, though not entirely for the reasons she thinks. Lacking in this though is an actual critique of comparing comics to what we now know about language. So, her argument against "comics as a structuralist type of language" rings true, but the broader question of "comics relationship to language" is ultimately not addressed. I see recognizing this distinction as crucial for getting the most benefit out of Miodrag's work, even if the book does not intend it.

Structure vs. Literature

The orientation towards semiotics as a paradigm in general is what complicates Miodrag's work the most. Pervading the whole work is tension in whether Miodrag is trying to discuss how comics work in a structural, cognitive sense—i.e. how is the medium structured and how do minds comprehend it—or how comics can create "meaning" as works of art/literature (where appropriate)—i.e. what are the techniques used to create artistry.

Miodrag appears to conflate these questions, as does much of "comics theory" with a semiotic orientation (including Groensteen, Postema, and others). This is quite apparent in her advocacy for the idea that comics be understood as a "network" rather than a sequence. The "network" idea that each panel connects with every other in a comic is most often attributed to Groensteen, and is one that I summarily rejected on the account of it being entirely cognitively unfeasible. Miodrag in fact admits the logic in my counter-argument against this position, yet defends it as a worthy idea to hang onto nonetheless because it allows a discussion of the "aesthetic" features of a comic.

Now, I do not think that Miodrag holds onto this idea irrationally. The whole of the book shows just what a quality thinker she is. Rather, I think she does it because she ultimately struggles with separating the study of comics as aesthetic art/literary objects from the basic understanding of the mechanisms that underly their comprehension, here couched in the "comics as language" comparison (or as "formal theory"). Yes, the network idea is unrealistic when viewed from a cognitive perspective concerned with comprehension. But, if the desire is to describe how comics work as "literature" or "artwork," then such basic cognitive understandings may not be necessary to discuss what is important at the level of aesthetics and "artistic message" (though I might argue that those cognitive distinctions can better inform aesthetic analysis, but that's beside the point in this case).

Miodrag does not outright advocate for this separation between cognitive understanding and artistic interpretation, though she does at least acknowledge that such ideas exist on separate planes. After reading the book though, it is clear that such a separation should be acknowledged more forcefully throughout scholarship on comics.

"Comics theory" as literary analysis should not attempt to explain how they work in any cognitive or structural sense any more than literary treatments of novels should try to explain how language works. Research on how language works has distinct fields already: linguistics, psychology, and cognitive science—and it is those fields that should be the ones to study how the visual language used in comics operates as well. "Semiotic" theories like Groensteen's (and Miodrag's and Postema's) often only confuse and muddle the waters of this distinction.

Thus, if the take home message of Comics and Language is that "comics studies" should focus on the capacity of some works to be artwork/literature and described in terms of their own characteristics—leaving behind describing their formal structure (or at least leaving it to the cognitive sciences)—then I fully agree with these conclusions, even if they take a circuitous and bumpy route to reach.



** In my own upcoming book, I directly address the "design features" of arbitrariness, minimal units, and a visual vocabulary from Hockett that Miodrag finds troubling. This review is already long enough as it is, so I felt directly addressing here how the cognitive viewpoint differs from the structuralist viewpoint would be too much. If people want these arguments though to be made on the blog, let me know and I'll cover them in subsequent posts. As always, my recommendation for learning about actual contemporary linguistic theory is Foundations of Language, by my mentor, Ray Jackendoff.

Thursday, October 10, 2013

Programming with Grawlixes

Marc van Elburg has sent along a paper he's written about "grawlixes," which are the random symbols used often in comics to omit profanity, like $#!+! You can read his essay on it here.

He follows many of the categories proposed by Mort Walker's Lexicon of Comicana, though he departs a bit from that as well. His discussion also expands from grawlixes specifically to discuss aspects of the "visual language lexicon" such as motion lines and others. He then expands the discussion beyond what is found in comics, to exploring how these signs could undergo additional rules to make them more systematic.

It's definitely an interesting read. I don't completely agree with his categorization and conclusions in their entirety, but I like the approach of systematically cataloging various graphic signs and testing how they work in various contexts. As I argued before, I think that these are the only ways you can really get at understanding how the system of the visual language used in comics works.

Edit: And curiously timed, here's an article by the linguist Ben Zimmer on the origins of grawlixes!