Thursday, February 28, 2013

Science and Eddie Campbell's rules of comics comprehension

In Eddie Campbell's recent article at The Comics Journal, he described several potential "rhetorical rules" that authors of comics can follow in order to make them more understandable to inexperienced readers. In a previous post, I tackled the broader issue of what factors go into limitations to understanding. In this post, I discuss what scientific research tells us about his actual stated rules.

Rule 1: All the information necessary to understand the drama of a sequence must be contained in every panel of the sequence.

This rule actually reflects a very "Western" dialect of visual language that I would claim is even stronger in European comics than American comics (though I don't have data to support that). In two separate studies (first, second) I coded comics from America and Japan for how many characters appeared in each panel (the second breaks down American books into Indy and Mainstream genres).

Overall, I found that American comics used far more panels showing multiple interacting characters than Japanese manga, which used overwhelmingly more panels of single characters or close ups. This would support that American books use more sequences following "Rule #1" than Japanese books.

This difference has an impact on comprehension. Being provided with only parts of a scene (single characters) forces you to infer the larger scene. This requires more machinery in the narrative grammar (what I call "Environmental-Conjunction"), i.e., the rules in people's heads that allows them to comprehend sequential images. Yet, this does not necessarily lead to poor comprehension. Rather, it simply reflects a different grammar along with the need for a different type of fluency. Neither is better or worse. Just different.

So, as described in my last post (and the comments), the problem isn't that sequences like this are "incomprehensible" in some "universal" sense, but rather that those that have difficulty with them either 1) lack fluency in this grammar, or 2) have a different set of patterns in their heads from being fluent in a different visual language (such as European VL vs. American VL vs. Japanese VL).

Nevertheless, for Campbell's purposes of whether this would help an inexperienced (i.e., non-fluent) reader: There is no data at this point suggesting that the "Western" way leads to easier comprehension. I wouldn't doubt that this might be the case though, because it forces less inference.

However, it is worth also considering that were an actual author of comics to change their dialect in this way, it may have an effect on experienced readers. I would bet that doing as Campbell suggests would actually have an adverse effect on the reading experience for a Japanese manga reader, and possibly for a reader of mainstream American comics. So, for an author considering "changing their dialect" to that of Campbell's, they may have to weigh these issues (and for which audience is intended).

Rule 2: Ordering of speech balloons and Rule #3: Speech balloons should follow a system that can be intuited and doesn’t need to be explained.

Campbell claims that "After reading the contents of one balloon, the eye is likely to go to the next nearest balloon, even if that balloon is in another panel and the eye has not yet taken in all the balloons in the current panel. "

Some eye-tracking studies give us insight on this...

First, one study found that balloon position did have an impact on how often people skipped over content. They found that panels were skipped if they followed a panel that had a balloon with a dense amount of text. Breaking apart that balloon into smaller balloons with less text lead to less skipping over panels. This implies that alterations to balloons can have an impact on reading behavior.

Another study compared the eye-movements of "novice" and "expert" readers as they navigated through comic pages. The inexperienced comic reader had erratic eye-movements across a page and focused much more on the text. In contrast, the experienced comic reader had a very deliberate order of reading, and focused on the images much more than the text. This implies that an experienced reader would not jump around to whatever balloons are closest, but an inexperienced reader might.

Thus, in this case Campbell's rules might, as is their aim, help an inexperienced reader.

Rule #4: Timing only exists in comics if the reader agrees to play the game.
Unless I am mistaken, this rule has to do with people who read ahead in a book so that crucial information is known before it's read in the narrative. This has little to do with the structure of the narrative, and has to do with people flipping through a book beforehand. Contrary to Campbell's claim, I don't see how it's any different with skipping ahead in a novel except that images show you content. 

I'd be curious for Campbell to expand here on just how authors should prevent readers from skipping ahead. The example he gives is fairly constrained and clever for preventing people from getting too much information by inadvertently reading ahead. How would this work for something like his example of "Magneto [showing] up surprisingly on the last page"?

It's worth mentioning here that how we interact with a comic as a physical object (flipping pages, accidentally looking ahead to the next page, etc) is different from our actual comprehension (the patterns in our heads that allow us to understand/produce sequential images). I don't think that Campbell confuses this issue, but it's a point worth remembering.

Rule #5: In a visual medium, a thing does not exist unless it is seen to exist
This rule applies to characters off-panel and indicated with the tail of a balloon, or to implied aspects of a depiction. Campbell notes in his example that we are to assume a character has his arms tied behind him because we don't see them (and it matches the context of the page), and that this is something that should be depicted. 
However, this rule applies also to the legs of the characters: No character on that page is depicted below the waist, yet Campbell doesn't have an issue about that except to say that having a panel where the whole figures shown (feet and all) would help provide good spacing (i.e., Rule #10: have a panel with feet on every page). He doesn't say necessarily that if not shown, we won't believe they have feet.
It is certainly the case that "undepicted" elements are part of the conventional grammar of panel framing. Studies of children show the ability to treat a panel as a "window" on a scene is correlated with experience reading comics. Comic industry lore also tells of interactions with indigenous people who did not read comics who wonder why figures in panels without legs "had no legs at all." Unless you fully lack some type of basic fluency in the visual language, then this shouldn't be an issue. 
Nevertheless, there is some validity in providing at least some notice of an element in a panel and not leaving too much to be inferred. People do track elements and concepts across panels. An element even subtly depicted once can then pervade inferentially across a sequence.

In contrast, another constraint on sequences aims at reducing how many elements need to be tracked across panels. Including too much information can lead to overload in working memory and can adversely impact comprehension. Thus, balancing these issues—what should or should not be shown—can be a delicate issue.
---------
So, Campbell's rules cover a wide range of issues. In most cases though, he is correct that inexperienced readers may have trouble with these issues. It is not clear though whether following such rules would help those people further comprehend sequences or if some of these alterations may have adverse ramifications on the reading experience of people who are actually fluent already. However, experiments could easily test these ideas...

1 comment:

Jones, one of the Jones boys said...

"Rule" #1 struck me as particularly odd, like insisting that a sentence can't use pronouns