Thursday, December 20, 2007

Event segmentation in panels

I've been reading up lately on research related to how people segment events and their boundaries, particularly the brain areas associated with their processing through fMRI. In one study, they first showed people videos of events, then on subsequent trials asked them to identify fine-grained and coarse-grained event boundaries. In all trials, they found brain activation coinciding with the boundaries that were identified.

The results support a hypothesis that events are hierarchically organized, as fine and coarse grained responses in the passive viewing did yield differences. The brain activity in response to coarse grain event boundaries was stronger than for fine-grained boundaries, indicating modulation for hierarchical structure.

Reading this got me curious as to whether there are different cognitive effects for the representations within comics' panels for showing an event at different stages of its enaction. The "Marvel" style always pushed for people to be at an exaggerated state of the action, reflecting the event boundaries rather than their internal parts (I remember a vivid image from How to Draw Comics the Marvel Way with a drawing at all stages of a guy's punch).

Does the stage of the drawn action have an effect on cognitive reading? Since the Marvel exaggeration pushes the event to its boundary, does this mean that it may be processed easier, because it demarcates the segmentation as opposed to the fine-grained inner parts of the action? Or, would a drawn slice of an action create a boundary effect no matter what, since the more fine-grained parts are left out of the representation anyhow?

This would not be that hard an experiment to perform in fMRI, especially using comics as stimuli. The trick (as usual with experiments) would be creating sufficient stimuli that represent actions at different stages in their enaction. So, a punch would be shown in one condition at an exaggerated pose, and in another condition in a relatively unexaggerated pose. Would we find the same differences in fine vs. coarse grained processing of perception of event structure?

Monday, December 10, 2007

Some Peanuts patterns

It's the end of the semester, and as usual things are crazy. I've finally got my Peanuts experiment up and running, which means people are coming in to participate. A lot of people. The experiment lasts an hour, and between last week and this week, I'll run 24 subjects, which means I'll have lots of data to pour over during winter break.

In the meantime, I also finished coding several strips from Peanuts, and have found several interesting things. For this experiment, I culled 180 strips from the first two Complete Peanuts volumes (kindly donated by Fantagraphics), which were either silent or I altered to become silent. I then coded them all panel-by-panel. That's 720 panels, and yes, it took me all semester.

So, what did I find in my sample? Well, there is some interesting stuff...

Most of what I coded for has to do with narrative structure, or what I would call visual grammar. I'm hoping my redone terminology is transparent enough to follow here.

Out of 180 strips, 140 of them (78%) used "Establishers" to set up information in the first panel. Conversely, 123 of them (68%) finished with a "Release" where the tension of the narrative dissipates. 135 (75%) also use an "Initial" as the second panel, which initiates the actions of the strip. 112 (60%) finish the strip with a "Peak" — the height of narrative tension.

Of 180 strips, 50% (90) use the overall structure of "Establisher-Initial-Peak-Release." The next highest isn't even close, with only 13 strips using the pattern "Establisher-Initial-Initial-Peak."

The "E-I-P-R" pattern is what I think of as the canonical narrative arc (which on a macro scale resembles the traditional "narrative arc" of plotlines). All this aligns even more interestingly to coding I did of event structures for each characters' actions, but describing all that here might be a little overkill.

Just as a reminder, this is a very specific sample of strips and shouldn't be construed as making any sort of claims overall about Peanuts. Nonetheless, its fun to see what info the strips alone hold. Now I'm even more excited to see what the results of my study show about people's behavior in relation to these coded predictions.

Sunday, December 02, 2007

New Essay: Japanese Visual Language

For the first time in a long while, I've got a new essay up for download. This one discusses the visual language that underlies manga, and will be part of the Manga: The Essential Reader collection published next year by Continuum Books. Here's the abstract:

Over the past two decades, manga has exploded in readership beyond Japan, and its style has captured the interest of young artists all over. But, what exactly are the properties of this "style" beyond the surface of big eyes and "backward" reading? This paper explores the structural elements of the Japanese Visual Language (JVL) that comprises the "manga style" — ranging from looking at the “big eyes, small mouth” schema as a “standard” dialect, to examining the graphic emblems that form manga’s conventional visual vocabulary. Particular focus will be given to JVL grammar — the system that creates meaning via sequential images — and how it differs from the visual languages from other parts of the world. On the whole, manga provide an excellent forum for understanding the scope of the visual language paradigm.