Thursday, August 30, 2012

Eye-movements for comic panels vs. photos

In the recent article, "Inferring Artistic Intention in Comic Art through Viewer Gaze," the authors examined whether people's eyes are more directed to parts of comic panels than they are when looking at other types of visual phenomena (particularly photos). The aim of the study was to investigate the claim among comic artists "that the artist is able to purposefully direct the visual attention of readers through the pictorial narrative."

The authors used an eyetracking device to measure where people's eyes look when they are reading  individual comic panels (as opposed to across a whole page), as well as photographs taken by experts, amateur photographers and a robot.

They found that participants had far more directed and consistent eye movements towards specific portions of comic panels than were found for photographs, where gaze was far more general. They suggest that these findings show that comic panels direct the flow of attention of their readers. 

I am not sure that these findings fully support their goal to see if "artists purposefully direct the visual attention of readers through the pictorial narrative." This is a fairly vague hypothesis (direct visual attention to what? To the whole image? What does that mean?). For full evidence of this hypothesis, they would need to see the relationship between eye movements across a larger page layout with those in individual panels, assuming that this is what they mean by directing a reader's attention through the narrative.

What's appealing about these data though is the idea that panels—being created to be in sequence—hone a reader's attention to specific parts of panels over others. This is an important finding, and invites follow up experiments that might better explore just what portions of panels might be important or not for the comprehension of a sequence.

However, there are some limitations of their design that may make their overall conclusions a bit premature. My fear in the comparisons they make is that the compared stimuli are not equivalent or counterbalanced appropriately. For example, they use comic panels from Watchmen and an Iron Man book that likely depict figures and actions. In contrast, the photos taken by amateurs were of a variety of topics both of figures and places (online photo albums). In contrast, the robot's photos were almost entirely of static environmental information about the interior of a building. These comparisons in subject matter are not equivalent in their subject matter.

More equivalent stimuli might be able to ask: Would photo versions of panels (as in a photo novella) elicit the same types of eye movements as those in drawn panels? What if the photos also showed figures engaged in actions instead of places)? How are eye movements of comic panels different from other artwork or film shots (where all are designed, but only comics and film intentionally have a sequence)?

It seems that these would be more equivalent comparisons, otherwise it seems like comparing apples and oranges: the stimuli are totally different from each other in nature to begin with. The more important comparison shouldn't be comic panels vs. photos, it has to bear in mind the content of those images.

Full Abstract:
Comics are a compelling, though complex, visual storytelling medium. Researchers are interested in the process of comic art creation to be able to automatically tell new stories, and also, summarize videos and catalog large collections of photographs for example. A primary organizing principle used by artists to lay out the components of comic art (panels, word bubbles, objects inside each panel) is to lead the viewer's attention along a deliberate visual route that reveals the narrative. If artists are successful in leading viewer attention, then their intended visual route would be accessible through recorded viewer attention, i.e., eyetracking data. In this paper, we conduct an experiment to verify if artists are successful in their goal of leading viewer gaze. We eyetrack viewers on images taken from comic books, as well as photographs taken by experts, amateur photographers and a robot. Our data analyses show that there is increased consistency in viewer gaze for comic pictures versus photographs taken by a robot and by amateur photographers, thus confirming that comic artists do indeed direct the flow of viewer attention.

ResearchBlogging.orgJain, Eakta, Sheikh, Yaser, & Jessica Hodgins (2012). Inferring Artistic Intention in Comic Art through Viewer Gaze SAP '12 Proceedings of the ACM Symposium on Applied Perception, 55-62 : 10.1145/2338676.2338688

Tuesday, August 07, 2012

Manga research meeting

I had a great time yesterday meeting with several manga researchers in Japan at Chiba University. I got to present my work to them, and then discussed several projects by other people.

For example, my host, Jun Nakazawa from Chiba University, presented a new study he did in collaboration with an American researcher. In this study, Nakazawa-sensei showed that Japanese participants have better comprehension of sequential images—both strips, Western comics, and Japanese manga—than American participants. This continues his work on looking at how different populations of people (different ages, levels of expertise, etc.) comprehend sequential images, which I have previously discussed.

Another study was presented by a graduate student, Hiromasa Hayashi from the University of Tokyo, who I met at the Cognitive Science Society conference last week. His study examined how the length and number of motions lines affects the perception of speed an object is moving. They had the image of a ball appear to move across a screen, only to have it disappear and the participant press a button when they thought it reached another location on the screen. Faster reaction times appeared to objects with longer lines than shorter lines, and to those with more lines (5 or 8) than less lines (1 line), implying that participants viewed those balls as moving faster.

After our meeting, we went to dinner with Fusanosuke Natsume (blog), who is a former manga artist and well known "comic theorist" in Japan. Natsume-sensei is a wealth of interesting ideas and stories about manga, and is also a martial artist to boot! On Friday, I'll be going to a manga museum with him and Nakazawa-sensei, which should be great fun.

Wednesday, August 01, 2012

Back to the land of manga

It's been 9 years since I was last here, but I just arrived in Japan for a few weeks. I'm currently in Sapporo, on the island of Hokkaido, where I'm attending the conference of the Cognitive Science Society. I'll be speaking on three separate panels about the cognitive implications of differences between American comics and Japanese manga, about the grammar of sequential images, and about evidence from the brain that we organize sequential images into chunks (as opposed to linear transitions).

Later on in my trip, I'll be heading down to Tokyo, where I'll be meeting several Japanese manga researchers for the first time. I'm looking forward to connecting with these international researchers!