Wednesday, April 30, 2014

Visual Language research galore!

Can I brag for a moment? Not about me, really, but about my students!

This quarter I'm teaching a course on the "Cognition of Comics" for the UCSD Cognitive Science Department. While the classes do discuss experiments and studies that have been done related to how people understand comics, and how they differ across cultures, I wanted this class to give students the opportunity to do science rather than just learn about it.

So, the course actually revolves around students doing their own research projects related to topics in visual language. While I've tried to help guide them towards interesting projects, they've mostly come up with them on their own, and boy do they seem exciting!

We have 7 experiments that people are doing, investigating the cognition of "upfixes" (symbols that go above characters heads), how people use systematic representations in drawings, how people navigate page layouts, and even how color might affect emotion in comics.

We also have projects coding across many comics to see what the properties of various structures are. Before this class, there were less than ten studies doing this sort of corpus analysis on the structure of comics. There was a dissertation by Neff, McCloud's study of panel transitions, a few done by Charles Forceville, and two done by me. In this class alone, there are roughly 14 corpus studies!

The comparisons of populations range from looking at how various structural aspects of American Visual Language change over time in superhero comics, how structures might be similar or different between genres of Japanese manga (shoujo, shonen, sports, etc.), how structures might differ between comics done by English vs. French vs. Spanish speakers, and differences between American, Japanese, and/or Chinese comics. (among others)

These comparisons are looking at how panels frame information, the constraints on how manga use super-deformation, how "visual morphemes" and symbols are used across comics of different cultures, how schematic graphic information (like the way people draw eyes or hair) are systematically used across different authors, how page layouts are structured, and how text and images interact with each other. Plus, quite a few other topics!

Hopefully, many of these papers will find information that will be significant enough to publish, suddenly increasing what we know about the structure of different comics across the world by over 200%! It's going to be a very exciting quarter!

Tuesday, April 22, 2014

Post-talk and new Spanish article

Thanks to everyone who came out to hear my talk at WonderCon on Sunday! I got great questions and really enjoyed the discussion after the talk with people who hung around. The Con itself was pretty fun as well. As I commented with several people, it felt like ComicCon was back 20 years ago.

And, as you can see in the photo... I found someone walking around with an upfix! I think he was confused why I was so excited. Are you confused why I'm so excited? Then go check out my book, where discussion of "upfixes" (stuff floating above character's heads) receives a nice lengthy discussion!

In other news... the latest issue of Revista Exégesis is now out! This Spanish-language comics anthology has lots of good short stories, and, like all their issues, it features an article by me, translated into Spanish! This issue has a Spanish version of part of my Visual Language Manifesto, which discussed how visual language theory can inform a restructuring of the comic industry (originally written way back in 2004). Enjoy!

Wednesday, April 16, 2014

Wondercon and updates

Just a reminder that I'll be speaking at WonderCon in Anaheim this weekend! My talk on the Cognition  of Comics is going to be at 11:30 on Sunday. Hope to see you there!

Here's the description:

What happens in people's minds and brains when they read and create comics? Neil Cohn (University of California, San Diego) will present an overview of his new book, The Visual Language of Comics: An Introduction to the Structure and Cognition of Comics, which provides an extensive introduction to the cognitive science of comics comprehension. This discussion will cover the systematic components that make up unique and different panels, the grammar of sequential images and page layouts, cross-cultural differences in structure, and the newest neuroscience research on what the brain is doing while comprehending comics.
Sunday April 20, 2014 11:30am - 12:30pm 
Room 210BCD

Thursday, April 10, 2014

The Visual Language Fluency Index

One of the interesting findings throughout many of my experiments is that the comprehension of sequential images seems to be modulated by participants' "comic reading expertise." These effects are predicted by my theory of "visual language"...

If drawings and sequential images are indeed structured like language, then we should expect varying degrees of "fluency" across individuals based on their experience reading and drawing comics. Previous studies in Japan have supported this, finding that various aspects of comic understanding correlate with age and frequency of reading comics. Not only does this support my idea of "visual language," but it flies in the face of the assumptions that all (sequential) images are universally understood by everyone equally.

In order to study this type of "fluency," I created a measurement that calculates a number that can then be correlated with experimental results. In the first use of this metric, I found that brainwaves and reaction times correlated with people's fluency, and several studies since then have also found similar correlations. This study was predated in time (though not publication date) by my study of page layouts, which also found differences based on people's backgrounds, which was a precursor to the changes in the way I gathered this type of information.

I've now decided to name this metric the "Visual Language Fluency Index" (VLFI) and have decided to make resources available to anyone who might want to use it in their own experiments. Hopefully this can be helpful to anyone who is doing research or is planning to do research on sequential image comprehension.

You can now download a zip folder (direct link) from the Resources page of this site which contains a questionnaire for participants to fill out and an Excel spreadsheet to enter in this data, which will also calculate the VLFI scores. There is also a "read me" file providing documentation about the metric.

I'll make a final note as well that, although the VLFI score as it currently stands is very useful and has been proven to be a reliable predictor of comprehension in several studies, I'm not satisfied to leave it alone. Studies are already underway looking into how to improve the measurements and scale, which will hopefully make it even more reliable. Should anything change, I'll post about it here and update the files on the Resources page.