Monday, July 20, 2009

(^_^) ... Emoticons and the Brain

Masahide Yuasa, Keiichi Saito,Naoki Mukawa. 2006. Emoticons convey emotions without cognition of faces: an fMRI study. Conference on Human Factors in Computing Systems. April 22-27, 2006. Montréal, Québec, Canada

Related to the previous post on a testing of McCloud's "Cartoon Identification Theory" on cartoony vs. realistic images in the brain, here's a study using fMRI (brain scans) to look at emoticons which at this point are perhaps the most simplified signs for faces we use.

Faces have been the focus of a lot of debate in cognitive neuroscience, particularly about the "face area" in the brain. One side says it's an area strictly devoted to processing human faces, the other side says that it's an "expertise" area and it activates because humans are experts at recognizing faces. It is one of the most fiery debates in cognitive neuroscience, and learned about in most all intro classes.

Amazingly, this study shows no activation of this "face area" when looking at emoticons. :-O

Using fMRI, the authors compare Asian style emoticons (non rotated) with averaged faces (photos of multiple faces that have been blended to be more "generic") that were expressing the emotions of happiness and sadness. Emoticons appeared first on their own, and in a second study embedded within sentences, while non-emoticon signs using the same characters were also used as fillers (i.e. ":O*-<").

They found that photos of faces activate both areas pertaining to emotional valence (right inferior frontal gyrus) and facial recognition (right fusiform gyrus), while emoticons only activate emotional areas but not face areas. That is, as the authors say, "Remarkably, emoticons convey emotions without cognition of faces."

This finding has very interesting consequences for understanding how brains process varying degrees of complexity in images. The implication here at least is that more simplified faces become tied more explicitly to a "symbolic" meaning as opposed to their iconic meaning of resembling what they look like. That is, more simplified images strip down the meaning to its core meaning disconnected to the iconic reference that they are framed within.

It would be interesting to see a graded approach to this — such as taking different degrees of representation from McCloud's gradient of "cartoonification" (or to use my term, "haplosis"). Are there different degrees of activation for different representations? Does activation for the fusiform gyrus all suddenly drop off at a certain level of simplification? How does this affect the debate that the fusiform gyrus is an "expertise" area rather than a face area?


Tuesday, July 14, 2009

Comic-Con 09: Action Stars!

I am excited to say that my panel at Comic-Con next week should be a fun one. I'll be on the Comics Theory panel in room 30AB on Thursday at 11:30. Here's the official description:

11:30-1:00 Comics Arts Conference Session #2: Comics Theory— Dru H. Jeffries (Concordia University) argues that Zack Snyder’s film 300 mimics the form of comics by manipulating film styles, particularly slow motion. Neil Cohn (Tufts University) presents the results of psychology experiments investigating how our minds make sense of the sequence of images in comics. David B. Olsen (St. Louis University) uses examples that include Winsor McCay, Paul Pope, and Alan Moore to demonstrate how we negotiate the rhythm of comics reading. Room 30AB

More specifically, I'll be presenting my studies on "Action Stars", which I first posted last Fall. If you took my online experiment earlier this year where you had to fill in the blank describing various strips, you'll now get to learn what that was all about. I'm really excited about this presentation, so come on out!

Tuesday, July 07, 2009

WGBH Boston: One Guest Interview

A few months back I recorded an interview with WGBH Boston that has finally come online:

They went with the "not just for kids" lines a bit, but it was a quick tantalizer into my work I thought. The most challenging thing I thought was how fast it was. We had a number of topics we wanted to hit, and only 5 minutes to do it with no editing. So, that meant everything went by very fast. Enjoy!