Thursday, September 11, 2014

New Paper: You're a good structure, Charlie Brown

Wow, big week for new papers! I've got another one officially out now in the latest issue of Cognitive Science, "You're a Good Structure, Charlie Brown: The Distribution of Narrative Categories in Comic Strips" (pdf). This one actually reports on the first studies I did on comics during grad school, way back in 2007! (Sometimes science takes awhile to get out...).

This study examines whether the narrative categories I've proposed for sequential images use patterned roles in sequences. There are four experiments we used:

Experiment 1: People took 4 unordered panels and ordered them into a sequence
Experiment 2: We measured how long people viewed panels in sequences where two panels were reversed in order
Experiment 3: People took 4 unordered panels, ordered 3 into a sequence and deleted 1
Experiment 4: People viewed 3 panel sequences where one panel had been deleted, and guessed where it was deleted from

Across all tasks, we found complementary results for how different types of panels behaved, showing that there are certain "distributional trends" for the roles that panels play in a sequence. For example, panels that were freely chosen to be deleted were not recognized when they were missing, but panels that were not often deleted were noticed when gone.

Plus, some panels can play multiple roles in a sequence, but not all panels have this ability. This confirms that some types of panels are flexible in the role they play in a sequence, but not all panels can go in any location in a sequence. This goes against the idea that "any panel can go in any position" in a sequence and still be meaningful.

Here's the full abstract:

Cohn's (2013) theory of “Visual Narrative Grammar” argues that sequential images take on categorical roles in a narrative structure, which organizes them into hierarchic constituents analogous to the organization of syntactic categories in sentences. This theory proposes that narrative categories, like syntactic categories, can be identified through diagnostic tests that reveal tendencies for their distribution throughout a sequence. This paper describes four experiments testing these diagnostics to provide support for the validity of these narrative categories. In Experiment 1, participants reconstructed unordered panels of a comic strip into an order that makes sense. Experiment 2 measured viewing times to panels in sequences where the order of panels was reversed. In Experiment 3, participants again reconstructed strips but also deleted a panel from the sequence. Finally, in Experiment 4 participants identified where a panel had been deleted from a comic strip and rated that strip's coherence. Overall, categories had consistent distributional tendencies within experiments and complementary tendencies across experiments. These results point toward an interaction between categorical roles and a global narrative structure.


Full Reference:

Cohn, Neil. (2014). You’re a good structure, Charlie Brown: The distribution of narrative categories in comic strips. Cognitive Science, 38(7), 1317-1359. doi: 10.1111/cogs.12116

Monday, September 08, 2014

New paper: Building a better "comic theory"

I'm happy to say that I have a new paper (pdf), "Building a better "comic theory," in the latest issue of the journal Studies in Comics. In this one I critique the existing theories about "how comics are understood" and provide a framework for better research to be undertaken. Longtime readers of this blog will certainly recognize some of my advice for researchers, now presented here in a coherent fashion.

Here's the abstract in full:

Research on the understanding of ‘how comics work’ has grown tremendously over the past twenty years, with more articles and books emerging each year. Much of this research has discussed comparisons between comics and language, and/or has speculated on comics’ cognition. However, much of this research faces limitations, which hamper the seriousness of the endeavour and reflect the youth of this emerging field. This article points out these deficiencies that pervade theories about comics. These include inadequate background research, overly general and unsupportable claims, a lack of adequate evidence, and limitations for research methodologies. To address these concerns, I draw from over 50 years of research from linguistics and cognitive science to inform how the field of ‘comic theory’ can move forward. In particular, I outline two primary ways of progressing with this line of research: (1) explicit manipulation of the component parts of the structure used in comics and (2) cataloguing actual comics for various theoretically relevant phenomena. This data-driven approach is offered as a guiding vision for future works on the understanding of ‘how comics work’.



Full Reference:

Cohn, Neil. 2014. Building a better “comic theory”: Shortcomings of theoretical research on comics how to overcome them. Studies in Comics. 5(1), 57-75

Wednesday, September 03, 2014

New book on Australian Sand Narratives

Anyone who has read my new book or followed my work online for the past several years knows that I frequently talk about sand narratives created by Australian Aboriginals as a counterpoint to the structures found in the visual language of comics. I find these drawings made in sand to be extremely interesting and important for the study of sequential images and drawing in general.

In fact, learning about them has been pivotal in forming my broader ideas of visual language, which is why I included a whole chapter about them in my book.
I first learned about Australian sand narratives from my post-undergrad advisor Dan Slobin, who introduced me to David Wilkins, a linguist who had written about the sand narratives of the Arrernte. This led me to read lots of papers by anthropologist Nancy Munn about the similar systems created by the Warlpiri. I'm happy to say that seminal papers by both of these authors will appear in my upcoming edited collection on visual narratives, out in late 2015/early 2016.

However, for the past several years I've corresponded with Jennifer Green, who has actively been doing extensive research on sand narratives and how they interact with spoken and signed languages. Her work is amazing and thorough, and is wonderfully now available in a new book:


In the chapter on my book, I attempted to distill the information about sand narratives regarding the features of the graphics alone. However, as with images in comics, they rarely appear isolated, and are usually embedded within a broader system of speaking, drawing, and signing. It is these broader multimodal interactions which Jenny is most concerned.

If you are interested in how these systems work beyond the broad overview I present in my book, I highly recommend checking out Jenny's new book. It is extensive, enlightening, and is now the first comprehensive work discussing these systems. Hopefully it won't be her last.

Here's the synopsis:

"Sand stories from Central Australia are a traditional form of Aboriginal women's verbal art that incorporates speech, song, sign, gesture and drawing. Small leaves and other objects may be used to represent story characters. This detailed study of Arandic sand stories takes a multimodal approach to the analysis of the stories and shows how the expressive elements used in the stories are orchestrated together. This richly illustrated volume is essential reading for anyone interested in language and communication. It adds to the growing recognition that language encompasses much more than speech alone, and shows how important it is to consider the different semiotic resources a culture brings to its communicative tasks as an integrated whole rather than in isolation."

Thursday, August 28, 2014

An update: Talks of future past

I've got roughly a month left of my summer (yay quarter system!), and it's been a wild ride so far. I've given talks in Germany, Québec City, and Seattle, and by the time summer is over, I'll have given roughly 10 talks over three months!

To cap off this tour, I'm about to head back to Germany for a few more talks at the end of September. The first will be for the "Empirical approaches to comics" workshop in Berlin. They've recently posted their schedule online and it looks like a very fun line up!

After that I'll be speaking at some researchers' labs in other parts of Germany, before heading back to Berlin for the academic ComFor conference on comics. There I'll be giving the keynote discussing the relations between comics and the brain. (For those of you going, if you have requests for which research you want to hear about, let me know!)

Speaking of talks, I'm excited to say that my recent talk up in Seattle at Microsoft's FUSE Labs is now online! You can watch it with additional slides on their site here, but you can also just watch it below.

Finally, Happy Birthday to the "King" of comics Jack Kirby! He'd be 97 today.





Friday, August 22, 2014

"Bad anatomy" and body objectification in comics

Greg Land cover and redrawn version from www.themarysue.com
The recent uproar over the covers of Spider-Woman books by Marvel has lead to cries that they are both sexist and poorly drawn. I think that these covers, and especially the responses to them, make for a good opportunity to examine the idea of "bad anatomy" in drawings and how bodies are drawn.

Concern #1: Bodies are depicted "unrealistically" in comics

The first critique of drawings like this is that they show "bad anatomy." This critique is common of course to many comic artists—whether there are claims that their work is sexist or not. For example, Rob Liefeld is basically mocked on regular occasion for his drawing of bodies and feet to the point its become a culturally accepted meme that he's a bad artist (he's not).

From a cognitive perspective, none of these works are "bad drawings." As I argue in my recent book, The Visual Language of Comics, and in several articles, drawing is not now—nor has it ever been—about drawing "realistically." The idea that drawings should somehow mimic our perception and align with "the way things are in the world" is pure fallacy (I previously called this idea "Iconic Bias").

This can only be addressed by considering "what are drawings?" in the first place. Drawings are a collection of graphic patterns stored in the head of an "artist" who combines them to form depictions. People draw faces by combining their learned head patterns with eye, nose, and mouth patterns. Feet (including those by Liefeld) are drawn using feet patterns, as are everything that a person draws. The same is true of "cutting off" legs at the knees to imply they are behind a person—it's just a pattern stored in the head of the artist. I have several examples of these—from many well respected comic artists—throughout my book.

This has nothing to do with how things look "in real life." It is about the patterns that an artist has learned over time and the ways in which they then use them in drawings. Why do so many people feel they "can't draw"? It's because they haven't built up a "visual vocabulary" of these patterns to be proficient at drawing.

In this cognitive point of view, there is no "bad anatomy" in these works. These artists—including Liefeld—are merely using the patterns of their "visual language." And they are doing so with high "fluency." The same is true of Peanuts characters or The Simpsons, which are more cartoony representations—where's the outrage that they only have three fingers? Or giant eyes? Or have elbows that appear and disappear? The cognitive fact is that drawing is not about "accuracy" to the "way things look in the world," be it in comics, manga, airplane safety guides, or art from any culture of the world and any time in history.

So... what's the issue with these bodies then? It's an issue of societal opinions about the representations of bodies—particularly women's bodies. This is not an issue of proficiency or anatomy—they are not "bad drawings" and in fact these artists (including Liefeld) show extremely high proficiency.

Rather, it's the fact that the way that artists of the "American Visual Language" used in superhero comics draw women is offensive to some people. This is directly analogous to the way that various words in spoken languages are offensive to some people. The "visual vocabulary items" used to draw women are on par with calling women any number of distasteful words.

People must accept that these patterns are merely a part of the learned visual vocabulary within a broader "visual language" in which that artist partakes. And, of course, that visual language is embedded in a broader culture in which women most certainly are marginalized and objectified. In order for that artist to draw women differently, they would have to learn new patterns, in the same way that they would have to use different words to speak about women in less offensive ways (which I discuss in length in this post).

Note, we've seen this before. African-Americans in the early 20th century were drawn in wholly offensive ways as "tar babies." These artists were not necessarily trying to be racist, but they used a graphic pattern that was inherently racist—and they were embedded in a culture that was also inherently racist. As people realized this about the images, the culture reacted to change the visual language such that people thankfully no longer draw this way. This is directly parallel to ostracizing the various racist words for African-Americans in our spoken language.

Again, this is not an issue with "anatomy" or proficiency in drawing—it is about the sociocultural use of different patterned representations which may or may not be offensive to people.


Concern #2: The depiction of bodies in comics

This leads us to the second issue, which is that women's bodies are drawn to be objectified in comics. As it happens, we have some data about this...

Last quarter at UCSD, I taught a class on the "Cognition of Comics" where all my students did original research projects. One student, Bianca de la Garza, decided to study the sexualized depictions of both female and male bodies in 15 American superhero comics randomly selected from the 1940s through the Present. She coded bodies using a scheme that ranked the "sexiness" of whole characters and their body parts (chest, butt, face, arms, legs, hip to waist ratio, etc.) and assessed the feasibility of their poses. Her 1-7 scale ranged from a 1 being "unhealthy" both obese or undernourished (i.e., the Blob or Plastic Man), a 4 being "normal", and a 7 being grossly hypersexualized (i.e., gigantic muscles, enormous breasts, etc. to the point of disfigurement). Thus, a 5 or 6 would be a highly sexy but not disfigured depiction.

Now, going into this, we both predicted that, as is the cultural belief, women would be far more sexualized than men in their depictions. In fact, we were totally wrong. In almost every category, females and males appeared to have the same ratings of sexualization, and this relationship between genders did not change over time (though the sexualization of certain body parts did change over time).

(Note that these findings differ slightly from those in other studies, discussed below.)

Interestingly though, across all time periods, the numbers of women represented in our sample were vastly lower than men: Women were underrepresented in the books (including some where a woman was the title character!). While we did not code for the content of the stories, my suspicion is that the underrepresentation of female bodies (i.e., female characters) also correlates with women playing less significant roles in those books, which of course reflects a marginalization of them.

So, our data showed that while women's bodies may indeed be drawn in objectified and sexualized ways, the same is true of men's bodies. Furthermore, I'll go so far as to say that this is not an issue about comics, but about the depiction of bodies—regardless of sex—in media throughout our culture. How often do you see ugly people of either sex on television or movies? Did I miss the uproar about the sexism about Chris Pratt having to bulk up his body for Guardian's of the Galaxy? His shower scene was clearly a gratuitous look at his body, and Dave Batista spent that entire movie without a shirt on! (Of course, as in our sample, there was also only one woman on that team, and she wasn't the lead star).

Now, I happen to be particularly sensitive to these issues because my parents have written and published books about eating disorders and body image issues for over 30 years. They were the first to write a book for laypeople about bulimia, my mother was the first openly recovered bulimic on television, and my father wrote the first book about men's body issues (which happens to be dedicated to me).

While the objectification of women's bodies in our culture is certain true, it is also true that men's bodies have become increasingly objectified and scrutinized over the past decades. The question for comics then, is whether it's acceptable to maintain a visual language where all bodies are shown in this manner, or whether people want to push for "visual language reform" along the lines of changing American culture's visual vocabulary.



-------------------------

Addendum: Two other studies have looked at bodies in American superhero comics (citations below), and here is what my student Bianca noticed about them: Cocca finds that many women are depicted in unfeasible poses. However, her analysis largely ignores mens bodies. Healey and Johnson meanwhile found that male bodies were closer to "average" than females', which were more underweight, according to the "stats" for male and female bodies for Marvel characters by calculating their BMI. However, this study didn't look at actual drawings (just character stats), nor did it differentiate between body types—an underweight character could be malnourished and an overweight figure could be fairly muscular (a problem with the BMI in general).

If you want to explore this topic more, I encourage you to do your own corpus study!

Cocca, C. (2014). The‘Broke Back Test’: a quantitative and qualitative analysis of portrayals ofwomen in mainstream superhero comics, 1993–2013. Journal of Graphic Novels and Comics, 1-18.

Healey, Karen, and Terry Johnson. Comparative Sex-Specific Body Mass Index in the Marvel Universe and the "Real"World.

Tuesday, August 19, 2014

Violating your drawings!

Here's a fun blog post by Chris Schweizer on spotting "compositional tangents" while drawing comics. For those unfamiliar, Chris does the excellent "Crogan's Adventures" series (Crogan's Loyalty, Crogan's March, and Crogan's Vengence) which I highly recommend.

So, what are these "tangents" that Chris is talking about? He describes tangents as "when two or more lines interact in a way that insinuates a relationship between them that the artist did not intend."

These include things like when one line runs into another line (as in the example image, taken from Chris's article), or when lines all come to a single point unintentionally. He lists quite a range of them.

While it isn't true of all of them, what he's describing are essentially "illegal" line junctions of "graphic structure"—the component part of the drawing system that guides how we produce lines and shapes. The tangents that Chris discusses aren't horribly bad junctions, which I'll talk about below, but they are illegal combinations nonetheless.

In my book, I've compared the way that lines combine in drawings with each other to the way that sounds combine together in speech. For example, in English, it's illegal to combine a "t" and "g" sound at the start of a word or syllable.

Similarly, the way that lines combine in line junctions is how graphic structure creates shapes, and different junctions imply different meanings. For example, a T-Junction looks like a "T", and implies that one object is placed in front of another object. You can see some of these in the lines of the barn above, and of the interaction between the barn and the background shrubbery. The "T" shaped junction between these lines gives the meaning that the barn is in front of the background.

Certain combinations between lines thus lead to "illegal" junctions analogously to illegal sound combinations in speech. For example, the "continuous line" junction shown in the image above is weird because one line runs directly into another as a junction. You can very much mess up an image by altering its line junctions, as in the image below, which comes from my book.


I always like pointing these out, because they nicely fly in the face of the idea that you can't make an "ungrammatical" drawing. Something is clearly wrong here! These junctions are discussed at length in the excellent book, Art and Representation by John Willats.

So, most of these problematic cases reflected in Chris's article are problems with line junctions, and, as he says, they are easily remedied. The easiest solution is simply to make one thing overlap another—to create a T-Junctions—so that one thing appears to be in front of another. Go read the article!

Wednesday, August 13, 2014

Visual style and visual morphology

While I was on the University of Bremen campus in Germany last month, I stumbled on a very appropriate poster hanging in the linguistics department (right) which sponsors a nice thought experiment about the relationship between conventionalized graphic "morphology" (like motion lines, thought bubbles, or lightbulbs/gears/hearts floating above people's heads) and the visual "style" they are represented in.

Of course, what caught my eye was that it plays on the "lightbulb above the head" convention, which I call "upfixes." In my book, I've argued that upfixes form a class of "visual morphology" where some sort of object floats above a person's head to create an even larger meaning. These can be gears for thought, hearts for love, ?! for surprise, lightbulbs for inspiration, and many others.

What caught my eye here is not just that it playfully evokes an upfix by having a person holding a lightbulb, but that the act of holding it doesn't interfere with the meaning. In fact, the holding of the lightbulb somewhat allows for the fact that it's a photo to be ok.

For example, we might suspect that having a realistic photo of a person trying to evoke a drawn convention might look weird. And, of course, someone holding a lightbulb does seem a little weird, but not horribly bad. We still get the sense of the "inspiration" meaning out of it.

However, compare this to the poster when we fully get rid of the hand (left)—now resulting in a realistic lightbulb floating above his head. To me at least, this looks a little weirder, even though it's actually more "accurate" to the upfix!

My suspicion is that we're more ok with the use of visual morphology in realistic photos when they retain their "realism" using the physics of the "real" world. However, doing the same thing when the morpheme is more similar to its actual graphic equivalent is weirder (i.e., just floating), because it violates the principles of the realism? (I dunno... Maybe it would be better if the lightbulb was lit up?)

I don't think that it worked all too bad with the upfix cosplay that I stumbled on at WonderCon earlier this year with the exclamation mark, but I'm not quite sure how far we can push this... Clearly, drawn hearts replacing eyes ("eye-umlauts") are a convention to mean love. I think it'd just look strange to have hearts placed over someone's eyes in a photo, right? What about holding actual gears above someone's head?

There seems to be something interesting here about how far one can go with non-iconic elements of visual morphology and their "stylistic" realism. Perhaps this is also why superhero comics don't use morphology like eye-umlauts or upfixes: they often attempt to maintain some degree of "realism" despite their fantastical powers and those associated conventions (like motion lines, x-ray vision, etc.). This seems like a corpus study and/or an experiment just waiting to happen...


Wednesday, August 06, 2014

Page layouts with the Golden Ratio

I was recently sent a link to this blog post from back in May that discusses the "Golden Ratio" in the creation of comic pages by Bizhan Khodabandeh. He has some interesting ideas about how basic geometric shapes can underlie page composition in a way that's pleasing to the eye. He therefore advocates using the "rule of three" to divide up pages and space, coming at the issue from a very "graphic design" perspective.



I'm not sure if I'm convinced that these pages are aesthetically more appealing or easier to understand than pages that may not follow these exact ratios, but the idea is interesting nonetheless. Following such ratios may lead to some unintended consequences though... by so rigidly following the underlying shapes in the example above, it looks like the bird's legs are at an awkward angle such that the body weight wouldn't be supported by them. But, this is a very minor issue I'd think, and the ideas are definitely worth checking out.

Essentially, what the author is talking about is the relationship between External Compositional Structure (ECS)—the physical relations of panels on a page—and Internal Compositional Structure (ICS)—the organization of elements within a panel. The ideas of making these elements interact has long been a focus of both comic creators and comic theorists (Groensteen talks about this a lot).

What I find interesting here specifically is how far it pushes this interaction in treating the page as an aesthetic canvas. In research on ECS specifically, one of my students coded American superhero comics from the 1940s through the Present and argued that page layouts have been growing in their treatment of "pages as aesthetic canvases" rather than treating them like containers of "panels in a flow of information."

That is, authors are becoming more sensitive to how the page as a whole works as an aesthetic whole. The ideas in this blog post make such ideas very explicit.

Monday, July 28, 2014

Recap: Cognitive Science 2014

This past weekend I got to enjoy the annual Cognitive Science Society conference in Québec City, Canada. Unfortunately, this year's conference conflicted with ComicCon, but in the end it was definitely worth heading up to Canada than just going a few miles down the road here in San Diego.

I had several presentations throughout the weekend, but the highlight was the "Cognition of Comics" symposium on the first morning. Over the past few years, I've started collaborating on visual language research with a few other researchers, and I assembled everyone together to talk about our work.

I offered both an opening overview of my/our line of research, and at the end of the symposium I presented a wides scale view of how cognitive science issues can be studied using comics. In-between, my collaborators presented. David Wagner (University of Stavanger) and Tom Foulsham (University of Essex) both presented the work they've been doing using eye-tracking to see where people look while reading comics. Then, John Drury (Stony Brook University) presented work of ours looking at brainwaves in the comprehension of visual narratives, math, sentences, and music. This work seeks to explore whether the same brain mechanisms are involved in processing all of these domains.


Beyond the incredibly interesting talks themselves, as far as I know, this symposium is the first time multiple empirical psychology studies about the cognition of comics has ever been put together. To me, it felt like the starting point of the nascent field of visual language research—very exciting!

This feeling was only reinforced because we had ongoing discussions about research and comprehension of visual language for basically four straight days. Looks like there's going to be some great research up ahead from all of us, and a lot of work to be done! Because of this, I imagine we might look back on this conference as a seminal point in the history of this research.

The other special thing about this conference was that my mentor, Ray Jackendoff, was awarded this year's Rumelhart Prize, which is the "Nobel" of the cognitive science field. It's hard to think of a scientist who is more deserving. The award not only honors his extensive scholarly contributions to areas across cognitive science (language, music, perception, consciousness, social cognition, etc, etc.), but also his teaching and mentoring. This is also quite deserved, as I can say as one of the few individuals to graduate with a PhD with him as my primary advisor.

As part of the award, Ray gave an impassioned talk about the need and benefit of "theory" in the cognitive sciences. I recorded video for the talk, and we're going to try to get it posted online for all to see. Link to come.

Sunday, July 20, 2014

Rules of Emoji

A friend passed along this article recently which describes research by Tyler Schnoebelen that explores the rules people use when writing with emoji/emoticons. When people hear that I work on "visual language" they often think of things like this. However, Schnoebelen actually did study these things, scouring through roughly 500,000 tweets and analyzed the types of productions that people made.

For example, he noted that people tend to place emoji at the end of clauses and sentences rather than the beginning. This makes sense, because people won't want to divide up their syntactic clauses. Instead, they'd prefer to use emoji like punctuation to divide up those clauses.

There also seems to be at least some linear causality when people are trying to communicate in emoji alone. The article notes that a story told through emoji would have to go in sequence, like this:
But, rearranging the order creates a different story, possibly less comprehensible:
This type of ordering could possibly use the rules in my visual narrative grammar, but in fairly stripped down and choppy form. Scrambling the order of units though is a common technique we've used before in studying the structure of sequential images.

These are particularly interesting for me because this type of things is what people often mistake my research for. However, emoji are not actual "visual language" as is laid out by my theory. Rather, they are at the intersection of visual language with writing—a conversion of sound into graphics (a learned synesthesia). They can act like "visual gestures," supplementing or enhancing the expressions of the text. Or, they can be like a pidgin, a hybrid communicative resulting from the intersection of rules of different systems, often using combinations based on basic rules of meaning without a complex grammar.

Nevertheless, emoji are particularly interesting because they still can reveal various preferences that people have for ordering meaningful information that is likely shared across various domains. For example,  another thing discussed in the article is that people will generally show an emotional state first, then show the cause of it, as in the laughing-monkey-poo example above and this one, where sadness precedes the reason why (a broken heart):
These examples use a structure common across communicative systems, which is to express the Agent (the doer of an action) prior to the Action. In this case, the Agent is sad, and the sadness is caused by the action of a broken heart. Often, this pattern will have a Patient (a receiver of an action) as well, in a canonical Agent-Patient-Act order. This pattern occurs very frequently in both full languages (such as Subject-Object-Verb ordered grammars), and in communicative systems that lack a full grammar (such as when non-sign language speakers are asked to communicate with only gestures).

I think there could likely be some interesting work done using experiments that ask people to communicate with emoji alone. That might be a nice follow up to Schnoebelen's work. He's done the corpus analysis of looking at what people have done in naturalistic expression. An interesting next stage would be to test various rules in experiments.

So... are emoji "visual language"? Not really in any full blown sense. However, they can tell us about verbal and visual languages, and the rules of communicative systems more generally.


(Wayback machine note: A long time back I did this related post about emoticons and the brain)

Thursday, July 10, 2014

Germany workshops recap

I'm a few weeks late to post on it, but my workshops in Germany were wonderfully fun! I've spent the intervening week traveling throughout the East Coast, so I'm only now able to get to posting photos and whatnot from the trip.

It was a great time, and I greatly appreciated everyone's enthusiasm, energy, and curiosities. The workshops were especially fun since I really only planned an introductory slideshow, and then I took requests for whatever the attendees wanted to learn the remainder of the day.

My first stop was Freiburg where I had a packed house of students and faculty from lots of different fields. We focused a lot on narrative grammar and page layout (as we did later in Bremen), but also quite a bit on cross-cultural studies. Here's a group photo of those who survived the whole 8 hour workshop!


A few days later I made my way up to Bremen, where we had my second full day workshop. We again focused on narrative grammar and page layouts. But, I got to present some as-yet-unpublished theoretical work on multimodality (the paper for which I later ended up writing on the plane ride home). Alas, by the end of the 8 hours, I forgot to take a group picture, but it was quite the fun time. It also got covered by a German radio station, which you can listen to here.

I then gave my last talk as part of a series on film, which meant I had to discuss at least some ideas about how to apply my narrative grammar to film. That talk was in an amazing circular room at the University of Bremen that made me feel like I was an old Greek scholar or something (bring me a toga!). Also quite a fun experience, but only 2 hours so not nearly the marathon of the workshops.



All in all it was a fun and educational trip, definitely for me and hopefully for the people who came out to learn and work with me. So, special thanks to John Bateman, Janina Wildfeuer, and Stephan Packard for bringing me out there and for being such great and hospitable hosts, and thank you to everyone who came out to participate and listen to the talks!

If you couldn't come see me this trip to Germany, never fear! I'll be back again for a conference on "Empirical approaches to comics" in Berlin this September 19 - 20. I would of course love to do more workshops during that time, so contact me if you'd like to have me!

Monday, July 07, 2014

New article: The architecture of visual narrative comprehension

I'm happy to announce that I have a new article out in the journal Frontiers in Psychology titled "The architecture of visual narrative comprehension: the interaction of narrative structure and page layout in understanding comics."

This is a "Focused Review," so it summarizes the research that has been done on narrative structure, page layouts, and panel framing… and then tries to integrate them together! The intent is to show that each of these structures operates independently of each other, but that they all interface to create a larger whole than just the parts.


Abstract:

"How do people make sense of the sequential images in visual narratives like comics? A growing literature of recent research has suggested that this comprehension involves the interaction of multiple systems: The creation of meaning across sequential images relies on a “narrative grammar” that packages conceptual information into categorical roles organized in hierarchic constituents. These images are encapsulated into panels arranged in the layout of a physical page. Finally, how panels frame information can impact both the narrative structure and page layout. Altogether, these systems operate in parallel to construct the Gestalt whole of comprehension of this visual language found in comics."

Cohn, Neil. 2014. The architecture of visual narrative comprehension: the interaction of narrative structure and page layout in understanding comics. Frontiers in Psychology. 5:680. doi: 10.3389/fpsyg.2014.00680