Thursday, February 15, 2018

New Paper: In defense of a “grammar” in the visual language of comics

I'm excited to announce that my new paper, "In defense of a 'grammar' in the visual language of comics" is now published in the Journal of Pragmatics. This paper provides an overview of my theory of narrative grammar, and rigorously compares it against other approaches to sequential image understanding.

Since my proposal that a "narrative grammar" operates to guide meaningful information in (visual) narratives, there have been several critiques and misunderstandings about how it works. Some approaches have also been proposed as a counterpoint. I feel all of this is healthy in the course of development of a theory and (hopefully) a broader discipline.

In this paper I address some of these concerns. I detail how my model of Visual Narrative Grammar operates and I review the empirical evidence supporting it. I then compare it in depth to the specifics and assumptions found in other models. Altogether I think it makes for a good review of the literature on sequential image understanding, and outlines what we should expect out of a scientific approach to visual narrative.

The paper is available on my Downloadable Papers page, or direct through this link (pdf).

Abstract:

Visual Language Theory (VLT) argues that the structure of drawn images is guided by similar cognitive principles as language, foremost a “narrative grammar” that guides the ways in which sequences of images convey meaning. Recent works have critiqued this linguistic orientation, such as Bateman and Wildfeuer's (2014) arguments that a grammar for sequential images is unnecessary. They assert that the notion of a grammar governing sequential images is problematic, and that the same information can be captured in a “discourse” based approach that dynamically updates meaningful information across juxtaposed images. This paper reviews these assertions, addresses their critiques about a grammar of sequential images, and then details the shortcomings of their own claims. Such discussion is directly grounded in the empirical evidence about how people comprehend sequences of images. In doing so, it reviews the assumptions and basic principles of the narrative grammar of the visual language used in comics, and it aims to demonstrate the empirical standards by which theories of comics' structure should adhere to.


Full reference:

Cohn, Neil. 2018. In defense of a "grammar" in the visual language of comics. Journal of Pragmatics. 127: 1-19

Tuesday, January 23, 2018

My friend, Martin Paczynski

It was with much surprise and a  heavy heart that I learned last week that my friend and colleague Dr. Martin Paczynski suddenly passed away. Martin and I met in 2006 when I entered graduate school at Tufts University, and he was the first graduate student working with our mentor Gina Kuperberg (I was her second). He quickly grew to be a close collaborator, a mentoring senior student, my first choice for brainstorming, and my best friend throughout graduate school. Here, I'll honor his place in the sciences and my work.

It's always a nice benefit when your closest colleagues are smarter than you, and that meant Martin's influence on me and my research is everywhere. He essentially trained me in using EEG, and helped me formulate and analyze countless studies. Though he started the program a year before me, we graduated together, which I think made it all the more special.

Though he initially studied computer science and worked in that field, Martin's graduate work at Tufts focused on the neurocognition of linguistic semantics, though he was knowledgeable in many more fields. His early work focused on aspects of animacy and event roles. He later turned to issues of inference like aspectual coercion—where we construe an additional meaning about time that isn't in a sentence, such as the sense of repeated action in sentences like "For several minutes, the cat pounced on the toy." His experiments were elegant and brilliant.

Our collaborative work on my visual language research started with my first brain study, for which Martin was the second author. After graduate school we co-authored our work on semantic roles of event building, which united our research interests. This continued until just recently, as my most recent paper again had Martin as my co-author, directly following our earlier work, almost 6 years after we left graduate school together. And it wasn't just me: he is a co-author on many many people's work from our lab, which speaks to both his generosity and insightfulness.

Authorship wasn't his only presence in my work. If you've ever seen me give a talk that mentions film, you'll see him starring in the video clips I created as examples (him walking barefoot down our grad office hallway... a frequent sight). If you look at page 85 of my book, there's Martin, shaking hands with another friend:


After graduation, Martin's interests moved away from psycholinguistics, more towards research on mindfulness, stress, and other clinical and applied aspects of neurocognition. For many years he talked about one day studying architecture and design using EEG, but hadn't implemented those ideas just yet. There seemed to be no topic that he couldn't excel at when he applied himself.

He was warm, kind, creative, funny, brilliant, and intellectually generous. I like to especially remember him with a mischievous grin, foreshadowing a comment which would inevitably be both hilarious and astute.

The sciences have lost a spark of insight in Dr. Martin Paczynski, and the world has lost a progressive and compassionate soul. I've lost that and more. Safe travels my friend.