Thursday, December 20, 2007

Event segmentation in panels

I've been reading up lately on research related to how people segment events and their boundaries, particularly the brain areas associated with their processing through fMRI. In one study, they first showed people videos of events, then on subsequent trials asked them to identify fine-grained and coarse-grained event boundaries. In all trials, they found brain activation coinciding with the boundaries that were identified.

The results support a hypothesis that events are hierarchically organized, as fine and coarse grained responses in the passive viewing did yield differences. The brain activity in response to coarse grain event boundaries was stronger than for fine-grained boundaries, indicating modulation for hierarchical structure.

Reading this got me curious as to whether there are different cognitive effects for the representations within comics' panels for showing an event at different stages of its enaction. The "Marvel" style always pushed for people to be at an exaggerated state of the action, reflecting the event boundaries rather than their internal parts (I remember a vivid image from How to Draw Comics the Marvel Way with a drawing at all stages of a guy's punch).

Does the stage of the drawn action have an effect on cognitive reading? Since the Marvel exaggeration pushes the event to its boundary, does this mean that it may be processed easier, because it demarcates the segmentation as opposed to the fine-grained inner parts of the action? Or, would a drawn slice of an action create a boundary effect no matter what, since the more fine-grained parts are left out of the representation anyhow?

This would not be that hard an experiment to perform in fMRI, especially using comics as stimuli. The trick (as usual with experiments) would be creating sufficient stimuli that represent actions at different stages in their enaction. So, a punch would be shown in one condition at an exaggerated pose, and in another condition in a relatively unexaggerated pose. Would we find the same differences in fine vs. coarse grained processing of perception of event structure?

Monday, December 10, 2007

Some Peanuts patterns

It's the end of the semester, and as usual things are crazy. I've finally got my Peanuts experiment up and running, which means people are coming in to participate. A lot of people. The experiment lasts an hour, and between last week and this week, I'll run 24 subjects, which means I'll have lots of data to pour over during winter break.

In the meantime, I also finished coding several strips from Peanuts, and have found several interesting things. For this experiment, I culled 180 strips from the first two Complete Peanuts volumes (kindly donated by Fantagraphics), which were either silent or I altered to become silent. I then coded them all panel-by-panel. That's 720 panels, and yes, it took me all semester.

So, what did I find in my sample? Well, there is some interesting stuff...

Most of what I coded for has to do with narrative structure, or what I would call visual grammar. I'm hoping my redone terminology is transparent enough to follow here.

Out of 180 strips, 140 of them (78%) used "Establishers" to set up information in the first panel. Conversely, 123 of them (68%) finished with a "Release" where the tension of the narrative dissipates. 135 (75%) also use an "Initial" as the second panel, which initiates the actions of the strip. 112 (60%) finish the strip with a "Peak" — the height of narrative tension.

Of 180 strips, 50% (90) use the overall structure of "Establisher-Initial-Peak-Release." The next highest isn't even close, with only 13 strips using the pattern "Establisher-Initial-Initial-Peak."

The "E-I-P-R" pattern is what I think of as the canonical narrative arc (which on a macro scale resembles the traditional "narrative arc" of plotlines). All this aligns even more interestingly to coding I did of event structures for each characters' actions, but describing all that here might be a little overkill.

Just as a reminder, this is a very specific sample of strips and shouldn't be construed as making any sort of claims overall about Peanuts. Nonetheless, its fun to see what info the strips alone hold. Now I'm even more excited to see what the results of my study show about people's behavior in relation to these coded predictions.

Sunday, December 02, 2007

New Essay: Japanese Visual Language

For the first time in a long while, I've got a new essay up for download. This one discusses the visual language that underlies manga, and will be part of the Manga: The Essential Reader collection published next year by Continuum Books. Here's the abstract:

Over the past two decades, manga has exploded in readership beyond Japan, and its style has captured the interest of young artists all over. But, what exactly are the properties of this "style" beyond the surface of big eyes and "backward" reading? This paper explores the structural elements of the Japanese Visual Language (JVL) that comprises the "manga style" — ranging from looking at the “big eyes, small mouth” schema as a “standard” dialect, to examining the graphic emblems that form manga’s conventional visual vocabulary. Particular focus will be given to JVL grammar — the system that creates meaning via sequential images — and how it differs from the visual languages from other parts of the world. On the whole, manga provide an excellent forum for understanding the scope of the visual language paradigm.

Thursday, November 29, 2007

Big Game Poem, 2007

Ode to The Play
By Neil Cohn, November 2007

Hello stanfurd Cardinal, this Big Game is a treat
As it marks the occasion of an incomparable feat.

Twenty-five years, we celebrate this night
Of giving the sports world its most riveting sight

After all of these years, we still watch its reprise,
Shouting and screaming like it’s before our eyes.

“Amazing! Sensational! Dramatic! Heart-rending!”
Starkey summed up a most unthinkable ending.

Five laterals you say? A clock down to nil?
A band on the field, later feeling quite ill?

You thought that you’d won after that field goal,
And with four seconds left, had your eye on a Bowl.

But you can’t blame the Bears, for acting the dethroner
Of making your school look like a well-placed tromboner.

For the Bears don’t quit when you might think they’re done,
They’ll take your squib kick and fight til they’ve won.

Now twenty-five years have gone and past,
But once again at the end, Cal will hold the Axe.

Somehow it’s fitting, despite all of your cries,
For you to receive sport’s most dubious prize.

And so stanfurd, “Thank you”, in your own stupid way,
For making Cal look as good as we did in The Play.

Tuesday, November 27, 2007

Projects galore

Wow, three blog posts in three days, and I'm not even on winter break yet! Lots of stuff has been going on.

First off, my buddy Alexander alerts me to a strip over at the Comics Reporter that somehow slipped my eye when I was over there this morning. It's a good treatment on graphic fluency and why being able to understand comics involves cognitive skills — that some just don't have.

Anyhow... I'm extremely close to being able to start running subjects for my Peanuts comics experiment. This has taken me waaaaay longer to get up and running than I expected, so it'll be good to finally start getting some data. In hopes of not running into this problem again, I'm already planning to set up my next experiment, which will use newly created sequences made of various panels from Peanuts strips. I'll talk more about it once things get closer.

Additionally, there's a good likelihood I'll have a major new paper posted online by next week. Unfortunately, its not the paper on page layout (which is still undergoing editing... hopefully over winter break), but it is another paper I did over the summer that's due to be published in a collection next year.

Oh, and I have to do a book review for a class I'm in, so it looks like my critique of Groensteen's System of Comics is finally going to be finished sometime soon.

Monday, November 26, 2007

Reruns: Fight the Comic Aristocracy

I have a "rerun" of my older article Fight the Comic Aristocracy over at Sequart. It deals with the the "aristocratic" structures that are in the comic industry and the democratizing force that the notion of "visual language" can have in contrast to it.

I used the term "aristocratic" there pretty much to stand in for "Bourgeoisie" or "elitist", but in a somewhat broader more abstract sense. In retrospect it does admittedly sound a bit hokey, but I couldn't really think of something better. Ah well.

When it was originally posted over at (the-then-) Comixpedia, they broke it up into two separate articles, while here I've retained it in its intentional one big piece. I've also cleaned it up a bit and junked a few parts that I thought were clunky. So, if you didn't get enough when it came out a couple years ago, enjoy it once again!

Sunday, November 25, 2007

Eye-movements reading comic pages

Omori, Takahide, Taku Ishii, and Keiko Kurata. 2004. Eye catchers in comics: Controlling eye movements in reading pictorial and textual media. In 28th International Congress of Psychology. Beijing, China.

A team of Japanese researchers perform two experiments examining eye-movements across comic pages to show that both page layout and balloon placement factor into how readily comic pages are read.

They found that, for an average of 8.5 panels per page, there are an average of 20.3 fixations. Most of their study focuses on panels that were skipped over for one reason or another, and examining modifications made to see whether they would still be skipped over.

There were two major changes that showed significant effects in decreasing the rates that they were skipped: balloon position and panel layout.

The first factor in skipping is if a panel is followed by another panel with dense text. They altered the "dense balloon panels" by distancing the balloon further away from the preceding panel. This change resulted in a significant reduction in the times that the preceding panel was skipped.

The other major factor was when panels were vertically stacked next to a long adjacent panel (what I call "blockage"). The lower panel was often skipped so the reading follows the horizontal path. When altering these layouts to make the panels horizontally arranged, the rate of skipping decreased. However, this phenomenon was only observed in a couple of scenarios (6 instances) and they don't mention how many of these skips lead to going back and rereading the skipped panel. They also don't state how many times "blockage" occured and didn't lead to skips.

Slight decreases in skipping were shown for moving characters' positions within a panel, though not to high percentages (significance is not shown).

Additionally, a recognition task asking whether various panels were or were not in the comic showed significant increases in accuracy for the modified versions. No differences were shown in accuracy of reading comprehension for the story.

While they state that their participants all had comic reading experience, I wonder the degree of "comic fluency" that they have. The desire to jump towards panels with dense text insinuates a focus more on text than on the visuals, which was characteristic of a naive comic reader's eye-movements compared with an expert reader in Nakazawa's eye-tracking study.

Further, this study supports an idea that "blockage" situations are harder to process (evidenced by the skipped panels). However, I have empirical evidence from my own experiments on page layout (to be posted soon hopefully) that following the vertical path of panels is the prefered reading path, and that preference for it does depend at least partially on expertise in comic reading. Also, their studies used only the results from 25 subjects (half seeing modified versions half not), whereas mine used 145, so looking at a broader populace would be good here.

Hypothetically, I could tackle this issue myself, since the lab next door to mine has an eye-tracker at my disposal. We'll see... I have a few other things on my plate right now.

Monday, November 12, 2007

Capturing vs. Generating Comics

A good friend of mine who works for the company that produces Second Life sends over this link about using the Comic Life software with Second Life screenshots. I've expressed my displeasure with Comic Life before, but I haven't really thought about comic creation of video game clips before.

Something about it rubs me the wrong way... And I think its the same issue that I have with why "photo comics" don't work, and why only some CGI comics feel comfortable.

The problem is that they don't come from some sort of conceptual basis. They are just capturing events in the (virtual-)world and the displaying them in segmented parts. But, contrary to regular comic sequences, they aren't produced to be sequential.

(This may be the same reason that pin-up/cover artists don't always translate to being good "storytellers": they are used to drawing single images, not sequences. Or: they have good visual vocab, not so good visual grammar.)

The capturing vs. generating sequences makes a huge difference, since in one you are actively setting out to express concepts visually, and the other you're just collecting whatever actions might be given to you. In fact, I'm guessing that the CGI comics that read the best (and there are some good ones) are the ones that were first drawn in thumbnails or layouts. The actual "visual language production" occurs at the thumbnail stage. The rest is all just refinement. These "event capturing" comics bypass the stage where visual grammar is deployed.

Of course, the grammar could be deployed "online" in the processs of that CGI comic being created, but I doubt most who do this have much capacity for visual grammar in the first place. They use it thinking that it is an alternative to having graphic fluency, only their non-fluency then shows through in CGI instead of poor drawings.

In many ways this issue is similar to an Internalist vs. Externalist debate in linguistics/philosophy as to where meaning comes from. Traditional philosophy/linguistics (and I think? a commonsense view of meaning?) has held that meaning of sentences is derived from the truth value of how that sentence relates to the "real" world. The Internalist side (including my advisor) says that those meanings only connect to concepts in a person's head, regardless of their truth value to the world.

"Capturing of events" for comics is much like the Externalist viewpoint — sequences of images are depictions of some form of events, and it doesn't matter how they get depicted. The Internalist side would be the opposite: Sequences of images are derived from the conceptual expression of a human mind, and reflect the fluency of that mind.

Sunday, November 04, 2007

Is this a Comic?

Patric Lewandowski joins the club of discussing the definition of "comic" with a new column over at Comixtalk. He has yet to mention my split between comics and visual language, but did use the magic VL words, so perhaps he's on his way there? Seems to be the start of a potentially interesting treatise at least, and I look forward to seeing where he's going with it.

At the very least, I'm glad Comixtalk has some other people writing about formalist-ish topics, since I'm far too busy to write things these days.

Sunday, October 28, 2007

Graphic non-fiction links galore!

Wow, all of a sudden I have a ton of great links to share, all about graphic non-fiction. And sometimes you find them in the darndest of places...

Last night I was hanging out in a pub on the MIT campus and happened upon Joost Bonsen, the creator of an interesting new book called Howtoons that aims to get kids excited about engineering. It has all sorts of projects for things that people can make at home with commonly found items. There is also a Howtoons website featuring lots more online.

Also there was Joost's friend Drew, who pointed out to me that the latest edition of Nature features the graphic "Adventures in Synthetic Biology" teaching about DNA and engineering biology. It starts with some very basic concepts, and then ramps up to some very complex stuff (a pdf. is also downloadable from here). Very cool!

Finally, several months ago a student named Shane Smith had emailed me about an essay he was writing about "essay comics," and, practicing what he preached, that's exactly what he's created! "Academaesthetics: How the essay and the comic can save each other" is a (long!) graphic essay arguing just what that title says (linked site leads to pdf). Definitely worth reading, especially for its comparative analysis of mine and McCloud's definitions of "comics" and as a well executed use of the medium it's advocating.

Go now!

Tuesday, October 16, 2007

Ambiguous signage in words and pictures

Things have been crazy busy lately, so blogging has been a bit sparse. However, here's a few signs I found around the Boston area that struck me as interesting for their ambiguity. First off...



This sign posted outside a fence to a lumber-yard features a line I've noticed more and more around here, which is "Police Take Notice." What initially amused me about the sign, and this phrase in particular, is that this can be read in two ways.

The first, what I assume is the intended way, says something along the lines of "No Tresspassing. Police will notice if you do."

The other interpretation is directed at the police: "No Tresspassing. Hey Police, that means you! Stay out!" I just pictured some guy with no shirt, overalls, and a shotgun looking out at his lumber hoping no cops come around.

Here's the other sign, from a street post:



This one is clearly trying to prevent domestic abuse, fairly clearly stated by the text. However, the image can be a little ambiguous. Given that only two hands are shown without connection to other bodies, it is technically unclear whether they belong to the same or different people. The different readings give very different interpretations.

Again, starting with the intended view, they belong to different people. The fist of one person is being stopped by another person — the illustrated stopping of the fist of domestic violence.

However, if you perceive the two hands as belonging to the same person, it seems like the common gesture of one person grinding or punching their fist against their own open hand as if itching for a fight. This gets just the opposite meaning, since it implies a desire/threat for violence.

Sunday, October 07, 2007

Panoramic Comic

Via the TalkAboutComics blog comes a link to this very cool comic where the entire story is depicted using successive panoramic viewpoints. It's a very interesting use of the Infinite Canvas. Narratively, most of the comic is what I'd call "passive/negative entities" — just a non-active cityscape — though it oscillates with scattered glimpses of the character in the apartment (though does not show him doing a sustained activity. The text is primarily dominant, but the panoramic certain gives a narrative feel that feels fairly unique. Check it out!

Wednesday, October 03, 2007

Visual language in society

As I've mentioned before, I think that the meshing of the notion of "comics" with sequential images (a simple glossing of what I call visual language) actually hurts the perception of the graphic form by pigeon-holing it into a specific cultural context.

If this form of communication was actually used like a language there would be no reason people would call it "comics," and I certainly wouldn't have to be arguing that it is literally a language. Its recognition as a language would be self-evident from its usage.

This is why I always correct people who claim that we have a visual culture, or that people these days have a vast visual literacy. They have a familiarity with technological or cinematic representations, certainly. But, when 2/3 of America can likely not draw a coherent narrative sequence? Visual fluency, I doubt they have.

Truly, the need to argue for a "visual language" only comes out of a society where such usage is exceptional, not the norm. You can't have a culture where people claim visual communication is vast and prevalent, while at the same time have books arguing for increased usage of it.

Can you imagine books arguing for why people should be using language more? It wouldn't happen, because language is so prevalent and pervasive in society that to do so would be boring.

That's the extant of visual language I'd like to see — where someday people will look back on my writings and think its bizarre that someone had to argue for someone to even need to make the argument that its language.

Saturday, September 22, 2007

Status report

School is back in full swing, which means my time to devote to other projects (like blogging) has received a decimating blast. I'm still planning to post my research project on page layouts sometime soon — I just need to find the time to make some edits at this point.

On the good side though is that I have multiple projects in the works. My experiment using Peanuts strips is well underway in the coding and preparation of stimuli, and should hopefully be rolling with subjects in a month or so.

I'm also working out a coding schema to analyze the various donated books I've got on my shelves. Hopefully that will tell us some interesting things about the ways different populations encode information in their comics. Most excitingly, I have some students interested in working on these projects with me, which means the potential to get a whole lot more done that isn't just reliant on my time constraints. (yay!)

And, having just experienced a rather boring summer rehabbing my hip from surgery, I'm now trying to plan ahead to next year by designing a "Cognition of Comics" course. We'll see if it gets picked up for the summer school, but if so, it should be a lot of fun.

Also in the news, I'm now slated to be a fascilitator at the VizThink Conference in San Francisco in January. There's an interesting and diverse line-up of speakers, so it should be quite the event.

Oh, and I have a meeting with Noam Chomsky in a week. That should be interesting.

Wednesday, September 12, 2007

"Comics" needs less nouns and more adjectives

In the comments of a previous post, I started riffing about the different uses of the terms "graphic novel" and "comics", and what might be a better terminology. It seems like people are constantly looking for that upgrade in terms so "comics" won't be looked at disparagingly. But, as I discussed in this essay, just giving a new term won't necessarily solve the problems — what you really need is a whole new network of associations that is brought about by the new term.

We kind of have this with "graphic novel." For some, it insinuates something different than "comics" — it's long-form, non-pamphlet, non-mainstream genre of a serious topic, etc. It evokes a different subculture and literary movement.

But, for others, its just an upscale synonym for "comics", and many who see that status difference want to capitalize on it by bootstrapping "comics" into it. Companies like Marvel and DC don't give a damn about the alternative movement of "graphic novels", but they do see the term as a way in which they can give their products a respectable label. (and, I'd guess, this is the way that most outside the comics community view it)

As I've said before, I see "comics" and "graphic novels" both as simply social contexts in which a "visual language" (of sequential images) is written. This visual language is used in different avenues, the same way that we use English to write "articles" versus "novels" — such is the (potential) difference between "graphic novel" and "comics."

However, I think perhaps this whole terminology game has been played wrongly. If you want to get across this different viewpoint — which truly does give an alternative network of ideas — then what we don't need is an alternative term to talk about different works of "sequential images." Any time that a new term is created it will just be a synonym for "comics" with a little different flavor, be it graphic novels, comix, sequart, or strip lit.

Really what we need is not a noun, but an all purpose adjective. And, I think that adjective should be common parlance — not something new that is made up. Personally, I like "graphic", since this visual language is inherently graphic representation. So, while "graphic novels" might stand, instead of "comics", etc. you get:

Graphic books
Graphic stories
Graphic essays
Graphic fiction
Graphic non-fiction
Graphic humor
Graphic short story
Graphic short
etc.

Rather than trying to identify both medium and form wrapped up into one term (and thus also subculture, etc), you just get an overarching description of the manner in which that form is written (graphically, instead of just text). Not only does this fix the terminology issues, but it also puts these things on equal footing to text. It's not "comics" vs. "books": "graphic books" are just another type of book.

Edit: As noted by Eddie Campbell in the comments... this view well meshes with the use of "author" as the person creating this "graphic book." There's really no need to use some separate term like "cartooonist," especially if, as many have said, really all we're doing is "writing in pictures." The less sepratism we have in our vocabulary, the more integrated this visual language will become in society.

Video of Ray Jackendoff

It's not quite topical to the things I usually talk about on this blog, but my advisor recently gave a talk at Google that has now been posted to video:

Jackendoff's talk on "The Peculiar Logic of Value" centers on how humans conceptualize systems of value. He hypothesizes that value is conceptualized as an abstract property attributed to objects, persons, and actions. There are several distinct types of value, including Affective value (does it feel good or bad?); Utility (is it good for me?); Prowess (is so-and-so good at doing such-and-such); Normative value (is it good of so-and-so to do such-and-such?); Personal Normative value (is so-and-so a good person?); and Esteem (does so-and-so have a good reputation?). Each of these kinds of value plays a different role in the ecology of the value system.


Monday, September 10, 2007

ACLU comi... I mean "graphic novel"!

Apparently the ACLU has an "online graphic novel" titled Defenders of Freedom up at their site. I find their use of wording interesting. The piece itself states, "We are not trying to disguise a civics lesson in a comic book" — though their tagline calls it their "first graphic novel" (apparently more will follow?).

This seems like another instance of "graphic novel" being used as an upscale synonym for "comics" — without regard for format (it's on the web!) — as opposed to using it to denote a separate categorical frame/artitic movement. The quote in the piece bears this out, since "comic book" is used fairly negatively here (and straight-up ties it to the notion of superheroes), when the work is obviously done in the "comic medium."

Here again a notion of a "visual language" would be useful. What the ACLU is trying to say (I think) is that they want to communicate this valuable information in a graphic form that is accessible (visual language), but they don't want it to have the stigma of "comics" (the social construct associated to superheroes, etc.) biasing people's opinions of it.

Wednesday, August 29, 2007

Video: Visual Grammar

The conference proceedings for the Visual and Iconic Languages Conference from a few weeks ago are now posted online. On the site you can download the slides from my talk, and they've also posted full video of the conference proceedings. I've embedded my talk below, and also to my site.

I highly recommend watching this video of if you are at all interested in my overall theories or in how sequences of images communicate. I describe what exactly I mean by "visual language" fairly clearly, and why it is different from "comics."

Most of the talk though is essentially a snippet of my developing theory of visual grammar — how sequences of images communicate — including my arguments for why panel transitions don't work and my alternative model. I don't plan to post an essay of this work online for awhile (I've been tinkering with it for four years), so this is the best place to find these ideas.

My talk runs for the first 45 minutes of this video, followed by 15 minutes of questions (the second hour is someone else). If you download the slides (large pdf) to flip through at the same time, they might be clearer than on the screen. Enjoy!

Monday, August 27, 2007

Trouble with categories in "Comics are not Literature"

Listening to the downloadable mp3 of the "Comics are not Literature" panel from Comic-Con is fascinating as a demonstration for how different people's categories are relating to the definitions of "comics," "literature," or even "reading" it seems. (all of the issues seem resolvable by accepting the notion of a visual language)

As I've been claiming for awhile, "comics" for these people seems to really be bound to the genre(s), so the notion of that as literature would indeed seem troubling. It also seems rather ridiculous in general for anyone to make a blanket statement about the notion that all comics are literature, as opposed to literature being something that individual comics can attain. Is this just a terminological straw man? Perhaps this is just a problem created by using "comics" as a singular noun?

Concerning the one person's notion that comics are not "read" just seems ridiculous. Calling the process that we do to decode comic sequences just "looking" is revealing as belonging to a paradigm of thinking holding images into a subjugated position. "Perceiving" or "looking" are passive processes compared to the active "reading." I'd argue, with empirical evidence, that "reading" is indeed the closer category.

I find it also very interesting that the panel has next to no one who is actually draws comics — i.e. no one with "visual language fluency." And, for a conversation that keeps going back to formal properties of the medium, perhaps they'd have done well with someone on the panel who actually knows about that stuff?

In general, it seems like most all of these issues that they struggle with are almost wholly resolved by accepting a paradigm that acknowledges that images in sequence are literally a visual language. No more struggling with whether someone is a "cartoonist" or "writer" or "artist" — they're a "visual author", or even better, just a "writer" who writes in pictures as opposed to words. I could go on and on (and, in fact, I have).

These were exactly the issues I was addressing in two of my older articles: The "Literature" issue is essentially encapsulated by the "Comics as Art"-debate, while the issues that they're struggling with in general are instances of the limitations of the network of concepts that "comics" encapsulates where these peoples' categories are running up against troubles.

Sunday, August 26, 2007

Manga facial symbols

I've been putting the finishing touches on my first draft of a book chapter for a textbook on Japanese manga recently. Mainly, the last things I had to do was draw my examples. Also being a "comic author" is a big advantage as a scholar, since I can create my own examples when I don't want to deal with issues of copyright and permissions.

For this one, the challenge was drawing in the "manga style," which I rarely do. For better or worse, here's some symbolic faces I've whipped up:

This section on graphic symbols/conventions was one of the hardest portions of the actual paper to write. While most of us know that manga uses a ton of wacky conventions, there aren't many places referencing them outside of informal listings like wikipedia. At most, various sources mention one or two different conventions, but I couldn't find any extensive type of cataloging. (though, if anyone is aware of such a thing, please let me know)

I started trying to make a cross-cultural list like this back when I used to have the forum, but that project seems to have stagnated. This is a research project just waiting for someone to take it up (like oh so many)...

Thursday, August 23, 2007

Lessons from manga

I found an interesting discussion in this thread over The Engine discussing this quote from a PW article:

Still, if there’s a manga feel to the books, it’s not an accident. “I went into my local bookstore, and there were two shelves for the entire history of American comics and 12 shelves for manga,” Crilley said. “Whenever kids come up to me and say, ‘Look at this drawing I made,’ nine times out of 10 it is in the manga style. For this generation, comics are manga. This is the language of this generation, and I’d better learn how to speak this language or I’ll never reach them.”

Recently I've been finishing writing a book chapter for a new textbook on Japanese manga, so these sorts of comments and discussions are particularly salient for me right now. This quote is revealing about a lot of theoretical issues, beyond simply calling it a "language." Or, perhaps more accurately, I think manga shows us many of these issues, which are kindly wrapped up into this quote. Among these are:

1) It acknowledges that the system of representation and expression used in manga is different than in American comics. Underlying message: Graphic systems are not universal

2) It notes that one graphic system can influence another one (just like languages do when they have contact with each other). Underlying message: Graphic systems (or rather, human minds that produce graphic systems...) are fluid and changing

3) People who draw in one manner can adopt additional manners of drawing. Underlying message: Multilingualism in visual language!

4) Manga are extremely popular, and that popularity is tapped into by adopting its system of expression. Underlying message: Having a consistent style might increase readership (ahem... among many other factors)

5) Children are choosing the "manga style" en masse to draw in — a consistent style which is beyond the scope of a single author and belongs instead to a community. Underlying message: Children learn to draw by imitating others

Wednesday, August 22, 2007

Understanding motion lines

Friedman, Sarah L., and Marguerite B. Stevenson. 1975. Developmental Changes in the Understanding of Implied Motion in Two-dimensional Pictures. Child Development 46:773-778.

This paper reports on an experiment testing how action lines are understood by children. They compared figures with postural cues showing motion (i.e. they look like they're running) with those with motion lines, and those that have polymorphic features (i.e. repeating legs over and over to show motion). They tested preeschoolers, first graders, sixth graders, and college students.

Massive increase in the understanding of “cartoon” conventions (i.e. motion lines) occured between the ages of first grade and sixth grade. (possibly due to developmental reasons... or to increased exposure to comics?). Reliance on postural cues decreases from first grade through college. Understanding of polymorphic representation increases greatly between pre-school and first grade, then levels out. The insinuation is that conventional cues are relied on more and more as people age.


Gross, Dana, Nelson Soken, Karl S. Rosengren, Anne D. Pick, Bradford H. Pillow, and Patricia Melendez. 1991. Children’s Understanding of Action Lines and the Static Representation of Speed of Locomotion. Child Development 62:1124-1141.

A similar study also tested children’s knowledge of action lines to determine whether body posture or action lines contributed more to understanding with seven and nine year olds, and adults. The stimuli showed a running figure (one silhouetted, one photo-like), without lines, with lines trailing it, or with lines behind it. They also had a task where subjects drew their own figures running.

They found that the relationship of action lines to the meaning of locomotion is non-arbitrary, though exposure to drawings using it is necessary to understand its convention. Again, children attenuated more to postural cues than adults did. The younger children did not distinguish between action lines and background lines to the same degree as older children and adults.

Interestingly, it also asked subjects what the lines represented, showing that children contributed meaning to lines where adults did not. Many children gave visible and invisible explanations for the lines – such as “air moving” or “wind.” Adults simply accepted the lines as symbolic representations. In cartoon drawings, adults treated lines as “path-of-movement metaphors,” but for photos were treated as non-arbitrary cues for perceptual movement.

---

(Updated 8/25): These conclusions are interesting for a few reasons... one, it shows that not all graphic things are universally or transparently understood by everyone. It takes some degree of development to reach that understanding for things that we take for granted as obvious. This is different than some of the findings on speech balloons or thought bubbles.

The fact that the development happened most in pre-puberty aligns with many other developmental changes, like language acquisition. Whether that development has to do with exposure or just age, it's hard to tell.

Also, its interesting that there was a change in how they considered what the lines were — from an iconic meaning (claiming the lines are "wind") to a purely symbolic meaning. This too is consistent with developmental changes in other domains.

Thursday, August 16, 2007

The Comic Strip and Film Language

Lacassin, Francis. 1972. The Comic Strip and Film Language. Film Quarterly, 26(1), 11-23.

In this piece translated by David Kunzle, French theorist Francis Lacassin discusses the similarities between the "syntax" of film and comics, noting that they both use "shots" as their base units. For him, this includes various things, like various degrees of framing (long shot, close up, medium shot), dynamic use of what could be multipanel representations ("panning"), as well as semantic alterations, like subjective viewpoints. (I would argue that this isn't "syntax" at all... but that's a larger post).

He argues that though film and comics emerged around the same time, these techniques came first in comics — not the other way around, as is often argued — and that they may have been autonomous developments not influencing each other at all.

He writes: "It is more reasonable to suppose that comic strip and cinema have both separately drawn the elements of their respective languages from the common stock accumulated in the course of the centuries by the plastic and graphic arts." (14)

To this I would question, is it really through historical development, or is this just a reflection of the structuring of people's minds/brains?

He hypothesizes also that film and comics both accomplish their sequential meaning by use of the film theory of montage, which for Lacassin appears to cover most things that do not appear similar to real-world perception.

In his own section at the end, Kunzle criticizes Lacassin for claiming comics were invented before cinema, while those framing techniques are cited being used by authors two generations before that "birth" of comics. Kunzle then discusses the work of 1800s artists Töpffer, Doré, and Busch, noting that they used various techniques like close-ups and polymorphic representation (where one character is repeated in a single frame showing the unfolding of action), among others.

Tuesday, August 14, 2007

A Time Sex Thing!

In response to this Language Log post, my best friend John sends along this wacky revamped movie poster:



As he put it "A google image search for 'pulp sci-fi novel' turns up some great material for this title."

Indeed.

Monday, August 13, 2007

Perhaps I should avoid the slopes...

After the conference in New Mexico let out, a bunch of us went on an excursion to the top of a mountain that overlooks Albuqurque. At the top, we found this sign on the side of a building:



Given that we'd all just talked about visual signs for a day and a half, many of us photographed it and cracked jokes. While one might expect the images are there to help give aid to those who don't speak the language or to reinforce the text, but really most of them are extremely ambiguous...

The top left one looks like the skiier is walking on cracks in the earth, and the bottom middle one looks like it's telling you to hang out in the middle of the road (when the text says exactly the opposite). The top middle one seems like the skiier is dropping his skiis from the lift.

My favorite though, is the upper right. It could easily be construed that, when you're skiing and approaching someone else, you should go around them on both sides at the same time (except for its physical difficulties). I kept picturing some Looney Tunes cartoon, where someone hikes up their body to elongate their legs so they weave around an object to avoid hitting it.

If one can only read the very large text at top (since the smaller text is a bit tough to read), these "signs of safe skiing" paint an awfully strange picture of what skiing is like.

Anyone got any more good interpretations of these?

Friday, August 10, 2007

Biblio clean up

Just some housecleaning to report... I've updated the Reference Bibliography with some long-overdue entries that have been building up for quite awhile. I've also added a whole page listing all the papers I've reviewed here on the blog (also accessible in full through the Bibliography tag).

Hopefully this will make it easier for people to find things they might be interested in.

Wednesday, August 08, 2007

VaIL '07

I recently got back from my keynote talk at the Visual and Iconic Languages conference at the University of New Mexico. The conference went very well, and a lot of very interesting conversation was had in a very short (1.5 days) amount of time. People seemed to enjoy my talk on visual language grammar, so hopefully some good will come of the event. Personally, I seem to have made some good contacts for potentially promising future works.



As I've mentioned before, I've been told that my talk may be posted online. If that's the case, I'll be sure to link to it.

One of the most interesting talks was given by Alan Stillman, who is the CEO and founder of a company called Kwikpoint. They make sheets of paper filled with simple images on it, designed for communicating with people when you can't speak the language. They were originally designed for travel, though now they're being used by the military in Iraq and Afganistan.

Originally I was doubtful of how effective this sort of thing would be, but he showed a great video of soldiers' testimonials saying how these guides helped them save people's lives, both military and civilian. And, hey, who am I to argue with very tangible success like that? He was also a very nice guy, and we've talked about my maybe doing some advising for them. I certainly hope that happens, since they really seem to be making a substantial difference.

And, just cause it's a great photo, here's me and the conference organizer, Sunny, taking in some New Mexico flavors...

Sunday, August 05, 2007

Thought Bubbles, Children, and Autism

These are several articles I found particularly interesting looking at the understanding of thought bubbles, and using them to help autistic children.

Wellman, H. M., Hollander, M., & Schult, C. A. (1996). Young Children’s Understanding of Thought Bubbles and Thoughts. Child Development, 67, 768-788.

Several experiments on 3 and 4 year olds show that thought bubbles are understood at fast rates by both age groups as depicting thoughts. This is interesting, because at similar ages, other visual conventions such as speed lines are acquired over age (I'll post on these articles next maybe).

This contrasts with the findings in the Yannicopoulou study, which showed that preliterate children could not distinguish speech from thought balloons – which was not tested in these experiments. They suggest that this shows evidence for “something recognizable from our everyday understanding or experience of thoughts themselves” tapping into "theory of mind" knowledge...

Wellman, H. M., Baron-Cohen, S., Caswell, R., Gomez, J. C., Swettenham, J., Toye, E., et al. (2002). Thought-bubbles help children with autism acquire an alternative to a theory of mind. Autism, 6(4), 343-363.

Children with autism have specific difficulties understanding complex mental states in other people like thought, belief, and false belief and their effects on behavior (what are known in psychology as "Theory of mind").

These children benefit from focused teaching about thoughts, where beliefs are likened to photographs-in-the-head. Here two studies, one with seven participants and one with 10, tested a picture-in-the-head strategy for dealing with thoughts and behaviour by teaching children with autism about cartoon thought bubbles as a device for representing such mental states.

This device led children with autism to pass not only false belief tests, but also related theory of mind tests. These results confirm earlier findings of the efficacy of picture-in-the-head teaching about mental states, but go further in showing that thought-bubble training more easily extends to children’s understanding of thoughts (not just behaviour) and to enhanced performance on several transfer tasks. Thought-bubbles provide a theoretically interesting as well as especially easy and effective teaching technique.

Tuesday, July 31, 2007

The Art Perspective

A friend of mine got an advanced copy of a book on comics that will be coming out this summer. I'd prefer not to identify it by name here, but it had a great paragraph summarizing what I would call a driving viewpoint of the "Art Perspective":

"The fact that drawing style is the most immediate aspect of comics means that what you see when you look at a comic book is a particular, personal vision of its artist's vision—not what the artist's eye sees, but the way the artist's mind interprets sight. That's not unique to comics of course: it's true of any artist... Since comics are cartooned instead of conventionally drawn, though, they're more obviously distorted by the artist's vision."

Despite expressing that my interest in the Art vs. Language perspectives regarding drawing are only analytical, I have been accused of deriding the Art Perspective. I am "descriptive" insofar as I am not actively "advocating" the practice of one belief set over the other. People are free to do whatever they want in practice (and certainly, as a product of my culture, my own graphic development took the Art perspective).

However, I am most certainly condemning the Art Perspective as an ineffective paradigm for thinking about graphic creation. Here's why...

I don't believe that drawing has anything to do with "someone's vision of the world." I think drawing has to do entirely with formulating a mental storage of "structures of drawing" (i.e. "Photological" structures) that are actively outputted when the context arises.

It works just like the sounds of language. You take in the graphic patterns, store them cognitively (creating a stock of basic schemas), and output them as necessary. If you're taking in only visual perception, you're not providing your mental structures with the building blocks it needs—you have to create them on the fly, which is far harder — especially given that human's (and especially children's) minds/brains are "pattern seeking machines." The main difference is that the structures in speech are symbolic sounds, while those of drawing often resemble what they mean.

Indeed, the belief that children naturally grow their graphic "personal vision" from some pure innateness is ridiculous if you consider a perspective that compares it to other aspects of child development. While the ability to draw is innate, it's maturation is not. This "pure development" stops around puberty for most cultures, Japan being the most notable exception. What seems to be going on here? This is the apex of a learning period, like many other developmental learning periods (the most prominent being language).

Why do Japanese children overcome this drop-off in ability (and seem to have a higher proficiency at drawing than other cultures)? As I discuss in my book, I suspect it's because they 1) read a lot of comics (i.e. have mass exposure to the visual language in manga) and 2) are consisitently drawing in this visual language by imitating it. This is not "talent" or something special about Japanese people — it's purely about stimulus-response.

This is just like any other language: if you have exposure, imitate to learn, and put in the time for learning, you develop naturally. If it's just left up to "nature" to run its course without outside influence and/or consistent practice, you get marginal results.

Sunday, July 29, 2007

Comic-Con Report '07

As ever, I had a great time at Comic-Con this year, seeing lots of friends and giving a few talks. My convention floor highlights were pretty subdued I think, as my favorite was perusing the gorgeous original Alphonse Mucha prints at the vintage art dealer. I would have liked to meander the hall a bit more, but having hip surgery two months ago took a little skip out of my step. Though, I did get to saunter about with my grandfather's old pimp-like cane (the top unscrews to reveal a compass and a shaft for a flask!).

Inevitably at the Con I have some random encounters, especially since I'm originally from San Diego and have been at this Con in some professional capacity for 13 years now. Topping the list this year was a friend I'd lived with in Japan and hadn't seen for six (!) years.

My panels seemed to have gone well too, and thanks go to everyone who turned out in attendance! You all are what make the experience a good one. The Comics and Education panel was fun, with a nicely vocal exchange between audience and presenters that made the discussion far more communal. I felt like my theory talk had a lot of good energy, largely drawing off the 100+ people who turned out. Durwin Talon's presentation on color theory that immediately followed mine was fairly cool too.

A couple who heard my talk told me a fun story afterwards. They were next door at a workshop where the instructor said that layouts are read just like text, left-to-right and down, which they found to be a little boring. So, they came next door and heard me, where I presented my experimental data showing that people don't just read comics left-to-right and down! Yay science!

In any case, for all you that missed it, I'm aiming to have the paper for this talk online in a month or so. Next on the docket I'll be flying out to New Mexico to talk about visual grammar. I've heard they'll be taping the talks, so an online lecture may be coming soon...

Wednesday, July 25, 2007

Pre-literate understanding of speech balloons

Yannicopoulou, A. (2004). Visual Aspects of Written Texts: Preschoolers View Comics. L1- Educational Studies in Language and Literature, 4, 169-181.

This study assessed preschool aged children’s ability to understand various features of Carriers (i.e. thought bubbles and speech balloons), despite not being able to understand written language. Tests showed:

- 87.1% recognized angular balloons meant anger.
- 83.7% correlated a flowery border to politeness.
- 78.7% recognized increase in volume by increase in size.
- Speech vs. thought balloons were distinguished for their meaning at chance (49.7% speech/47.5% thought)
- Most poor was recognition of other languages as indicated by variation in text.

These results were fairly interesting, especially since they imply that children can recognize aspects of manner of speaking (politeness, anger, etc), yet can't differentiate plain thoughts versus speech. Part of this might relate to a general developmental trajectory though — that children don't yet have "theory of mind," the recognition that other individuals' have thoughts of their own. Preschool children are roughly at the age where this ability is developing, so the problems they had recognizing thought bubbles might be due to their lack of understanding thoughts in other people in general.

However, these data contrast with other studies in this regard that I'll be posting sometime soon.

Monday, July 23, 2007

Blasphemy!

As I've mentioned before, I'm currently designing a series of experiments that will use Peanuts strips for stimuli (kindly donated by Fantagraphics). Lately, I've been doing the arduous process of scanning them in and coding them.

Since I'm only trying to look at how sequences of images create meaning, and because the inclusion of text changes things, I've been trying to only use silent strips. Luckily, Peanuts has a lot of those. Additionally, I've also been using strips that have minimal text or could be (gasp!) manipulated to have no text.

I've been taking these strips and deleting the text, then touching up the mouths, etc. so that they work alright silently. It's been quite fun to make sure it all looks like Schulz's style. On the one hand I keep thinking "this is so cool!" and on the other I can't help but think, "Blasphemy!" for defiling the originals.

Most of the time I cut and paste from one panel to another, like putting a frown over an open mouth. In most cases, not much at all is lost in the meaning. It's usually like erasing a word balloon saying "Good Grief!" to just Charlie Brown frowning. The meaning (and humor) stays pretty much the same. I think this is actually a testament to how great Schulz's visual humor was, that I could go in and muck with things, yet the original meanings still come through.

Now... once I start swapping panels around or creating new strips out of panels from a variety of strips, then that might be another story...

(Amusingly, my father actually asked me if there were ethical issues with doing that... my advisor just thought it was pretty amusing. I'd have to think there are less ethical issues involved with retouching comic strips than experimenting on animals like the lab next door, but if I start seeing protests out front I'll reconsider).

Monday, July 16, 2007

Talk talk talk... at ComicCon '07

Just like every year, I'll be talking next week at ComicCon. This year I've got two appearances lined up (with a booksigning on Friday):

Thursday- 10:30-11:30 Comics Arts Conference Session #1: Comics in Educational Settings—Comics have long been stigmatized as a lesser medium of communication than text. However, for many years comics have been making inroads to classrooms as an effective medium for learning, from analyzing graphic novels as literature to using superheroes to teach philosophy and writing textbooks in the comic medium. Neil Cohn (Tufts University), Diana Green (Minneapolis College of Art & Design), Leonard Wong (Templeton Secondary School, Vancouver), and Danny Fingeroth (Disguised as Clark Kent) discuss the various roles comics and the comics medium can and do play in education. Room 30AB

And then my theory talk....

Friday- 2:00-3:30 Comics Arts Conference Session #8: Storytelling and Visual Language—Neil Cohn (Tufts University) reports the findings of a psychology experiment showing that readers navigate comic layouts by using systematic rules that rely on subtle cues of panel sizes and relationships while often defying the stereotypical Z-path, "left-to-right and down."...Room 30AB

This one is actually just my talk on a bigger panel, but I'm actually really looking forward to it. I did this experiment from a booth at ComicCon a couple years ago, and I'm excited to have finally analyzed the data. I've actually got cold-hard numbers that hint at how people read comic page layouts, so the talk is going to be a fun one (as if they're all not!). I'm still tinkering with the essay, but I hope to have it online not too far off.

Following this Friday talk I'll be signing at the Comic Relief booth, hawking my usual wares and looking to engage in lively conversation. Hope to see you there!

Saturday, July 14, 2007

Huh?

Every now and then I come across people who really resist the sort of work I'm doing. They dislike the idea of academic investigation of "comics," as if it somehow degrades the value of the works that are produced.

I'm also confused at the general meme that claims academics are so enveloped in their work that they have "lost touch with the real world." I've often found that people in academia are more in touch with the issues going on in the world (or at least try to be), and often are committed to making it a better place, not holing up and ignoring everyone in their study in their non-existant "Ivory Tower."

What I find even more amazing is the vitriol that some people enamoured with McCloud's work sometimes have against mine. I've heard that I'm "overanalyzing" or "missing the obvious" etc. The funny thing is that McCloud's work was accused of exactly the same thing when it came out. Many people did (and still do) hate it for trying to open a dialogue about this stuff. Now McCloud's work has such a dogmatic following that people have once again closed off their minds to that "debate" that he so willingly opened to everyone.

At this point, I've spent as many years out of academia doing this work as I have in it. And, I'll say that doing it within academia sure beats out, because of the availability of resources to use, people to discuss with, time you can allot to the work, etc. It's great to be on the outside changing the system, but sometimes being on the inside is a good thing. (and sometimes you'll always be on the outside, no matter how inside you are).

It baffles me, what sort of rational thinker would believe that any type of scientific or scholarly investigation is a bad thing?

It is the nature of discovery and exploration that help define being human... Even exploration into the cognition of the "comic" medium.

Thursday, July 12, 2007

I'm a winner!

Apparently, I've won an award. I just got an email saying that my "paper, The Grammar of Comics, was selected as the winner of the M. Thomas Inge Award for Comics Scholarship for the Comic Art & Comics Area at the 2006 Popular Culture Association conference in Boston."

So... yay!

Monday, July 09, 2007

Blindfolded patterns

Dirk links to this interesting post where cartoonists were asked to draw their characters blindfolded (versions on the right). This Chester Gould one is pretty impressive...



What's interesting to note is where most had problems and where they were fine. Most of the linework between the blindfolded and sighted versions are the same, it's putting them all together that they had trouble with. I like this a lot since it shows that authors are using consistent patterns when drawing (as if people didn't know this), even when they can't see the page!

This all reminds me of some citation I saw that claimed blind people also use/conceive of speed lines to indicate motion, but I never fully got the chance to read up on it. Perhaps I'll hunt that one down...

Sunday, July 08, 2007

Essay in The Public Journal of Semiotics

Way back in January, I had a piece published in the innaugural issue of the online Public Journal of Semiotics, yet kept forgetting to post about it. The essay there is an expanded version of my downloadable essay "A Visual Lexicon", which people seem to find as one of the most interesting of the papers posted (at least, that's what the vocal feedback says).

Naturally, I think the PJOS version of the paper is much better than the original, though the interface is a little funky since they're experimenting with electronic delivery beyond simple pdfs. Unfortunately, that means they lost all of my formatting, in addition to putting my images in places that don't always make sense for the flow of reading. Grr...argh.

Neverthless: an expanded essay for your reading pleasure.

Tuesday, July 03, 2007

Yowza!

As I've probably mentioned before, I'm currently building a corpus of comics to be used in research related projects. I've had some donations from companies already, and Dark Horse just about doubled my collection in one fell swoop! They are thanked tremendously (as are everyone else who's donated). I'm still interested in expanding the collection, especially with international books, so if you can help out with that, shoot me an email.

The ultimate goal, which we're inching towards implementing, is to create a massvie database of books. We will code various properties of the books, then be able to mine the data to come up with lots of interesting statistics about the structure of comics — cross-listed by country, genre, company, etc. This can tell us about the trends and patterns in visual languages of the world. Plus, all the information will also store coding information about the panels, strips, and books that I will be using in experimentation.

Hopefully I'll be able to get this set up to start having data entered by the Fall (most likely by enterprising undergrads... anyone out there go to Tufts or nearby schools and want to do research for course credit??).

Sunday, July 01, 2007

Comics definitions and distribution

A friend of mine and I were talking about comic distribution and some interesting points came up, including how McCloud's definition of "comics" might actually hurt the growth of the medium.

I'll get back to that one.

My friend noted that "comics" are coming to be split into two major groupings of mainstream works and graphic novels (largely dominated by memoir), and that "comics" were once again coming to refer to the latter as a more niche label of superhero/fantasy/sci-fi/-ish genres.

Part of our discussion focused on the inadequacy of the distribution to "comic stores" for appealing to customers who might not have any interest in that niche. I've argued at length before about how if someone were to write a compelling baseball "comic" that would appeal to baseball fans (which outnumber "comic" fans by a lot I imagine, when they aren't overlapping), only selling it in a "comic" store would not reach the real audience intended to buy it — you'd need to sell it in sporting-good stores, batting cages, baseball games/stadiums, etc. (in addition to non-niche places like grocery stores).

Now, when McCloud tried to define "comics" back in 1993 as "sequential images," I think (?) his intent was to move the label beyond its stereotyped niche. Using such a formalist/structural definition would seemingly let so many other things into the fold that superheroes/etc. would become just another genre.

But instead of "comics" becoming the superordinate category to the benefit of the medium, I think this has actually had a negative impact for those that have adhered to it. Instead, it seems as though it only cast a wider net for all those things that break the stereotype to be sucked into the associations of "comics." My hypothetical "baseball comic" would only get hurt by being called a "comic" and being carried in "comic" outlets because readers of said genre already have predispositions toward things called "comics."

Indeed, the only real growth areas in the industry right now are things that have both evaded those stereotypes and use new labels altogether: "manga" or "graphic novels."

With those new labels they have appealed to audiences outside the "comics" markets. I've heard stereotype-avoiding readers say "it's not a comic, it's a graphic novel" — which is why Marvel and DC are now trying to give a post-hoc association of their products ("comics") as being "graphic novels" so they won't be left behind by the new wave of readership. (I doubt it's working)

I actually think it's fine for "comics" to refer to a niche, since it gives it a reliable label. And, there's nothing wrong with that, especially if we have a notion of the sequential graphic communication system that is separate from the notion of how that graphic system appears socially. It's also fine for that niche to be found reliably in "comic stores," while other graphic works that don't need to be called "comics" can viably be sold in other marketplaces.

We shouldn't limit potential graphic stories and books to the labels and distribution venues of a niche they don't belong to. Doing so would only ensure that they never sell to their potential and that the medium never reaches beyond niche works.

Thursday, June 28, 2007

Abstraction

Dirk links to this blog post with a scanlation of the manga "Abstraction" by Shintaro Kago. It's a pretty incredible formalist experiment, and definitely worth checking out. (beware...it's a bit "lewd" to say the least...) Go now!

Wednesday, June 27, 2007

I'll show you my paradigm if you show me yours

I'm shocked at the response that my quote in yesterday's ¡Journalista! (thanks Dirk!) has generated. Is it really so hard to believe there's a separation between the structure of a system of expression and how it's used in "literary" contexts?

Ok... maybe I'm not so shocked...

For spoken language, this is simply the difference between studies in lingusitics/cognitive science and literary studies. Not seeing that split for "comics" is one of the issues wrapped up in the muddled understanding of graphics and "comics" in our society.

And, I should say, I don't believe that literary and linguistics/cognitive analysis don't or can't complement each other, but recognizing the division is important at the outset. One is involved with idenitifying cognitive processes and patterns of usage and behavior. The other looks at how those patterns are used to create some sort of expression (and possibly, larger level patterns). One discusses meaning for the base semantic understandings of cognition. The other discusses meaning layered on top of those cognitive processes (often "interpretive").

The structural analysis lends itself to informing the literary quite easily. If I were to propose a method for categorizing panels based on how many "characters" they contain (which I have), even as a structural analysis, literary works can use it to discuss the sorts of concerns they have. This doesn't work as well in reverse — categorizing panels (or layouts, as Peeters did) as "decorative" versus "rhetorical" does not offer me a way to study processing or structure (though I'm open to being proven wrong). However, it does work within the contexts of literary analysis.

Neither approach is inherently better or worse within the contexts of its own intents. However, the motivating factors behind the paradigms need to be recognized as very different. To this end, the label "comic theory" is being used in very disparate ways that do not necessarily inform each other. While I've found many who go ga-ga over it, I find French "comic theory" as largely unusable and uninformative to my conception of "comic theory"—because it is of a totally different paradigm. (...and also why, I'm guessing, that they tend to dismiss McCloud's work, which I find to be far more interesting than theirs)

My point is, that like the divisions between any paradigms, it's not so much the answers being provided that are different, but also the questions. The clearer this can be made, the more there is potential growth for "progress" on all sides.

Tuesday, June 26, 2007

Photo Technology

This is pretty amazing new technology for photos that aggregates the information in many photos to create virtual environments, not to mention a pretty impressive navigational tool for photos and data alone.

The potential for applying it to comics is huge. On the one hand, you could use all the information stored in various panels to create a virtual environment of the fictional landscape. Moreso though, I imagine a tool like this would offer huge ground for those who are experimentally minded about online comics and the Infinite Canvas.

Thursday, June 21, 2007

ImageText Vol. 3

The latest issue of the online journal ImageText just came out with a couple articles of note (to my tastes):

The first one examines the role of comics in education, a subject dear to my heart (and the topic of one of my Comic-Con panels this year). The experiment is interesting, though I would have liked to see actual reporting of the statistics that were run, which seems like it'd be important in a piece like this.

Also, the background research seems blissfully ignorant of most of the research out there pertaining to this sort of work. Perusing the references, the books about comics are largely historical or literary treatments, and the books about psychology/education are mostly 20 to 30 years old—none of which are about comics or related research in text-image relations in educational contexts. Given a section is called 'Comics and Cognition,' it has next to no research about comics and cognition, much less the directly pertinent work that has been done in educational contexts (despite the clear overlaps in methods).

I also found it quite odd that while testing students' background experiences with images versus text, there was no question about how often they read comics. This seems like a major oversight in design, since it could correlate levels of expertise into the statistics.
---

The other articles of note are Jesse Cohn's (no relation) translation of a chapter of Benoît Peeters' Case, Planche, et Recit pertaining to comic page layouts, as well as his commentary about the chapter.

Peeters proposes a taxonomy for different types of page layouts, serving decorative versus rhetorical functions (among others). I find it a bit difficult to review these theories, simply because of the very apparent paradigmatic differences between this approach and my own. We are asking very different questions to generate our different answers.

French works like Peeters or Groensteen seem to be concerned with how things function in the contexts of the "work of art." In contrast, proposals like McCloud's (and mine) do not care what the work is about, but hypothesize structural principles at work in the medium and (hopefully) cognition. To invoke an analogy: I'm interested in identifying how "nouns and verbs work." They're interested in how people "use nouns and verbs in writing."

Along these lines, I'm currently analyzing data from an experiment I did looking at page layouts (I hope to have it readable by the end of summer... this is what my other ComicCon talk will be on). Immediately apparent is the difference in the intents: I don't care what the "design" of the layout "conveys"; I want to know about the strategies people use to navigate the layout irregardless of the content. As much as people might hypothesize what order people read panels in, or how the "layout is read as a whole," no data has yet shown what people actually do. Hopefully I can show some of that.

Thursday, June 14, 2007

Don't!

My department here at Tufts is hosting a big linguistics/psychology conference this weekend that everybody is going crazy organizing and getting ready for. I'm not really doing that much though, since I'm not presenting or participating that much (I'm actually heading to Santa Barbara on the weekend to watch my brother graduate college!).

Anyhow, my advisor asked me to draw a few comics for his upcoming talk and book so I figured I'd share. I'm not sure what the full discussion will be, but he said these will be used to talk about how sometimes reference occurs in places outside of sentences themselves:

Tuesday, June 12, 2007

Links anyone?

Life's been good lately. After being a bit debilitated from my hip surgery, I'm now starting to walk short distances again, though the crutches will be with me for a few more weeks still. More pertinent to my research concerns, I've finally finished writing my first year project paper, which was a bear to write. Now I can turn my focus to other visual language and comics related projects. Here's some links that have fallen in my lap recently...

Through my ComicSpace account, I was alerted to The Cape Symposium. While most of it seems to be a message board for talking about superhero comics in particular, the overall tone tries to probe a little deeper than the usual surface discussions, and some threads seem to aim towards praxis based theory. One thread has a downloadable pdf exploring the relations of text and image in comics, largely from a semiological-ish view.

Also, Tor from Comic Book Innovation emails me with word of Storytron, which seems to be a company working on technology for interactive storytelling. Seems to be pretty interesting.

Finally, Alan David Doane has a great essay about why the mainstream comic market is doomed to extinction. I think it's a great piece, and echoes many of the things I said in my first essay for Comixpedia, and in many of the writings in the "Visual Language Manifesto". Worth reading if these are concerns of yours.

Monday, June 11, 2007

Review: The Language of Comics by Mario Saraceni

Saraceni, Mario. 2003. The Language of Comics. New York, NY: Routeledge.

Saraceni's The Language of Comics is one of the few books that attempts to present a holistic theory of how comics work, and draws upon work from "applied linguistics" no less. The book is actually a stripped down version of his dissertation (as is his article "Relatedness" from the Graphic Novel collection). (And good luck finding the dissertation... I had to print it from microfilm on interlibrary loan).

Unfortunately, much of his approach seems to feel of grafting McCloud's work to ideas in applied linguistics in a simplistic (and uncited) way. For instance, he proposes a gradation between semiotic types (like symbols and icons), and can well be compared to McCloud’s Big Triangle.

He treats the sequential aspect of panels as equal to sentences, giving them a “discourse theory” type analysis (like the dissertation by Stainbrook). Saraceni claims meaning is created through commonality between elements in panels, alternating with successive new and given information. He also uses "semantic fields" (connected meanings: like how "snow, caroling, pine trees, and presents" invokes "Christmas") to unite panels not encompassed by this information structure.

However, in doing so, he eschews the role of linear sequentiality, yet provides no argument for why people do indeed read in consistent sequences. The result is essentially a watered down version of McCloud's closure — which it is: his dissertation has the theory in full, and exactly does shoehorn discourse theory onto McCloud's transitions. Some of his insights here are useful and enlightening, yet they deal entirely with "exceptions." He rarely discusses "run of the mill" things like the depictions of events, instead culling his examples from very experimental comics storytelling, like Peter Kuper's The System.

Other chapters cover things like word balloons (perceived as equivalents to direct quotes) and drawings of eyes to buttress a discussion of subjective and objective viewpoints. The final chapter is about computers, which seems out of the blue and has next to nothing to do with comics.

Since it uses applied linguistics, much of the book feels attuned to what might be useful for literary studies. To this extant it might work very well. However, as a theory of "meaning" it falls short, largely because it does not address any type of cognitive system, and lacks even McCloud's precision of surface categorization.

My biggest gripe about the book is that it is presented as an introductory textbook as part of the Intertext textbook series, has no citations outside a "recommended reading" list in the end, and is written with a matter of fact tone that presents it as an authoritative stance on the body of knowledge of this field. The truth is that no such body of knowledge exists at this point for "comics theory." Right now, we're in that period of science where lots of different viewpoints are popping up, just waiting for an encompassing paradigm shift to sweep in and take over.

Even if I were to come out with a book of my full theories, it wouldn't be a de facto textbook because it would be my views drawing upon that body of research. As a result, to those "in the know," the format and style make this book seem misleading in its intents for fronting Saraceni's views as well established scholarship.

To end on a good note: Though I think it fails in not using a cognitive approach, I do like that the book tries to use concepts from linguistics with "comics." It shows that this type of approach is not just intuitive to me, but to others as well, and locating the book in the broader field of linguisics is good for the field as a whole.

Tuesday, June 05, 2007

More on Imitation in Drawing

Brent Wilson and Marjorie Wilson have done a lot of great work on child drawing that has influenced my thinking, especially since they take comics and manga into account a lot. I got to meet Brent last year at a manga exhibit, and he was a really nice guy. Here's another good (old) article that I recently read (it won't be the last):

Wilson, Brent, and Marjorie Wilson. "An Iconoclastic View of the Imagery Sources in the Drawings of Young People." Art Education 30.1 (1977): 4-12.

Wilson and Wilson present findings that contrast the century-old belief that imitation is bad for learning to draw. This view focused on the belief that children have an innate purity to drawing that emerges out of their natural tendencies. Imitation is thought to defile this purity. Additionally, due to their iconicity, drawing has been seen as the correspondence between the world and mental equivalences of those objects.

The Wilsons' data counteracts this with the observations that of the hundreds of drawings gathered from high school students, virtually all of them could be traced back to imitation of some other source of representation (especialy comics and cartoons, but not much fine-art). They note that the learning of drawing might be more similar to learning words (though they don't seem to really know what that means beyond common sense knowledge).

They propose that people are using/creating mental models for drawing, and that minor modifications to generalized structures can aid in creating specific representations. At times, these models would be begun to be employed, yet abandoned part way through if the drawer couldn't produce the desired result with that schema.

People might also have mental models for one type of representation, but be unable to do drawings outside of that model. So, drawing novel objects either results in sub-adaquate ability or deploy existing models from other domains.

In total, people seem to be able to store hundreds of these types of mental models. For instance, one subject who drew comics could create figures in innumerable ways. When drawing a figure he hasn't before, they hypothesize that he "averages" several of the models together to produce the new form. To me, this raises the question whether the aggregation of these models creates a singular more abstract schema or whether it remains a catalogue of numerous "malleable" models.

---
This means the mind has a place to store such visual schemas, a "photological" component akin to a "phonological" component for spoken language. As I've said before, if perception is the desired stimulus for drawing, models aren't created from existing models — so the system never creates a conventional set of signs.

And just to riff on my previous post, this is also the reason why drawing perspective might be "awkward" for learners — because it sidesteps metal modeling in order to exact a system of depth through measurement (as opposed to schemas... though "loose" drawing of perspective might involve some level of mental modeling created by learning how to do it. I'll have to see if there's research on that...).

I had a 13-ish year old student when I taught "drawing comics" in an afterschool program, who could not draw perspective for a tile floor in a hallway. He drew the receding lines of the hall walls with us as a group, then when drawing the floor on his own he drew it flat — like an aerial view. My interpretation: he couldn't override his existing mental models for spatial representation with new ones for perspective. That's not to say he couldn't if he worked at it, but at that moment he couldn't do it.

Finally, note that perspective and schemas are both learned — the difference is that one is acquired "effortlessly" through imitation (as they'd say in language acquisition) and the other is taught explicitly.

Monday, June 04, 2007

Art 'n' stuff

Apparently, I've inspired Nathan Sandler enough that he devoted his inaugural blog posts to critiquing my thoughts on Art vs. Language. I'm flattered that he's spent the time to write out his thoughts, and even more that he found my work thought-provoking enough to mull so much about them.

Unfortunately, I don't think he quite got what I was aiming at with the ideas of Art vs. Language. In lieu of what would assuredly be annoying, I won't make a direct rebuttal, but try to clarify my position a bit since any confusion out there might just be because I haven't explained it well enough.

There are actually two separate issues involved in my discussions of Art vs. Language. The first involves socio-cultural labeling, the second is about how people treat images in our culture especially in regard to learning to draw. Let's start with the first one...

1) Art as an identifying label to justify comics' worth

My notion of visual language is largely about drawing — and moreso about drawing sequences of images. Those drawings (and sequences) are governed by mental rules, and by nature are what I'd call a "visual language."

Whether you want to call that use of the visual language "Art" largely depends on its context. Not all drawings are conceived of as "Art" (though they are sometimes terminologically conflated) — but nor are all uses of the English language called "Art." (...unless you're post-modern enough to say that everything is "art," which is fine, though I think it fails as a useful notion if it's so all-inclusive).

The point is that the label "Art" is applied interpretively to whatever it's talking about — be it process or form (issue #1).

I have no real desire to "define" the notion of "art" — I agree with the cliché that "I know it when I see it," and I generally think that's true for most people. But, for a notion so categorically vague, how can it inherently apply to anything? Why should this notion of "comics" (the socio-cultural context that this visual language is used in) be inherently "Art" at all?

And, perhaps more important to people's concerns usually... how can such an ephemeral concept be used to defend the notion and status of "comics" in society?

This idea is prescriptive: Playing the role of a critic, I think defining "comics" as "art" is less effective toward their gaining public acceptance than a notion of visual language.

2) "Art" as a sociocultural frame for treating drawings

Issue #2 of my "art vs. language" track has to do with the forces of influence on how people (and especially children) learn to draw. This is where I trot out the "Art Perspective" as a notion of the cultural forces of a particular mentality that work to temper the way people in our society draw, but I'll leave that to other posts.

This idea is descriptive: Playing the role of an academic, I'm only making analysis of the various evidence and concepts involved, and am not advocating for any type of practice per se.

In any case, I love that Nathan devoted such a long post to all this (and seems to continue to prod my stuff), and look forward to reading what else he comes up with.

Thursday, May 31, 2007

Point perspective

Austin Kleon has a nice little post about point perspective in comics, noting his like of artists who don't use it at all (link via Derik). The preference for point perspective is of course wrapped up in the whole desire for iconicity that readers of this blog are probably sick of hearing me rant about.

His post got me to think about some other related issues. For instance, point perspective was developed in the Renaissance, which I imagine coincided with the Enlightenment's focus on discovery about the world and the rise of science. (Though, I have no idea since I'm not an art historian.)

What is worth remembering though is that point perspective was truly a discovery. The human mind/brain may be able to see in perspective, but we don't draw in it. There was a study** I read that talked about South African children (ages 5-9) who had trouble with understanding certain parts of images. The parts they misinterpreted the most were perspective, depth, and shading — all highly iconic and things that must be explicitly taught to people learning to draw.

Interestingly, when looking at the data, the means for misinterpretation drop for children at Grade 3 (and 9 yr. olds) in almost all categories. The conclusion of the author is that schooling teaches children how to understand images, but this could just be a coincidence in that children’s exposure to images comes in a school setting. That is, it’s not about instruction, but about exposure.

Whatever the case, perspective is not a built in part of the human graphic system. This again goes to the point that drawing is less about mimicking the perception of the world as piped through an individual's mind, and more about the way minds are enabled to convey concepts visually.

Update: I feel I should add, that there's nothing wrong with learning how to draw with point perspective, only that our minds' graphic system is not predisposed to it. As an academic, I'm not prescribing anything, just analyzing. Learning perspective requires iconic understanding that doesn't just come out of imitation of other people's drawings. That is, it once again skirts conventionality and the establishment of mental models for drawing in lieu of imitating perception.



** Liddell, Christine. 1997. Every Picture Tells a Story—Or does it?: Young South African Children Interpreting Pictures. Journal of Cross-Cultural Psychology, Vol. 28, No. 3. Pp. 266-283

Wednesday, May 30, 2007

All new look, same great taste!

Today marks five full years that emaki.net has been online. That's quite a span for me, covering four states, three bodies of water (2 oceans, 1 lake), and three universities. So, I figured it marks a good enough occasion as any to give a massive remodeling to the site (it had it coming).

I'm hoping this new and improved version should make the site all the more navigable and pretty. Beyond the spiffy new look with color codings for each major section, the changes include...

New "comics" for each major explanatory section of the pages for:
The Home Page
What is Visual Language?, and...
What are Emaki?

I've also removed a few of the less popular downloadable essays to give more focus to the others, while adding a downloadable pdf of the edited compilation of the The Visual Language Manifesto — my writings railing on the comic industry's faults and how I think it can overcome them.

And, you'll still find the same resources for research like my ever growing Reference Bibliography, with new entries being added frequently.

I'm very happy with the new look, and hope you enjoy it too. Please let me know if you face any errors, typos, problems, etc. with it. Thanks to everyone that has supported me and my work, and I hope to see you around for another five years and beyond!

Tuesday, May 29, 2007

Burnt City bowl

Apparently the 5,000 year old bowl with a graphic sequence that was unearthed from the Burnt City is now going to be on display in Tehran. I love examples like this, and I'd just like to remind those interested about my previous post analyzing the bowl's graphics and critiquing the reporting of it. (via Journalista)



---EDIT 11/2013---
The comments section on this and all posts related to the Burnt City Bowl are now closed, due to the inordinate amount of anonymous and slanderous comments left by people clearly bearing some type of political agenda (however construed). All comments made on this blog of such a nature will be deleted during moderation prior to being published.