Wednesday, August 29, 2007

Video: Visual Grammar

The conference proceedings for the Visual and Iconic Languages Conference from a few weeks ago are now posted online. On the site you can download the slides from my talk, and they've also posted full video of the conference proceedings. I've embedded my talk below, and also to my site.

I highly recommend watching this video of if you are at all interested in my overall theories or in how sequences of images communicate. I describe what exactly I mean by "visual language" fairly clearly, and why it is different from "comics."

Most of the talk though is essentially a snippet of my developing theory of visual grammar — how sequences of images communicate — including my arguments for why panel transitions don't work and my alternative model. I don't plan to post an essay of this work online for awhile (I've been tinkering with it for four years), so this is the best place to find these ideas.

My talk runs for the first 45 minutes of this video, followed by 15 minutes of questions (the second hour is someone else). If you download the slides (large pdf) to flip through at the same time, they might be clearer than on the screen. Enjoy!

Monday, August 27, 2007

Trouble with categories in "Comics are not Literature"

Listening to the downloadable mp3 of the "Comics are not Literature" panel from Comic-Con is fascinating as a demonstration for how different people's categories are relating to the definitions of "comics," "literature," or even "reading" it seems. (all of the issues seem resolvable by accepting the notion of a visual language)

As I've been claiming for awhile, "comics" for these people seems to really be bound to the genre(s), so the notion of that as literature would indeed seem troubling. It also seems rather ridiculous in general for anyone to make a blanket statement about the notion that all comics are literature, as opposed to literature being something that individual comics can attain. Is this just a terminological straw man? Perhaps this is just a problem created by using "comics" as a singular noun?

Concerning the one person's notion that comics are not "read" just seems ridiculous. Calling the process that we do to decode comic sequences just "looking" is revealing as belonging to a paradigm of thinking holding images into a subjugated position. "Perceiving" or "looking" are passive processes compared to the active "reading." I'd argue, with empirical evidence, that "reading" is indeed the closer category.

I find it also very interesting that the panel has next to no one who is actually draws comics — i.e. no one with "visual language fluency." And, for a conversation that keeps going back to formal properties of the medium, perhaps they'd have done well with someone on the panel who actually knows about that stuff?

In general, it seems like most all of these issues that they struggle with are almost wholly resolved by accepting a paradigm that acknowledges that images in sequence are literally a visual language. No more struggling with whether someone is a "cartoonist" or "writer" or "artist" — they're a "visual author", or even better, just a "writer" who writes in pictures as opposed to words. I could go on and on (and, in fact, I have).

These were exactly the issues I was addressing in two of my older articles: The "Literature" issue is essentially encapsulated by the "Comics as Art"-debate, while the issues that they're struggling with in general are instances of the limitations of the network of concepts that "comics" encapsulates where these peoples' categories are running up against troubles.

Sunday, August 26, 2007

Manga facial symbols

I've been putting the finishing touches on my first draft of a book chapter for a textbook on Japanese manga recently. Mainly, the last things I had to do was draw my examples. Also being a "comic author" is a big advantage as a scholar, since I can create my own examples when I don't want to deal with issues of copyright and permissions.

For this one, the challenge was drawing in the "manga style," which I rarely do. For better or worse, here's some symbolic faces I've whipped up:

This section on graphic symbols/conventions was one of the hardest portions of the actual paper to write. While most of us know that manga uses a ton of wacky conventions, there aren't many places referencing them outside of informal listings like wikipedia. At most, various sources mention one or two different conventions, but I couldn't find any extensive type of cataloging. (though, if anyone is aware of such a thing, please let me know)

I started trying to make a cross-cultural list like this back when I used to have the forum, but that project seems to have stagnated. This is a research project just waiting for someone to take it up (like oh so many)...

Thursday, August 23, 2007

Lessons from manga

I found an interesting discussion in this thread over The Engine discussing this quote from a PW article:

Still, if there’s a manga feel to the books, it’s not an accident. “I went into my local bookstore, and there were two shelves for the entire history of American comics and 12 shelves for manga,” Crilley said. “Whenever kids come up to me and say, ‘Look at this drawing I made,’ nine times out of 10 it is in the manga style. For this generation, comics are manga. This is the language of this generation, and I’d better learn how to speak this language or I’ll never reach them.”

Recently I've been finishing writing a book chapter for a new textbook on Japanese manga, so these sorts of comments and discussions are particularly salient for me right now. This quote is revealing about a lot of theoretical issues, beyond simply calling it a "language." Or, perhaps more accurately, I think manga shows us many of these issues, which are kindly wrapped up into this quote. Among these are:

1) It acknowledges that the system of representation and expression used in manga is different than in American comics. Underlying message: Graphic systems are not universal

2) It notes that one graphic system can influence another one (just like languages do when they have contact with each other). Underlying message: Graphic systems (or rather, human minds that produce graphic systems...) are fluid and changing

3) People who draw in one manner can adopt additional manners of drawing. Underlying message: Multilingualism in visual language!

4) Manga are extremely popular, and that popularity is tapped into by adopting its system of expression. Underlying message: Having a consistent style might increase readership (ahem... among many other factors)

5) Children are choosing the "manga style" en masse to draw in — a consistent style which is beyond the scope of a single author and belongs instead to a community. Underlying message: Children learn to draw by imitating others

Wednesday, August 22, 2007

Understanding motion lines

Friedman, Sarah L., and Marguerite B. Stevenson. 1975. Developmental Changes in the Understanding of Implied Motion in Two-dimensional Pictures. Child Development 46:773-778.

This paper reports on an experiment testing how action lines are understood by children. They compared figures with postural cues showing motion (i.e. they look like they're running) with those with motion lines, and those that have polymorphic features (i.e. repeating legs over and over to show motion). They tested preeschoolers, first graders, sixth graders, and college students.

Massive increase in the understanding of “cartoon” conventions (i.e. motion lines) occured between the ages of first grade and sixth grade. (possibly due to developmental reasons... or to increased exposure to comics?). Reliance on postural cues decreases from first grade through college. Understanding of polymorphic representation increases greatly between pre-school and first grade, then levels out. The insinuation is that conventional cues are relied on more and more as people age.

Gross, Dana, Nelson Soken, Karl S. Rosengren, Anne D. Pick, Bradford H. Pillow, and Patricia Melendez. 1991. Children’s Understanding of Action Lines and the Static Representation of Speed of Locomotion. Child Development 62:1124-1141.

A similar study also tested children’s knowledge of action lines to determine whether body posture or action lines contributed more to understanding with seven and nine year olds, and adults. The stimuli showed a running figure (one silhouetted, one photo-like), without lines, with lines trailing it, or with lines behind it. They also had a task where subjects drew their own figures running.

They found that the relationship of action lines to the meaning of locomotion is non-arbitrary, though exposure to drawings using it is necessary to understand its convention. Again, children attenuated more to postural cues than adults did. The younger children did not distinguish between action lines and background lines to the same degree as older children and adults.

Interestingly, it also asked subjects what the lines represented, showing that children contributed meaning to lines where adults did not. Many children gave visible and invisible explanations for the lines – such as “air moving” or “wind.” Adults simply accepted the lines as symbolic representations. In cartoon drawings, adults treated lines as “path-of-movement metaphors,” but for photos were treated as non-arbitrary cues for perceptual movement.


(Updated 8/25): These conclusions are interesting for a few reasons... one, it shows that not all graphic things are universally or transparently understood by everyone. It takes some degree of development to reach that understanding for things that we take for granted as obvious. This is different than some of the findings on speech balloons or thought bubbles.

The fact that the development happened most in pre-puberty aligns with many other developmental changes, like language acquisition. Whether that development has to do with exposure or just age, it's hard to tell.

Also, its interesting that there was a change in how they considered what the lines were — from an iconic meaning (claiming the lines are "wind") to a purely symbolic meaning. This too is consistent with developmental changes in other domains.

Thursday, August 16, 2007

The Comic Strip and Film Language

Lacassin, Francis. 1972. The Comic Strip and Film Language. Film Quarterly, 26(1), 11-23.

In this piece translated by David Kunzle, French theorist Francis Lacassin discusses the similarities between the "syntax" of film and comics, noting that they both use "shots" as their base units. For him, this includes various things, like various degrees of framing (long shot, close up, medium shot), dynamic use of what could be multipanel representations ("panning"), as well as semantic alterations, like subjective viewpoints. (I would argue that this isn't "syntax" at all... but that's a larger post).

He argues that though film and comics emerged around the same time, these techniques came first in comics — not the other way around, as is often argued — and that they may have been autonomous developments not influencing each other at all.

He writes: "It is more reasonable to suppose that comic strip and cinema have both separately drawn the elements of their respective languages from the common stock accumulated in the course of the centuries by the plastic and graphic arts." (14)

To this I would question, is it really through historical development, or is this just a reflection of the structuring of people's minds/brains?

He hypothesizes also that film and comics both accomplish their sequential meaning by use of the film theory of montage, which for Lacassin appears to cover most things that do not appear similar to real-world perception.

In his own section at the end, Kunzle criticizes Lacassin for claiming comics were invented before cinema, while those framing techniques are cited being used by authors two generations before that "birth" of comics. Kunzle then discusses the work of 1800s artists Töpffer, Doré, and Busch, noting that they used various techniques like close-ups and polymorphic representation (where one character is repeated in a single frame showing the unfolding of action), among others.

Tuesday, August 14, 2007

A Time Sex Thing!

In response to this Language Log post, my best friend John sends along this wacky revamped movie poster:

As he put it "A google image search for 'pulp sci-fi novel' turns up some great material for this title."


Monday, August 13, 2007

Perhaps I should avoid the slopes...

After the conference in New Mexico let out, a bunch of us went on an excursion to the top of a mountain that overlooks Albuqurque. At the top, we found this sign on the side of a building:

Given that we'd all just talked about visual signs for a day and a half, many of us photographed it and cracked jokes. While one might expect the images are there to help give aid to those who don't speak the language or to reinforce the text, but really most of them are extremely ambiguous...

The top left one looks like the skiier is walking on cracks in the earth, and the bottom middle one looks like it's telling you to hang out in the middle of the road (when the text says exactly the opposite). The top middle one seems like the skiier is dropping his skiis from the lift.

My favorite though, is the upper right. It could easily be construed that, when you're skiing and approaching someone else, you should go around them on both sides at the same time (except for its physical difficulties). I kept picturing some Looney Tunes cartoon, where someone hikes up their body to elongate their legs so they weave around an object to avoid hitting it.

If one can only read the very large text at top (since the smaller text is a bit tough to read), these "signs of safe skiing" paint an awfully strange picture of what skiing is like.

Anyone got any more good interpretations of these?

Friday, August 10, 2007

Biblio clean up

Just some housecleaning to report... I've updated the Reference Bibliography with some long-overdue entries that have been building up for quite awhile. I've also added a whole page listing all the papers I've reviewed here on the blog (also accessible in full through the Bibliography tag).

Hopefully this will make it easier for people to find things they might be interested in.

Wednesday, August 08, 2007

VaIL '07

I recently got back from my keynote talk at the Visual and Iconic Languages conference at the University of New Mexico. The conference went very well, and a lot of very interesting conversation was had in a very short (1.5 days) amount of time. People seemed to enjoy my talk on visual language grammar, so hopefully some good will come of the event. Personally, I seem to have made some good contacts for potentially promising future works.

As I've mentioned before, I've been told that my talk may be posted online. If that's the case, I'll be sure to link to it.

One of the most interesting talks was given by Alan Stillman, who is the CEO and founder of a company called Kwikpoint. They make sheets of paper filled with simple images on it, designed for communicating with people when you can't speak the language. They were originally designed for travel, though now they're being used by the military in Iraq and Afganistan.

Originally I was doubtful of how effective this sort of thing would be, but he showed a great video of soldiers' testimonials saying how these guides helped them save people's lives, both military and civilian. And, hey, who am I to argue with very tangible success like that? He was also a very nice guy, and we've talked about my maybe doing some advising for them. I certainly hope that happens, since they really seem to be making a substantial difference.

And, just cause it's a great photo, here's me and the conference organizer, Sunny, taking in some New Mexico flavors...

Sunday, August 05, 2007

Thought Bubbles, Children, and Autism

These are several articles I found particularly interesting looking at the understanding of thought bubbles, and using them to help autistic children.

Wellman, H. M., Hollander, M., & Schult, C. A. (1996). Young Children’s Understanding of Thought Bubbles and Thoughts. Child Development, 67, 768-788.

Several experiments on 3 and 4 year olds show that thought bubbles are understood at fast rates by both age groups as depicting thoughts. This is interesting, because at similar ages, other visual conventions such as speed lines are acquired over age (I'll post on these articles next maybe).

This contrasts with the findings in the Yannicopoulou study, which showed that preliterate children could not distinguish speech from thought balloons – which was not tested in these experiments. They suggest that this shows evidence for “something recognizable from our everyday understanding or experience of thoughts themselves” tapping into "theory of mind" knowledge...

Wellman, H. M., Baron-Cohen, S., Caswell, R., Gomez, J. C., Swettenham, J., Toye, E., et al. (2002). Thought-bubbles help children with autism acquire an alternative to a theory of mind. Autism, 6(4), 343-363.

Children with autism have specific difficulties understanding complex mental states in other people like thought, belief, and false belief and their effects on behavior (what are known in psychology as "Theory of mind").

These children benefit from focused teaching about thoughts, where beliefs are likened to photographs-in-the-head. Here two studies, one with seven participants and one with 10, tested a picture-in-the-head strategy for dealing with thoughts and behaviour by teaching children with autism about cartoon thought bubbles as a device for representing such mental states.

This device led children with autism to pass not only false belief tests, but also related theory of mind tests. These results confirm earlier findings of the efficacy of picture-in-the-head teaching about mental states, but go further in showing that thought-bubble training more easily extends to children’s understanding of thoughts (not just behaviour) and to enhanced performance on several transfer tasks. Thought-bubbles provide a theoretically interesting as well as especially easy and effective teaching technique.