In this paper I should like to begin an exploration into how, and what kinds of things, pictures mean (Note 1). I also want to explore how the way that pictures mean differs from the way such things as "words" or "languages" mean. In order to explore these things, I will compare them along dimensions which I believe are central to the existence of both verbal and visual signs as communicative modes. I will first present an outline of two kinds of meaning, which I have called communicational meaning and interactional meaning, and relate these meanings to the kinds of interpretations that can be made from visual (pictorial) signs and verbal (linguistic) signs. Second, I will argue that pictures--paintings, movies, television, or sculpture cannot be either true or false signs, and that therefore they cannot communicate the kind of statement the meanings of which can be interpreted as true or false. I shall then argue that picture interpretation is very different from word interpretation as regards its so-called pictorial code, convention, or "grammar," and also that syntactic, prescriptive, and veridical aspects of verbal grammar are very difficult to apply to pictorial events. I shall further attempt to show that differing assumptions accompany these different strategies of
interpretation (Note 2), and that these assumptions--of intention, in the case of communication, and existence, in the case of interaction--determine how we deal with the concept of truth or falsity in pictures.
Before I begin to discuss the problem of communicational meaning,
I should like to clarify the distinction I will be making between
pictorial signs, or visual signs, and the term pictures.
Semiotic literature either ignores "pictures" or equates
them with something called visual or pictorial signs. Signs, as
we know, can occur in a natural state. Wind, and trees bending
before it, can become a sign or signs of an impending storm. That
is, we can abstract and separate from the natural flow of events
a set of units which we call, and treat as, a sign. Anything may
become a sign in that way, provided it fits our particular criteria
for the use of signs. A sunset in general or a particular sunset
may become a sign, but it can never become a picture, because
a picture is not a natural event. It may, if the viewer so thinks
about it, indicate certain relationships to natural events, but
a picture is a symbolic event and therefore a created
social artifact.
Pictures may mean in many different ways, as well as mean different things, and I shall argue later that the meaning of a picture on one level of interpretation determines what signs go to make up that picture on another level. I shall therefore expect you to treat the term picture with some notion of a distinction between it and the term sign.
It is obviously beyond the scope of this paper to deal extensively
or widely with the term meaning I shall try to be as narrow
as I can, hoping to keep as tight a hold as I can on a slippery
concept. In order to compare pictures and language (or visual
signs and verbal signs) along dimensions of meaning, it is necessary
for me to distinguish at least two differing kinds of meaning:
interactional meaning and communicational meaning.
Pictures, and words for that matter, mean in both ways, but how
they mean in each of these two interpretive modes or strategies
is completely different. This difference in the way--in how--we
arrive at meaning determines to a large extent the kind of meaning
and the level of meaning we live with, both on an articulatory
and on an interpretive level.
Let us start with interactional meaning, since to some extent it
is a larger and more encompassing class than communicational meaning. One position taken by some theorists (Charles Morris 1946 would be the paradigmatic example here) holds that signs "set up" in observers some "response" or "disposition to respond," and that that response is the meaning of the sign or is closely connected to what the sign means. This is similar to some communication theories what define communication as a situation in which two or more entities are mutually interdependent. Thus genes, muscles, and atomic particles, as well as people across generations and spaces, "have meaning" for each other--are in communication. It is true that if one object or event causes, or is correlated with, a corresponding bit of behavior, we can say that somehow event X "has a meaning" in relation to event Y: In some sense every stimulus means its response, and the experiments with Pavlov's dog, as well as subsequent research in operant conditioning, show that some responses do indeed mean their stimuli. However, mutual interdependency, a stimulus response relationship, or any other form of interaction in which entities or people merely engage their environment do not seem to me to be cogent dimensions upon which to build an understanding of symbolic behavior, communication, and meaning.
Let us look at interactive meaning from another point of view, that of interpretation. We observe an event in the "real world" and sometimes are able to say "that means" In this sense, the meaning of the world around us is an interpretation that we must make "correctly" in order to stay alive. Gustave Bergmann frequently made a similar point in his logic classes: "I will," he said, "stake my life on the fact that the sun will rise tomorrow--but not my philosophical reputation, since I can never prove an induction." He would, and did, attribute meaning to the movement of the sun in exactly the sense that I shall be trying to distinguish.
In a larger and nonrestrictive sense, we can, of course, observe and interpret both natural sign-events and symbolic sign-events. That is, we can look at the tree bending in the wind, or a novel, a play, a movie, a painting, and so on. We can say about any of them that they "mean." Let me, however, limit myself in this paper to symbolic--man-made--events. Figure 5-2 (see chapter 5) is a schematic and hierarchical representation of some different ways in which we can make meaning from symbolic events. The major difference that should be noted at this point is the distinction between the interpretive strategy of attribution and the interpretive strategy of implication/ inference. The hierarchical nature of these two processes must also be emphasized. When one learns to make inferences, one obviously does not lose the ability to make attributions. Implication/inference
procedures are built upon earlier stages of recognition. For the moment, let us concentrate upon what I have labeled "interaction" in Figure 5-2--that particular part of the interpretive process that is connected with the strategy of attribution.
By using the term attributional meaning, I wish to distinguish a process by which people largely impose, impute, put onto symbolic events knowledge they have within their psychocultural selves, as opposed to other processes that I shall discuss as communicational, in which people interpret meaning from symbolic events using knowledge they acquire outside themselves and from within the symbolic event itself.
Communication, as I shall use the term here, is defined as a social process, within a sperifed context, in which signs are produced and transmitted perceived and treated as messages from which meaning can be inferred.
Going back to Figure 5-2, we can think of the concepts of articulation and interpretation as comparable to the production and transmission of signs, as well as to their perception and subsequent treatment. While the perception and subsequent treatment of symbolic events might be thought of as acts of interpretation, and production and transmission thought of as acts of articulation, they can most fruitfully be considered as parts of a process, which will here be called articulation/interpretation. This process will be explained as being similar to implication/inference. By drawing attention to the processual nature of these cognitive strategies we can deal with the fact that one cannot be understood except in terms of the other; we articulate in terms of the subsequent interpretations we expect, just as we imply only in those terms which we expect others to use when they infer. Conversely, we interpret (in a communicational sense) in the terms which we recognize as articulated, and we infer what we assume was intended to have been implied.
I will subsequently argue that the inference process cannot take place without an assumption of intention on the part of the interpreter and that implication likewise demands intention. Further, it will be necessary, in order to investigate this process, to understand that the roles of implier and inferrer may and probably must be held simultaneously by people in a communication situation. In such a situation, one shifts, in one's mind, back and forth between articulation and interpretation, asking oneself, "If I say it (paint it, or sequence it in a film) this way, would I make sense of it--given the conventions, rules, style, and so forth, in which I am working?"
It should be clear now that I am suggesting that meaning cannot be inherent within the sign itself, but exists rather in the social context, conventions, and rules within, and by which, articulatory
and interpretive strategies are invoked by producers and interpreters of symbolic forms. It is my view, then, that only when an interpretive strategy assumes that production and transmission are articulatory and intentional can communicational meaning be inferred. In a communicational sense, therefore, articulation is symbolic, and interpretive strategies are designed within social contexts in order to enable us to make inferences from implications.
It is my purpose to narrow rather than enlarge upon common usage, so the argument that will be made further is that any and all interpretations of meaning (consistent with psychocultural process within the individual) are possible, from any mode, using an interactional-meaning strategy. Comparisons of pictures and words, for example, along interactional or attributional lines are bounded only by the creative power of the interpreter, rather than by the articulatory power of the creator.
Let us therefore examine a strategy for the interpretation of meaning which is communicational, and which uses the social process of implication and inference. My use of the terms implication/ inference is not inconsistent with their use in formal logic but is not, on at least one level, the same. I use these terms in an ethnographicsense, referring to an intentional use of symbolic material in ways which are shared by a group precisely for the purpose of inferring meaning from signs and sign-events. This process--communication--that I am proposing as a dimension along which comparisons of meaning may be made between the verbal and visual modes is somewhat similar to what others have proposed as the process through which painting or art may attain meaning (Boas 1955; Gombrich 1961; Kris 1952; Panofsky 1959).
When I use any event as a symbolic event, a sign, with the intent of sharing it with others, I am using it in an implicational manner. I only use this implicational strategy, however, when I expect that others will know the rules and conventions by which I have structured my signs and will draw the inferences from them that are "proper" to the structure. When I use signs this way, I structure them, and I expect that others know and acknowledge that structure and use it in order to make inferences.
However, the evidence for the use of communicational-meaning strategies lies in the reasoning that leads to an interpretation, rather than in the accuracy or correspondence between inference and implication. It is not necessary for an interpretation to be "right" for a communication situation to exist. It is merely necessary that both parties actually share a set of conventions about communication. The
articulator and/or interpreter must assume an intention to communicate. The articulator must structure within a social context, and the interpreter must assume such an intention to structure.
Let us now deal with the assumption of intention that, it is my
argument, is necessary for communicational meaning to exist. When
a child, or an adult for that matter, fails to recognize structure,
he will make interpretations through attribution. When he recognizes
structure, he may know that implication/attributional strategies
of interpretation are called for, or he may wish to invoke such
strategies in some particular context. The reasons for this knowledge
are imbedded in structural recognition, and basic to that recognition
is what I have called the assumption of intention.
Intention is not, in this use of the term, an empirical datum or a perceptual process such as seeing color, hearing sound, or feeling heat and cold. Nor, as I have already suggested, is it verifiable by some result of an interpretation, such as making the "correct" interpretation.
Communication, and its attendant implication and inference of meaning, is a process in which one produces a set of symbolic forms or signs in some mode--in words, pictures, or sound--as well as in some code. The social nature of this process is imbedded in the assumption of intention. That assumption is basically that the signs people choose are coded and that the relations between signs or elements are conventional.
This assumption of intention is based upon and supported by a variety of "knowledges" arising from one's membership in a group: we tacitly know how to speak our group's language, or how people "like us" behave on the street, in classrooms, and in talking on the telephone, as well as how they write books, make movies, and prepare papers for learned journals. It is most likely that a first reason for some specific assumption of intention is that the form we choose to interpret is socially coded as being possibly, or certainly, an intentional form. We are all held socially as well as psychologically accountable for certain aspects of our behavior, particularly for our symbolic behavior, and we all know that under certain conditions and contexts other members of our group will expect us to recognize this and to behave accordingly.
Interpretation consists, in a communicational sense, of a process by which X (the interpreter) treats Y (the utterance, sign, or message) in such a way that the assumed intention is for X a reason for, rather than a mere cause of, his interpretation. Evidence for the assumption
of intention is not an isomorphic matching between intended statement and interpretation, but rather the reason and reasoning given by the respondents.
This conception of intention and meaning differs from many referential, behavioral, and semiotic theories which treat meaning as, if not divorced from, at the least unconcerned with, implication, inference, and intention. Basically, we are arguing that a stimulus response theory of meaning which concerns itself with tendencies or dispositions to behave in certain ways upon the presentation of signs is totally inadequate for a theory of interpretation which takes it as axiomatic that human symbolic behavior can be communicational and is concerned seminally with meaning.
Our use of the term intention here is quite close to that of Grice (1957), who says that the statement "'A meant something by X' is roughly equivalent to 'A uttered X with the intention of inducing a belief by means of the recognition of this intention' " and that "not merely must [X] have been uttered with the intention of inducing a certain belief but also . . . the utterer must have intended an 'audience' to recognize the intention behind the utterance."
Verbal language and the grammatical structures through which all native speakers learn to recognize linguistic form are, in general, the closest paradigm to the structural recognitions we use to verify our interpretations of meaning. In other contexts and modes, we may proceed to very complex strategies, however. We may recognize poetic structures such as rhyme, rhythm, and meter, by using other conventions and rules which we know (such as those of poetry or music), and proceed from there.
If, to carry our argument further, we wish to explain why any one would want to invoke inference as opposed to attribution (clearly easier, and even more fun) as a way of making meaning from pictures, we should be able to show that signs of implication/inference are present within the recognized structure of the sign-event itself. Grice puts it quite precisely when he points out that not merely must a meaning-event have been articulated "with the intention of inducing a certain belief but also the . . . utterer must have intended an 'audience' to recognize the intention behind the utterance.'' Although this concept of meaning has most frequently been used to describe meaning in the verbal mode, Grice offers us some particularly instructive examples from the visual realm which can serve both to clarify my own arguments as to the assumption of intention and to act as a bridge between these necessary preliminary concepts of communication and meaning and the following discussion of the use of the negative in visual signs.
Compare the following two cases:
- I show Mr. X a photograph of Mr. Y displaying undue familiarity to Mrs. X.
- I draw a picture of Mr. Y behaving in this manner and show it to Mr. X.
I find that I want to deny that in (1) the photograph (or my showing it to Mr. X) meant anything at all; while I want to assert that in (2) the picture (or my drawing and showing it) meant something (that Mr. Y has been unduly familiar), or at least that I had meant by it that Mr. Y had been unduly familiar. What is the difference between the two cases? Surely that in case (1) Mr. X's recognition of my intention to make him believe that there is something between Mr. Y and Mrs. X is (more or less) irrelevant to the production of this effect by the photograph. Mr. X would be led by the photograph at least to suspect Mrs. X even if instead of showing it to him I had left it in his room by accident; and I (the photograph shower) would not be unaware of this. But it will make a difference to the effect of my picture on Mr. X whether or not he takes me to be intending to inform him (make him believe something) about Mrs. X, and not to be just doodling or trying to produce a work of art. [Grice 1957:382-83]
It seems to me correct to say that "it will make a difference
to the effect of my picture on Mr. X whether or not he takes me
to be intending to inform him. . . ." Grice, however, follows
this phrase with what I believe to be a completely misleading
final clause: ". . . and not to be just doodling or trying
to produce a work of art." Clearly he must mean that a doodle
and a work of art are in some way not used for, or are incapable
of, carrying signs of intention and meaning. I think that here
Grice falls into a habit of thought that has seemed to prevail
largely unexamined in both linguistics and the philosophy of language,
and assumes that the linguistic mode is capable of meaning and
that things called "art" (doodles) either verbal, visual,
or musical, are not capable of carrying meaning.
It is clearly beyond the scope of this paper to enter into even a brief review of this academic, or philosophical, sad state of affairs. Some sense of the context within which my own thinking has developed might, however, prove clarifying. I am thinking of concepts about language identified with early logical positivism which not only argued that nonlinguistic expressive modes were inherently meaningless but added to the bag of meaningless activities "poetry" and "art" as well (Note 3). Much discussion of language in that period was concerned with concepts of Truth and Falsity and with the ways (truth tables) that language made statements that could be
characterized as True or False or with the ways in which we can characterize the referents of language as True or False. I am also thinking of that stream of research in linguistics stemming from Bloomfield through Zellig Harris which made such an effort to define language without using the concept of communication or meaning at all.
Art critics, historians, and aestheticians, in particular, were so powerfully influenced by these notions about language that they developed elaborate rationales about what pictures were for, if (as they came to believe) they were in fact meaningless. It was in large part, I believe, in response to suppositions of meaninglessness that modern aestheticians developed theories of pictures as expression, emotion, semblances or structures of emotion, and so on. "Pictures" as a class--differentiated from something labeled "art"--were almost never studied or thought about by art historians, critics, and philosophers of art. Just as the study of language by linguists rarely concerned itself with such things as "poetry" or "literature," so in the visual realm we studied "painting" rather than pictures, "architecture" rather than buildings, and "sculpture" rather than statues. It seems to me that if we are to begin a study of how pictures mean, we must study pictures rather than painting, movies rather than cinema, drawing (on paper as well as walls, by children as well as by adults) rather than graphics, and visual structures rather than composition or design.
It is probably true that the resemblance or correspondence with
"reality," and therefore the recognition of "representational"
or "referential" pictorial signs, occurs on a basic
biological level, and occurs earlier developmentally than the
recognition of word signs. It would, of course, be quite tempting
to argue that pictorial events are iconic in relation to the real
while verbal events are symbolic and arbitrary, and that since
recognition is easier with pictures, and assumptions of existence
are more reasonable with pictures, that therefore attribution
naturally follows. Professor Gombrich (1962) has forever spoiled
this temptation for us through his analysis of the conventional
nature of pictorial "realism." Gombrich shows that it
is difficult to take the position that what we call "representational"
drawing is in fact representational because that's the
way the eye sees. What we are concerned with here is an interaction
between convention and correspondence. In the case of words, the
knowledge of convention seems more important than the ability
to recognize correspondence. In the case of pictures, on the earliest
recognihon level, a knowledge of the rules of correspondence seems
more important.
As soon as we move beyond this early recognition stage,
however, and begin to deal with communicational meaning and its attendant recognition of order and structure, these differences that might seem to exist between pictures and words become trivial. As long as we stay on an attributional-meaning level, questions of representation, correspondence, and biological similarity of operation become the most important dimensions of comparison, and the difference between the two modes assumes major proportions. When we move to the symbolically complex levels (as opposed to the biologically complex), these questions seem less important. Methods of recognizing order and structure in both words and pictures seem to be similar cognitive processes. The way we perceive such things as order and structure are, it seems to me, "meta" operations that the human mind imposes upon both the symbolic and the "real" universe. It is my position (which I shall not argue here) (Note 4) contrary to, for example, Rudolph Arnheim, that the world does not present itself to us directly; that in the process of becoming human we learn to recognize the existence of objects, persons, and events that we encounter, and to determine the strategies by which we articulate, interpret, and assign meaning with and to them.
Communicational meaning, involving the recognition of order and structure, is, I believe, similar for words and for pictures. Although it builds on the physical and biological correspondences between symbolic modes, it is a meta-recognition, a cognition applicable to all modes as mental operations, rather than as judgments of physical or biological correspondence. I am suggesting that the recognitions I talk about here are applicable to mathematical, musical, logical, verbal, and pictorial modes and that the conventions attendant on communicational meaning (metaphor, rhythm, rhyme, similarity, repetition, analogy, and so on) may also be found in all modes.
Not only do we tend to believe that most pictures represent the world closely (are similar to it), but words in our culture have lexicon definitions that limit the attributions one can "reasonably" make; pictures do not.
Parents and grade school teachers socialize our children into the belief that pictures have few rules for either qualitative judgment or the interpretation of meaning. It is a rare father who would watch his son swing at a ball in a Little League baseball game ten consecutive times without hitting it, and shout, "That's a good boy, Joey. You're doing great." Yet the same father, upon Joey's presentation of an 8 x 10-inch piece of paper filled with variously colored shapes, will
say, "Say, that's wonderful. What is it--an elephant? How nice." Joey might reply proudly, "No, it's not an elephant, it's a picture of me and Mommy playing baseball, and I hit a home run." "Oh," says the proud father, "that's right. That's great, Joey. Ask Mommy to hang it up."
Not only is the above somewhat bizarre-sounding conversation not unusual for parents and children, it is not unusual among poets and painters either. They also recognize the distinction between the two strategies for the interpretation of meaning and often take sides in a prescriptive dispute, demanding that their "audience" employ either one strategy or the other. Although Paul Valery is hardly a typical proponent of the proud father, a child's drawing, or a sandlot baseball game, it might help to clarify my point about the similarity of words and pictures in terms of communicational meaning to quote Picasso arguing with Valery on the distinction I am proposing.
Valery used to say, "I write half the poem. The reader writes the other half." That's all right for him, maybe, but I don't want there to be three or four thousand possibilities of interpreting my canvas. I want there to be only one and in that one to some extent the possibility of recognizing nature, even distorted nature which is, after all, a kind of struggle between my interior life and the external world as it exists for most people Otherwise a painting is just an old grab bag for everyone to reach into and pull out what he himself has put in. I want my paintings to be able to defend themselves, to resist the invader, just as though there were razor blades on all the surfaces so no one could touch them without cutting his hands. A painting isn't a market basket or a woman's handbag, full of combs, hairpins, lipstick, old love letters and keys to the garage. [New York Times, 9 April 1973]
Note that one can substitute poem, novel symphony, dance, or any other symbolic form for the word painting above. No artistic implication should, as Picasso sees it, become a grab bag for everyone to reach into and "pull out what he himself has put in."
On an attributional-meaning level, however, the listener may indeed be able to do as Valery hopes; write half, three-fourths, seven-eighths, or any and all proportions of any work. He may, if we do not constrain attribution by personality and culture, put anything into a work and happily extract anything out of it. On a communicational level, on the other hand, the reader (again within some constraints of form, content, and so forth) does not write any part of the poem, any more than the viewer paints the picture or makes the film. The reader (viewer), if he can participate in a communications event, recognizes the work's structure, assumes an intention to mean on the
part of the creator, and proceeds to his extremely complex job of making inferences from the implications he can recognize.
What is it, then, that pictures cannot do that words can do? Not
only are words able to deal with negatives, but some linguists
(for example, Sapir 1921) have speculated that the ability of
words to deal with what is not is one of the central functions
of language. Pictures, I shall argue, cannot deal with what is
not. That is, they cannot represent, portray, symbolize, say,
mean, or indicate things equivalent to what verbal utterances
of the type "This is not a . . ." or "It is not
the case that . . ." can do.
On a trivial level, we can construct picture symbols or signs with negative meanings that resemble language, as do some parts of Egyptian wall paintings or other hieroglyphic forms; "languages" for traffic signs and advertising have been developed which have wide usage across verbal language groups. A red crossbar over any image means "forbidden," or "do not," so that a crossbar across an image of a car means "no cars," and so on. These uses in posters, traffic signs, and even price tags are not pictorial uses but rather linguistic uses of visual forms which become sign elements in a special language. They differ from what Gombrich (1961) calls a schema for picturing. In that sense, we are talking about ways of picturing, ways of structuring the universe through visual symbolic forms. In the former, trivial, sense we are talking about specific pictures or visual forms to which we assign some particular lexical meaning or function. The crossbar becomes a stimulus sign to which birds, dogs, and other animals, as well as man, can be conditioned to respond.
In some sense, also, every positive or existential statement carries along with it the statement that it is not any other. The statement "this is a cow" or "I am a man" carries within it the conventional understanding that "this is not something that is other than a cow or other than a man," or, more exactly, "this is not a noncow" or "not a nonman." So also does a picture of a cow or of a man carry along with it the knowledge or understanding (in this culture at least) that it is not a picture of something else. Again, I believe that this aspect of pictorial negation is trivial.
However, on a value level, what is pictured is often valued precisely for what it negates by leaving out--so that in modern art it is possible to be nonrepresentational, either by being nonobjective or by being an abstract expressionist. In music, I suspect that certain notes are expected in certain codes. In earlier periods, depicting vulgar images was a rejection, and in that sense a negation, of other
conventions and prescriptive rules. We therefore can, by means of our rules of picturing, accept as negations the absence of such social concepts as representation, illustration, sentiment, imitation, contrivance, vulgarity, nobility, dynamism, and so forth (Note 5).
I have introduced the value level not because I wish to make a point of evaluation, but to make a point about "meta" levels of interpretation. When we make judgments about what a picture maker did not do, as well as upon what he did do, we make a judgment based upon our knowledge of choices that the picturemaker had available to him, both as psychological individual and as a member of a society performing a social act. We do not, however, make these judgments based solely upon what is in the picture itself. For example: in looking at The Raft of the Medusa, we "know" that Gericault could not picture "I am not at home in my comfortable easy chair." He could paint the picture he did and expect us to recognize that, but everything else is not happening on the Raft of the Medusa. A picture-maker cannot specify, out of all the things that his picture does not show, which he means to say are not the case. There is no pictorial means that a painter has of indicating that a color, a shape, or an object is not something, or anything, else. All that pictures can show is what is--on the picture surface.
It is for that reason that it seems reasonable to argue that True False criteria cannot be applied to pictures and that, further, pictures cannot be said to "make" propositions. We say of verbal statements that they are "not true" or are "false," or even are "full of baloney." We rarely if ever, in ethnographic fact, talk that way of pictures.
Let us then first examine what we do say of pictures on a continuum of correspondence to something called "reality." On one end of this continuum, we have the motion picture/television image, a supposed correspondence to reality, in color, with motion and sound. At the other end we have the picture of the abstract expressionist or perhaps even that of the conceptual artist who uses only words and produces no picture at all. In between, we have paintings in a variety of styles and conventions, such as caricatures, cartoon strips, or the sort of abstractions produced by Picasso and Braque that imply some correspondence with the "real" world, but portray that correspondence in nonrepresentational ways, or in less representational ways than do photographs or movies.
Let us also examine how people actually talk about hand produced, as opposed to machine-made, pictures. At one end of the continuum of hand-produced pictures--the abstract,
nonrepresentational spectrum--a viewer not versed in conventions of abstraction might say that such pictures are "silly," "make no sense," "are not understandable," and so on. Our unsophisticated viewer will almost never say that a Picasso is false or that a Phillip Guston (in his abstract-expressionist period) is false. Hand-produced pictures, a viewer "knows," are somehow supposed to correspond to some concept he has about reality. If pictures do not correspond or are not judged similar enough to this "reality," the picture-maker is judged to be inept, a child to be humored, a "modern" artist to whom attention need not be paid, or some other form of incompetent or deviant. Rarely, however, is such a person considered to be a liar. Rarely is he understood to be deliberately trying to lie to the viewer. Unlike words, pictures may be thought to show it the way it is, but pictures are rarely thought of as telling lies in the way that words do.
The cliche has it that pictures cannot lie, and this is, even today, a largely acceptable statement, albeit with confusing counterexamples which lead to all sorts of angry responses that I shall deal with below.
In the case of Pollock, the unsophisticated viewer makes judgments of both deviance and incompetence, often saying, "He's crazy!" and, "Why, he can't even draw a face!"
In machine-made pictures--photography and film--we have supposed a value-free picture producer. It (the machine) tells neither truth nor falsehood but, again, tells it "like it is. " The machine is to be trusted to produce an image that corresponds to that portion of the world to which it is pointed. What happens when you see a photograph of a familiar face and you fail to identify the subject of the photograph? Contrary to how you would act if it were a painted portrait, you do not doubt either your ability to identify the face nor the honesty of the photo-taker who said that this was a photograph of someone you know. You blame "reality," "the photographer," or the machine and the process. In the first case (let us say it was a photograph of your wife) you might say, "She doesn't look like herself these days," or "From that angle (in that light), she just doesn't look like herself." If you blame the picture-taker, you might assign some of the same reasons as his fault: "You should know better than to take a picture of Mary from that angle." The comment, "It's so underexposed that you can barely see anything" is another way that the picture-taker receives the blame when instant recognition does not occur. In the last case, when the process is at fault, you may get comments ranging from "What can you expect from a dumb machine!" to "You have to spend so much time fiddling that you have no chance to think about the person you're taking a picture of." In
some studies of home moviemaking and exhibition, as well as snapshot albums and commentary about them, Chalfen (1975) found that most commonly, negative evaluations blamed the mechanical aspects of picture-making, rather than the human. If a photograph was over or under-exposed, "that stupid exposure meter" was to blame, and if the heads of people or other important parts of the picture are cut off, "that stupid viewing system" or "these lousy cameras" were to blame.
All of this is based, of course, on an assumption of intention to portray and depict a scene that corresponds to that which the camera was pointed at. If we assume another intention, we have a choice of an intention to produce "art," or a deliberate attempt to produce a product that will fool us. I have already discussed, in the section on Grice, the possibility that art and "doodles" are equivalent in relation to some concepts of meaning. But what about an intention to trick or fool us--an intention to show it the way it isn't--and to make us think that is the way it is?
If we ask people what a "false photograph" is, almost everyone immediately asks, "You mean a fake?" A photograph that doesn't correspond (in the accepted way) to reality is not a lie, because we tacitly "know" that the medium has no conventionalized procedure for stating lies. The only way a photograph can be understood not to correspond to reality is when we change something in a hidden, secret, and hence tricky, manner. If I superimpose a picture of the honest senator who swears he didn't know the gangster upon a scene of the gangster having dinner with his cronies, so that it appears that the senator is toasting the gangster, I have produced not a lie but a fake. The attributions one might make from such a photograph would be empirically false, but the picture would in all respects correspond to what it would look like if the senator had been there. If I paint a picture of one woman (Mrs. A) and present it to a viewer as a picture of another (Mrs. B), it is not the picture that lies, but the picture presenter.
A movie of a boy with green hair is also not seen as a lie, and barely as a fake; it is mostly admired as a clever manipulation. It falls into the realm of fantasy, rather than fake, but is not judged by criteria of truth or falsity. In a very deep sense, I am sugeshnp that the real world is symbolically inviolate. If noncorresponding messages about it are made verbally, they are either mistakes, lies, or false statements. If noncorresponding pictures are produced, they are "fakes" or "tricks." The real world is, and is in a sense that supersedes symbolic manipulation. We would rather change our concept of the "real" to match our images or myths, if need be, but in any event we rarely allow a
conflict between a pictorial symbol and "reality" to go on for long.
If we are faced with a conflict between that which a picture shows and that which we know cannot be the case, we do not shout, "Lie!" but instead say firmly, "It cannot be so." The case of the so-called "impossible figures" of Penrose and others offer us an almost perfect paradigm for how we treat noncorresponding pictures (Figure 7-1). They are in fact "impossible." Gregory (1970) has confounded the issue further. Almost all impossible figures are drawings. Gregory has provided us with a photograph that purports to be the figure in Figure 7-1. When this photograph is shown to a class of high school students who have listened to a lecture on perception, we find two responses: one is anger, and the other is a happy demand for an explanation of "how that trick photo was done."
The angry response is a common one; it is not the anger of someone who has been lied to, but the anger of someone who has been tricked. It is an anger that I call media rape. It occurs with greater and greater frequency as artists in all modes of symbolic activity-- painting, moviemaking, television, novel writing, and news reporting --are trying more and more to explore the distinctions between the real world and their symbolic world. In a movie such as W. R.: Mysteries of the Organism, Makavejev, the Yugoslavian master, uses old documentary films, current documentary films, and old as well as current reenactments of public and private events. He has photographs of Stalin and photographs of actors playing Stalin intermixed and so juxtaposed with current acted and documentary footage that it is
almost impossible to tell which is which. Many members of the audience, particularly in socialist countries where such experimentation is relatively new, become quite angry.
In America, the film Medium Cool, which mixed acted and documentary footage in ways never before attempted, created genuine rage on the part of many audiences. They felt that this film went beyond trickery. Mixing actors and the riots at the 1968 Democratic Convention in Chicago in a way designed to make an acted plot seem real, or to make reality seem like acting, was not "right." The schemata of movie representation demands clear separation between that which is to be thought of as acted or unacted. We want to be able to say that an acted film was almost real enough for us to believe, but we want a performance, not confusion. We must be able at all times to "know" the difference.
Again, in films such as Medium Cool, the audience's response is not that the film is, or tells, a lie, or that the film is false. The response of many people is that the images on the screen are impossible. What they see before them cannot be. The feeling of "cannot" is so strong that many people move without thinking from "cannot" to "should not." And thus, it seems to me, their anger. I have seen the work of Escher responded to in the same way. If one cannot take pleasure in the manipulation and the creation of a structure that cannot be, one tends toward nonacceptance of the medium, or genre, itself as legitimate, and thus may become angry (Note 6).
Pictures, as we understand them in this culture, depict, or picture, what is. They are, in a visual mode, somewhat similar to the verb "to be" in its existential, not veridical, sense. Pictures, however, cannot depict conditionals, counterfactuals, negatives, or past-future tenses. Neither can they make passive transformations, ask questions, or do a host of things that a verbal language is designed to do. Pictures depict the present even when they depict fantasies such as fairy godmothers, myths, unicorns, and gods. When a picture depicts the star Venus, its context (day or night) might help us to label it the "moming star" or the "evening star," but the classical problem of the referential meaning of morning stars is not involved when we deal with pictures. Instead we may wish to ask how the context of the picture affects our labeling of that particular star. That is: how dark or how blue must the sky be for the star to be "morning" or "evening"?
Since pictures do not have the formal capability of expressing
propositions of negation, it follows that pictures cannot be treated as meaningful on a dimension of truth and falsity. If pictures cannot depict the proposition that something is not so, or is not the case, it would hardly be reasonable to suggest that pictures are designed to depict only those things that are the case.
What then do pictures depict? It seems that all we can say is that what they depict is. They depict events for whose existence they are the sole evidence. Pictures in and of themselves are not propositions that make true or false statements, that we can make truth tables about, or that we can paraphrase in the same medium. Pictures, it must be remembered, are not representations or correspondences, with or of, reality. Rather, they constitute a "reality" of their own.
But if pictures are not propositions, and not even representations, and if pictures cannot be dealt with on True-False dimensions, how then are we to deal with them? What is the "logic" by which meaning is interpreted from pictures? Leaving aside attributional strategies, from which, as we have seen, any meaning may be made, how do we understand the logic by which implications are made and inferences drawn? By removing the propositional property from pictures, I seem to have removed the possibility of grammar or syntax as we know it.
Truth tables can only be constructed for sentences. What are sentences in pictures? Logical syntax requires precisely defined connectives such as "and," "either . . . or," "if . . . then." Is there anything like that for pictures? I am suggesting that the basic difference between words and pictures lies in the fact that our use of speech is based upon a convention that requires a clearly defined syntax which allows us to articulate propositions about truth or falsity, while our use of pictures, on the other hand, is based upon conventions which, while linguistic in nature, have no clearly defined syntax, no ability to articulate propositions (and therefore no ability to depict negation), and no ability to make "meta" statements about lower-level statements of the picture system. A picture cannot comment on itself. A picture cannot depict "This picture is not the case," or "This picture is not true."
Before continuing, I must digress, or at least stop for a moment, to consider the argument that although pictures cannot deal with negation, they can deal with something called truth. Let me briefly mention some of the arguments that lead me to feel that the subject is indeed a digression, although much attention has been paid to this issue (Price 1949; Urban 1939; Reid 1964; Casey 1970).
The theory that there is truth in pictures usually rests on the argument of correspondence, and correspondence usually means
either similarity or correctness, or both. The notion of similarity rests to a large extent on the iconic-digital distinction between pictures and speech. Even linguists, in recent years, have given up their certainty about the arbitrariness of linguistic symbols (Friedrich 1974). Semioticians, art historians, anthropologists, and psychologists have also come to realize that a "copy" theory of pictures is simplistic, misleading, and probably just plain wrongheaded. A look at Chinese painting or another look at Figure 7-1 should make the weakness of the copy, or similarity, theory of picture-making obvious. Truth is not to be found in that direction.
Second, similarity is not, and cannot be, necessary to correspondence (Note 7). A conventionalized code (the Morse code) can make dots and dashes correspond to letters of the alphabet, although their degree of likeness to letters is very small and their degree of similarity to the sounds of speech is almost negligible. Thirdly, similarity is an almost impossible criterion of correspondence. Similarity, or verisimilitude, asks that we match one thing (the picture) against another. How close a match makes a match? Clearly only a picturing convention or schema can tell us whether the picture is similar to something in reality.
If we take correctness as our criterion for correspondence, we fall into a set of problems that confuse, rather than clarify, the issue. Here we usually mean scientific correctness, or accuracy. As a matter of speech, we refer to "accuracy" colloquially as "scientific truth." Science carries high status and is the closest approximation we have to a method of determining something called empirical truth. But in order to be correct, we must have a measure, a standard. We must have something to measure against. "Reality" is too vague. Do we compare to a standard of our eyes what we see--or to our cognitive capacities what we know? In either case, we are again confronted with conventions, rules, and schemata. If we pass that problem, we find ourselves back to the problem of nonrepresentational pictures and what they correspond to. Truth, as it concerns pictures, is indeed a digression.
Pictures and speech are different precisely because pictures are
not a language in the verbal sense. While words mean, primarily
or basically, because of lexicon and syntax, pictures have no
lexicon nor syntax in the formal grammarian's sense. And yet I
am suggesting that we can interpret meaning from pictures. It
is clear, however, that
if pictures have no grammar in the strict linguistic sense, they have something like it: they have form, structure, convention, and rules. It is clear that even though a theory of correspondence is not sufficient to deal with truth in pictures, pictures must nonetheless correspond to something. Even the most un- or non-representational painting must refer to something, or it would make no sense at all. Although attributional strategies are convenient for the unskilled, no picture-maker likes to think his picture is totally up for grabs.
Earlier I suggested that the strategies we employ to interpret meaning from pictures--that is, how pictures mean--are largely responsible for what pictures mean. I suggested that if we use attributional strategies, pictures can mean almost anything. The limits placed upon our interpretation in attributing meaning are dependent mainly upon our individual psychological, social, and cultural histories. These histories interact with the sociocultural limitations we place upon what may be interpreted from pictures in specified contexts.
If, on the other hand, we use communicational strategies, a particular set of meanings can be developed for pictures, as well as for that which we have often defined as "art." In general, what we imply and infer through pictures are, first, an existential awareness of particular objects, persons, and events that are ordered, patterned, sequenced, and structured so as to imply meaning by the use of specific conventions, codes, schemata, and structures. And second, in what Larry Gross (1973) has termed the communication of competence, pictures, more than speech, and perhaps like some of the special modes or codes such as poetry, the sonata, or the story, communicate the competence and skill with which their structures have been manipulated according to a variety of rules, conventions, and contexts.
I have briefly outlined earlier why I think the notion of matching to the real world is insufficient to explain how pictures mean. Now I can say that I don't believe this is what one matches pictures to at all. Correspondence, if it makes any sense as a concept, is not correspondence to "reality," but rather correspondence to conventions, rules, forms, and structures for structuring the world around us. What we use as a standard for correspondence is our knowledge of how people make pictures--pictorial structures--how they made them in the past, how they make them now, and how they will make them for various purposes in various contexts. We do not use as our standard of correspondence how the world is made.
Our notions of correspondence, of similarity, and of correctness in the meaning of pictures are evidenced more in statements such as "That's a movie?!" "That's no mural!" and "I don't even call that a
picture!" rather than by statements such as "That's not true," "That's a false picture," or "That picture means that it is not about a sunset." What we mean when we say "That's not a movie" is that the articulated symbolic event before us does not correspond to our conventionalized knowledge of the way signs in movies are manipulated in our society. In effect, pictures are a mode that best communicates a dialogue with the "real" world that Picasso called "a proposition; to the viewer in the form of traditional painting violated." "I want," he continued, "to give my work a form that has some connection with the visible world, even if only to wage war on that world" (New York Times, 9 April 1975). That dialogue in the form of traditional painting violated is similar to what some painters have meant when they said that painting was about painting, and what Malraux meant when he said that painters did not copy nature: painters copied other painters. Pictures, in this sense, picture conventions, forms, structures, and so forth. Pictures are a way that we structure the world around us. They are not a picture of it.
Although pictures do not have a grammar by which to structure how the world is, they are clearly not without forms, genres, styles, conventions, rules, and systems of usage. The concept of the "language of pictures," the "grammar of art," the "syntax of the cinema" must be understood as a metaphor, at best. In another paper (chapter 1) I have discussed the problems of describing a film grammar and have shown that certain concepts that make sense in speech or verbal usage are simply not used in motion pictures. Notions of "grammaticality," "native speaker," "paraphrase," and grammatical or syntactic transformation, while powerful enough to be forced into applicability in relation to almost all symbolic usage, actually make very little explanatory sense when applied to pictures. We can always say of pictures that grammaticality refers to that set of rules that allows even an unsophisticated viewer to "know" that a drawing is unacceptable because the perspective is "not right." We can stretch the concept of grammar to say that the recognition of "impossible figures" demands a native speaker's knowledge of the grammar of visual representation. To some extent, of course, an ability to interpret perspective is necessary in order to infer meaning from a perspective drawing, but it seems to me to be a distortion, or at least not very helpful, to refer to such conventions as perspective as a language or grammar of pictures.
It seems to me rather that pictures operate both within the framework of language knowledge within us, and outside the framework of language in itself. That is, the pictorial mode (from drawing to motion pictures) does not have a rigorous set of rules employing
a lexicon, a grammar, an ability to construct paraphrases, or an ability to produce translations within its own formal devices. But we, the viewers, do have a facultéde langage in general, about all symbolic materials, so that in motion pictures, for example, where sequence and time become parameters to be manipulated, we can instantly bring to bear linguistic rules for implication and inference. In other research (Worth and Adair 1972) I have shown that people who are native speakers of Navajo will frequently use Navajo syntactic rules as justification for the structuring of films that they themselves have photographed and edited.
Metz (1970) has shown quite convincingly that the acceptability of film content most often depends on its adherence to film convention rather than on its adherence to "reality." For example, a shopgirl is depicted in films in a certain way. Everyone "knows" that real shopgirls do not look or act that way. If a real shopgirl were to be cast in a film, we might recognize her correspondence to life, but would reject her because of noncorrespondence to film. What we call "true to life" must be a stereotype if it is to be recognized, and therefore becomes the least, rather than the most, valued as "art."
What is communicated by pictures, then, is the way picture makers structure their dialogue with the world. What is meant by pictures, when we use a communicational strategy of interpretation, is how we should put the pieces together. First, as in Figure 5-2 (see chapter 5), we recognize some object, person, or event in movies. It may be a "tree," or a "man." In a painting, it may be a representational object, or a color, shape, or juxtaposition of elements. We can, and many people often do, stop right there. They start attributing-- putting onto and into the picture. Others, however, are able to go further, both in the articulatory as well as the interpretive process. They recognize and can articulate structure, assume purposeful manipulation and, therefore, social behavior, and treat that manipulation as a set of instructions by which meaning may be inferred.
When people are speaking, participants are able to be speakers as well as listeners. In picture-making, as in reading novels, "dialogue" or "discussion" does not exist. What we appreciate is the manipulations that the picture-maker or writer performs on his materials and our ability to recognize and to understand the conventions, rules, styles, and usages within which his particular dialogue with the world has been carried out and in which we may share.
To sum up then, I have tried, in this paper, to begin an exploration into how and what pictures mean--by themselves, as particular modes of symbolic event, and in comparison to the verbal modes. I have described a method for discussing what symbolic events mean
x
through the use of a communicational as well as an interactional strategy for the interpretation of meaning. In comparing speech and pictures, I have argued that on the communicational level, the major differences lie in the fact that pictures provide for the articulation of existential rather than veridical events, and that therefore pictures cannot provide us with propositions or propositional statements; that pictures do not have the formal capability of depicting negative events, and that therefore the dimension of truth or falsity is a fairly useless dimension with which to think of and about pictures. I further argued that therefore the meanings implied and inferred from pictures cannot be on a true-false continuum but rather on an exist do-not-or-cannot-exist (impossible) continuum.
I then argued that if the way in which pictures mean is thought of as either interactional or communicational, an interactional strategy makes it possible to have any picture mean anything, while communicational meaning was primarily constituted by an interpretation of competence in the presentation of a dialogue between picture event and "reality" concerning the very act of structuring that reality.
Neither of these strategies of pictorial interpretation, it seems to me, is or can be formally concerned with truth or falsity. Speech and verbal behavior, on the other hand, may not always be concerned with truth or falsity but are formally and socially designed to be so.
A semiotic of pictures, then, must begin to describe the structures by which visual communication, in its many codes and modes, presents its own unique dialogue with the world.
1. This paper originally appeared in Verses 12 (1975)-85-108. The paper has been edited to eliminate sections that essentially repeat material presented in chapter 5--Ed. Return.
2. Much of the work on strategies of interpretation reported in this paper, both theoretical and empirical, was done with the collaboration of Larry Gross and is reported in Worth and Gross (1974). I have also benefited greatly from discussions with him about most of the material in this paper. Return.
3. It was Carnap who urged that unverifiable statements be thought of as merely expressive and not be confused with meaningful statements. Return.
4. See chapter 4, above. Return.
5. For an interesting discussion of this issue, see Gombrich (1963a). Return.
6. See Chukovsky (1963), who points out a similar response: the anger of adults at children's fantasy verse. Return.
7. I have relied heavily on the excellent review of this concept that appears in Casey (1970). His final conclusions, it should be noted, differ markedly from mine. Return.