Images as the Text: Pictographs and Pictographic Logic

Johanna Drucker and Jerome McGann

	Beauty is momentary in the mind
	The fitful tracing of a portal;
	But in the flesh it is immortal.
		Wallace Stevens, "Peter Quince at the Clavier"


Ignace Gelb, the renowed Assyriologist and authority on the history of writing, made a distinction between writing and other forms of notation.<1> Writing, he stated, is a system capable of transcribing linguistic statements while other forms of graphic notation allow for human intercommunication through a stable or conventionalized set of marks or signs. Pictographs fall quite clearly under the second set of descriptive requirements, and thus are often distinguished from writing by definition.

A true pictograph functions as an image whose meaning is communicated through its visual form as a picture of something, whether the communication is effected through substitution or translation into language or not. Historically speaking, we know of very few fully developed pictographic systems. Ancient writing systems such as Egyptian hieroglyphics, Babylonian pictographic precursors to cuneiform, Hittite hieroglyphics, and other scripts in the ancient near east, used pictorial forms but usually in what are known as "mixed" systems. The pictographic representation of an animal, object, or person functioned as a sign in a system that also used ideographic, logographic, syllabic, and phonetic principles to represent concepts, words, or sounds. Mixed systems are also characteristic of the new world languages and writing systems from mesoamerica, such as Mayan and Aztec glyphs. Many of these systems used the sign in multiple ways (so that, for instance, the rebus of an eye would be readable as "I", as the concept "to see", and so forth).

What all of these systems share with each other, and with other modes of graphic representation from the most schematic mark-making to the most elaborate of pictorial instances of visual representation, is a higher order of graphic representation that puts the elements of the system into logical relations with each other. Take for example the placement of signs into columns, into directional orientation with respect to each other and their order of reading, or the use of the graphic surface -- stone, papyrus, clay, paper, bark, or whatever -- as a space in which that order is articulated: all these involve conventions whose structure is rarely given attention as a set of formal principles. These conventions can be described as elements of a pictographic logic. Diagrammatic in nature, they are founded on two fundamental principles: the description of an entity and the articulation of relations among entities. These principles, critical to writing systems, pictographic and otherwise, also underlie the structure of graphs and logical diagrams of various kinds. Furthermore, they can be used to make explicit the logical structure of inscribed or printed texts in their embodied, visual terms.

In this paper we are proposing an abstract logic of graphic relations for all such graphic forms. This logic provides the primary groundwork for conceptualizing information design in visual and textual environments, not on the basis of their specific content or form, but at another level of organizational logic usually left inexplicit. Our premise is that the concept of a pictographic logic, or metagraphic logic, is a useful way of making explicit the abstract ordering of elements in texts, images, and graphic forms. First we want to show how this logic can be elicited from pictographs and graphic forms that are fundamentally non-linguistic in nature. Then we want to step away from current interpretive norms for studying the relation of the linguistic and the graphic features of text, along with their inherited conceptual assumptions and premises. Such norms are codified in various ways by specialists in different disciplines. Bibliographers, for example, regularly work from a distinction between the "substantives" and the "accidentals" of text, the latter representing what are perceived as largely secondary, formal features - like punctuation -- that merely support the conceptual core or "substance" of the text. A variant of this distinction is replicated in the more recent field of electronic textuality as the distinction between the content of text (which is logically marked) and its bibliographical forms (which go unmarked and hence are left outside the analytic framework of the electronic text). Perhaps the most widespread and manifest instance of the general distinction is the one all but universally made between a text's conceptual content and its layout, page design, and typography. In this frame of reference, graphic features of text are ancillary and transient forms -- apparitions, as it were, that house textual essences as fleshly frames support immortal souls. We bring that metaphor forward here because in our view this approach to textuality is riven by an ideology explicit in that figure of speech.<2> This habit of thought is widespread in discussions of the history of writing systems as well as in information design. Take as an example Edward Tufte, whose work is often taken as an emblem of clarity of purpose in its presentation of apparently seamless "envisionings" of designed information. A key assumption underlying his work imagines that the information has an essential form needing to be revealed. Successful information design is measured by its capacity to bring that revelation into a form as clean and transparent as any crystal goblet. But the fact is that the graphic designs celebrated by Tufte are not vehicles for delivering coherent and pre-established information content, they are the creators of such content.<3> This vehicular approach to material signifiers pervades our thinking about language and writing systems.

In order to shift our view away from that kind of thinking we offer here a set of programmatic examples. We begin with a set of five instances of pictographic inscription, each of which is fundamentally non-linguistic: an instance of a Sumerian clay sign-token from the late fourth millennium B.C., an Aztec pictographic narrative, Bishop John Wilkins' philosophical writing system from the 17th century, the existential graphs of Charles Peirce, and the logical diagrams of John Venn. Following this we shift register, and approach the problem through a study of the "metagraphic" logic operating in a text that appears to be purely linguistic. This case offers a poetic text by Byron. In making this move, we are suggesting two things. Pictographic logic, inexplicit but present in visual form, can be described at a level where images become abstracted into a logic of relations. This graphic logic also operates in texts, where it is particularly under-recognized as part of the semantic field. We are therefore insisting that these two principles of abstract logic – a logic of visual form as such and a graphic logic of textual practice -- are two aspects of the same general premise. Now to the examples.


Considered "protowriting", clay sign-tokens used in ancient Sumer to record business transactions are among the earliest surviving examples of systematic use of a stable writing system.<4> On these clay tablets signs representing things (jars of oil, animals, grain, or other goods) were inscribed alongside signs indicating numerical quantity through a more abstract system of notation. [Figure 1] The number of jars, oxen, or sheaves of grain was not represented by a literal repetition of its sign, but by the placement of the sign in proximity to a number sign. (Thus 33 jars of oil would be represented by the sign for the jar and another or others for the numerals 33). In this context, the first signs are quite literally images of things. To be understood they do not need to go through a translation into language, though they lend themselves quite readily to clear restatement in words. Nonetheless, as images, these signs also carry a certain amount of information in their own right: the shape and size of a jar indicates something about the artifact and its cultural context, for instance, just as the way the signs are drawn provide a certain amount of documentary evidence that inflects our reading of the sign. We can read through the sign, to its purported meaning, or we can read the sign itself, as an image and artifact.

In this example, the logic of the pictograph is quite straightforward. It contains two elements: signs of items and signs indicating quantity. These two systems are different from each other. The sign for the thing is a pure pictograph. The sign for the quantifier is part of an abstract code, not necessarily bearing any iconic resemblance to the quantity it represents. The intellectual leap that allows for abstraction frees written signs to function as a highly powerful code. This is the "double articulation" that distinguishes writing from other notations or visual representations and permits it to function according to Gelb's definition of writing, as a system capable of recording any utterance. The two systems, a pictograph and an accounting system in abstract code, function together in these sign-tokens, but the logical order operates graphically through differentiation and juxtaposition. The juxtaposition in this instance permits the two systems to operate in relation to each other, as distinct but related orders of representation: pictographic and numerical. The signs are disposed in a graphic space, their zones demarcated by the boundaries of the signs themselves (they are constituted entities, just as cuneiform, alphabetic, hieroglyphic, or other graphic signs). This might seem overly obvious, but the example contains the two fundamental elements of a pictographic logic: entities and their relations. These elements become enmeshed in far more complicated equations in other examples of graphic, pictorial, and typographic format.

The pictographic representation of a travel narrative in an Aztec Codex demonstrates the use of a spatialized disposition of signs whose relations are more scenographic than those in the Sumerian sign-token.<5> [Figure 2] The graphic space is pictorial, and does not rely on the conventions of columns, lines, or other constrained arrangements of marks that characterize writing. The scene depicts Aztecs leaving their homeland, and pictographs that represent travel (footprints) literally connect the place of departure (on the left) with that of destination (on the right). Elizabeth Hill Boone's discussion of the image notes the considerable efficiency with which the pictographic signs depict the story.<6> Her description of the actions and events takes about five hundred words, detailing the use of glyphs to indicate place, identity, dating, and other specifics that are indicated in the condensed form of pictographs.

But the density of the information in the individual glyphs is choreographed into meaningful relations by the use of several logical principles. The direction in which the figures face, that is, the orientation of the signs within the delimited field of the surface, is a dynamic element, one that instructs the reader to interpret the signs in a specific way. The boundaries of the two locations (origin and destination) cut them out from that surface so that the gap between them can be crossed. Because these spaces are delineated, they become entities capable of being connected. As Boone points out, a sequence of events clearly indicated by the way the different spaces are linked. She notes further that the "conventionalized meaning of the images, and their spatial or directional relation to each other, must be known before narrative is understood."<7> The key point is that these conventions make use of spatial and directional relations, since these relations demonstrate an active use of the diagrammatic capabilities of the graphic space.

Inventing entirely pictographic writing systems, a number of 17th century philosophers created densely coded visual sign systems. These idiosyncratic notational schemes, such as that of Joachim Becher and Bishop John Wilkins attempted two things.<8> They wanted to find a set of signs that could encode a well-ordered (philosophical) representation of all the elements of the known universe. To this end they struggled to create sign systems that would inform the reader about this universe through the decoding of the sign. This peculiar isomorphism depended on an elaborate hierarchical outline of knowledge represented in a schematic code. In both Becher's and Wilkins's systems, a set of bars, lines, curves, and dots gain meaning through a set of formal conventions.<9> [Figure 3, Figure 4a, [Figure 4b] These are meant to mimic the formal relations of the knowledge scheme that renders the universe in an orderly, rational manner. Position, location, and relative length of line, all become meaningful because the scheme is carefully controlled. The signs are not transparent or immediate– they have to be learned, though the expectation was that they might come to be sufficiently familiar so that a reader could acquire knowledge about the order of the universe through decoding the signs. But the mythic conception that inspired these sign systems -- of the hieroglyph as an image whose meaning immediately struck the mind through the eye – wrong in its own right, also failed in these circumstances. The signs were a code of a code, too complex for easy reading, too reductive to carry the semantic richness of ordinary language or the pictorial repleteness of visual images.

The realization that pictographic logic is conventionalized, not natural or intuitive, merely reinforces the useful insight that the bases of the conventions are graphic, rather than linguistic, and thus use the kinds of properties of space, location, and orientation that work in these very different visual systems. When Charles Peirce, the American philosopher and logician, sought a means of representing logical propositions through graphic devices, he clearly recognized the limitations to other methods of symbolic notation, especially those that followed the conventions of algebraic formulae or linguistic form.<10> The existential graphs he invented were conceived as a method of doing logic and not merely for representing its propositions or conclusions. In this regard, though his graphic imagination remains closely tied to the formal questions he was posing, he leapt to make use of the potential of diagrammatic operations at their most powerful. [Figure 5]

Peirce's existential graphs relied upon three elements: a surface, definition of entities, and articulation of relations among these. The surface Peirce termed a Sheet of Assertion, the entities being defined by outlined borders, and the relations by lines of connectivity, with various graphic devices engaged in nuancing these. Peirce worked with this visual format in part because he saw the possibility of using iconic principles of resemblance between the ideas he was representing and the graphic shape of their appearance. Given the potency of belief in formal logic as an analogue of thought, Peirce may well have imagined that this iconic correspondence had at its foundation a potential to be a diagrammatic representation of thinking processes in their logical form. Peirce's work with the graphs remained his own, too idiosyncratically complex to be taken up by others. But logicians and mathematicians pursued other graphic representations with similar convictions about their usefulness for presenting information.

John Venn, by inventing the simple but powerful devices for describing relations and intersections of classes of objects in the diagrams known by his name, abstracted the formal isomorphism latent in Peirce's diagrams and concretized it into a system for simultaneously showing and performing relations of interconnection, exclusivity, and inclusion.<11> [Figure 6] Once again, the visual properties of the diagram are very powerful at the level of articulating relations. But here they are less useful in indicating specificity than the most meagre and schematic of pictographs. Venn diagrams, though they could be made to contain any number of kinds of information put into relation, are themselves rather minimal as visual forms. They serve well, however, to remind us of the tension in pictographic logic between abstraction of formal relations and the communication of replete and specific material information. The diagrammatic field, as an instance of pictographic logic, functions as a schematic presentation of information. The diagram is a logical structure, capable of holding various (replaceable) values within a structured set of relations. These relations, abstracted from the visual properties of the image, are in turn capable of giving rise to a diagram, table, graph, or template that can be used for other purposes (as any schematic form is able to accommodate a number of possible values within it while preserving the relational structure of the formal arrangement it embodies). We could go back to the simplest example here, the cuneiform tablet, and note the way the separation of object from quantifier has an internal structure, its distinction of entity from quantity, that operates through visual means. The clarity of Venn's images resides in their "self-evident" configuring of intersection, overlap, and exclusion. The principles are at once abstract and embodied, concretized into figured form, and capable of functioning schematically to hold values within these configured relations. This is the power of the diagram when it functions as an abstract but visual form of pictographic logic.


Our discussion of pictographs, of Pierce, and of "envisioned information" might seem peripheral to considerations of the presentational character of alphanumeric text. We bring these examples forward, however, for two reasons: first, as instances determined to exploit graphical forms for clearly "intellectual" ends and purposes; and second, as stimuli for encouraging us to re-see and re-think traditional textual forms in new ways - or rather, in ways that bring us to see what we have not heretofore seen very well: that the conceptual "content" of scripted or printed alphanumeric texts is by no means largely, let alone exclusively, confined to the linguistic system - any more than the expressive and conceptual content of spoken discourse is largely, let alone exclusively, linguistic in form. Sentences, paragraphs, chapters; in verse, lines, inter- and intra-linear forms (such as rhyme and metrical forms, not usually thought of as "graphic" though they are embodied and instantiated through inscription), stanzas, cantos; the page, the opening, the book: using these divisions, we can describe an alphanumeric text in terms of its relational segmentation, down to the level of the character marks that comprise word units. By deploying different kinds and scales of segments we can create fields of metasegmentation, and expose their performative efficacy within the production of textual meaning.

This way of conceptualizing a book has a history reaching into antiquity. Mary Carruthers's work on the medieval book structure as a memory device demonstrates the link between the architectural models of memory activity and the structure of the book space.<12> In this view, graphical devices serve cognition even when their meanings are not textually rendered. Cognition develops within a display of shifting symbolic relations spatially arranged. Departing from Frances Yates's classic work, Carruthers thus argues that memory theatres are not static forms. She compares the mnemonic function of book space to the cathedral, which is organized to give dynamic access to the liturgy. Book layout, she argues, functions similarly. Spatialized and graphic characteristics are fundamental to cognition even when they are not rendered explicit or called to conscious attention. The distribution of text into graphic format did not produce a fixed structure into which items for recollection were placed. Memory was not rote and reading was not an orthodoxy (with a set explication of texts). The "organized schemata" served as the basis of orthopraxis, a "matrix of reminiscing cogitation" that was always a reenactment of the text.<13> Graphic structure thus comprised an integral and dynamic feature of the semantic field.

Here is a simple illustration taken from the modern period – a poem by Lord Byron.<14> We will look at two different incarnations of the opening section of this poem in order to expose crucial graphic features that regularly go unnoticed.

To the Po. June 2nd 1819.
River! That rollest by the ancient walls Where dwells the lady of love, when she Walks by thy brink, and there perchance recalls A faint and fleeting memory of me, What if thy deep and ample stream should be 5 A mirror of my heart, where she may read The thousand thoughts I now betray to thee Wild as thy wave and headlong as thy speed? What do I say? "a mirror of my heart"? Are not thy waters sweeping, dark, and strong, 10 Such as my feelings were and are, thou art, And such as thou art were my passions long. Time may have somewhat tamed them, not forever Thou overflow'st thy banks, and not for aye The bosom overboils, congenial River! 15 Thy floods subside, and mine have sunk away, But left long wrecks behind us, yet again Borne our old career unchanged we move, Thou tendest wildly to the wilder main And I to loving one I should not love. 20

The poem continues for another 32 lines but this initial part suffices for our purposes. What we give here is a diplomatic transciption of the text of Byron's holograph manuscript. He never published the poem, which he wrote - as the title indicates - in early June 1819. The work was first published in 1824, shortly after Byron died, in Thomas Medwin's book about Byron, and it was brought into the official corpus of Byron's works in 1831 when it was printed as follows:

Stanzas to the Po.
River, that rollest by the ancient walls, Where dwells the lady of love, when she Walks by thy brink, and there perchance recalls A faint and fleeting memory of me; What if thy deep and ample stream should be 5 A mirror of my heart, where she may read The thousand thoughts I now betray to thee, Wild as thy wave, and headlong as thy speed! What do I say -- a mirror of my heart? Are not thy waters sweeping, dark, and strong? 10 Such as my feelings were and are, thou art; And such as thou art were my passions long. Time may have somewhat tamed them, -- not for ever; Thou overflow'st thy banks, and not for aye Thy bosom overboils, congenial river! 15 Thy floods subside, and mine have sunk away - But left long wrecks behind: and now again, Borne our old unchanged career we move, Thou tendest wildly onwards to the main, And I -- to loving one I should not love. 20

The linguistic changes that come into the 1831 text shall not detain us, interesting as they are. What we call attention to is the striking change in the text's format, a change signaled in the changed title. Byron's editor has printed the poem in four line stanzas.

The consequence is a graphical transformation of the shape of the poem as Byron originally wrote it. Setting the two shapes next to each other calls attention to this primary fact: that the graphic deployment of text is every bit as significant in terms of "meaning" as the linguistic elements of text.

Let us study this situation with some care. Note that Byron wrote the poem in metrical quatrains rhyming abab. By long-standing typographical convention such a verse form is normally arranged as the "Stanzas" are arranged in the 1831 printing. But when he wrote the poem Byron departed from that convention, nor is it difficult to see why: running the quatrains without stanza breaks forces an approach to the affective pace of the poem that turns the graphic form into a figure of the dominant linguistic figure (headlong passion and headlong river). The eye registers the presence of the quatrains and the refusal of the stanza convention and then "reads" the relation between the two.

Much can and should be said about the implications of what we are seeing here. At the most general level we remark on the signifying character of even the most elementary typographical conventions of verse presentation. What we observe here of this four line stanzaic form should be extrapolated to the presentational character of all stanzaic forms, as well as to the visual segmentions executed at the text's lower - if no less significant -- levels. Punctuation is the most crucial visible form in this case. We want to recall that this highly evolved set of marks represent signs that were originally introduced as notations both for oral articulation and syntactic differentiation, and that they function in both registers to this day. As a set of oral cues - whether in silent or in articulated reading - punctuation is a foundational element in the affective (as opposed to the conceptual) ordering of the poem. As a set of syntactic cues it is a signifying system for foregrounding dominant sets of conceptual relations in the text. We can see some striking illustrations of these matters if we look again at the Byron poem. Compare, for example, the terminal punctuation of lines 4 and 13 in the two versions. In the 1831 text the editors introduce an end-stopped punctuation that isn't present in Byron's manuscript, and they further alter the movement of the text by breaking Byron's word "forever" into two words. The two texts signal very different pacings - what Keats famously called "unheard melodies" – for the reader. Furthermore, the 1831 punctuation of line13 also directs the reader to a very different conceptual understanding of the poem. In Byron's manuscript the enjambment at the end of line 13 places the phrase "not forever" in a zeugmatic structure such that we are asked to read it simultaneously in two different syntaxes (as modifying "tamed" and as modifying "overflow'st"). The 1831 punctuation is a visible signal to read the phrase in only one syntax. In this respect it replicates the meaning signaled in the 1831 decision to print the poem in four-line stanzas: to do that is not only to tell the reader that the poem's basic metrical unit is a quatrain, it is also to say that the quatrain and its textual presentation stand to each other in a relation of symmetry rather than a relation of tension. And that "statement" imbedded in the text's visible form plainly goes to the very core of this poem's intellectual and affective meaning.

We want to point out that in this example we have consciously chosen a poet who is not known for any special interest in exploiting graphical forms for poetical effect. We leave aside Blake, pattern and concrete poetry, Rossetti, Dickinson precisely because we wish to show that the standard presentational and graphic features of text are meaningful as such. In the effort to articulate meaning, affective as well as conceptual, one necessarily installs the visible resources of language. For every text is comprised simultaneously of a beauty of inflections as well as a beauty of innuendoes.


The presentational form of texts and images usually masques their logical operations in a surface rhetoric that dominates and controls our conscious attention. So it happens that in reading texts we are seduced from attending to graphical features and codes. The physicality of textual marks and shapes disappear in an act of reading determined to highlight certain kinds of conceptual references (content). In a similar, if seemingly antithetical way, the holistic repleteness of images keeps us from perceiving their conceptual order. In place of that formal order, a powerful figurative gestalt (or figurative reference) stands imperiously before us, hiding the methodical procedures by which the image's ideas or arguments are being developed -- as if the graphic character of the work were completely obvious. In each case a rhetoric of transparency makes it difficult to see beyond the moves within the text and the image to understand the metagraphic logic organizing them. In such a situation, the study of pictographs, which hover at the borderland of text and image, can be especially useful. They help us to see that at the next level of abstraction, of texts and images as information, similar logical mechanisms are at work. Each instance of text and image is an incarnation of such a metalogic, but it can be articulated according to its own rules and principles if it is rendered explicit.

One final matter is work remark. These systems of graphic presentation are operational, not merely passive schematic structures. They are active agents for creating meaning, instructions for reading, viewing, comprehending information. This is as true in the pictographs and other diagrammatic forms (Wilkins, Peirce, Venn) as it is in a conventional alphanumeric text such as we saw in the Byron example.


  • 1. Ignace Gelb, A Study of Writing, (Chicago: University of Chicago Press, 1952) Back
  • 2. A general philosophical note is perhaps in order here. It is not our intention to attack the ancient philosophical distinction between body and mind. The distinction seems, in fact, to capture fairly well the inherent tension and Eros of human experience. What we bring into question is the moral meta-structure laid upon the distinction, a meta-structure that brings confusion to our efforts to understand human powers and virtues and how they function. In this respect our argument appeals to much recent work on "embodied mind": the work of Lakoff and Johnson, e.g., or of Maturana and Varela; or, in specialized fields interested in visuality, of Gregory. Back
  • 3. Ian Hacking, Representing and Intervening (Cambridge: Cambridge Univeristy Press, 1983). Back
  • 4. Denise Schmandt-Besserat, "Two Precursors of Writing: Plain and Complex Tokens," in The Origins of Writing, Wayne M. Senner, ed., (Lincoln and London: University of Nebraska Press, 1989), pp.27-42. Back
  • 5. Elizabeth Hill Boone, "Aztec Pictorial Histories", Writing Without Words (Durham and London: Duke University Press, 1994) pp.50-76; Back
  • 6. Boone, op.cit., p.52. Back
  • 7. Boone, op.cit. pp.52-53. Back
  • 8. Murray Cohen, Sensible Words, (Baltimore: Johns Hopkins University Press) Back
  • 9. Bishop John Wilkins, An Essay Towards a Real Character and Philosophical Language (London: 1668, Printed for the Royal Society). Back
  • 10. Don D. Roberts, The Existential Graphs of Charles S. Peirce (The Hague and Paris: Mouton, 1973) Back
  • 11. Martin Gardner, Logic Machines and Diagrams (New York, Toronto, and London: McGraw-Hill Book Company, Inc., 1958), see especially pp.28-59. Back
  • 12. Mary Carruthers, The Book of Memory (New York and Cambridge: Cambridge University Press, 1990). Back
  • 13. Mary Carruthers, The Craft of Thought (New York and Cambridge: Cambridge University Press, 1998). p. 1. Back
  • 14. Byron's MS text (Berg Collection, New York Public Library) is the fair copy he made in April 1820 from the draft MS (Morgan Library) written in early June 1819. For Thomas Medwin's text see his Journal of the Conversations of Lord Byron (Henry Colburn: London, 1824). The 1831 text is the six volume edition of Byron's Works published by John Murray and edited by Thomas Wright and Thomas Moore. For fuller information see Lord Byron. The Complete Poetical Works, Vol. 4 ed. Jerome McGann (Oxford: Clarendon Press, 1986), 210-212. Back