John Zuern: Toward a Materialist Semiotics of Web Animation

In one of the fragments on photography in the materials associated with his Arcades project, Walter Benjamin comments that the "chronological specifiability of the development time of a photograph contains in nuce the political significance of photographic technology" (691). Such formulations are typical for Benjamin, whose analysis endeavors to reveal not just the significance of the immediately perceptible features of a world saturated with technical systems and processes, but the meaning of those systems and processes themselves. What I am going to very tentatively sketch out here as a materialist semiotic approach to web-based motion graphics attempts to follow that example, suggesting that work in literary and visual art that comes into being in application programs and programming languages bears legible and significant traces of its material conditions of possibility, and that the meaning of these signs in turn conditions whatever meaning we might want to extract from the work as a whole.

Having spent a lot of time as a teacher in both a graphic design program and an English department trying to help people make application programs and scripting languages do what they want them to do, I have developed a real appreciation for the artisanal aspects of new media productions and for their status as artifacts of particular material cultures. I am always willing to ask flat-footed questions about how things are made, and I want to argue here that looking very closely at the construction of particular instances of artistic and literary production in digital forms--asking small, old-fashioned questions about craft--is one avenue toward a better understanding of some the newer and bigger questions we want to ask about the materiality of the cultural productions we find in the online arcades of our world.

In his useful study The Language of New Media, Lev Manovich also invokes the craft of new media objects, but he does so in a way that raises a problem I would like to focus on in this brief essay. Manovich states that "if medieval masters left after themselves material wonders of stone and glass inspired by religious faith, today our craftsmen leave only pixel sets to be projected on movie theater screens or played on computer monitors. These are immaterial cathedrals made of light. . ." (201). This is an especially lovely version of a fairly typical formulation in new media criticism, but it offers only a partial, idealized account of new media architectures. It is a statement of faith that would seem to be contradicted by its own recognition of the labor invested in both pre-capitalist and late capitalist cathedrals. A number of people, including N. Katherine Hayles and Mark Hansen, have already pointed out the limitations of seeing information and the artifacts of information technology as ""immaterial." In my view, one of the factors that seems to contribute to such a critical faith in the immateriality of new media productions is a tendency to view programming-"the code" -in an abstracted and universalized way. "Code" then serves as an umbrella term that overshadows the particular languages, scripts, and development environments that have particular characteristics and particular itineraries in the history of software engineering under commodity capitalism. This tendency to generalize doesn't only undermine the kinds of "medium-specific analyses" of electronic texts that Hayles has exemplified in her work on hypertext fiction, it also obscures a category of materiality that has traditionally been the focus of materialist critical approaches: the category of relations of production. Writers of codes, whether at Microsoft or at the Multimedia Art Asia-Pacific Festival, are producers of culture, agents in the still-unfolding history of what Manuel Castells has described as a global, and profoundly stratified, network society.

One way to re-materialize our reading of new media is to keep our eyes on the specific application of particular programs and codes to the workaday problems that arise in the production, distribution, and reception of digital art. In what follows I examine two examples of web animation that in one way or another incorporate within their structures responses to the technical constraints of their medium, in particular the problem of bandwidth. They have been optimized in ways that are legible in relation to the parameters of their development environments--the "code"--as well as in the sensuous surface of their appearance on the screen. I want to look at the way time matters for them as a dimension of their aesthetics as well as the way they employ the resources of their respective development environments--design strategies in Macromedia Flash and coding techniques in DHTML--to grapple with technical problems related to download time. These practical solutions put a certain pressure on their surface aesthetics and at the same time can be read as productively ambiguous signs of the pressure of the material conditions of possibility for this kind of art.

Dakota

My first example is a Flash animation entitled Dakota produced by Young-hae Chang Heavy Industries, a team of two artists, Young-hae Chang and Mark Voge, who are based in Seoul, Korea. After their introduction to Flash at the Multimedia Art Asia-Pacific Festival, they started n 1999 to produce a series of compelling Flash pieces that coordinate narrative text, jazz music, and fast-paced animation. They use only the Monaco typeface, a system font, and include neither images (apart from the typography) nor interactive features in their work. In their interviews and articles Chang and Voge make it clear that their aesthetic refuses certain received forms of web art: hypertext, interactivity, intensive graphical interfaces. Their movies simply play, doling out fragments of a coherent, more or less linear text at a rate that always seems to me just a little bit faster than a comfortable reading speed. The texts themselves juxtapose motifs from global popular culture with sardonic criticisms of corporate culture and politics in Korea--Samsung is one of their favorite targets. The text of Dakota is a inspired by Ezra Pound's first and second Cantos and opens with a visual allusion to the image of breaking dawn in Canto I: a series of "blank" screens modulate from black through the spectrum of grays to white. What follows this opening sequence is a gritty, on-the-road narrative of misspent youth with cameos by Elvis and Marilyn Monroe that segues to a hommage to the jazz musician Art Blakey (whose music accompanies the piece) and then ends with a depiction of late-night traffic on a street in Seoul, with executives being chauffeured home from hostess bars and moped drivers delivering heating oil and take-out food.

While they reject a number of the conventions of digital art, Chang and Voge do make use of a range of optimization features and techniques that are either built into the Flash environment or that have become standards of good practice for Flash developers. While the most striking characteristic of the piece is the synchronization of the text-bearing screens and the driving beat of the accompanying music, the feature I want to point to is the series of black-to-white screens with which the piece opens. This aesthetic element refers to--in fact, rewrites into another medium--the classic set-piece of a description of the breaking day; it connects intertextually to Pound as Pound's own dawn-image connects to Homer and other classical writers. Viewed within the context of the Flash environment, these opening screens can be seen to contain another allusion--to the "loading sequences" that introduce any number of Flash productions currently on the Internet. These often comprise a simple animation that offers some indicator, frequently a progress bar or the word "loading," that runs until enough of the opening frames of the movie have been downloaded to the reader's computer to begin playing without stalls. (The Flash application includes a "bandwidth profiler" feature which generates a histogram of the movie's files as it plays, allowing developers to predict which frames are likely to cause download delays.) Young-hae Chang Heavy Industries takes a more subtle approach, front-loading their animation with frames that will download and start playing right away. While such optimization techniques are often characterized by designers as concessions to a still-sluggish medium, in many instances, as in Dakota, they are incorporated into what art directors in American advertising agencies call, with more insight than they may realize, a work's "look and feel."

Optimization strategies function not only as instrumental "back end" solutions in the production of digital literature; they gain complex semiotic values when they are interpreted alongside the aesthetic functions of the visible, legible features with which they are aligned in the work as a whole. At various points in Dakota the black and gray frames return, interleaved with frames bearing text, so that they literally flash between the screens of words. As the only non-alphanumeric visual element in the piece, these internal pulses of color recall the opening sequence, drawing the gray dawn of Dakota's beginning into the evening and night of its end, reinforcing both the work's intertextuality with Pound's poetry and the strategic opening gesture with which it negotiates its invocation by the reader's browser and its entry into the data-stream.

The pared-down purity of Dakota and other Young-hae Chang Heavy Industries productions, perhaps initially the most striking characteristic of these artists' approach to their medium, is a purity that exposes a contamination. As a literary text, Dakota is in a sense haunted by its non-verbal but semiotically overdetermined starting-point. In order to "begin" as a work of art, as the literary narrative of a journey, Dakota must launch itself into the still-resistant medium of the global Internet and travel toward whatever horizon of functionality its reader's system has predetermined for it. As it manifests itself to that reader as a work of art, a story, the recurring signs of the dawn sequence bind the driving narrative--from the American boys on the road to the Korean delivery boys on their mopeds--into the material network of technical systems that is delivering it. I do not want to suggest here that the well-articulated artistic sensibility of Young-hae Chang Heavy Industries is determined by the constraints of their medium, but that a critical reading of their work as a new media object cannot easily separate the realization of that sensibility from the technical parameters in which it unfolds.

Iris

My second example is much more directly a response to the constraints of bandwidth. iris is a piece coded solely in HTML and JavaScript (a combination known as dynamic HTML) by the web designer Josh Santangelo. In 2000 it was one of the winners of the 5K Contest, a web design competition that requires entries to be under 5 kilobytes in file size. After the initializing click on the title, the iris opens and closes to reveal, line by line, a stanza from D. H. Lawrence's 1916 poem "Flapper" which compares the glint of love in the iris of a woman's eye to the shimmer on the wings of a bee "before he flies." Like Dakota, the text is presented at a fixed pace. In his artist's notes, Santangelo writes that he was interested in forcing his audience to read the text at a rate which "might be faster or slower than your normal reading speed." After the poem has been displayed, the words "blah, no more 'art,'" and "time to play!" appear, and the piece switches to a chase-the-cursor game in which the cursor's x-axis position controls the speed, direction, and radius of the circle, and clicking resets the center. In the middle of the circle, the piece proudly displays its file size: 3,264 bytes.

Where optimization in Dakota relied on design strategies and preprogrammed features in the Flash environment, Santangelo's optimization takes place in the script itself. Reading his source code reveals that to minimize his file size Santangelo has compressed the variable names in his JavaScript into two-letter abbreviations and used iteration functions to make every possible character do as much work as possible. Such techniques are standard practices in industrial software engineering, where application programs called obfuscators are used to abbreviate and compress elements of a company's proprietary code with the aim of reducing the size and improving security of the product--obfuscated code is difficult to steal and rewrite.

The on-screen experience of iris emerges from clean, compact code that has one interesting--and deliberate--flaw. The character that composes the "iris" on the screen appears as a question mark in the Macintosh operating system and as the general currency symbol on a PC. In his commentary on the work, Santangelo explains that he could have built in a workaround to this incompatibility, but chose to let the inconsistency stand as part of the "mystery" of the piece.

While it is perfectly possible to read iris as an amusing commentary on how a constricting high literary culture (D.H.Lawrence's poem) has been superceded by the liberating forces of interactive gaming (the chase-the-cursor diversion that follows), I think it is also possible to read it as a cultural commentary on bandwidth and cross-platform compatibility. That tiny scripting detail that ironically reaches across platforms to link the general currency symbol with the question mark instantiates the enormous tension between the drive to standardization of the global IT infrastructure (the work of the World Wide Web Consortium, for example) and competitive engineering and business practices in the globalized free-market economy (Microsoft and Netscape vs. Open Source initiatives, for example). Read as a structure of signifying surfaces (in a version of what Roberto Simanowski has called a "hermeneutics of deep information"), the source code of iris and the text it displays on the screen argue for a definition of poetics that can encompass both the "technical" achievements of D. H. Lawrence and those of Josh Santangelo.

Info-Aesthetics

I think the relatively small and largely extra-textual details like the ones I have been pointing out are important not only because optimization remains a major consideration at all levels of new media production, but also because workaday technical worries about factors like bandwidth, browser compatibility, and, in the U.S. context, compliance with the Americans with Disabilities Act guidelines for information architecture represent the terrain on which the crassest forms of instrumental reason, the most high-minded ethics of accessibility, and all the compromised positions in between go to battle for the control of a certain kind of poetics: the creative yet rule-bound, time-bound, earthbound making of new media.

In his essay on the liminal figure of the "programmer-artist," Reinhard Storz suggests that the critic's nostalgic search for "the brushstrokes of the artist on the canvas" (1) is frustrated when "code" becomes the artist's creative medium. Even if we disregard the problem of using our already flawed, conventional notions of what artists were in trying to figure out what new media artists are, we can recognize in the enormous amount of energy that is currently directed at elaborating the craft of digital media production--in manuals, help pages, developer's listservs, workshops, and classrooms--that our time is probably better spent trying to figure out what a new media brushstroke is. I argue here that a critical reading practice that can account for something like a brushstroke--or the mark of a digital chisel--will be more enabling to us as readers, teachers, and producers of new media than a generalized genuflection to "code" or a fetishistic faith in immateriality.

Such a critical practice will have to struggle on the one hand against the dematerialization, dehistoricization, and disembodiment of our technological world and on the other against a reinstallation of humanist models of the subject and sentimental formulations of labor. Here's where I concur with Manovich and his belief that ". . .we need something that can be called 'info-aesthetics'-a theoretical analysis of the aesthetics of information access as well as the creation of new media objects that 'aestheticize' information processing" (217). While they are not "immaterial cathedrals of light," digital productions like Dakota and iris might be seen as crystal palaces--historically specific configurations of the technological systems and processes that have emerged since the Industrial Revolution and our crafty and compromised, frustrated but also faithful efforts to make those systems and processes mean something.

Acknowledgments: Thanks to Young-hae Chang Heavy Industries, Josh Santangelo, and Jay Snortland for helpful discussions as I was working on this paper.

Benjamin, Walter. The Arcades Project. Trans. Howard Eiland and Kevin McLaughlin. Cambridge, MA: Harvard UP, 1999.

Simanowski, Roberto. "Hermeneutik der Tiefeninformation. Zur Psychologie des digitalen Bildes." Kodikas/Code 24 (2002), 127-134. (vgl.

online)