Roots of Performatology: From Uber-Marionette to Embodied Performative Agent

Roots of Performatology: From Uber-Marionette to Embodied Performative Agent
Maraffi, Christopher, University of California, Santa Cruz, topherm@soe.ucsc.edu
This poster presents humanities research the author completed in Digital Arts and New Media that has led to the development of a novel high-level Performatology approach to designing embodied agents in Computer Science. Agent research for interactive narrative and games have incorporated some performative dramatic theory (Mateas, 2004; Seif El-Nasr, 2004; Tannenbaum, 2008; Perlin, 1996), but being primarily influenced by literary approaches (Laurel, 1991; Murray, 1998; Austin, 1962; Searle, 1969), the focus has been towards developing a Neo-Aristotelian Poetics approach to interactive drama. Some agent work has been done on modeling improvisational performers from Theatre Arts (Magerko, 2010), but their micro-agent designs did not incorporate embodied gesture. Although additional research has been done on modeling the gesture of professional speakers for enhancing the personality of embodied conversational agents (Neff, 2008), little work has been done to procedurally model professional performers from the arts to enhance the gestural quality of embodied agents.

By and large, the emphasis for previous interactive narrative research has been to provide story authoring tools rather than tools for embodied performers to represent their craft in computational media. Avatars, Non-Player Characters (NPCs), and Intelligent Virtual Agents (IVAs), all tend to be designed to function primarily as embodied conversational agents, with gestural performance being supportive to speech acts that drive the narrative forward. Thus, interactive drama in games is in stark contrast to how drama developed in both classical live theatre and moving pictures, where gestural performance historically preceded speech acts, and where physical drama and visual spectacle dominated textual narrative.

When cinema started, the camera was the entry point for professional performers to migrate out of the proscenium and into the screen mediums of film and animation. Actors effectively digitized their embodied 3D gesture into animated 2D representations that translated their already developed fictive personas into the plastic time and space of analog media, where the characters could repeat the original performances indefinitely, even after the death of the performers. Multiple takes, editing, and visual effects allowed them to iteratively improve their linear performances for audiences. The dramatic personas or icons created by media stars of the last century arguably eclipse the narrative elements in any single story, and indicate a development path for embodied performative agents in New Media today.

From Uber-Marionette to Avatar Theatre: Imagining the Ideal Performer
Interactive drama has the potential to extend the actor’s craft further than all previous acting mediums, but due to a technological divide, actors have been prevented from playing on the new stage. The current absence of performing artists in computational media, as well as the potential solution to the problem in a procedural performer, was anticipated by theater practitioner Edward Gordon Craig at the turn of the last century. An influential British theorist working out of Florence, Craig self-published his controversial theatre reformation opinions in a periodical called The Mask (1907-1929). He advocated the rise of the Director-Designer as a visionary artist in charge of all aspects of production, over both the author and players, and in this role he faced character believability problems that caused him to question the viability of the actor as an artistic medium. Craig particularly struggled with the unpredictable personalities of live actors in his productions, which he attributed to a fundamental problem of belief in the theatre. He conceptualized the perfect actor as one who had a single-minded belief in the idea of the character, in the ideal Platonic sense, which would in turn make the audience believe in the characterization enough to have an emotional response (Craig, 1963).

A former actor himself, Craig’s views on acting as a belief problem was influenced by Neo-classical ritual acting techniques from masked theatre and puppetry. These art forms also moved him towards symbolic gesture, along with contemporary influences such as the stylized movement of actor Henry Irving and dancer Isadora Duncan, as well as the anti-realism of the Symbolists (Ayat-Confino, 1987). His innovative productions were larger than life spectacles of moving lights and set pieces, which he intended as a new type of immersive Kinetic Stage. Since he was working against realism or naturalism, but still wanted to create a believable experience for the spectator, designing a consistent theatrical experience was critical to him. But the one thing Craig couldn’t control was the live actors in his productions, who he claimed did not have the disciplined belief required to reliably portray his desired characterizations. So in 1907 he published his infamous essay The Actor and The Uber-Marionette, where he proposed a technological solution to his acting problem, and proclaimed that for the artistic future of the theatre, all live actors should be replaced by autonomous puppets (Craig, 1907).

In his essay, Craig compared acting to other art forms, and found it came up short. His main complaint was against the live actor as a performing medium. When playing in front of an audience, Craig claimed that actors allowed their personality or mind to get in the way of their characterization. Through either nervousness or ego, they would tend to frequently act out of character, breaking the believability of their portrayal for the audience. Painters and musicians, as visionary artists, could abstract and refine their art forms with complete control. Actors, who often relied on their own personality to carry a role, in Craig’s opinion, did not qualify as artists. His ingenious solution was to remove the actor entirely from playing in front of a live audience. He proposed, for the good of the theatre, that actors had to be banned from the stage and replaced with an autonomous puppet he called the Uber-Marionette. Craig already used techniques from masked theatre and puppetry to insert a performing object between the actor and audience, which was known to create a distancing effect for the performer. The Uber-Marionette concept was an extension of these techniques intended to entirely remove the actor in form and personality, leaving only a refined version of their gesture as an embodied performative representation.

Craig’s argument may be sound, but his purposely provocative delivery didn’t receive a positive response from the theatre community, especially from actors. So there is some theater, to champion the inclusion of live performers in the field of computational media today. Clearly ahead of his time, and highly perfomative in writing The Mask (Taxidou, 1998), he was often misunderstood by his contemporaries who either took his words at face value, or assumed he was writing in metaphors.The argument can be made that Craig was actually proposing a vision of the ideal actor intended for a future performance medium not yet invented. He said as much in his Uber-Marionette essay when he wrote, “If you can find in Nature a new material, one which has never yet been used by man to give form to his thoughts, then you can say that you are on the high road towards creating a new art. For you have found that by which you can create it. It only remains for you to begin. The Theatre, as I see it, has yet to find that material.” (Craig, 1907). Though technologically impossible in his lifetime, Craig never gave up trying to realize his concept, nor did he ever admit that it was impossible to build. Though highly criticized by his contemporaries, he showed an unshakeable belief that it would be invented someday by discovering a new control mechanism, “What the wires of the Uber-Marionette shall be, what shall guide him, who can say?” (Craig, 1963).

Although Craig’s banishment of the actor has inadvertently been realized in computational media, the author contends that his vision of the perfect actor as an Uber-Marionette can also be realized by modeling the behavior of embodied agents on the gesture of live professional performers. If the essence of good acting is semiotic gestural technique, or a vector of poses and movements that convey a symbolic attitude of the intended persona, then the ideal acting medium is one that can iteratively refine a trained performer’s gesture to a singular clear purpose in real-time for a live audience. Craig’s imagined “wires” are procedural character algorithms trained on motion capture data, and his desired new “material” for Theatre is the virtual stage of games and interactive drama. The author’s MFA thesis and accompanying Avatar Theatre performances were intended to translate Craig’s vision to a New Media context, and to informally assess contemporary audience reaction to seeing a live performer interacting with a 3D character in a shared performance space (See Figure 1; Maraffi, 2010).

Figure 1

Full Size Image

Audience feedback suggested that, in the context of a live dramatic performance, believable interaction and expressive appeal can possibly solve Computer Science problems for embodied agents, such as passing a gestural Turing Test and offsetting the Uncanny Valley Effect. For instance, there were performances when audience members believed that a remote performer was controlling the Avatar character, as well as a positive general audience response to comedic interaction that countered any strangeness from seeing the avatar mimicking the performer’s natural movement.
From Avatar Theatre to Performatology: Getting the Performer In the Game
So how does the above MFA research apply to games and interactive drama? It is the proof-of-concept for developing a Performatology approach to designing embodied performative agents at UCSC’s Computational Cinematics Studio. Using this approach the author has designed a novel performer modeling agent architecture, IMPRSONA, which uses machine learning and motion capture to build a performer profile from a knowledge base of procedural gesture prototypes (see Figure 2; Maraffi, 2011).

Figure 2

Full Size Image

We are mapping a performative ontology from Performance Theory (Barba, 2005; Schechner, 2002; Aston, 1991), and from the author’s experience as a performance artist and technical animator, to simulate the apprentice training process used by professional performers when learning and practicing their craft through mimicry and improvisation. The author’s hypothesis is that the performer’s technique, developed over many years of disciplined training, results in a semiotic quality of gesture that is integral to portraying believable, expressive, and appealing fictive characters.
Puppetry and animation over the last century have shown that abstracting principles from live performers can simulate the illusion of life in moving images (Thomas, 1981), as seen in Disney’s Mickey, Henson’s Kermit, and many other iconic characters, creating personas that persist in linear time-based media. These icons indicate that the key to solving embodied agent believability problems may be discovered in simulating the performer’s gestural quality. Embodied agents in games have the potential to become interactive New Media icons if we can represent the performer’s craft as procedural algorithms. Our embodied agent architecture is intended as a performative component in a broader Nara-Performa-Ludic system design (see Figure 3), and is the first step in a formal Performatology study of real and simulated embodied performers interacting together in a shared performance space. Our goal is to get the professional performer in the game, and get the actor back on the stage of interactive drama.

Figure 3

Full Size Image

Acknowledgements: The author would like to thank his current Computer Science PhD advisors: Arnav Jhala (UCSC Computational Cinematics Studio) and Noah Wardrip-Fruin (UCSC Expressive Intelligence Studio). Also, a very special thanks to his Digital Arts and New Media MFA Thesis Committee: Kathy Foley (UCSC Theater Arts), Michael Mateas (UCSC Expressive Intelligence Studio), and Ted Warburton (UCSC Theater Arts). Additional thanks to his UCSC Theater Arts faculty advisors for the 2010 DANM Performative Technologies Group: Jim Bierman, David Cuthbert, and Kimberly Jannarone.

References:
Aston, Elaine and Savona, G. 1991 Theatre as Sign System: A Semiotics of Text and Performance, New York Routledge

Austin, John 1962 How to Do Things With Words, . MassachusettsCambridge

Ayat-Confino, Irene 1987 Beyond the Mask: Gordon Craig, Movement, and the Actor, . Southern Illinois Univ. Press

Barba, Eugenio Savarese, N. 2005 A Dictionary of Theatre Anthropology: The Secret Art of the Performer, New York Routledge

Craig, Edward Gordon 1907 On the Actor and The Uber-Marionette, The Mask, (Self-Published Periodical Florence)

Craig, Edward Gordon 1963 The Theatre Advancing, Benjamin Blom, Inc. New York, NY

Laurel, Brenda 1991 Computers as Theatre, New York, NYAddison-Weseley

Magerko, B Fiesler, C, et al. 2010 ““Bottoms Up: Improvisational Mico-Agents”, ” 3rd Workshop on Intelligent Narrative Technologies, , FDG (INT3 2010)

Maraffi, Christopher 2010 Avatar Theatre MFA Thesis Performances, (Video Documentation) UCSC Digital Arts and New Media Program (link)

Maraffi, Christopher Jhala, A. 2011 Computational Cinematics Studio Performatology Project: IMPRSONA Performative Agent Architecture, UCSC Computer Science Program (link)

Mateas, Michael 2004 ““A Preliminary Poetics for Interactive Drama and Games”, ” First Person, Wardrip-Fruin and Harrigan, eds.Cambridge, MassachusettsThe MIT Press

Murray, Janet 1998 Hamlet on the Holodeck: The Future of Narrative in Cyberspace, Massachusetts The MIT Press

Neff, M Kipp, M, et al. 2008 “Gesture modeling and animation based on a probabilistic re-creation of speaker style, ” ACM Transactions on Graphics, 27

Perlin, Ken Goldberg, A. 1996 ““Improv: A System for Scripting Interactive Actors in Virtual Worlds”, ” Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, New Orleans

Schechner, Richard 2002 Performance Studies: An Introduction, New York Routledge

Searle, John 1969 Speech Acts, New YorkCambridge University Press

Seif El-Nasr, Magy. 2004 “A user-centric adaptive story architecture: borrowing from acting theories, ” ACE '04 Proceedings of the 2004 ACM SIGCHI International Conference on Advances in computer entertainment technology,

Tanenbaum, J. Tanenbaum 2008 ““Improvisation and Performance as Models for Interacting with Stories”, ” ICIDS '08 Proceedings of the 1st Joint International Conference on Interactive Digital Storytelling: Interactive Storytelling, (2008 ICIDS)

Taxidou, Olga 1998 The Mask: A Periodical Performance by Edward Gordon Craig, University of Edinburgh UK

Thomas, Frank Johnston, O. 1981 Disney Animation: The Illusion of Life, New YorkDisney Editions