PLAYStrong–Engineering POETICstates:On Facilitating Mental Health through Bio-Signal Interfaces and Brechtian Learning Strategies
Antje Budde* and Gustavo Sol**
Abstract
Our essay discusses Gustavo Sol’s bio-data performance interface for actors developed in and through the solo performance entitled Discontinuous Object, in which he explored the relationship between Poetic States of Presence (PSP), bio-data real-time performance and the craft of acting. We plan to expand and utilize this experimental research for social actors like university students. This is situated in the context of real world problem solving and the current research project between the Digital Dramaturgy Labsquared and the Interactive Media Lab entitled “PLAYStrong—Promoting Student Resilience through Interactive Prototypes for Embodied Self-learning,” investigating the potential of psycho-physical and playful forms of mental health support for our university students by using interactive performative interfaces for self-learning inspired by Brechtian theatre dialectics of learning.
Keywords: Bio-data interactive performance, digital dramaturgy lab, student mental health, agency and social actors, Brechtian learning, artistic intelligence (A/I), social actors
PLAYStrong: Context, Funding, Relationship-building
Our collaborative paper has three goals: 1. Introduce the PLAYStrong research project and its context in support of student mental health at the University of Toronto as a real-world problem application. 2. Introduce the acting-informed, bio-controlled interface and iterative design process of Gustavol Sol’s Discontinuous Object (Objecto Descontínuo, 2013–22) and its potentials for Being for Others technologies in the performing arts (Dixon) and PLAYStrong student-centred dramaturgy. 3. Introduce the Brechtian embodied concept of learning and self-learning as a method that inspires our project. With this, we hope to facilitate mental health and playful forms of world-building among university students, encourage alternative thinking/physical movement and build critical awareness in situations of stress or inhibitive self-doubt in everyday performance.
In a workshop planned for August 2022, we will test and further develop Sol’s experiments with Poetic States of Presence (PSP) as a responsive human-machine feedback system. The hypothesis is that in this iterative process which employs collaborative strategies of Artistic Intelligence or A/I, we will be able to make less visible states of mind and body, as well as patterns of mental stress indicators, more accessible to student users based on their own bio-data and provide them with an instrument of building resilience through creative and critical forms of self-learning.
Real World Problem Solving
Our project is an experimental attempt to explore and generate art-specific ideas of what engineers and neuroscientists refer to as “real world problem solving” (RWPS). Vasanth Sarathy insists on the importance of interactions with the environment as a crucial factor in facilitating RWPS, beyond the narrow focus on neural network analysis prevalent in cognitive neuroscience.
Real world problem-solving (RWPS) is what we do every day. It requires flexibility, resilience, resourcefulness, and a certain degree of creativity. A crucial feature of RWPS is that it involves continuous interaction with the environment during the problem-solving process. In this process, the environment can be seen as not only a source of inspiration for new ideas but also as a tool to facilitate creative thinking.
Sarathy
The real-world problem we aim to address is the emerging mental health crisis among university students, acknowledged in a report by the Presidential and Provostial Task Force on Student Mental Health (University of Toronto). This report applies definitions of mental health and mental illness on a student mental health continuum ranging from healthy to reacting to injured and ill (University of Toronto 6). Our project is focused on the first two categories that allow for preventive and empowering integration of physical and mental engagement and where medical diagnosis or intervention can still be prevented through alternative forms of art-inspired interactive environments. Our project is both an offer to students and a critique of the current healthcare system.
PLAYStrong: Intersecting Creativity
The two-year project, in which workshops with Sol’s and other performative interfaces are experimentally embedded, was conceived by Antje Budde and Mark Chignell and is called “PLAYStrong—Promoting Student Resilience through Interactive Prototypes for Embodied Self-learning.” On the most practical level of research, the project is a collaboration between the Digital Dramaturgy Labsquared (DDL2), led by Budde and affiliated with the Centre for Drama, Theatre and Performance Studies (CDTPS), and the Interactive Media Lab, led by Chignell at the Mechanical and Industrial Engineering Department (MIE).
The PLAYStrong project, inspired by Bertolt Brecht’s learning plays (discussed later), focuses on three interfaces or “embodied technology scenarios to facilitate self-learning and psychological resilience”:
- immersive hybrid physical/virtual AR/VR environments;
- physical-digital motion-tracking interactive spaces;
- EEG-interfacing physical live movement/action and multi-media interaction. (Budde and Chignell)
For the purpose of this paper, we will discuss interface number three, a research project titled Discontinuous Object developed by Sol (pen name of Gustavo Garcia Palma) as part of his doctoral research examining states of poetic presence mapped through Electroencophalography (EEG) and heart rate (BPM).
ACTOR-MACHINE: Embodied Interactive Pattern-Making of Poetic States of Presence (PSP)
The purpose of Sol’s interface, demonstrated in/as his performance experiment Discontinuous Object, was originally to explore how direct and real-time bio-data input from an actor’s body in a theatrical performance can be employed dramaturgically in a live performance. Simultaneously, both the actor and the audiences received data of a feedback loop that indicates what Sol calls the Poetic States of Presence (PSP) experienced by the actor while performing, based on the bio-data they produce as they progress through a solo performance. We can think of this as a dashboard of the actor’s psycho-physical processes in real-time. These meta-data of acting and being are usually not accessible in this way by both actors and audiences as part of a performance. We think that this interface is particularly well suited to employ Brechtian performative ideas of self-learning by a student-user, who will be able to engage as an observing, physically and mentally engaged social-actor when playfully and critically attuning to their complex ways of being. We hope students will gain important insights into their psycho-physical processes and build resilience and agency in an informed manner.
Sol identifies three types of measured bio-data that are fed into his digital theatre system’s feedback loop: the actor’s brain waves, heart rates and facial expressions. These data inputs/outputs have two functions: first, to raise the actor’s awareness of their embodied states of poetic and physiological presence at any given time in the performance. Second, to allow these bio-data to influence the playback of performance media (images, video, sound) and regulate levels of distortion of such media that are not predictably cued or operated by a technician but are “controlled” directly through the actor’s bio-data and PSP as they unfold. In addition, the computational setup is visible to both performer and audiences. It thus becomes an integral part of the live performance stage design, digital dramaturgy and embodied storytelling. This offers both the performer and audiences an opportunity to raise critical awareness about how an actor’s body operates while performing. They can also contemplate what types of often involuntary psycho-physiological processes are at play and what neurophysiological patterns are emerging as the actor engages in different types of theatrical storytelling and body control. At the same time, visibly present human-machine interfaces are problematized as many audience members know them in settings of medical diagnosis (brain wave and heart rate measurements) or camera-based surveillance systems (facial recognition). This offers them great opportunities to make critical connections between how and to what end emerging technologies are used in the arts and how they are present in their everyday lives.
Discontinuous Object
Sol’s art-science research uses sensing technologies, facial recognition technology and artificial intelligence/machine learning for two intersecting purposes. One is to artistically create live interactive audiovisual performances and the other to scientifically map possible neurophysiological patterns that can emerge according to different kinds of acting and performance techniques. For this purpose, three sensing technologies and their bio-data measurements were employed for data input, including: an EMOTIV[1] device worn on the actor’s head, which continuously creates electroencephalograms as the actor performs; an Arduino-based Oximeter,[2] monitoring the actor’s heart rate, worn on the actor’s hand; and interactive facial recognition interactive software, FaceOSC, which also produced data input. Incorporating this latter technology is particularly interesting because audiences traditionally read actors’ facial expressions in performance, but now they can simultaneously experience how machine sensors read such expressions and—via computational AI processing and media control—impact both the performer and the audience.
In Discontinuous Object, the real-time measurements of bio-data taken from the actor’s body mark responses to specific stimuli, such as images, sounds or mental tasks that the actor is engaged in. Each measurement indicates a point in the creative process of engagement and signifies a particular moment of the PSP the actor experiences. Such measurements allow for real-time insights into an actor’s organizing creative process when cognitively and affectively performing tasks associated with acting. Early research findings suggest that this setup has its challenges because the available technology is very sensitive to noise (in data measurements), especially noise produced by muscular activation, such as movements of the arms, torso or even eye movements (Flores-Vega 34–39). Thus, it can be difficult to use such technologies in a performance context because physicality is essential to actors’ work on stage.
Interaction challenges with biometric systems and other technologies needs to be considered in digital dramaturgy of performance, particularly when using low-cost equipment such as the EMOTIV device where low noise filtering needs to be anticipated when thinking about actors’ movement choreography in acting and the accuracy of the bio-data produced. Moreover, each technology offers implicit parameters and limitations that need to be studied and understood in the process of artistic creation. For example, the classification of six emotional states built into EMOTIV devices, which were designed for other purposes, need to be accepted. In addition to six emotional states, the EMOTIV device classifies signals of facial expressions and electrical intentional patterns that the actor’s body creates when focusing on stage actions or images and sounds that stimulate emotion during the performance.
How does this work out on the dramaturgical, acting and performance level? Discontinuous Object is a non-linear performance (object) designed for experimentation and critical observation that is dramaturgically built on a discontinued, non-Aristotelian narrative which is split into further discrete sub-objects (segments or scenes). In each sub-object, the emotions of the actor are stimulated or triggered by autobiographical videos, images and a personal letter, based on memories of family events that speak to emotional co-dependency. The performance investigates and measures the affective dimension of such stimuli in an active live performance. Each sub-object or scene is organized based on a group of acting techniques or procedures, such as a combination of a breathing pattern along with a particular muscular effort, that stimulates sensations and emotions. These techniques provoke images and memories in the actor’s body and mind, which can lead to changes of voice projection, articulation and tones. They can also induce stage actions and movement or activate symbolic associations in the audience. The techniques employ a combination of actor training principles developed by theatre artists such as Stanislavsky, Brecht, Grotowski or Artaud, including emotional engagement in a scene, narration and storytelling methods, work on impulse, corporeal work, breathing techniques, active imagination, memory recovery and so on. This mixed approach creates a sensory-motor and imaginative feedback loop for the performers’ physiological processes on stage that allow them to create body-mind states, or presence, in a poetic artistic context.
Work. Flows. Sensing Machines
Sol’s purpose-built interactive system creates unexpected flows of audio and visual material due to the programming of output data, which trigger unexpected body reactions and PSP that are continuously creating feedback loops between the technological system, the acting body and, by extension, the relationship with audiences. This process is open to new possibilities, provoking ambiguity and unexpected outcomes. For example, sometimes there are overlapping projected images and performer gestures, suddenly accelerating audio voices, or figures on video are completely erased. In this environment, actors must constantly be on their toes, communicating simultaneously with human audiences and machine co-actors, and dealing with the unpredictable challenges thrown at actors by their own psycho-physical processes all at once. Discontinuous Object is a high-calibre exercise machine for the twenty-first-century multi-tasker.
The computation for each sub-object, or scene, was based on visual object-oriented programming software such as Isadora (Troikatronix), Max/MSP/Jitter (Cycling ’74) and Pure Data (Pd) (Puckette). This integrating approach of software use allows one to build specific sequences for each sub-object, connecting sensors and data with computational processing and data output organization, which control presentational technologies like video projection and sound amplification.
Besides experimenting with inducing PSP based on dramaturgical choices for each sub-object, Discontinuous Object was also interested in the statistics-based visualization of how PSP manifest throughout the performance. Thus, the project tried to experimentally map patterns of different states of poetic presence that an actor experiences while performing. For this purpose, four distinct states were envisioned and tested:
PSP 1: Non-Visual Focus
Performer task: “Listening to your body,” focus and “stop thinking” beyond the present moment, produce awareness
Technique: deep breathing, listening
Sensory body feedback: relaxation and feeling a little vertigo
Biometric aspects: reduced level of prefrontal cortex activity (a process called alpha modulation), reduced level of muscular activation, which also means low artifacts in biometric data
PSP 2: Visual Focus
Performer task: “Just looking”
Technique: build awareness through the observation of objects (form, light reflections, dimensions) rather than focusing on their function
Sensory body feedback: relaxation and feeling a little vertigo, awareness of eye rhythm coordinating focus of attention
Biometric aspects: reduced level of muscular activation, which also means low artifacts in biometric data
PSP 3: Movement Focus
Performer task: visual, spatial and temporal planning and coordination of body movement and specific movement of shoulders
Technique: control the materiality of body; memory and emotions are implicated but not essential
Sensory body feedback: sensation of rhythm and gravity
Biometric aspects: changes of voice, facial expressions, heart and respiratory modulations, focus of attention related to action planning
PSP 4: Memory Focus
Performer task: read a personal letter of emotional significance
Technique: inspired by Stanislavsky’s concept of “emotion memory”
Sensory body feedback: sensation of rhythm and gravity, breathing pattern changes, increased closeness to scene, memories and emotions are activated
Biometric aspects: facial expressions captured by EMOTIV are crucial to detect the State, which means muscular activation in facial sensors is more significant than the muscular bio-data coming from excessive movements in the torso, arms or legs. Variations in skin sweat and heart rates are frequently related to situations with emotional engagement. This state tends to have a longer duration in relation to other States.
Experimenting with bio-data from PSP serves two purposes. First, it employs a data-based investigation of how actors’ psycho-physically move through the complex tasks of a solo performance beyond anecdotal evidence. Second, the experimental interactive setup actively designed components of acting and artistic communication as part of the larger concept of digital dramaturgy. The actor is not simply a lab rat exposed to scientific inquiries but takes an active role in crucial discoveries about their embodied craft and mental strength. Furthermore, the generated bio-data analysis offer feedback (for example, through distortions of sound or video projection) to the actor or user of the system that maps the current state of psycho-physical being in surprising ways. It can serve as indicator of physical and mental health and also support the actor/user to gain more agency and insight by understanding their own embodied existence in a deeper sense.
How is the interface organized? In area A, above, one can set the interface to record or classify states. Area B is a simulator and a system that receives the biometric data from the EEG and the Oximeter. According to the amount of time defined in C, these signals go to the ANN (Artificial Neural Network), creating a small delay between action and classification. The ANN will work according to these settings (in area D reading numbers of neurons, internal layers, limits, ranges, etc.). The interface shows the classification of the main state in the blue area (E). F shows the classifications of the other states recorded previously. The interface was able to map successfully all four states described above. Sol recorded a video[3] that demonstrates each state during a live performance. To set this up took Sol many months of self-learning, trial and error. The visualization in figure 3 below shows color coded data representations of all four PSP.
The experiment confirmed the initial artistic intuition and hypothesis that PSP can be triggered through specific sets of mixed performing techniques (procedures) representing related emotional patterns and cognitive organization during acting on stage. Further statistical analysis showed recurring correlations in this process, which points to a particular structure underlying PSP. These measured instances arguably represent different moments on the continuum between acting and not-acting suggested by Michael Kirby (3–15). In Sol’s experiment, a sliding scale between these poles of the continuum could be identified which also means that both practices of the performance art on the one hand and representational theatre on the other—a distinction Kirby makes—are activated. This biometric sensing process is partly inspired by methods such as Alba Emoting (Bloch 121–38) as a scientific method of emotion induction but also proposes that the artistic configuration (sub-objects built as scenes in a solo performance) and the creative self-learning process in a poetic context can simultaneously serve as both an artistic and scientific experiment.
With this method, individual patterns of emotion revealed in biometric data can be related with accuracy to the performer’s subjective perception of their PSP. This experiment allows the performer to evaluate, classify and work with four basic operative states of poetic presence in a process that mixes artistic and scientific research with self-learning using sensing technologies that originally were purpose built for medical diagnostics but have been used—in different ways—in bio-data and bio-controlled interactive art for decades.
Doing with Others: Where to Go Next
Sol’s interface can help with the playful understanding of the actor’s own agency and potentials in the performance process. It is precisely these aspects that make this human-machine performance interface so interesting to the PLAYStrong project as an experiential environment of self-learning, which aims at facilitating students’ discoveries about themselves as they are dealing with a range of stressers in a competitive university environment. We hope that Sol’s experiments as an actor can be meaningfully transferred into interactive student play and game-based experiences of how their bodies and minds work and where to go from there. This potential transfer will be the focus of our upcoming workshop in August.
The August workshop will feature several new improvements to the technological interface.[4] For facial recognition of performers or student users, we will further explore FaceOSC open access software which produces less data noise than the EMOTIV device. Sol had implemented this new technological configuration in 2021, and our workshop benefits from these improvements. The processing of the data will run through the OSC protocol and be sent for further processing to Max MSP,[5] a non-structured canvas for interconnectivity, commonly used for programming in interactive performance contexts.
With regard to AI, we need to be cautious about inherent biases produced in AI data processing and pattern recognitions as we will be working with a diverse pool of student users. In Sol’s early research, he only used his own biometric data and facial expressions to build his custom-made interface for the purpose of his experiment in the form of a solo performance. We cannot simply use these data to train AI because that would ignore users with gender, race or body identities that differ from Sol’s. His biometric data sets cannot be considered as representative of a universal “model human.” Technology is not neutral in its uses and many ethical problems, reinforcing unjust social power relations, have occurred particularly with AI.
Another important development in the current phase is the improvement of the heart rate monitor process and the final reintegration of all these elements in the latest version of the interface that will be used in the PLAYStrong Project. First, the hardware to capture the heart frequency was completely redesigned. The current system uses the AD8232 module to capture heart beats, the MPU6050 module to measure movements and angular velocities of the body in space, and the ESP32—a more robust microprocessor in comparison to the original Arduino Uno—to send all this data to the computer via Wi-Fi in real time. In addition, we will be using the latest model of the EMOTIV device, which is a simpler, almost plug-and-play version. This will allow for a less complicated setup process, a very important improvement that prepares the system to be used with different persons.
We aim to build a multidimensional interface and performative system that allows student users to observe, understand and impact their own creative and learning processes, their own PSP, building mental resilience and alternative thinking in real life settings—rather than in a theatre show. The goal is to build an interactive experience that strengthens the student user’s understanding of self and awareness of being, allowing for insights in their strategies dealing with life challenges, while also being critical of the root causes of such challenges.
We hope this evocative experience creates decentering sensations of body-machine extensions and a balanced sense of belonging. To this end, we will encourage student users to create their own data sets with videos and sounds taken from their everyday life that reflect on their affective experiences with friends, fellow students and family. This will be a rather different use of self-generated audio-visual materials than is often employed in the oppressive settings of social media and surveillance capitalism. Student-users will also have the chance to experience wearable technology and put it to a use that is hopefully beneficial to them.
PLAY. Entertain. Brecht. The Art of Learning
Arts, Science and Mixed Methods
The previous part discussed the process and purpose of the Poetic States of Presence (PSP) interface in its iterative design process, as well as future plans in collaboration with PLAYStrong research. A major modification we have to develop is the shift from a professional theatre actor to a student user—a social actor—for whom we want to make this performative interface accessible. The next iteration of this project benefits from artistic strategies but is not an artistic project per se. We hope this shift can be achieved by employing Brechtian concepts of participatory self-learning further enhanced by play and game-based methods of controlled interaction that allow for a meaningful understanding of individual users’ patterns of PSP. We cannot yet discuss the specific transfer of a game-based design or concrete Brechtian applications we are planning to investigate since we are in the early conceptual stages of such development. For now, we will present frameworks of alternative methods of learning and research in which our experimental praxis for the PLAYStrong project is situated.
Overall, praxis in DDL2 work aims to integrate experimental research methods informed by practices of critical making (Ratto 252–60), critical engineering (Oliver et al.), feminist queering (Halberstam) and digital dramaturgy as experimental performance (Budde, “Reality Show” 21–35; Budde, “Affecting the Apparatus” 189–200; Budde and Samur 83–101, Budde and Babayants 111–43, Kleber and Trojanowska 99–112; Budde, “Making. Critical Pleasure. A/I.”), as they can apply to praxis-informed research in the arts and engineering, both of which are based on praxis and creativity. In recent decades, such methods have been critically discussed across the arts, sciences and engineering, particularly as they have become relevant in discussions of the integration of the sciences, technology, arts and mathematics (STEAM) but also in intersecting methods of praxis-led, practice-based or practice-as-research (PaR) methods in the performing arts (Kershaw and Nicholson; Nelson; Hughes and Waterfield 114–35; Arlander et al.; Duffy et al.; Nielsen 164–70).
There is a growing mutual interest across the arts, sciences and engineering in each other’s methods and even purpose of research. The DDL2 has taken an integrative, mixed method and multi-lingual approach to research methods as they emerged as a translational necessity in the work and its critical reflection. A major influence on Budde’s artistic research direction have been Brechtian ideas on the relationship between the performing arts and the sciences from an artistic, queer-feminist and critical Marxist viewpoint, as well as Brecht’s groundbreaking experiments in participatory theatre and radio aimed at learning and self-learning. He often responded to the latest scientific discourses, political challenges (rise of Nazi Germany), economic conditions (industrial capitalism), technological developments and paradigm shifts in cultural communication (radio, film, new forms of acting). While it is well known that Brecht wanted to create a theatre for the scientific age, it is lesser known that he also tried in 1936 to form the Diderot Society as a platform where the arts are not separated from the sciences but can learn from the scientific spirit (Gorelik 111). Meanwhile, cognitive science has developed a deeper interest in Diderot’s performer training concept while Diderot in his time was trying to insist that the labor of acting is art (Dennis and Lewis 36–47).
Learning. P(l)ays.
The PLAYStrong project proposal identified Brecht’s learning plays as a major inspirational conceptual framework and interaction method that informs our strategies of digital dramaturgy as experimental performance—a negotiating process of embodied human-machine and performance-technology interaction. In this context, it is necessary to dive more deeply into crucial elements of this approach and why it makes sense in our current moment in a neoliberal university, such as the University of Toronto. Related to institutions of higher learning, Jon McKenzie introduces the metaphor of the lecture machine (19–21) that in its configurations, causes much distress for complex reasons. The early critique of such lecture machines (both in education and the arts) is of paramount relevance to Brecht’s creative alternatives that offer audiences and participatory student social actors liberating possibilities towards personal and social change.
Brecht, discussing the question of learning in, about and through theatre, concisely identifies that learning motivations, purposes and methods in the broader social context are contingent on several factors. In the context of capitalist production, learning serves particular and utilitarian purposes which are still prevalent at the University of Toronto and that negatively impact students’ well-being. As Brecht observed, knowledge functions as a commodity (tuition fees, for example, are an actual stresser for students). Learning and its purpose are conditioned by rigid divisions of labor that make “comprehensive knowledge unnecessary and impossible” (Brecht, “Theatre for Learning” 24). Motivation to learn, according to Brecht, is inspired by the essential needs of those “who are discontented with the ways things are” (24). Many students are discontented with how their university operates and contributes to the mental health crisis. That was clearly addressed in the 2019 mental health report discussed earlier. Following Brecht, depending on the context and purpose of learning, “There is thrilling, joyous and militant learning. If learning could not be delightful, then the theatre, by its very nature, would not be in the position to instruct” (24).
It is our intention to build delightful learning environments through human-technology interfaces in support of student mental health and to facilitate self-confidence and awareness through attitudes and skills of comprehensive self-learning. Self-learning, experiential learning and learning-by-doing approaches are a major driver of innovation in the face of critical challenges. For us, entertainment or playful learning does not function as distraction or escape from the complications of the world (Pavis 67–68), including mental health challenges, but rather as an improvisational instrument to aid participatory modes of real-world problem solving and embodied knowledge-making.
Freddie Rokem argues that Brecht’s explorations offer “processes of ‘learning through the arts,’ as well as for ‘the art of learning’” (57). He mentions specifically the learning plays (Lehrstücke) that
present self-reflexive situations of learning while simultaneously highlighting the conceptual basis for such pedagogical and meta-pedagogical activities. Brecht’s dramatic writings often show situations of interrogation and acquisition of knowledge while at the same time exposing their own aesthetic practices as a method for gaining new knowledge about what “needs” to be “learned,” as well as about the process of learning itself—in short, they can teach us how to learn.
57
The idea of instruction or guidance in this context (for example, through game-based rules) does by no means entail any simple “messages,” which are usually central to militant learning methods—those that avoid comprehensive knowledge making—and that prefer to impose unquestionable answers rather than encouraging critical questions. Barnett, thinking through Brechtian approaches to the entertaining pleasures of learning, outlines two important characteristics: “it employs dialectical methods without which Brecht’s approach to learning and theatre cannot be understood and it cannot be expected to produce unambiguous simple messages as this would pervert any art that sets out ‘exploring questions that resist simple answers’ for example the question ’what is justice?’” (24). Nowadays, “serious games”—a modern means of learning through the arts—pursue building critical-thinking skills through dramaturgical methods such as “puzzle games that help students solve problems and think ahead, story-based games that help students understand and unpack local and global issues, and strategy games that get students to manage time and resources” (Emphasis AB, Common Sense Education). A crux of the discussion is how to have fun while working through serious problems and that is true for Brecht’s learning plays and epic theatre as much as for serious game design. Mildner and Mueller (57–70) assert,
Games, serious or just entertaining, are played because they are fun. This can be explained with different models that share common elements such as play, rules, storytelling, social factors and learning. . . . Serious games can provide an extrinsic motivation to players who do not have an intrinsic motivation to engage with a topic otherwise.”
61
Likewise, Dörner et al. provide two definitions for digital games and serious games that are both relevant to further research of our upcoming PLAYStrong workshops. They define serious games as follows: “A serious game is a digital game created with the intention to entertain and to achieve at least one additional goal (e.g learning or health). These additional goals are named characterizing goals” (3). Qualitative characteristics of entertainment and fun in serious games lead to a question of the function of “fun” and “entertainment” and whether it serves to passively forget about the complications of the world or to actively care about them.
There are interesting intersections between serious game design and Brecht’s original learning plays in addition to characteristics of the Brechtian epic theatre techniques (Gestus, Fabel, Verfremdungseffekt, historicization). An interdisciplinary interest in Brechtian praxis has been reflected in recent publications expanding their reach into video game design (Pinchbeck 399–408; Farman 96–109; Evanz 1–22; Beck; Pötzsch), role-playing games (Hoover et al. 213–26), engineering (Walther 450–62), neuroscience and consciousness studies (Roessler), social justice (Rokem 57–66), socio-cultural theories (Zazzali “The Role of Theatre in Society” 685-97) and performer-based discussions of embodied, animated and prosthetic extensions in human-machine relations (Elswit 380–410).
As we work through digital dramaturgies of relationship-making in performance-technology interfaces, what we need to be critically aware of are “immersive theatrical events (that) have also been criticized as money-making ‘experience machines’ where the critical objectivity of an audience’s experiential separation from the performance is traded for a hedonist pursuit of a desirable affective experience. . . .” (Hoover et al. 221). Learning plays are anything but money-making experience machines. They are highly innovative challenges to consumerist, “culinary” as Brecht would say, and passive capitalist culture and its socio-economic apparatus that disenfranchises its audiences. Hence, learning plays are participatory community-based plays without audiences, originally written for students and often musically performed to explore together hard and dialectically driven questions while building agency and life skill competencies.
Brecht’s collaboratively co-performed plays (1926–33) are, to some extent, theatrically designed multiplayer games in pursuit of strategies to understand and face social, political and ethical dilemmas that need a revolutionary perspective/praxis of playful learning. We will need to find out how we can address what Brecht coined “inploitation” (an inversion of “exploitation”) of audiences that are “ambivalently split into exploiter and exploited” (Mueller 102). They are both the cause and victims “of a system that divides both terms—on the one hand, it marks work and productivity as devoid of pleasure, and on the other it keeps pleasure free of productivity for the sake of work” (102). With his learning plays, Brecht had deliberately stepped outside the entertainment industry that sells manipulatable emotions rather than taking audiences seriously in more holistic ways; not as passive consumers but as active producers who enjoy both the emotions and thinking about them and many other things (Brecht “The Lehrstück”). For Brecht, an active role of audiences was paramount across his works in theatre, anti-opera and media (The Flight Across the Ocean being a most prominent example for interactive live radio in 1927 that still inspires today. See Budde, “Reality Show” 21–35).
Brecht, writing in German, plays with the two meanings of the word Unterhaltung, which, depending on context, can mean both maintaining/sustaining and entertaining. Theatre and the arts are understood as a daily bread, as an essential need rather than as the cherry on the cake. He argues that the arts along with the sciences are a vital part of maintaining human societies:
science and art meet on this ground, that both are there to make·men’s life easier, the one setting out to maintain, the other to entertain us. In the age to come art will create entertainment from that new productivity which can so greatly improve our maintenance, and in itself, if only it is left unshackled, may prove to be the greatest pleasure of them all.
Brecht, “Short Organum” 185
Of course, the arts are rarely left unshackled, be it in the age of industrial capitalism and electricity, in which Brecht lived, or the age of artificial intelligence and globalized capitalism’s neoliberal educational systems and cultural industry. Only recently, the futureStage Research Group at Harvard’s metaLAB published their futureSTAGE Manifesto which declares performance a human right and stubbornly insists: “Performance isn’t a commodity. It isn’t a luxury. It isn’t something that gets added on top of the flow of life. It doesn’t belong to the ether, to the state or to private funders. It doesn’t belong to the places where it’s performed.”
“New productivity,” however, has become a less liberating project than Brecht imagined. As McKenzie suggests, we have moved away from scientifically argued and oppressive Taylorism, as a governing performance management practice in industrial capitalism (rhythms of the assembly line and such), to the organizational practice of ars poetica in corporate culture, which basically leads to a 24/7 oppressive cycle of precarious labor: “The principles regularly cited in management are not uniformity, conformity, and rationality, but diversity, innovation, and intuition” (McKenzie 6). Using “enobling discourses” (7) and employing instead references to Renaissance paintings, improvisational jazz music and cultural diversity, which apparently “contributes to creativity and problem-solving” (7), effectively “generated practices so distressingly familiar to US workers. . . . These practices include reengineering, restructuring, TQM programs, downsizing, outsourcing, part-timing, flex-timing, hoteling, and multi-tasking” (7).
McKenzie understands performance as the contemporary formation of knowledge and power (19). His vehicle for performing this theory is the “lecture machine,” a polyvalent term used to “frame and embody a series of case studies” (20). This term is first drawing metaphorically on the actual thing of the lectern. The “lectern has become an emblem of knowledge and power, a symbol standing upright between lecturer and audience, separating the one presumed to know and thus empowered to speak the truth from those presumed not to know and thus empowered to seek the truth” (21). Secondly, lecture machine references Derrida’s understanding of entire academic institutions as a lecture machine. McKenzie expands the metaphor “to any system that processes discourses and practices, any assemblage that binds together words and acts or, alternatively, that works to disintegrate their bonds and erode their forms and functions” (21). And finally, the lecture machine is understood as the embodiment, the paradigmatic shift and radical transformation of our reading machines with the emergence of new technologies, most importantly the computer and digital technology (22).
Of course, 20 years after McKenzie’s discussion, resistance is forming, expressed in emerging labor activism triggering new labor legislation. In a March 2021 Backgrounder, the Government of Canada informed the public of the “Right to Disconnect” and the problem of hyper-connectivity—a problem that our students are facing. This document claims that “lines have been blurred between being ‘at work’ and ‘not at work.’” The main factors are: “the constant connection to mobile devices” and “the rapid rise in remote work” (ibid.) While this can be beneficial, “it also carries health risks for employees when it is not balanced against the need for rest. Risks include: anxiety, depression, burnout” (ibid.) The risks in human-technology and labor relationships are quite clearly issues of mental health that affect students in our university. It is our working hypothesis that Sol’s and other interfaces of our research can be modified in ways that help students to actively observe, analyze and combat the causes of mental health challenges on a personal and institutional level.
In conclusion, in our article we have discussed the problem, context, goals, methods, dramaturgical and research strategy, and one of the performative psycho-physical interfaces (Discontinuous Object/PSP) that are at the core of current research in the PLAYStrong project in support of student mental health, awareness and playful learning to care for themselves and others. We are in the process of re-thinking Brechtian ideas on participatory learning plays and their relevance today. While we are critically reflective of the discursive interdisciplinary fields of our research, our research is also deeply practice-based in search of real-world problem solutions. Both authors of this paper are working on our particular in(ter)vention also in the way Brecht would have appreciated: “One of the foremost experts on Brecht, his former assistant Carl Weber, suggests that Brecht had little interest in conventional actor training: ‘Brecht had scant confidence in theatre schools; he firmly believed in learning-by-doing. . . .’” (Zazzali, “Consciousness in Brechtian Acting” 26).
Endnotes
[1] Currently we are using wireless Bluetooth-connected and wearable devices like the EMOTIV Insight.
[2] This is a device similar to the Arduino-based open-source licenced oximeter project shared by chirukesh “Pulse oximeter using Arduino © GPL3+”. Accessed 3 Apr. 2022.
[3] See Sol’s “Neurocomputing of Performativity.”
[4] A recent YouTube video by Sol describes and documents a more recent stage of research in his interactions with sensor-based technology, which allows a better understanding of the main aspects of the interactive human-machine eco-system that we will experiment with: See, Sol’s “Neurocomputing of Performativity” and his“Interface de Reconhecimento de Estados de Presenças Poéticas.”
[5] See a demonstration of its capabilities on the company’s website.
Bibliography
Arlander, Annette, et al. Performance as Research: Knowledge, Methods, Impact. Routledge, 2018.
Barnett, David. “What is Brecht Teaching.” Brecht in Practice: Theatre, Theory and Performance, Bloomsbury, 2015, pp. 24–26.
Beck, Michael J. Shall We Play a Game? The Performative Interactivity of Video Games. U of North Texas P, 2014.
Bloch, Susana. “Alba Emoting: A Psychophysiological Technique to Help Actors Create and Control Real Emotions.” Theatre Topics, vol. 3, no. 2, 1993, pp. 121–38.
Brecht, Bertolt. “Short Organum for the Theatre” Brecht on Theatre: The Development of an Aesthetic, edited and translated by John Willett, Hill and Wang, 1992, pp. 179–208.
—. “Theatre for Learning.” Brecht Sourcebook, translated by Edith Anderson, edited by Henry Bial and Carol Martin, Taylor and Francis, 2005, pp. 21–28.
—. “The Lehrstück or Learning Play.” The Measures Taken and other Lehrstücke, translated by Carl R. Mueller, Ralph Manheim, and Wolfgang Sauerlander, Methuen Drama, 1977, p. 6.
Budde, Antje “Affecting the Apparatus: Queer Feminist Re/Decodings in the Digital Dramaturgy Lab, Toronto.” Performance, Feminism and Affect in Neoliberal Times: Contemporary Performance InterActions, edited by Elin Diamond, Denise Varney, and Candice Amich, Palgrave Macmillan, 2017.
—. “Making. Critical Pleasure. A/I.” Vimeo, uploaded by Antje Budde, 31 Jan. 2022, vimeo.com/669275251. Accessed 7 Mar. 2022.
—. “Reality Show–Show Reality: Theatre Life Media Politics and the Coolness of Contemporary Theatre in the Performances of Obama, Brecht and Kondek.” Teritorija. Sociālā plastika. žurnāls par têatri, vol.1, 2008, pp. 21–35.
Budde, Antje, and Art Babayants. “Homo Ludens—Common. Play. Grounds. A Praxis Report on Workshop-making by the Digital Dramaturgy Labsquared (DDL2) at Coburg University of Applied Sciences, Germany 2019.” Elfenbeinturm oder Kultur für Alle? Kulturpolitische Perspektiven und künstlerische Formate zwischen Kulturinstitutionen und Kultureller Bildung, edited by Julius Heinicke and Katrin Lohbeck, KoPaed Publishing, 2020, pp. 111–43.
Budde, Antje, and Mark Chignell. “PLAYStrong—Promoting Student Resilience through Interactive Prototypes for Embodied Self-learning.” Expanding Our Research Impact Through Inter-Disciplinary Collaboration 2021–23 JOINT EMHSeed AND XSeed APPLICATION FORM. University of Toronto, 31 Mar. 2021.
Budde, Antje, and Sebastian Samur. “Making Knowledge/Playing Culture: Theatre Festivals As Sites of Experiential Learning.” Theatre Research in Canada/Recherches théâtrales Au Canada, vol. 40, no. 1 and 2, 2020, pp. 83–101.
Common Sense Education. “Games for Building Critical-Thinking Skills.” Accessed 1 Nov. 2021.
Dennis, Rea, and Lisa Lewis. “Diderot’s Body and Cognitive Science: Sensation, Impulse and Action in Performer Training.” Studies in Theatre and Performance, vol. 38, no. 1, Jan. 2018, pp. 36–47.
Dixon, Steve. Cybernetic-Existentialism: Freedom, Systems, and Being-for-Others in Contemporary Arts and Performance. Routledge, 2020.
Duffy, Peter, et al. Drama Research Methods: Provocations of Practice. Brill Sense, 2018.
Elswit, Kate. “The Some of the Parts: Prosthesis and Function in Bertolt Brecht, Oskar Schlemmer, and Kurt Jooss.” Modern Drama, vol. 51 no. 3, 2008, pp. 389–410.
EMOTIV. “Software Development Kit User Manual for Release 2.0.0.20.” Epoc Emotiv User Manual. 2012. Accessed 3 Feb. 2022.
Farman, J. “Hypermediating the Game Interface: The Alienation Effect in Violent Videogames and the Problem of Serious Play.” Communication Quarterly, vol. 58, no. 1, 2010, pp. 96–109.
Flores-Vega, Chrisyian Humberto. Recognition of Cognitive States in EEG Signals [Reconhecimento de Estados Cognitivos em sinais EEG]. 2010. U of São Paulo, Master dissertation.
futureStage Research Group at Harvard metaLAB. “futureSTAGE Manifesto.” The Theatre Times. 19 Nov. 2021.
Gorelik, Mordecai. “Bertolt Brecht’s ‘Prospectus of the Diderot Society.’” The Quarterly Journal of Speech, vol. 47, no. 2, 1961, pp. 113–17, doi.org/10.1080/00335636109382464.
Government of Canada. “Backgrounder: The ‘Right to Disconnect.’” 18 Mar. 2021. Accessed 5 Nov. 2021.
Halberstam, Judith. The Queer Art of Failure. Duke UP, 2011.
Hoover, Sarah, et al. “Performance Studies and Role-Playing Games.” Role-Playing Game Studies, 1st ed., Routledge, 2018, pp. 213–26.
Hughes, Jenny, and Carran Waterfield. “Digging Deep: A Dialogue on Practice-Based Research.” Studies in Theatre and Performance, vol. 37, no. 1, Routledge, 2017, pp. 114–35.
Kershaw, Baz, and Helen Nicholson. Research Methods in Theatre and Performance. Edinburgh UP, 2011.
Kirby, Michael. “On Acting and Not-Acting.” The Drama Review: TDR, vol. 16, no. 1, 1972, pp. 3–15.
Kleber, Pia, and Tamara Trojanowska. “Performing the Digital and AI: In Conversation with Antje Budde David Rokeby.” TDR : Drama Review, vol. 63, no. 4, MIT Press, 2019, pp. 99–112.
McKenzie, Jon. “Introduction. Challenges.” Perform or Else: From Discipline to Performance, Routledge, 2001, pp. 1–26.
Mildner, Philip, and Florian “Floyd” Mueller. “Design of Serious Games.” Serious Games, edited by Ralf Dörner, Stefan Göbel, Wolfgang Effelsberg, and Josef Wiemeyer, Springer International Publishing, 2016, pp. 57–82.
Mueller, Roswitha “Learning for a New Society: The Lehrstück” The Cambridge Companion to Brecht, edited by Peter Thomson and Glendyr Sacks, 2nd ed., Cambridge, 2006, pp. 101–17.
Nelson, Robin. Practice as Research in the Arts: Principles, Protocols, Pedagogies, Resistances. 1st ed., Palgrave Macmillan, 2013, doi.org/10.1057/9781137282910.
Nielsen, Lara D. “The Oral History Project: Practice-Based Research in Theatre and Performance.” Mapping Landscapes for Performance as Research, edited by Shannon Rose Riley and Lynette Hunter, Palgrave Macmillan, 2009, pp. 164–70.
Oliver, Julian, Gordan Savičić, and Danja Vasiliev. “The Critical Engineering Manifesto.” CE, 2011–22, criticalengineering.org. Accessed 20 Jan. 2021.
Palma, Gustavo Garcia (Gustavo Sol). States of Poetic Presences Mapped by the Technique of Electroencephalography (EEG) and Heart Rate (BPM) and a Proposal for Performative Creation through Live Neurophysiological Sensing [Estados de presenças poéticas mapeadas pela técnica de Eletroencefalografia (EEG) e pela frequência cardíaca (BPM) e uma proposta de criação performativa por meio do sensoriamento neurofisiológico ao vivo]. 2017. U of São Paulo, PhD dissertation.
Pavis, Patrice. “Entertainment.” The Routledge Dictionary of Performance and Contemporary Theatre, edited by Patrice Pavis and Andrew Brown, Abingdon, 2016, pp. 67–68.
Pinchbeck, Dan. “A Theatre of Ethics and Interaction? Bertolt Brecht and Learning to Behave in First-Person Shooter Environments.” Technologies for E-Learning and Digital Entertainment, Springer, 2006, pp. 399–408.
Pötzsch, Holger. “Playing Games with Shklovsky, Brecht, and Boal: Ostranenie, V-Effect, and Spect-Actors as Analytical Tools for Game Studies.” Game Studies, vol. 17, no. 2, 2017, gamestudies.org/1702/articles/potzsch. Accessed 20 Nov. 2021.
Ratto, Matt. “Critical Making: Conceptual and Material Studies in Technology and Social Life.” The Information Society, vol. 27, no. 4, 2011, pp. 252–60.
Roessler, Norman, and Anthony Squiers. Philosophizing Brecht: Critical Readings on Art, Consciousness, Social Theory and Performance. Brill, 2019.
Rokem, Freddie. “The Aesthetics of Learning: Bertolt Brecht’s Die Ausnahme und die Regel (The Exception and the Rule).” Theatre Topics, vol. 25, no. 1, 2015, pp. 57–66.
Sarathy, Vasanth. “Real World Problem Solving.” Frontiers in Human Neuroscience: Hypothesis and Theory, vol.12, 2018, pp. 1–14. Accessed 20 Feb. 2022.
Sol, Gustavo. “Discontinuous Object.” YouTube, uploaded by Gustavo Sol, 18 Aug. 2017. Accessed 20 Dec. 2021.
—. Interface de Reconhecimento de Estados de Presenças Poéticas. YouTube, uploaded by Gustavo Sol, 8 Aug. 2021. Accessed 20 Feb. 2022.
—. “Neurocomputação da Performatividade.” YouTube, uploaded by Gustavo Sol, 7 Aug. 2021. Accessed 20 Dec. 2021.
—. “Neurocomputing of Performativity.” YouTube, uploaded by Gustavo Sol,15 Nov. 2021. Accessed 20 Dec. 2021
—. “Objecto Descontínuo.” (2013). 1 Nov. 2021. Web. Accessed 20 Dec. 2021.
University of Toronto: Presidential and Provostial Task Force on Student Mental Health. Final Report and Recommendations, December 2019. Accessed 30 Nov. 2021.
Walther, Joachim. “Understanding Interpretive Research Through the Lens of a Cultural Verfremdungseffekt.” Journal of Engineering Education, vol. 103, no. 3, Blackwell Publishing, 2014, pp. 450–62, doi.org/10.1002/jee.20053.
Zazzali, Peter. “The Role of Theatre in Society: A Comparative Analysis of the Socio-Cultural Theories of Brecht, Benjamin, and Adorno.” The European Legacy, Toward New Paradigms, vol. 18, no. 6, 2013, pp. 685–97.
—. “Consciousness in Brechtian Acting: Defamiliarizing the Self.” Philosophizing Brecht, edited by Norman Roessler and Anthony Squiers,Brill, 2019, pp. 24–46.
*Antje Budde is the artistic research director of the Digital Dramaturgy Labsquared and an Associate Professor at the University of Toronto. Antje is a queer-feminist experimental scholar-artist who explores intersections between participatory experimental performing arts, human-machine co-performance and interactive technologies (digital and analogue), the sciences, experiential learning praxis, cross-cultural performance and multi-linguality. As part of her lab explorations and digital dramaturgy research she has developed the concept of artistic intelligence A/I in critical conversation with artificial intelligence or AI developments.
**Gustavo Sol is an actor, performer, and director. He is a lecturer at the University Center of Beaux-Arts in San Paolo, Brazil, where he heads the Neurocomputing of Performing Arts Study Group. Gustavo collaborates with the Robotics and Rehabilitation Research Laboratory—LABORE / IF in San Paulo and the Digital Dramaturgy Labsquared from the University of Toronto, Canada. He is interested in neuroscience and interactive technologies applied to performing arts.
Copyright © 2022 Antje Budde and Gustavo Sol
Critical Stages/Scènes critiques e-ISSN: 2409-7411
This work is licensed under the
Creative Commons Attribution International License CC BY-NC-ND 4.0.
