Dancing in the Metaverse

Dancing in New Territories

The metaverse is still being defined. Yet, its potential to unleash creativity and give birth to new artistic practices is palpable.

In the realm of dance, technological development was accompanied by the emergence of a new generation of artists embracing the virtual to reinvent their practice. What is dancing in the metaverse? How can virtual spaces shape new experiences for artists and audiences alike?

For Jean-Marc Matos, choreographer and artistic director of K. Danse, “hybrid performances make use of various digital technologies to create live interactive environments: mobile phones, reactive clothing, physiological and haptic sensors, optical tracking sensors, internet networks for tele-presence between distant sites, AI neuronal networks and machine learning, interactive 3D, motion capture, VR, robotics.”

Dancing in the metaverse means dancing in new spaces created or recreated by technology. Technology redefines what is possible, pushing the limits of artistic possibilities: “It is of great interest to consider that a dance in virtual space is a dance that can enact what is impossible in real space.” Virtual spaces enable “easy change of gravity without specific physical tools, infinite multiplicity, and reproducibility, prodigious scales, perspectival changes of vantage points of view.” 

Ultimately, “dances in virtual spaces can reenact forgotten movements, ‘lost’ gestures, amazing movement qualities, indefinite expressions, a ‘life after death’” explains Matos. Multiplicity, reproducibility, and gestures are at the heart of Matos’ creation Myselves. Myselves is an interactive choreographic performance that stages a dialog between a dancer and her multiple selves, embodied in an autonomous and unpredictable visual and aural creature, which emanates from her psyche as interpreted by optical and body sensors.

Aoi Nakamura and Esteban Lecoq, founders of the Medway/Kent based dance company AΦE, define their projects as “performance,” at the junction of the virtual and the real: “We don’t create dance pieces. We create experiences that can use dance in it, as much as lighting, or virtual spaces. All those elements can be explored at different levels of the experience.”

In their work-in-progress Reboot, Nakamura and Lecoq invite audiences to step into a surreal future world where time and space bend and boundaries between audience and performers are blurred. AI technology interacts with the audience who becomes performers impacting the narrative, as invisible mechanisms translate their human emotions into actions.

Performers, dancers, and choreographers of the virtual also find themselves as translators, as Brandon Powers describes: “I personally define myself as a creative director and choreographer, but often I like to describe myself as a translator because I sit between teams of artists and teams of technologists. I’m able to translate the conversation from one side to the other, because I have a unique set of skills where I understand both sides, and I can understand how both sides can really create a deeper and richer impact on each other.”

Through technology, Powers reinvent movements, physical language, and what it means to connect with one another. His creation DUET transforms what VR and dance can be as well as who they’re for. In pairs, participants from the audience adorn headsets and are guided by interactive music and light through an immersive choreographic experience surrounded by an audience as witnesses.

Both playful and meditative, DUET seeks to reimagine virtual reality as a space for embodied collaborative interaction and shift how we are accustomed to watching others move in a headset.

Through technology, choreographers and designers may reinvent their own practice. For Matos, “technology can provide choreographic and creative tools.” “Technology acts as a revealer of the invisible, through augmented tactility, retroactive feeling of touch, and physiological feedback.” 

Technology becomes an essential tool to unveil the unseen, unleashing creativity and possibilities. “I see collaborating with technology as a way to supercharge my creativity” explains Powers. “I like to allow the technology to do what it does best and then ask myself to do what I do best as the artist. Artists are fantastic at understanding structure and dramaturgy. As a choreographer, I really understand physical space. Then, I can allow the technology to take us to new worlds or create infinite numbers of new possibilities, or to manipulate my movement data. When we allow the technology to do those tasks, it becomes really empowering to us as artists, and doesn’t feel like we’re being replaced by it.” 

Technology also redefines collaborations between choreographers and designers, between digital artists and dancers. The work of Jeanne Morel and Paul Marlier perfectly reflects this hybridity: “I would say that we are the two ‘choreographers’ in the primary sense of writing with the body (χορεία, khoreia: dance and γράφειν, graphein: writing)” shares Jeanne Morel. 

From the biometrics data of his muse captured during her dances in extreme environments and microgravity, Paul Marlier generates digital and immersive artworks in which each spectator is able to interpret his own lightness. Jeanne Morel and Paul Marlier question the rest, the imprint, the invisible essence of grace. 

Not only does technology allow new forms of collaboration, it also creates hybrid practices.

Using technology in dance allows the creation of an “augmented stage,” which “combines a physical stage with its virtual double and synchronizes the performance of the performers on stage in real-time in a digital universe” explains Marie Albert, Production Director of the science and technology production company Dark Euphoria.

Although central to her projects, technology remains a tool to execute an idea and an artistic vision notes Albert: “At Dark Euphoria, we are totally agnostic regarding technologies, meaning that we always start from the artistic vision before finding the right technological path. We experimented with real-time 3D volumetric capture, motion capture suits, but also handmade craft designs like costumes, or handmade jewelry mocap accessories.”

At the junction of hybridization, ecology, and experimentation, dancing in the virtual redefines the constraints and inconveniences of traditional shows.

Dancer and choreographer Blanca Li, creator of the immersive live performance Le Bal de Paris, produced by Backlight studio VR shares this view: “Immersive theatrical shows in a traditional way, mean that you must invest in large spaces, with a detailed scenography, accessories, and you have to get a lot of performers. It is very difficult to travel with these kinds of shows, and it requires a lot of material construction and costumes which cannot be recycled. Virtual immersive shows allow to expand dramatically the limits of space, the number of performers with nearly no limit. There is no need for material set or accessories, or costumes to be carried on a tour. I can replicate the experience as many times as I want, in different countries at the same time, with only a human crew to carry out and a set of VR headsets, trackers and computers. We also aimed at reducing our battery use, by developing a new technological set up to use our trackers wireless.”

Le Bal de Paris is an immersive live show enhanced by 35 minutes of virtual reality inside an artificial, fantastic, poetic and surprising digitally created world. Each member of the audience is invited to watch, dance, and interact with live dancers. 

“Movement is the last of the performing arts to be digitalized, it will be as revolutionary for our field as it has been for the music and image at the end of last century.”
Gilles Jobin

The pandemic accelerated the transition into the virtual as dancer and artistic director Clémence Debaig explains: “My projects tend to be a mix of storytelling, technological innovation and social experiments. This often leads to working on themes such as empathy, control and agency. The starting point of our work with VR has really been the necessities of the pandemic. Very early on during the first lockdowns, we managed to keep performing on Zoom and other livestreaming technologies. I love re-inventing formats and breaking out of pre-defined boxes!”

As artistic director of Unwired Dance Theater, Debaig develops projects at the intersection of dance and technology. In her new creation DISCORDANCE, she expands on the experience of a traditional dance performance by combining it with the interactive potential of immersive technologies.

DISCORDANCE combines live dancers, motion capture, VR technology, telematics, and immersive theatre. Audiences can choose their level of immersion and participation. In-person and remote audiences coexist in the virtual environment and interact with the live dancers. Each format interweaves visuals from each virtual and physical location, making DISCORDANCE a truly unique hybrid experience. 

Creating New Formats, Reinventing Bodies

Not only do choreographers and artistic directors of the virtual world re-invent formats, they also redefine the definitions and the limits of the body through technology.

“The performances question the borders between fiction and reality, the social construction of the body, and the psychological structures in human relationships” explains Matos. “Virtual technologies, if properly used, can magnify the body, augment its expressive modes, expand its poetic visions, and offer audiences in-depth means of perception, and empathy.”

Transformed, revealed, or magnified by technology, bodies are always at the center of choreographies.

For choreographer and performer Margherita Bergamo Meneghini, “immersion” and “participation” are key to her artistic process, and “XR technologies offer enormous possibilities for those purposes.” “I am a choreographer, and so my starting points are body expression, creation with bodies in movement, and observation on how moving elements relate. I am also interested in investigating the relationship between languages and the body. I’m interested in working with the human body as a communication tool.” 

Technology offers new possibilities to choreographers to present and represent the body through new perspectives: “The digital offers me the possibility to play with scales, multiply and modify bodies, play with gravity, and have an unlimited number of sceneries at my disposal” explains Gilles Jobin, founder of the Cie Gilles Jobin, compagnie de danse digitale. 

Technology also enables new body movements, which were unseen before, as explains Li: “It was also interesting to play with movements that were supernatural, and could not exist in reality, because you can have the dancers play with the trackers. I have my dancers flying, and it is just a trick. Exploring all the possibilities opened by VR and our tracking system was very interesting from a choreographic point of view.”

Bodies of the dancers and bodies of the audiences can also interact in new virtual spaces: “Dance is often at its best when bodies interact together.” “Having the spectators and dancers to be able to dance together in the real world and at the same time in the virtual world through their avatars was a big challenge.”

The Birth of a New Spectatorship

In those new virtual settings, audiences become co-creators of the show.

For Li, enabling audiences to dance throughout Le Bal de Paris “is also a very inclusive experience, even a therapeutic one for some people who have difficulties moving and get suddenly a new physical freedom when they perceive themselves as completely functional and beautiful persons.”

Hence, the participation of audiences is at the core of new artistic practices brought into the virtual world.

For Meneghini, “the bodies of the performers, in their expressive fullness, transport the spectators to be participants, and finally to be co-creators of the work of art. With its multiple forms, VR invites to experiment, to play, to simulate, and finally, to return to our everyday life with an extra gear: having danced wildly together with professional dancers and other participants, in dimensions unattainable in reality.”

Technology offers the possibility to dance in new spaces for dancers and audiences alike, modifying the border of the stage. EVE3.0, directed by Meneghini, is a live contemporary dance hybrid performance using VR devices. The performance invites twelve audience members to participate in a body-based storytelling by dancing with different real and virtual performers through a multi-sensory illusion experience that involves sight, hearing, touch, and movement. 

“Dance is the allegory of life, the universal poetry. It is the constant movement that connects us to this fundamental imbalance. And virtual universes need poetry, moments of contemplation, almost meditation. And once again one is not contrary to the other. The virtual cannot be devoid of humanity. It is where our intimate emotions intersect.” Paul Marlier

For Charles Ayats, author and designer of immersive experiences, audiences become “ubiquitous:” “As a VR Designer, my aim is to surprise the audience, giving them a new cognitive perspective”. “One of the virtuality’s power is ubiquity, you can be seated on a tier, be on the stage, be on the moon, or in your bed listening to a story. All these virtual spaces share the same space and time.” 

No reality now exemplifies this idea: in this hybrid project combining virtual reality and stage performance imagined by choreographer Vincent Dupont and Ayats, and produced by the production company Dark Euphoria, 100 spectators find themselves in a space where reality communicates with the virtual. Three dancers, some equipped with “motion capture” suits, take over the stage and a metaverse. In the stands, the spectators equipped with a virtual reality helmet can navigate between the virtual and real worlds. 

Dancing into the virtual and investigating new virtual and universal spaces may break the fourth wall, as explains Morel: virtual spaces “break elitism and give everyone the possibility of dreaming, of escaping. We have all experienced it during the lockdowns. I also see the possibility of finding our rhythm, our place as a spectator.” Morel continues: “When we create 360° artworks, the experiences are as well suited to VR headsets as to domes or planetariums. Solitude is therefore not induced but it can be chosen.”

Technology also enables audiences to experience the same show and create commonalities, even if audiences may be far away in space reflects Jobin: “When we develop pieces like Cosmogony a live piece under motion capture that we perform from our Geneva studio remotely, we cancel the geographical and political borders for diffusion, no visas or ESTA are necessary to invite Cosmogony, and no travel costs! And for some reasons, the live experience, the synchronization of the live action and the audience creates emotions.”

Cosmogony is a live performance featuring three dancers who are motion captured live at Gilles Jobin Company’s #Studios44MocapLab in Geneva. The movements of the dancers are captured and sent instantaneously through cyberspace to appear as avatars, on-screen and in real time for a live audience, thousands of miles away from the company’s studio. The dancers’ bodies and their avatars are entangled like particles in a quantum state, their movements instantaneously teleported across the world. The dancers act as the puppeteers of their own bodies, animating their avatars in real time and composing the cosmogony of a world in suspension.

Dancing in the Virtual: New Questions, New Inquiries

Bringing dance and its centuries-old traditions and etiquettes into virtual spaces raises questions. Will virtual spaces replace traditional ones?

“We live in an increasingly technological world, where new possibilities and ancestral ways of living and interpreting life coexist” mentions Meneghini. “It’s extremely exciting to feel like I’m at the beginning of a new art form being made and that I can bring the ancient to the highly technological” adds Powers.

For the interviewees, technology does not aim to replace. It is a medium used by the artists who remain the sole creator of their work: “I think choreographers are so vital to creation in the extended reality space, and we are such perfect leaders for it, because not only do we understand physical space, but we understand the body. And XR is a spatial medium” argues Powers.

The digitalization of the arts is a historical and continuous process from which dance is not exempted adds Jobin: “In the sixties when contemporary dancers started to perform on the roofs of New York or in gallery spaces, they were exploring the new territories for dance. Critics were saying ‘it is not dance if it is not in a theater!’ But today every contemporary dance festival has a performance in a public space. Digital spaces are new spaces that have become available for dance thanks to the acceleration of the processing power of the computers and advances in immersive technologies and real-time game engines. Movement is the last of the performing arts to be digitalized, it will be as revolutionary for our field as it has been for the music and image at the end of last century.”

However, resistance is still found within institutions, as explains Powers: “The last challenge I will offer is that of venues and funding. It is really hard for an artist to find homes for the work. That’s why I launched MTFxR, a program supporting the next generation of XR musicals, as part of the Musical Theatre Factory. I was also a co-founder of the 5th Wall Forum, which is a summit and international incubator that was founded amidst COVID in order to accelerate live performance in extended reality.”

A larger reflection on sustainability and the transferability of technological developments also requires further investigation: “The new hybrid forms of live performance are often developed within the framework of one specific artistic production, without any reflection on the sustainability and transferability of the technological developments made. It is essential to initiate a more structuring reflection on these questions and to create ‘common’ tools in partnership with the venues, usable in all types of theatre and stage and for several artistic creations” explains Albert.

For Debaig, technologies sometimes do not intersect with the context of a traditional theater: “One of the challenges is to navigate those worlds that don’t currently intersect. For example, bringing livestreaming technologies into a theatre might not interface nicely with the tools the technicians use to manage projections.”

“It’s extremely exciting to feel like I’m at the beginning of a new art form being made and that I can bring the ancient to the highly technological”
Brandon Powers

Virtual spaces are faced with new questions and inquiries, which require new modes of understanding.

Nonetheless, in the virtual as in real life, dance remains infused with humanity, as describes Marlier: “Dance is the allegory of life, the universal poetry. It is the constant movement that connects us to this fundamental imbalance. And virtual universes need poetry, moments of contemplation, almost meditation. And once again one is not contrary to the other. The virtual cannot be devoid of humanity. It is where our intimate emotions intersect.”

Far from being opposed, the virtual and the real are united in symbiosis explains Li: “I think it would be a mistake to believe these forms are competing, and that live show or theater is challenged by the new forms of reality and virtuality. Performing in a virtual space allows to enrich the spectator experience. Live stage show is a source of emotion being the result of the closeness between the spectator and the performer, a kind of symbiosis which can be made even stronger with immersive VR or AR.”

Jean-Marc Matos may have the final word: “In all experiences mixing dance and digital technology, contemporary dance recognizes one of its initial vocations, which is to always make the known world tip towards the unknown, to travel across the threshold of the unreachable, to always lead towards new horizons for both the eye and the body.”

Across spaces, virtual or real, dance embodies its initial mission: to rewrite the rhythm of life, in movement.

We would like to thank each interviewee for their time to answer our questions with sincerity and passion.

Jean-Marc Matos

Jean-Marc Matos is a dancer, choreographer, and artistic director. Trained at the Cunningham Studio in New-York, he has performed with David Gordon (Judson Church). He is interested in the impact of digital technology on society, in order to develop a meaningful relationship between dance and new media. He has choreographed more than 45 pieces which have been presented extensively in France (Avignon Festival, the Pompidou Center, etc.) and in many countries (Europe, Central and South America, USA, Canada, North Africa, India, Pakistan). 

Since 1983, K. Danse has been present in major festivals and cultural events in France and abroad: Festivals of Aix en Provence, Avignon In, la Rochelle, Châteauvallon, Métafort d’Aubervilliers, American Center of Paris, Georges Pompidou Center, Grande Halle de la Villette, Maison des Arts de Créteil, Maison de la Danse de Lyon, Centre National Art et Technologie de Reims, Centre National de la Danse, ISEA 2000 à Paris, Monaco Dance Forum, Espace Odyssud de Blagnac, Cité de l’Espace de Toulouse, Electrochoc Festival, Digital Art Center Le Cube of Issy Les Moulineaux, Arts Center of Enghien les Bains, the Ardenome Gallery in Avignon, the CENTQUATRE in Paris, Le Louvre Pyramid in Paris (with the BodyFail project, laureate of the 2017 Open Art Pulsar Prize), CURIOSITas Festival, Paris-Saclay (2019).

Full bio here

Aoi Nakamura & Esteban Lecoq

Aoi Nakamura and Esteban Lecoq are the founders of AΦE (AE), a Medway/Kent-based dance company. Their uniqueness lies in the use of technology to completely reinvent audience participation touring their productions worldwide. AΦE is Associate Artists of the Institute of Cultural and Creative Industries (iCCi)/University of Kent, moved into a building within the iCCi at Historic Dockyard Chatham and opened A+E Lab. A+E Lab is home to AΦE company and a new digital and cultural hub, producing innovative productions and deliver community and artist engagement programmes related to technology and culture.

Brandon Powers

Brandon Powers is a theatrical director and choreographer. His work focuses on capturing liveness in the digital and shifting culture towards a more embodied future. To achieve the latter, Brandon used his knowledge as a movement practitioner to devise Embodiment Design, a methodology for designing more human-centered XR experiences. He created volumetric interactive choreography for Queerskins: ARK (Venice International Film Festival 2020, Tribeca Film Festival 2020), a VR saga, co-produced by Intel Studios, and collaborated on the AI-powered immersive installation, Frankenstein AI (2018 Sundance Film Festival), choreographing an AI-powered dance performance. He has worked alongside Tony Award winner Andy Blankenbuehler for Bandstand (Broadway/Paper Mill Playhouse) and created live work at La MaMa, HERE, Ars Nova, and Performance Space NY.

On TikTok, Brandon has grown a community of over 50,000 followers sharing his work and tips on becoming a creative professional. His videos have been recognized by BroadwayWorld and Playbill, with the latter naming him the “unofficial director” of the Ratatouille Musical phenomenon.

Brandon is a proud graduate of Northwestern University, NEW INC member, Associate Member of SDC, co-Founder of 5th Wall Forum, and staff member of Musical Theatre Factory where he launched MTFxR, a program supporting the next generation of XR musicals. He has spoken about the intersection of arts and technology at the National Endowment for the Arts, Creative Tech Week, the TCG National Conference, Verizon’s 5G Lab and Lincoln Center.

Jeanne Morel & Paul Marlier

Jeanne is a dancer and actress with a degree in Philosophy and History of Art. Paul is an architect, researcher, and digital artist  

The work of Paul Marlier and Jeanne Morel shows the imprint of the fragility of life, the power of grace. From the biometrics data of his muse, Jeanne Morel, captured during her dances in extreme environments and microgravity, Paul Marlier generates digital and immersive artworks in which each spectator is able to interpret his own lightness. Jeanne Morel and Paul Marlier question the rest, the imprint, the invisible essence of grace. Between space quest, philosophy, science and new technologies, Paul Marlier and Jeanne Morel invite the spectator into the intimity of emotion. The dancer’s performance in extreme territories are recorded using brain and body sensors.. Paul Marlier and Jeanne Morel invite us into a journey between micro and macro in order to dream and escape, for a while, from the gravity of the world.  

Marie Albert

Marie Albert is passionate about transdisciplinary innovation and the crossroads between art, science, and technology. 

With 10 years of experience in the development of art & innovation projects (European program Horizon 2020 S+T+ARTS, art-science projects, fablabs and third places), she joins Dark Euphoria at the beginning of 2021 to accompany artistic projects (immersive and interactive experiences, digital art, new media creations…). She is also a journalist for the online media Makery.

Blanca Li

Blanca Li is a choreographer, dancer, filmmaker, and director of interactive exhibitions. Whether performance, opera, music video or feature film, she initiates and realizes a great number of projects: “I like to give life to all that’s in my brain”.

Never restricted to one style, she gets her inspiration from a broad spectrum of physical forms of expression (from flamenco to classical ballet and hip-hop).

With Blanca Li, everything begins and ends in the energy of movement and dance. Blanca Li has been elected as permanent member of the new choreography section of the French Académie des Beaux Arts in April 2019. Her ceremony is here.

She is the Artistic Director of Los Teatros del Canal in Madrid since November 2019.

Clémence Debaig 

Clémence Debaig is a dance artist, UX/XR designer and creative technologist based in London. 

And she is the Artistic Director of Unwired Dance Theatre. She creates work for remote audiences, galleries, the stage, and more unconventional sites, in the form of interactive installations, audio-visual experiences and performances. Her background in engineering and UX design, combined with her experience as a performer and choreographer gives her a unique perspective when working with dance and technology. Her work explores notions of control, empathy and intimacy, questioning how technologically mediated interactions influence human behaviours.

Since the beginning of the pandemic, she focuses on bringing remote performers and audiences together by exploring new forms of live interactive work online, including work with telepresence, networked wearables, Mocap technology and VR.

In February 2021, as part of the Open Online Theatre programme, created by IJAD Dance, she developed her latest work Remote Intimacy – a dance performance happening live between London and New York where telepresence is mixed with networked wearable technology. In 2021, she created Unwired Dance Theatre, a creative company that makes playable experiences around empathy and remote connections through immersive performances, technology and audience participation.

Margherita Bergamo Meneghini

Margherita Bergamo Meneghini has been dedicated to contemporary dance and completed studies in Choreography in Barcelona after a professional career with the Italian rhythmic gymnastics team. Her way of representing dance seeks unconventional lines, alternative contexts and experiences, always with the participation of the audience. She worked with the dance company Erre que erre from 2005 to 2010, and she was the artistic co-director of Les filles Follen from 2010 to 2015.

In 2017, she founded Compagnie Voix, and in this framework she began a research creation on dance and technological applications. Her piece “Eve, dance is an unplaceable place”, co-created with Daniel González, won the Laval Virtual Award Recto VRso, the Grand Prix VR at the Kaohsiung Film Festival, and it is selected in the Catalog of the Digital Department of the Institut Français. She currently creates choreographies in 3D, VR and XR for Trizz Studio, Atelier Daruma, Digital Rise, Tamanoir and is enrolled in a research creation dissertation at Edesta Université Paris 8 & Università di Bologna.

Gilles Jobin

Gilles Jobin is a choreographer living and working in Geneva whose productions have been performed all over the world since 1995. Based in London from 1997 to 2004 he created the groundbreaking pieces A+B=X (1997), Braindance (1999), The Moebius Strip (2001) and Under Construction (2002), relying on choreographic language outside of established aesthetic frameworks that included forays into visual arts and live art. In 2003 he created TWO-THOUSAND-AND-THREE for the 22 dancers of the Ballet of the Grand Théâtre de Genève for a performance that “transcends both classical and contemporary dance” (Libération).

In 2015, he received the Swiss Grand Award for Dance from the Office Fédéral de la Culture in honor of his career to date. Nowadays Gilles Jobin combines contemporary dance with his passion for new imaging technologies, resulting in the production of the film WOMB in stereoscopy (3D) in 2016, FORÇA FORTE in 2017, a duo using motion capture images and in 2017 VR_I, a location-based piece in immersive virtual reality. His creations are being presented in major film festivals such as the Sundance Film Festival (2018 and 2020) and the Venice Film Biennial (2018 and 2020), leading dance festivals such as the Lyon Dance Biennial (2018) or BAM (2019) in New York or museums such as Haus For Electronishe Kunst / HEK in Basel. With the same creative team and dancers behind Magic Window (2019) and Dance Trail (2020), two augmented reality dance pieces, followed by La Comédie Virtuelle a multiuser VR piece and La Comédie Virtuelle – live show a real time multiuser performance in VR. In 2021 he created Cosmogony a dance piece motion captured live in the company’s studios and streamed in real time globally.

Charles Ayats

Charles Ayats is an author, director, and designer of interactive and immersive experiences. He obtained a Master’s degree in Interactive Digital Experience from the École de l’image Les Gobelins in Paris.

As an interactive designer of web documentaries for Arte, he creates participatory experiences, often enriched with game mechanics to facilitate transmission of knowledge, exemplified by Type:Rider (2013) on the history of typography.

Always on the lookout for new formats to tell his stories, he is awarded with differents VR and AR experiences : SENS VR (2016), 7 lives (2019), Scream VR(2019) or M.O.A (2020).