AΦE - LILITH.AEON - Immersive storytelling using AI-generated interactive realtime motion
AΦE - LILITH.AEON
Immersive storytelling using AI-generated interactive realtime motion
PUBLISHED 11 JUNE 2025
AΦE (AE) is a dance company founded by accomplished dance artists Aoi Nakamura and Esteban Lecoq in 2016. AΦE creates original narratives and distinctive storytelling approaches that reinvent the audience experience while pushing the industry's boundaries and broadening the range of audiences. As trailblazers in this area, it was the UK's first dance company to tour Virtual Reality (VR) and Augmented Reality (AR) productions internationally. It explores and develops unique experiences while incorporating cutting-edge technology into performances. It has toured across 16 countries and performed to over 100,000 people.
AΦE is an associate artist of the Institute of Cultural and Creative Industries at the University of Kent and in 2022 it launched A+E Lab, a cultural and technology hub located at Historic Dockyard Chatham, UK, producing innovative productions and delivering community and artist engagement programs related to technology and culture.
Project Background
In 2018, following losses of family members, Aoi and Esteban began to explore concepts around transhumanism - the belief or theory that the human race can evolve, beyond its current physical and mental limitations, by means of science and technology. They were initially inspired by ‘To Be a Machine - Adventures Among Cyborgs, Utopians, Hackers, and the Futurists Solving the Modest Problem of Death’, a book by journalist Mark O’Connell, documenting his travels and meeting with transhumanists. This led to the discovery of Matheryn Naovaratpong, a 2-year old girl from Thailand, who after losing her life to ependymoblastoma, a rare form of brain cancer, became the youngest person to be cryogenically frozen, in the hope of future revival as medicine advances.
Taking inspiration from this story Aoi and Esteban imagined the potential future of a reincarnated child, Lilith, and set about trying to demonstrate what the transhumanism movement believes and how AI may force us to rethink our views of life, death and what is in between. They began their work with neuroscientist, AI researchers and creative technologists to attempt to create an AI model that could respond to human interaction. The result was a chatbot that was able to generate poems from word prompts. These resulting poems were then used to build a script and associated choreography for a live dance piece focusing on the experience of Lilith’s parents, their grief at losing their child and the quest to revive her. Although this live dance project is an ongoing work yet to be publicly performed (entitled ORPHEUS) the choreography resulting from it was motion captured and used as the basis for the next iteration of their AI choreographic model, resulting in LILITH.AEON.
Project Approach
For the LILITH.AEON show, Aoi and Esteban envisaged a scenario where Lilith's consciousness was reincarnated far in the future, making it possible to interact with her through an LED volume. The idea of a 3D cube would enable an audience to see all aspects of the Lilith graphics offering an interactive 360° space. Taking the data based on their dance choreography motion capture, they trained their AI model to understand the movement capabilities of Lilith and generate new movement as part of the experience.
The AΦE team collaborated with Creative Technology to build a 3m x 3m LED cube-shaped volume that would be the centrepiece of the show, yet still transportable to enable the production to tour. Four cameras, one on each side of the cube, were connected to Unreal Engine and integrated with nDisplay, the rendering mechanism within Unreal, in order to map each wall of the LED cube and allow seamless blending of the AI-generated imagery with audience generated feedback. Extensive 3D animations were then developed to complement the AI-driven character movement, including backgrounds, visual effects, and an automated ‘spider’.
To power the whole experience, the AΦE team used seven GPU-accelerated workstations, based around graphics cards provided by NVIDIA, to control and render the cube imagery. Four workstations render the graphics from Unreal Engine on the four sides of the LED cube in realtime; the fifth workstation is an AI node that powers Lilith’s movement. The sixth workstation capture audience motion from the four cameras, also allowing audience members to be merged as avatars within the cube; and the last workstation acts as an editing machine that provides synchronisation with all the others. As with other physical elements of the production, the workstations are installed in mobile flight cases. You can learn more about the show’s concept and behind the scenes processes by watching the short documentary below.
Project Results
LILITH.AEON had its world premier in October 2024 at KM0 in Mulhouse, France. The 60-minute XR (extended reality) immersive performative installation unfolds on a large LED cube as a direct result of the character’s realtime interaction with the surrounding audience of 40 people, made possible by seamless merging of ground-breaking technologies including motion capture, 3D animation, machine learning and AI. A selection of clips of the animations and performance are shown in the video trailer below.
The next performance was in January 2025 at Creative Technology’s TECHFEST event, at the Tobacco Dock, London, UK; followed by a lecture performance in March at SXSW Festival in Austin, Texas, USA, enabling the audience to learn AΦE's creative process and how to integrate these technologies into immersive productions.
Feedback from these initial performances was extremely positive, such as “It was brilliant, beautifully performed, funny, thought provoking & very engaging! Well done! A truly refreshing approach, thank you!” and “Loved it! So interesting to see a really creative and engaging insight to the process alongside the work itself.” Between each show the AΦE team fine-tuned the AI model to iterate Lilith’s unique interaction with audience members.
Further 2025 performances are scheduled for May at the Node Media Lab in Vienna, Austria, the New Drama Festival in Bratislava, Slovakia and many more to be announced.
Next Steps
Aoi and Esteban consider LILITH.AEON a ‘baby’ AI, as the amount of the dance motion capture achieved to date is relatively small and each model needs its own computer for weeks or months of training. At present Lilith’s movement involves some repetition, so the AΦE team intend to continue training the AI model, aiming to make each performance better, uploading additional choreography data, providing the model with greater learning potential. At present, the intention is for the LILITH.AEON production to have a lifespan of five years, offering huge potential to further refine the experience.
The Scan Partnership
Scan and its in-house system integration division, 3XS Systems, worked with the AΦE team to design the seven NVIDIA GPU-accelerated workstation solution to power the LILITH.AEON experience. This included not only configuring hardware and software specifications, but also understanding touring and transportability restrictions in order to enable the production to be staged in any space without the need for hiring equipment or leaving vital computational elements to chance.
Project Wins
Realisation of XR performative installation with AI-powered interaction with a live audience in realtime
Bespoke touring production set-up enabling worldwide performances without external support
Aoi Nakamura
Choreographer, Dancer, Co-Founder & Co-Artistic Director at AΦE
"“We are really grateful for the support we received from Scan for all the workstations needed to run the AI model and interactive experience”"
Esteban Fourmi
Choreographer, Dancer, Co-Founder & Co-Artistic Director at AΦE
"Without support from partners like Scan we would struggle to take a concept like LILITH.AEON and successfully develop it with just public funding alone."
Dan Parkinson
Director of Collaboration at Scan
"We are proud to support such artistic innovators like Aoi and Esteban, helping to demonstrate how AI can be employed in use cases beyond what we see in the media everyday."
Speak to an expert
You’ve seen how Scan has helped the AΦE team deliver an interactive AI-powered experience Contact our expert AI team to discuss your project requirements.
phone_iphone Phone: 01204 474210
mail Email: [email protected]
Related Content
AΦE (AE) Dance Company
Learn more about Aoi and Esteban’s projects, merging dance, storytelling and immersive technologies.
Read More >
Upcoming Events from AΦE
See the latest schedule of upcoming immersive performance events across the globe, delivered by the AΦE Dance Company.
Read More >
A+E Lab
Discover A+E Lab, curated by AΦE to develop and deliver an ambitious programme of innovative educational workshops.
Read More >