Myself and I in the Stream of Virtual Subconsciousness
在我與我的翼識之間
Photo by Wang Li
A live, real-time interactive new media performance in collaboration with South Korean new media artist Jung Hyun Moon.
In the unfolded space of the subconscious, one encounters…self, and another self, and another.... Is it dreams within a dream, memory, past lives, or the cyborgian butterfly mind? Combining digital media coding design, motion capture, dance, and various settings of human-machine “improvisatory” frameworks, the performance explores the limits and fusions of sensorial and creative intelligence between human and technology. “The butterfly mind” comes from new media artist Roy Ascott’s theorization of variable reality.
In the huge gallery space, bodies, virtual images, and technology come together to bring to touch the experience of such a complex, expansive, and multi-dimensional reality, as well as the question of human beings’ perpetual fascination or likely oblivion — the self.
Concept: Jess Chiayi Seetoo and Jung Hyun Moon
Performance: Jess Chiayi Seetoo
Digital Interactive Media Art & Design: Jung Hyun Moon
Venue and Commission: Yuz Museum, Shanghai (“Hyundai Art + Tech” project, 2020)
Technical Development
Algorithmic animation
Body segmentation model (BodyPix)
Pointcloud using kinectV2
Visualization custom shader on WebGL (three.js)
Computer vision
Motion Tracking System: Developed a real-time motion tracking and networking system
Visualizing dancers' movements on stage using three Kinects and a network system.
“Virtual Memory”: Created a 3D space for storing body shapes of performers and audiences using BodyPix, Firebase, and three.js shaders.
Synchronization: Designed and controlled the synchronization of media content using Firebase Realtime Database.
3D Visualization: Simulated and controlled media equipment installations and live performance operations with 3D sketches.
Sound Design: Provided frameworks for composing sounds and collaborated with the project’s sound designer.
Installation: Supervised the installation of media equipment and managed the operation of the performance..
Space/Stage Plan
The performance took place in the 1st-floor gallery space of Yuz Museum, which was treated as a chamber of the subconscious mind. The performance included:
5 projectors and motion tracking systems for immersive experiences.
A single darkroom for sound immersion.
Audience participation as they followed the performer through space.
Digital Interactive Media Art & Design by
Jung Hyun Moon
Photo by Wang Li
