CodeKlavier’s ‘ARquatic’ is a live piano music and Augmented Reality experience in which the pianist is programming generative virtual plants with an aquatic theme by improvising and performing music. The audience can experience the virtual objects and their development live on their mobile devices in AR.
The pianist is embedding code into her playing via the CodeKlavier programming language. The code is interpreted as Lindenmayer Systems, a formal grammar invented by biologist Aristid Lindenmayer to describe plant structures, which is nowadays widely used in computer graphics and generative art. These sets of rules are streamed live via the Internet to the audience's mobile devices where the 3D objects are generated and rendered in AR. The movements of the virtual plants are directly linked to the pace and dynamics of the pianist's performance. The audience can immerse itself within the virtual objects, circle around them or observe them from a distance. The pianist is accompanied by a live coder who is adding an electronic soundscape and live electronics to the piano sounds and further controlling parameters of the AR visuals.
ARquatic is both an user interface and live coding experiment as well as a sensual multimodal experience for the audience with both direct and indirect audio-visual coupling. It is unique, as the musician is not only controlling the 3D visuals with her playing but also giving shape to the objects via the code expressed in the music. Performed in gardens and parks, ARquatic is juxtaposing natural plants with computer-generated ones and is inviting the audience to find relations between real and virtual plant life, to imagine more speculative or fantastic plant species and to immerse itself in the augmented audio-visual landscape.
For the Düsseldorf AR Biennale we propose a short 5-8 minute pre-recorded version of ARquatic as an installation piece that can be played back by the audience (preferably with their headphones) at its destined location at any time. The AR visuals will be played back synchronised with the music. Their shape and movement will have the same source as in a live setting: The piano performance.
ARquatic was created from 2019 to 2022 by Anne Veinberg (piano, CodeKlavier creator), Felipe Ignacio Noriega (live electronics, CodeKlavier creator) and Patrick Borgeat (generative 3D visuals, app developer).
ARquatic was awarded the Joke ‘t Hart Prijs 2022 and was extensively touring botanical gardens and parks in Fall 2022 in the Netherlands. Another instance of the ARquatic installation will take part in a location-based AR exhibition in Liepaja (Latvia) as part of the Update Festival in November 2022.