ARCube

The ARCube augmented reality interface.

ARCube is an ongoing project on the use of augmented reality interfaces in combination with hardware musical instruments and controllers. We are exploring new ways of composition and performance, with a scope on spatial sound and electronic music.

The AR interface consists of a semitransparent cuboid with edge lengths of about 30 cm. The augmented cubid can be freely placed in space by the user. Four spheres with a diameter of about 2 cm can be freely placed inside or outside of the cube. Their position relative to the cuboid is used as control data, turning the interface into a multi-value X-Y-Z slider.

Moving virtual objects.

ARCube in Action

SinusTon

Performing with ARCube at the SinusTon electroacoustic music festival in Magdeburg, Germany.

Live Modular Performance

First performance with the ARCube by Hyunkyung Shin.

Summer School Demo

30 high school students having fun with the ARCube.

Usablity Opportunities in Hybrid Spatial Interactions

In a first user study we were exploring two questions:

     1: Is the AR interface suited for spatial aound control?

     2: Can the AR interface be used in combination with a traditional MIDI controller.

Participant interacting with the hybrid setup during the UX study.

A paper + poster with results of this early stage user experience experiment was presented at AC SUI 2024 in Trier, Germany.

Performing using the ARCube at the SinusTon electroacoustic music festival in Magdeburg, Germany.

SC Symposium 2025

In March of 2025, the lab presented a paper and performed at the Supercollider Symposium in Washington, D.C..

Performing live at the SC Symposium.

Hyunkyung Shin and Henrik von Coler. 2024. ARCube: Hybrid Spatial Interaction for Immersive Audio. In Proceedings of SUI ’24. ACM, New York, NY, USA

First person view during the user experience study.
Publications4

Hyunkyung Shin, Henrik von Coler: 2CUBES: A Multi-User Augmented Reality Performance, 50th International Computer Music Conference, 2025

Hyunkyung Shin, Henrik von Coler: A SuperCollider-Based Spatial Audio Performance System with AR Interfaces, SuperCollider Symposium 2025, 2025

Hyunkyung Shin, Henrik von Coler: AR Matchmaking: The Compatibility of Musical Instruments with an AR Interface, Proceedings of the International Conference on New Interfaces for Musical Expression, 2025

Hyunkyung Shin, Henrik von Coler: ARCube: Hybrid Spatial Interaction for Immersive Audio, Proceedings of the 2024 ACM Symposium on Spatial User Interaction, 2024