AR music jamming application for remote music collaboration.

Overview

This application is part of an ongoing research project with the ACME Lab at CU Boulder. AR Drum Circle is an application that enables drummers to play together in a drum circle using physical electronic drum pads and augmented reality technology. Unlike previous remote jamming applications, musicians who use this application will see AR avatars of each of the other drummers they are playing with which they can place in their own environment via AR fiducial markers.

The application runs on android mobile devices with plans to port the application to the Nreal mixed reality headset. Two versions of the app exist which serve separate functions. In the version outlined in the system diagram above, users drum using a MIDI drum pad. The MIDI notes are sent over a server and translated to a 3D avatar which mimics the motion and sound of the drummer. A second version of the application allows users to play on a real drum as opposed to a MIDI device. The live audio is then picked up and sent over the server where it is then converted to MIDI for playback on the second users device. The audio to MIDI conversion is done using a Pure Data script.

Click and drag to view video in 360 degrees, physically move device if on mobile

The app makes use of particles, character customization, and other visual effects to make for a more visually interesting experience. Another feature of the app is the ability to record and loop your audio. This function works much like how a traditional looper pedal does with players able to record multiple loops at once for many layers of sound. This feature can be seen being demonstrated below.

Features