Is there an optimised UX for syncing sound to visual cues?

Question

I'm building an (Android) app that displays realtime visuals of musical timing, which is supported by audio cues (metronome click).

I'm looking to implement a mechanism for the user to correct for any audio latency in the system, such that the audible click synchronizes with the visuals as accurately as possible.

My first pass at this was to create a screen with a large block of color that sharply flashes with a sound click at 120bpm (twice per second). I included a slider to allow the user to offset the audio by up to 200ms either side of zero.

I think this is satisfactory, but it had me wondering if there is a more intuitive way to match audio cues with visual cues - and have a better sense of when they are in-sync vs. not in sync.

A high degree of accuracy is a goal here, as any synchronization issues will become painful when using other features of the app.

So far I'm thinking along the lines of including a variety of differently styled visual cues, in an effort to give the user more visual information - for example, making a sequence of objects appear (or vanish) one by one with each click.

---

Perhaps a somewhat broad question, but is there any research or wisdom known of what visual information our brains need to be best able to sync with audio cues?

Answer 1

cool question!

I'm not aware of academic research of this; and I think your implementation is fine. For the best possible result I'd ask the question: For whom exactly and what use case exactly is your app for?

If you are marketing your app to pretentious grand piano players they probably expect a different UI than a friend group using this for a ukulele jam session.

To give you something to work with, have you checked out how various audio and video players/editors have solved this problem? Because most of them have a functionality like this, be it syncing audio/video, subtitle settings in streaming or aligning tracks when cutting video...just look around what eminent software does to solve this (and what the community thinks of the feature) and steal what fits best :)