Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

I've only taken some basic signal analysis courses, so I might be missing some things.

Purely theoretical question:

What methods exist for representing audio?

What methods could be made for representing audio, more specifically musical audio?

So far, I'm aware of:

  • Viewing the waveform (Soundcloud does this), mostly useless except for seeing "loudness"

  • Spectral analysis (Example), good for seeing frequency and "loudness"

Essentially I'm wondering if there is a way one could "see" the notes, beats, and so on of a song, visually.

Right off the top of my head I can think of displaying 3 differently colored waves over time representing treble, mid, bass in a soundcloud-like container with the section playing (or moused-over) being magnified, with the surrounding waveforms being compressed into the corners (like a wide-angle lens effect).

EDIT: I don't know where this could be used, this was just born out of my frustration with current audio visualizing technology.

I imagine having a 3d graph of a spectral analysis over time would be the "best" solution since you see everything but it might not be the most elegant and it might not be portable to places like soundcloud.

Even current spectrum analysis is hard to decipher (Too low level for images):

FL Studio wave editor

I'm essentially wondering what might work for casual users, and for people wondering ahead of time how the song will play out.

THIS HAS BEEN CROSS-POSTED TO DSP: linksauce

share|improve this question
    
MP3 software 'visualizers' might be something to look into. Lots of examples there. –  DA01 Jun 29 '12 at 16:33
3  
I'm intrigued by this question, but "best" is a bit problematic without further context. Best for what type of user? A casual user who is trying to understand what a song 'sounds' like? For a professional who's trying to 'fingerprint' a song? etc. A bit more information will help you get better answers. –  Daniel Newman Jun 29 '12 at 16:34
    
Refined my question. –  dudeofea Jun 29 '12 at 18:01
2  
See also What would be a good way to thumbnail audio –  Ben Brocka Jun 29 '12 at 18:15
    
Yes, good thread and useful for displaying the time aspect into a single square picture. But it's hard to know what you're looking at if all you see is the waveform. –  dudeofea Jun 29 '12 at 19:17

3 Answers 3

up vote 2 down vote accepted

Bit of a wild-card (two, actually), but have you seen this?

Music with color and fire

At the risk of sounding even more idiotic, I think Disney could be a good source of inspiration (have you watched 'Dumbo' or 'Fantasia' recently?).

share|improve this answer
    
Can you summarise the content of that link? A link isn't an answer in itself, it's just a link. You should provide details of the link and summarise the relevant aspects so that your answer is of benefit to people. –  JonW Jul 13 '12 at 20:06
    
The guy in the TED Talk was thinking the same thing as me: he summed up a song in a single picture that you could understand and you can even find similar songs in an instant. –  dudeofea Jul 17 '12 at 21:00

I guess there are two problems you're asking; how to make a sound file look like sound before it's playing, and how to display audio while it's playing.

I can't help much in the first case (although the waveform a you mention is relatively intuitive as an amateur), the convention I'm aware of for the second case is a VU meter (either a digital representation of an old-school needle moving rotationally, or a simple field or a few that fills/lights up as the audio plays).

Those are often used here in Australia on newscasts that have phone audio playing to denote "audio only".

share|improve this answer

To be able to visually identify elements in the song, you will need to perform some sort of pattern matching between time frames (filtered windows in technical lingo) and a larger number of reference samples.

This process is indeed better asked in http://dsp.stackexchange.com, however, assume this can be dones, you could use a predefined visual symbol for each reference samples, then show a time line with the identified symbols (and this returns the answer to the domain of UX).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.