4

I've only taken some basic signal analysis courses, so I might be missing some things.

Purely theoretical question:

What methods exist for representing audio?

What methods could be made for representing audio, more specifically musical audio?

So far, I'm aware of:

  • Viewing the waveform (Soundcloud does this), mostly useless except for seeing "loudness"

  • Spectral analysis (Example), good for seeing frequency and "loudness"

Essentially I'm wondering if there is a way one could "see" the notes, beats, and so on of a song, visually.

Right off the top of my head I can think of displaying 3 differently colored waves over time representing treble, mid, bass in a soundcloud-like container with the section playing (or moused-over) being magnified, with the surrounding waveforms being compressed into the corners (like a wide-angle lens effect).

EDIT: I don't know where this could be used, this was just born out of my frustration with current audio visualizing technology.

I imagine having a 3d graph of a spectral analysis over time would be the "best" solution since you see everything but it might not be the most elegant and it might not be portable to places like soundcloud.

Even current spectrum analysis is hard to decipher (Too low level for images):

FL Studio wave editor

I'm essentially wondering what might work for casual users, and for people wondering ahead of time how the song will play out.

THIS HAS BEEN CROSS-POSTED TO DSP: linksauce

Improve this question
7
  • MP3 software 'visualizers' might be something to look into. Lots of examples there.
    – DA01
    Jun 29, 2012 at 16:33
  • 3
    I'm intrigued by this question, but "best" is a bit problematic without further context. Best for what type of user? A casual user who is trying to understand what a song 'sounds' like? For a professional who's trying to 'fingerprint' a song? etc. A bit more information will help you get better answers.
    – Daniel Newman
    Jun 29, 2012 at 16:34
  • Refined my question.
    – dudeofea
    Jun 29, 2012 at 18:01
  • 2
    See also What would be a good way to thumbnail audio
    – Ben Brocka
    Jun 29, 2012 at 18:15
  • Yes, good thread and useful for displaying the time aspect into a single square picture. But it's hard to know what you're looking at if all you see is the waveform.
    – dudeofea
    Jun 29, 2012 at 19:17

3 Answers 3

Reset to default
3

Bit of a wild-card (two, actually), but have you seen this?

Music with color and fire

At the risk of sounding even more idiotic, I think Disney could be a good source of inspiration (have you watched 'Dumbo' or 'Fantasia' recently?).

Improve this answer
2
  • Can you summarise the content of that link? A link isn't an answer in itself, it's just a link. You should provide details of the link and summarise the relevant aspects so that your answer is of benefit to people.
    – JonW ♦
    Jul 13, 2012 at 20:06
  • The guy in the TED Talk was thinking the same thing as me: he summed up a song in a single picture that you could understand and you can even find similar songs in an instant.
    – dudeofea
    Jul 17, 2012 at 21:00
0

To be able to visually identify elements in the song, you will need to perform some sort of pattern matching between time frames (filtered windows in technical lingo) and a larger number of reference samples.

This process is indeed better asked in http://dsp.stackexchange.com, however, assume this can be dones, you could use a predefined visual symbol for each reference samples, then show a time line with the identified symbols (and this returns the answer to the domain of UX).

Improve this answer
0

I guess there are two problems you're asking; how to make a sound file look like sound before it's playing, and how to display audio while it's playing.

I can't help much in the first case (although the waveform a you mention is relatively intuitive as an amateur), the convention I'm aware of for the second case is a VU meter (either a digital representation of an old-school needle moving rotationally, or a simple field or a few that fills/lights up as the audio plays).

Those are often used here in Australia on newscasts that have phone audio playing to denote "audio only".

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.