Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

I want to represent an audio file in an image with a maximum size of 180×180 pixels.

I have been thinking about something similar to what Soundcloud does, although it will require some chopping and stretching to mash it into the square:

Screenshot of Soundcloud's player

I wonder if any of you have something for this. I have been searching around for a bit, mainly audio visualization and audio thumbnailing, but I have not found anything useful.

share|improve this question
    
Interesting problem! –  Ben Brocka Feb 8 '12 at 21:11
4  
Do you need to visualize any audio? Or are you working with songs? Or sound effects? Or voice memos (all the same speaker)? Or voice messages (from various speakers)? Bird calls? Word pronunciations? Instrument samples? Car horns? Radio transmissions from distant stars? –  benzado Feb 8 '12 at 22:16
    
Very good points @benzado! I am working with audio in general. I intend for this to end up in GNU MediaGoblin. –  joar Feb 8 '12 at 22:30
4  
Why would you want to? Can any useful information about a song be human-consumable through a visual representation of it? Perhaps. Maybe varying colors could mean something ... but certainly a graph of the waveform doesn't do any good other than being cute. –  xanadont Feb 8 '12 at 23:34
    
@xanadont - I want the visual representation to be as conveying as possible about the characteristics of the song. More like a spectrogram. –  joar Feb 8 '12 at 23:54

8 Answers 8

What about taking a waveform resolved to stretch only a short distance, say 200px, then wrap it around a ring so it fits inside the square thumbnail constraints. So... length of waveform graph would be... related to pi somewhat.

Like this, which I made in photoshop using the polar coordinates command.

Sample waveform on a circle

share|improve this answer
3  
Nice suggestion! –  Vitaly Mijiritsky Feb 8 '12 at 21:22
5  
I'm curious though, would this be practical? It's hard to quickly identify and distinuish audio samples by looking at their waveforms, and it would be especially hard when distorted like this. It seems like a short audio preview on mouseover would be more practical than audio thumbnailing. –  Peter Olson Feb 8 '12 at 23:20
1  
This looks like the visual hashes Justin mentioned in his answer. –  Nick Chammas Feb 8 '12 at 23:30
    
I was just about to suggest a circle or ring! I would make the waveform go closer to the center. –  Jarrod Roberson Feb 9 '12 at 4:16
    
@PeterOlson It would be great to quickly recognize different soundbytes. Whether that would be a usefull feauture is up in the air. –  Barfieldmv Feb 13 '12 at 8:46

A smooshed-up waveform doesn't seem very useful. For all practical purposes it's meaningless, and it'd be hard to tell two 180x180px waveforms apart at a glance. Instead you might consider a visual hash like Identicon. That would still be meaningless, but the thumbnails would be visually distinct. A clever algorithm could even make similar files (e.g. songs from the same album, or from the same artist) visually similar.

Digging around on Don Park's blog a bit turned up Frederic Brodbeck's Cinemetrics project, which may be useful. It's a visual hash intended for films that you might be able to adapt for use on audio.

share|improve this answer
3  
I love the Cinemetrics project! There's also The Echo Nest Musical Fingerprint –  Roger Attrill Feb 8 '12 at 21:17
2  
Cinemetrics reminds me of moviebarcode.tumblr.com –  kraymer Feb 9 '12 at 13:34
    
The visual hash will identify the track, allthough it will not convey any informatin about the track if you don't know it since before. The Cinemetrix reference is more about what I am looking for. –  joar Feb 10 '12 at 10:00

I didn't understand whether by "chopping and stretching" you meant taking a segment of the audio data and stretching it so it fills the square, or rearranging it. The rearranging approach sounds like a good idea.

enter image description here

share|improve this answer
    
That is exactly what I meant :) and while it would work, I wonder if there would be any other alternatives. –  joar Feb 8 '12 at 20:04
1  
You can try showing one frame of the segment and then playing the piece (both audio and video) on mouseover. That could be a nice touch. –  Vitaly Mijiritsky Feb 8 '12 at 20:09

Thumbnails are meant as a preview for identifying the object without getting into the details.

Using the wave form in a thumbnail would not help identify the audio track.

What you could do is:

  • Use a vocal thumbnail (e.g. 10sec preview) played on mouse over

  • Use a visual representation of the contents e.g. use a database or pre-identified and tagged soundtracks or use audio recognition. Once you have identified the elements of the contents you could include small thumbnails that represent those elements in one bigger thumbnail.
    E.g. a thumbnail containing a picture of the singer, a guitar or a picture of a train if the soundtrack is a sample of the noise a train makes.

share|improve this answer
    
What about spectrograms? freesound.org/data/displays/145/145491_1841925_spec_L.jpg They at least provide some information about rhythm, harmony and intensity. –  joar Feb 10 '12 at 10:02
    
@joar only for a single time frame, wouldn't help. To properly represent a sound track, you would need a 3D image (time, intensity, frequency) and still, only a small percentage of people would understand what they are seeing. –  Danny Varod Feb 10 '12 at 12:00
    
The spectrogram is a three-dimensional representation of the audio, X, Y, color represent time, frequency and intensity. –  joar Feb 10 '12 at 12:10
    
@joar in that case, as long as the viewer is not color blind, it would be informative, however, it would still be hard to differentiate between different tracks from a small thumbnail. –  Danny Varod Feb 17 '12 at 16:57

You might be knowing this already , this might work with the song name in center of the image or below.

This is how Apple creates thumbnails for Audio files

share|improve this answer
    
Yes, I know about the tradition to have a cover image, and then fall back on a default "media" or "media type" image. I am looking to visualize the audio data in some way - I have realised this might be the wrong forum for such a question and I have cross-posted it to stackoverflow. –  joar Feb 8 '12 at 19:43
    
Cool , Hope you find some thing interesting there !! –  Pratheep ch Feb 8 '12 at 19:56

Good question. In terms of UX there are a few different things you want in a thumbnail.

  1. Thumbnails should be easily distinguished so you may wish to use some sort of visual hash (eg. here)

  2. Similar items should look similar and different items should look different. Ideally you will want to encode a few things that are likely to remain similar across a sensible group of sounds. Here are some ideas as well as what they might help differentiate:

    • Length (eg. podcasts, short samples); this should probably be shown logarithmically
    • Some Frequency domain representation (deathmetal vs pop vs talking)
    • volume histogram (procedurally generated vs 'organic' sounds)
    • meta-data if it is available (filetype, stream-rate etc)
  3. I imagine the naming (assuming these files have names) will be up to the user/consumer, this is the fallback if the user can't interpret the thumbnail so make sure it's clear and prominent enough just in case. (this problem appears with icons all the time, see UX Myth #13: Icons enhance usability)

  4. Ideally the thumbnail should be memorable, though how to do this with sound I have no idea.

Finally you want all of this to be clear on your 180x180 thumbnail. Doing this well will doubtless be very challenging.

Perhaps something like this (though perhaps this is overkill or too cpu intensive):

  1. Map the default hue to a mixture of some or all of the items in 2 above, this should give you similar color files for groups of files.

  2. Frequency spectrogram in the background for saturation

  3. Volume histogram can be represented by changing the hue of buckets (ie. squares in your 180x180 representation, 4 in my example, though you should use more)

  4. Duration can be mapped logarithmically to a border surrounding the whole thing in the form of a bevel. (7 minute files will look almost exactly the same as 6 minute files, but completely different from 7 second files)

Here's an example of what it might look like:

Example of a sound file thumbnail

Hope this helps

share|improve this answer
up vote 2 down vote accepted

The solution I settled for was to use Freesound's spectrogram generation code, then applying the Wadsworth Constant to the position for the thumbnailing. This gives you a snapshot of the audio clip with the center at 33% into the audio clip.

Example ↓
two audio spectrogram thumbnails
Live examples

The code for the process can be found at mediagoblin.media_types.audio.transcoders.py, line #99.

I will not mark any answer as correct, since none of them, not even mine is universally correct.

share|improve this answer
1  
Thanks for reporting back with your chosen solution for this interesting problem! –  André Sep 5 '12 at 12:58
    
@André You're welcome :) –  joar Sep 6 '12 at 10:31

Along the lines of @Forthright's answer, I think mapping the volume intensity to a gradient (colour/grayscale) and plotting along a Hilbert curve could be interesting.

This would be similar to http://corte.si/posts/code/hilbert/portrait/index.html and would provide a rectangular icon that gives an impression of the sound patterns that would be distinct for different files and scale to any length of audio and any icon dimensions.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.