0

I have interviewed 36 people, divided in three groups of 12 people. For one group I asked to describe a song they just heard. The second group had to describe a song by observing only a graphical representation of the song and the last group had the song and the representation.

My intention was to measure how the graphical representation helped the users to increase the quality of their description.

Now, the problem is, are there works with methodologies similar to mine? How do you think I could measure the quality of a description? Number of words? Number of adjectives used? With an entropy calculator? I committed the mistake of not consulting first a methodology to inspire my process.

  • What do you measure the description against? What is the ultimate quality of the description? IS there one? – Nicolas Hung Jun 11 at 14:22
  • @NicolasHung Each group against each other. My initial hypothesis is that: "the visual representation improves the understanding and the quality of the description", i.e., it improves the description by the terms and the variety of terms used. – Hugo .L Jun 11 at 14:25
0

What do you measure the description against? What is the ultimate quality of the description? IS there one?

I think "descriptions" with the purpose to communicate effectively can be measured, such as # of words, grammar, word choice, sentence structure, etc. Similar to what Grammarly is doing.

However, "descriptions" of songs (an art) might follow different rules. Sometimes a sentence might be enough, sometimes a paragraph might not be enough, and many times the same text can mean different things to everyone. Similar to how lyrics speak differently to each listener.

Maybe you should have the same person read all 3 descriptions and ask them to judge it. I'm not sure if there is a quantitative method that would work. Or you could use both qualitative and quantitative and see if there is a correlation or not.

  • Thanks. Certainly I'll use the metrics like Grammarly does. I wish I could find any paper with such methodology, but in any case I'll ground each metric with its definition and a use case in the literature. Thanks again. – Hugo .L Jun 11 at 17:52
0

When collecting information you should consider whether you need qualitative or quantitative measures (or perhaps both) to answer the research question. In this case a qualitative answer might be an indiscreet value that is subject to individual interpretations (e.g. loudness, speed, how it sounds), or you can look at quantitative answers that are less subjective to individual differences (e.g. standard genres and descriptions of the tempo, key and other musical definitions).

You can also consider using a reference answer to baseline the responses that you receive, in which case you would be comparing a response relative to the quality of the 'correct answer'. But even in this case you might be comparing qualitative responses that need to be ranked to the reference answer, or measuring quantitative values (e.g. the number of words required to describe the answer) to derive at your definition of 'quality'.

The important thing is that you need to have a consistent measure, and that it should cover the potential range of responses that you might expect so that each response can be categorized and grouped together for further analysis. Something that will guide you in the process is to consider the type of values that the graphical representation provides and try to base the type of values you measure against it because it is the main prompt or comparison that you are using in this test.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.