1

That's the main question, but here is some background:

We created an app whose goal is to make raw health data easy to understand for the average person. This health data includes statistics about the number of people in each state who suffer from common diseases/disorders. The target of the app is young adults who are looking to move to a new area, and may be concerned about info like smoking rates, allergy alerts, HIV/AIDS rates, etc.

Now we are channeling our energy towards making this an academic research project. Our main goal is to find out if this app is useful (does it help people understand publicly available health data?).. But how can we measure this objectively?

3
  • From a UX perspective, ask users. From an academic research perspective, I'd hire someone in academia to help. Ideally someone in a public health related program. This isn't really a user experience question.
    – DA01
    Jul 9, 2012 at 20:58
  • I posted here because the question is about the user's experience with the app. How can you objectively measure the impact it has on a person with a simple questionannire, or something else, that can be obtained through an interview? What is the measure? I'm sure it's not as simple as 'Is this useful? Circle: yes or no' but I could be wrong. (Also, the team consists of cs, public health, and hci/ux people, and we are all facing this challenge.)
    – mdegges
    Jul 9, 2012 at 21:21
  • If the question is 'is this useful to you?' then you can objectively collect that data by asking if it is useful. If the question is if it is useful in a large social health standpoint, I think you'd need to bring in the social health experts.
    – DA01
    Jul 9, 2012 at 21:41

2 Answers 2

0

The answer to your question is not as complicated as you think.

First of all, what is "usefulness"?

"Usefulness concerns the degree to which a product enables a user to achieve his or her goals, and is an assessment of the user's willingness to use the product at all".

From Rubin and Chisnell

So, there you go. Lots of information in that definition. Taste each word and phrase and figure out what that really means. This boils down to two issues, right?

1) Define the criteria for "achieve the goal". That is your decision to make. This could be by asking the user, or it could be by checking if a task were completed, or it could be by setting a maximum time to accomplish a task.

2) Find out if the product meets these criteria. This might sound complicated, but imagine that your building a desk that must be able to carry the weight of 100 kg. The test is simple: Put 100 kg on the desk as see if it still stands. This is the same thing, but with two minor modifications.

  • You need a bigger sample. We're talking statistics, and you need to perform more than one test to verify your thesis.
  • You might consider some other questions. A bit more sophisticated test. Eg ask if he/she would use it again, or whether he/she would recommend it to a friend. You could even trow in some inverted questions that needs a negative answer to conform the usefulness. Eg "Do you think you would receive more useful information if you asked your doctor". You can confirm the reliability of your data by mixing "yes/no" questions.

Now, by getting loads of qualitative data from these tests, you can start analyzing the data and compare the results with the criteria you defined.

1
  • 1
    Thanks for your answer, it made it much easier to wrap my head around.
    – mdegges
    Jul 11, 2012 at 3:12
3

Instead of asking "Does it help people understand publicly available health data?", a better question may be "In what validated scenarios does public health data contain the answer to people's questions/problems/needs?"

The target of the app is young adults who are looking to move to a new area, and may be concerned about info like smoking rates, allergy alerts, HIV/AIDS rates, etc.

...seems like an assumption that needs to be validated. Are young adults concerned with this data when moving into an area? The success of the app rests on the truth of this assumption.

This is a question properly answered by doing user research before creating an app. Here's a few good thoughts from a recent article on UXMatters: How to Know When Your Product is Going to Fail

A primary purpose of UX research is risk mitigation. When we perform research we often look for indicators that let us know whether we are on the right or the wrong track with product development—and, if we’re on the wrong track, how we can get back on the right track.

If you put a product in front of users and they don’t tell you that they want it, you might be on the wrong track.

At the current point in your development cycle, it could it be as simple as: "Usefulness is indicated by usage" - analytics will tell you how useful your app is. If people do not use an app or continue to use it (providing it receives enough promotion/publicity) it is an indication of a flawed user need scenario.

The concept of validating assumptions (and user interest) early is a key idea in The Lean Startup. If your assumptions do not validate, it does not necessarily mean that the entire idea is faulty, but that a pivot may be needed to a different approach that better connects the useful data to actual user needs.

2
  • Good point about "Usefulness is indicated by usage", and that monitoring the usage would give useful information. Jul 10, 2012 at 1:57
  • Thanks. I totally agree with you: we need to address whether there is a need for the app and whether our target group is interested in health data at all. Hopefully we will get answers to these q's during the study when users actually interact with the app and compare it to alternatives (raw health data & other apps).
    – mdegges
    Jul 11, 2012 at 3:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.