7

When we do User Experience Research we test a single application or even a single feature of an application. But there are no users who only uses one application. Users today are using several different applications on several different platforms in work, while traveling and at home.

This could be a Microsoft environment at work using Windows 7, SharePoint intranet and the Office applications. On top of that there could be a Time Registration System, an Enterprise Resource Planning system and a Quality system. This could also be accompanied by a Smartphone and all of its accompanied applications.

At home the user might have a tablet and a private Smartphone different from that of work. Then the user have credit cards, bank cards, gas station cards and all other cards where the user needs to remember the unique PIN-code on each card, which might change when we get a new card. Not to mention all the username/password combinations the user have to remember and change every once in a while depending on IT settings.

This is all part of the digital environment the users of today faces. I've tried to find research that covers the total digital environment of users, how they deal with these environments altogether. So far I've been unsuccessful finding relevant research. Have this kind of research been made?

6
  • 2
    Its a good question: people won't use one communication channel which is difficult - if they've got another easier method in their other hand. You might want to look at Ethnographic approaches.
    – PhillipW
    Aug 11, 2012 at 20:30
  • 1
    I do not know of any research done looking at how people use one or more applications across devices in their everyday life but I do know that Nokia used to (and probably still does) a lot of user research about how people use cellphones in general. Now of course tablet usages has also been studied. Definitely look at work done by Jan Chipchase. Also have a look at work done by Rachel Hinman (she also has a book: Mobile frontier). Great question. Thanks for making us all think about this. Caveat: Most of the work in this area is location, culture, and time specific & things change rapidly.
    – Viraj
    Aug 29, 2012 at 2:18
  • @Viraj Thanx! I'll take a look at the work of Jan Chipchase and Rachel Hinman. Aug 29, 2012 at 7:16
  • 1
    Nice to see some research questions in here +1
    – Daniel Li
    Aug 29, 2012 at 14:49
  • @HopeIHelped Agree, since knowledge comes from research as opposed to politics, which comes from opinions :) Aug 29, 2012 at 16:16

2 Answers 2

4
+50

Digital living labs are an option, stuff like Digital Spaces Living Lab andAn internationally distributed ubiquitous living lab innovation platform for digital ecosystem research. Living labs are a sort of holy grail, but they're also extremely difficult to set up relative to simple tests.

Instead of hoping for a pre-baked solution, look to examples of psychological research "in the wild". You might consider a briefer form of Longitudinal Study where you give your users an app to test and either ask them to use it naturally or ask them to use it every so often and collect data as time goes on.

There's some good ways a Longitudinal study can gather information without being over the user's shoulder 24/7:

  • Intermittent surveys or diary-style reports from participants
  • Automatic data collection tools (used with permission)
  • One on one or focus group interviews before/after/during the study

Ideally you'd gather feedback from a wide variety of users who all naturally use the app, but there's already going to be plenty of selection bias, so it might be worth asking participants to just use the app every X days or for an hour every so often since you're probably dealing with a limited pool of participants in a limited time frame; if you're not conducting peer reviewed research that is intended to be rigorous and reproducible, it's more important that you get some data out of people than that you run a perfectly valid study. You want to be as valid as you can, but optimizing for validity and reliability is very difficult to begin with in these tasks. Keep the data as real as possible, but be practical.

Instead of trying to record every second of the user's life with the product, collect intermittent surveys of their interaction with it. It's good to have a mix of some numeric measures like Likert scales along with some direct, relevant questions and a free-response section. This allows you to measure long-term trends so you can see the apparent difficulty/quality of experience curve of the app while also gathering free response insights which don't fit neatly into your stock questions.

It's also ideal that your app itself collect some tracking data of the users; gain consent and fully disclose what data the app will collect and when/how you'll get the data from the app. Ideally you could monitor usage of related apps, but this is rarely practical. It's more likely you'll be able to produce a special test user version of your product which does all this tracking. The survey is there to help fill the gaps.

Finally a one-on-one session or focus group with the users before debriefing can help you gather as much insights as possible (don't say you had anything to do with designing the product!). Not everyone fills out free response sections (or gives useful information in them), talking to them personally and casually can help glean as many insights as you can.

Obviously much more rigorous than your standard "bring 40 people in and have them click through a few screens" usability test, but this doesn't have to be as hard as it initially sounds. Collecting analytic data is pretty common in apps today already, and drafting up some surveys won't kill you. Remember your longitudinal study doesn't have to last years, pick a reasonable length considering the learning curve you want to study (new user to proficient? New user to expert?) and project time constraints.

2

These are called living lab methods, imagine them like a scientific version of Big Brother.

Google has thrown me this Living Lab Method as first result, but I've read about them in the Interaction Design book.

On a sidenote, I wouldn't be surprised if real Big Brother or other live-together reality shows would be used for this (as part of a silent sponsorship), I mean, the cameras are there, it's not quite a natural environment, but a living lab always has deficiencies.

Or you can install it at all the phones and workstations at a company, but I'd be terrified a bit on this. Living lab is easy. You sign a paper, you get benefits, and you're watched, that's clean. I wouldn't participate in it but there are people who would do.

Beware of the privacy issues: always be clear about what you watch and for what purpose to all those you're watching. Perhaps release your findings towards them at the end of the experiment. People are sensitive about their private life.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.