Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

When conducting usability testing, a variable that comes up is the competence or familiarity of different users with Internet or software application technology. It seems strange to compare different usability studies and results if there are no indication of the type of users that are being tested. Is there a standard test used to establish a baseline for user competency or proficiency with general computer technology? I imagine this would be useful for older users that may not have had as much exposure to computers and working out how much different there might be in terms of looking at task completion rate and task completion time.

I am interested in whether this is a standard item provided in a usability report or carried out as part of usability testing screening, and if so then whether there is a standard that is commonly used or if it is generally product/project specific.

Update: Looking at the 2007 Microsoft Office Fluent UI Study Information Workers by Forrester, I noticed that they included a definition of General versus Advanced users by asking two questions about Microsoft Office Product usage and also showing them examples of advanced features. Only people who fit in the criteria for advanced users are classified in that category. Could something similar be developed and standardized for usability testing in general?

share|improve this question
1  
Related/possible duplicate: How to best ask for computer experience in a survey? –  Ben Brocka Apr 26 '13 at 15:09
    
It's difficult to establish during RECRUITING for a test what skill levels users have. Once of course you have your users it becomes rather obvious what skill levels they have... –  PhillipW Apr 27 '13 at 11:47
    
Usually instead of asking about proficiency I'll ask about number of hours spent on the web, using MS Office, etc. That seems a better indicator of expertise than tester-estimates proficiency, which is very qualitative and easy to over/under estimate (as others have pointed out). –  Ken Mohnkern Jul 24 at 19:46

2 Answers 2

There isn't any, because "computer proficiency" is a vague term. Is a programmer more computer proficient than a secretary who can lay out a colorful document in Word? Something as vague as computer proficiency cannot be measured and therefore cannot be part of a useful research question.

So, you need to make your proficiency test specific to the thing you're actually testing for. Does being a programmer make you a better web surfer? It's the same outside this specific field: e.g. is a carpenter better at painting than a musician?

If you need to establish a user's basic skill level, you can use a questionnaire like in How to best ask for computer experience in a survey?. However, it would probably best to test the actual skill level in the exact area you're interested in. Questionnaires have the risk of users over- or understating their actual skill level.

share|improve this answer
    
+1 for beeing a myth buster –  Gildas Frémont Apr 27 '13 at 10:05
    
Well said. For any given task, let alone computer literacy in general, users are "perpetual intermediates'. Chapter 3 in Alan Coopers book 'about face 3' covers this well and ties into the user research work I've done. It's next to impossible to hire someone based upon computer literacy as a so called expert simply will have no experience of many computer related tasks so be equally clueless as a 'novice'. –  Stewart Dean Apr 27 '13 at 11:36

Before the study ever begins, participants are recruited by asking them a series of survey-like questions (the "screener") which is designed to include the types of people you want to test and exclude the people you don't.

If the goal is to test a diverse audience, the screener will include questions that would flag people for various traits... for example, people who are unusually technical, people who have an average level of technical knowledge, and people who are fairly inexperienced with technology.

The report may or may not identify different user groups, depending on a variety of factors. For example, if 100% of participants didn't see the "sign up" button, identifying their criteria may not be helpful. However, if 0% of participants who were inexperienced or average in terms of technical skill saw the "sign up" button, and 100% of participants who were very experienced saw the button, that would definitely be mentioned in a usability report. (Exaggerated for the sake of simplicity.)

Here's a good overview of a typical screener process: http://www.uie.com/articles/usability_testing_three_steps/

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.