3

I'm about to start testing the memorability of a user interface the Nielsen's way, where UI memorability is the amount of time/number-of-tries needed for the user to reach the efficiency during a repeated exposure to the stimulus.

The question is what time would be appropriate between the first exposure and the repeated exposure to have valid and meaningful results? 1 hour? 2 hours? 8 hours? A day?

It would also be awesome if you have any references on the procedure.

Thank you in advance ;)

PS: If the question is not clear enough please tell, I'll try to rephrase.

4
  • The question is clear enough, but it would help if you provide more context - what is the application and how complex it is? Also, why did you opt to measure memorability in the first place?
    – Izhaki
    Oct 1 '15 at 20:23
  • 1
    And are you sure you mean efficiency, or did you mean proficiency? Because memorability/proficiency are paired, efficiency is a criteria of its on, per Nielsen.
    – Izhaki
    Oct 1 '15 at 20:38
  • @Izhaki I'm testing all 5 factors and the memorability is the only one I'm struggling with. I mean efficiency as one of the 5 factors defined by Nielsen in his "Usability Engineering".
    – LoomyBear
    Oct 2 '15 at 1:40
  • @Izhaki the context is a simple UI - a set of buttons and images. Far from real life implementation. One of the useless UIs that academic people use for their low core research purposes.
    – LoomyBear
    Oct 2 '15 at 1:43
3

That depends on your application. For example, I use gmail hourly (if not more). I use my bank's application weekly. Each case has different retention needs.

So, if you expect users to use your application every day, test daily. Etc...

6
  • Thank for your reply. Makes sense! Problem is the UI is a very specific build for the research purposes only and it is not supposed to be used outside the lab. In other words it doesn't have any usage frequency that I can refer to.
    – LoomyBear
    Oct 1 '15 at 7:55
  • Then it seems like any sensible duration would be acceptable. Are there any other research done using this process? What duration do that use? By sticking to the same duration, you at least have a way of comparing efficiency with another study.
    – nightning
    Oct 1 '15 at 20:47
  • @nightning define "sensible"? ))
    – LoomyBear
    Oct 2 '15 at 1:37
  • @LoomyBear A certain period of time has passed such that the user cannot rote memorize the steps involved in performing a task. A day is very safe. If you would like a test be performed as a single session. Then consider treating it like a standard memory study. Have the tester perform a cognitively intensive distractor task between the test sessions to clear their working memory.
    – nightning
    Oct 2 '15 at 17:20
  • @nightning thanks for your reply! The next day is the first thing I thought of but this way I need to ask participants to commit to 24 hours test and I'm sure some of them just won't show up so I was hoping maybe there's a confirmed shorter time scales. Cognitively intense task? that's really interesting! do you have any references to studies where they make these distraction tasks? and how much cognitively intense is enough? I guess it varies from case to case. Thanks.
    – LoomyBear
    Oct 3 '15 at 0:23
2

Confound variables

Since you are trying to run a controlled experiment, any time between two exposures that is not within your control are likely to introduce many confound variables that could render your results invalid.

For instance, what if the participant had little and terrible sleep over night. What if during the break they attempt using a similar product, or make conscious effort remember something?

I have done user testing in the past where during task 4 users could not repeat a procedure they have manage to carry in task 2 (about 5 minute gap). These sort of failures suggest some fundamental usability issues - but I guess that's not what you're after.

Anyhow, you are likely to gather more valid results if you reduce confound variables, which calls for a single session.

The two types of evaluation

It seems odd to me to carry out a single instance memorability evaluation.

UX evaluation is of two kinds:

  • Formative - done during the design process in order to highlight issues and see how different designs affect various measures.
  • Summative - done after the design has been delivered to ensure it meets a pre-defined fit criteria.

So the former assumes a repeated experiment after design changes; the latter requires a defined fit criteria. Your method seems to belong to neither.

2
  • I agree and that's why more participants you have during the study more valid your results get, so you don't depend your results on the sleepy guy ;) And yes this is not what I'm after. I have 2 stimuli with only one variable that differs and I need to define if all 5 of the usability factors are impacted by the variable. So I'm not testing only the memorability. The memorability is the only one that I'm struggling with right now. Also 5 minute gap is not my case and of that I'm pretty certain.
    – LoomyBear
    Oct 2 '15 at 1:37
  • Basically I have 2 exactly the same stimuli with the only one variable that varies. It's not an industrial but academic study, the design won't be revised after the exposures, thus makes no relevance in my case.
    – LoomyBear
    Oct 2 '15 at 1:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.