-1

I am currently calculating using Excel, some collected quantitative data I have from participants. I have so far arrived at this % score:

enter image description here

Note: I know that, the metrics for Task Completion Time is calculated as follows: 

(max_score - participant_score)/(max_score - minimum_score)

My main question: Is there some equation/formula similar for "participants number of clicks" on specific page?

Improve this question
4
  • 4
    What are you trying to do with this data? Do you have additional information besides just number of clicks? What is the context in which you're doing this study? Do you have a goal or feature you're testing? Can you compare that with any previous metrics to notice any changes? It's hard to offer advice on what to do next because there's very little context on what's going on provided in this question. (Also, I'm not sure how you're calculating the % score, but Participant 1's score seems like it should be ~33%, no?)
    – maxathousand
    May 7, 2020 at 14:01
  • 1
    @maxathousand, hi, thanks for your reply. Within the context of A/B testing, variant A & variant B of a site. The website was updated. Now, users see two different versions. I have, click metrics, SUS metrics, and a standalone question per site (Likert Scale). I want to get a score for the clicks, and compare between A&B. Thx!
    – John Smith
    May 7, 2020 at 15:13
  • 1
    @maxathousand, "Participant 1's score seems like it should be ~33%, no?", yes you are correct, my bad.
    – John Smith
    May 7, 2020 at 15:14
  • I would consider deleting the question completely. We should not waste people's time. At least it needs to be edited so the data in the question are correct (there are mistakes in the table) and the solution is crystal clear because right now it isn't.
    – Mike Mark
    May 12, 2020 at 8:54

1 Answer 1

Reset to default
3
+25

Assuming that the data that you are collecting for the number of clicks is meaningful and directly related to a small change you are measuring (which is probably what you should be using the A/B test for), I imagine that you would want to set up a comparison to show that there is a significant difference (i.e. significance testing) in the number of clicks between page A and page B.

That means I would expect to see two sets of data set out as follows:

  • Column A - participant ID
  • Column B - clicks page A
  • Column C - clicks page B

Or if you used two different participant groups:

Table one

  • Column A - participant ID
  • Column B - clicks page A

Table two

  • Column A - participant ID
  • Column B - clicks page B

Depending on if you have the same participants or not clicking on both pages A and B (you would have two different populations in that case, and a slightly different statistical analysis), you would then work out the average number of clicks for page A and B, then compared to see if there is a statistically significant difference between the average number of clicks. That is, you would end up with the average number of clicks for the participants for page A (e.g. 4.15) and compare it with the average number of clicks for the participants for page B (e.g. 5.23) to see if there is a significant difference between those two values.

The details are a bit too long to go into it here, but that would be the measure that I would report on. Here is a useful diagram from Investopedia (Image by Julie Bang © Investopedia 2019):

enter image description here

Where you had the same participants click on the two different pages, it would involve doing a paired t-test for significance between the page click values recorded.

If instead these were two different groups of participants, one looking at page A and the other looking at page B, then you would be doing an independent sample t-test (either assuming or not assuming equal variance).

Improve this answer
4
  • 1
    So, how would I compute an average, shouldn't I do like in a task completion calculation. For example, "how many clicks the participant had to do to reach a page", and in this context we have minimum required clicks is 4 clicks.
    – John Smith
    May 10, 2020 at 8:56
  • 1
    Thx. Could you update your answer, and show me how to accurately calculate score % of clicks or something per participant and then the average of all? I've been Googling for days now. Seems this fall into Performance metrics.
    – John Smith
    May 10, 2020 at 22:38
  • 1
    @JohnSmith The way to calculate the significance of the two different page designs would fit more into a statistics answer, but if you want to understand the basic concepts behind the test this is a good starting point (and there are also online calculators that will provide you with an output based on your input data): statisticshowto.com/probability-and-statistics/t-test
    – Michael Lai ♦
    May 11, 2020 at 11:23
  • @JohnSmith you'll have to provide some details about the actual task, otherwise it is difficult to tell what would be a sensible measure/score simply based on the number of clicks as I mentioned before.
    – Michael Lai ♦
    May 11, 2020 at 11:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.