Multiple choice question editor

Question

I'm building an editor for multiple choice questions. Question can either allow a single chosen answer by the user (radio buttons) or multiple answers (checkbox).

Each choice has a percentage value that describes its "correctness", for example, in a radio button question, you get all the points if you select a choice that has a value of correctness equal to 100%.

The editor looks like this:

[radio button question]

[checkbox question]

What I don't like about this editor is the fact that the user has to manually input percentage values, which comes with a high interaction cost: in most cases, the user only wants to be able to mark the correct answers as such, so having a checkbox with a label "Correct" next to the answer's text editor would be much better.

At the same time, some users do want to have control over "partially correct answers" (for example giving 50% of the score rather than either 100 or 0) or even give negative score.

While, for radio button questions, it's easier to reason about correctness of a choice (a choice is correct if it has a score value of 100%, partially correct if it's anywhere in between 100 and 0, and incorrect if it's 0 or lower), with checkbox questions, there is an additional difficulty: you get the maximum score if you select all and only of the "correct" choices, which means that the correct choices' correctness values must add up to 100%.

So, for radio buttons, you might have a checkbox that allows to quickly select the correct choices and automatically set their value to 100%. But, for checkbox buttons, setting a choice as "correct" also changes the value of the other correct ones, in order to keep their sum equal to 100.

For example, if I marked a choice as the correct one for the first time, it'd have a correctness of 100. But marking a second choice as correct would give them both 50% correctness.

What would be a good way to simplify the interaction with the editor and allow: (1) for most, simple use cases, to directly mark the correct answers as such and, (2) for users that want full control over the correctness values, to still allow for such advanced usage?

Answer 1

Basically, there are two main concepts to consider here:

1. whether incorrect answers affect the outcome of multiple choice.
2. whether the user should control the system or not.

In the first case, this is a very important concept. We were dealing with this with my wife and partner just a month ago, and it involves the following: Suppose there are two selected answers, one of which is incorrect and the other correct. Should we score the correct answer (even if it's random), should we penalize the incorrect answer (which again may also be random), should we consider the absence of the correct answer for a non-selected value?

We've not agreed on a final answer, but we agree on one thing: the punishment cannot go below 0. So there can be no negative values (as in your picture) and wrong answers can't go below 0 either. Suppose there are two correct answers, but the student chose a wrong one: logically, he got 0 points. But if you punish each wrong answer, he'll get -1 or -2 points or whatever.

With this in mind, let's say there are 10 questions, and the student did what I mentioned above. If each question is worth 1 point, instead of getting 9 points, he'll get 8 or even 7 points, which is clearly wrong because he'd 90% of the answers correct. (NB: please note our grading system is 0-10 or 0-100, but you may adapt it to your needs)

And this leads us to the second point: should the user control the system?

Based on common UX heuristics such as that of Shneidermman, the answer is yes. But if the teacher has to think about different possibilities for the questions AND, has to consider grading systems (which in many cases aren't part of her area of expertise), I'd make her life much easier.

To summarize

If it were up to me, I wouldn't let the teacher control the grading system, I'd provide a simple interface and leave everything up to the system.

But there's an alternative way: you can set up an option on the settings level where the user can choose whether to manage the grading system or not (with explanation of the pros and cons)