Accuracy of a text field VS ease of buttons

Question

Our users complete coding challenges. Each of these challenges has an estimated duration. For example, "Build a CSS Robot" is estimated to take 60 minutes. All of our challenges are designed to be completed in less than 120 minutes (roughly one sitting).

Our users have reported that some of our challenges are significantly shorter or longer than their time estimation. We'd like to collect data on how long it takes for users to complete each challenge, but are unable to reliably do this by programmatic means such as measuring session duration or time on a page. So we must ask our users.

When users mark a challenge complete, we congratulate them before moving on to the next challenge. We'd like to use this opportunity to optionally ask them how long the challenge took.

We could put a text field and have them simply enter the number of minutes it took, which would be more precise. Or we could put a button array that looks something like this:

Did this take you 60 minutes to complete?

[ less than 60m | about 60m | longer than 60m ]

This would be faster and easier for users to submit, and we could gradually increase/decrease the estimated time as we receive aggregate feedback.

My question is, how much more likely are people to give us feedback if we use the button interface rather than using the open-ended number submission? If such research exists, this will help us skip the A/B test process and move more quickly toward our goal of making the time estimations of these challenges more accurate.

Answer 1

Users aren't likely to fill out optional fields.

Most people's brains aren't wired to keep track of time while they're actively engaged in an activity. To have them quantify how long they spent on the project by entering a number will cause most users to short-circuit and skip that field entirely.

If you absolutely don't have any practical means for tracking their time for them, then do the buttons.

And if you can get rid of that 60 number altogether, and just say "an hour", there's even less cognitive overhead for your users. We know pretty intuitively what an hour feels like, but 60 minutes asks us for precision, something you won't get from the button feedback anyway.

Answer 2

I would use a button interface that will let you know how long I took. Its easy to know if it was more than 60 minutes, specially if it does not requires how many minutes after the 60 mark.

Expanding in the subject, here is my input:

Although 60 minutes time frame is a great push and pressure to complete the challenge, I think that time frame is not for first timers due to the following reasons:

I can imagine that most of the practitioners or designers in this field will have a visual learning method, but most likely there are several learning ways among all the participants. There are many that like to write things down and more so when there is no indication of how long the participant will have access to the lessons for future references. (In that way won't have to take notes. Not so for the ones that learn writing things down). Besides, if one would like to keep what is seeing needs to capture it and initiate an archive system to review and study or use it eventually for further projects during the program. Another delay completing the stages may occur according to the amount of knowledge about the subject, adding to that the interest in wanting to explore creating a not so typical output, taking advantage of the tools provided. Never mind daily schedules...

Academically speaking, the practice to learn technique will need a few other tools handy, as well as indications, so the participant will feel safe and secure to go on lesson by lesson not taking notes or worrying about where to find the details for reference, (some generic way that would apply), to finish the last stage of the challenge. Even if one can go back to look for details -since for a first timer would be hard to remember a code, not sure all others can either- the details are not made to be extracted/copied and place at the text editor. Then, for first timers taking notes or saving code details somehow (organized) is a most for future reference. Which it most likely delays the time frame for completion.

I am sure creators have taken this in consideration, but I answered in case the 60 minutes indication for completion maybe can be looked at some other way, or provide other tools to make it be completed in 60 minutes.

I have no objection with 60 minutes or 1 hr indication. When is 60 minutes it reminds me when script/screen writing for film that each page is equivalent to a minute and it made me think of a lesson per minute, which it would be reasonable if one can be sure that what learned it will be there to return to it -organized- for reference.

There are some skills references provided in case of need, but are not in the same way they've being applied which is what one wants. One can find the exact way, most likely, in w3schools that apply more to the need in time.

Nevertheless, 60 minutes or not, I finish each challenge the best way I can in order to retain the information. For me is more important to retain information regardless of all the references available, than pass a challenge and then later on encounter a tiny little thing that detains me and I can't figure it out cause I would not know how to debug it.