Quantifying Simplified Language

Question

If you wanted to be able to quantify how much easier it is for users when you reduce the reading level of you text, how might you do that?

For example, how would you figure out how much longer it would take someone who reads at a grade level of 8 to read a very similar text written at a grade level of 10, maybe 20% longer? I know that's assuming they understand the text written at a higher level (which is not a good assumption), but if they did understand it, we'd like to know how much time would it save them not to have to figure out the harder text.

Answer 1

we'd like to know how much time would it save them not to have to figure out the harder text

If time is your key metric just report on that dimension (time) somehow. I've never done it myself with reading comprehension tests so I can't be more specific.

A = Copy written in 8th grade reading level B = Same copy, but written for 10th grade reading level

You could either do a Cloze test where they fill in the blanks or do a memory test and give them a pop-quiz based on the content you just showed them. If your testing tool captures time to completion, then you have a number to compare as long as you capture enough data from apples-to-apples test participants.

As I type this out it sounds pretty involved, though. Not sure if there's an upside that makes it all worth the effort of testing. According to NNG, readability is usually scored via computer.

If you're trying to quantify the time cost difference between such subtle grades of reading level, hopefully there's a lot of $ at stake to justify the investigation. Good luck,

Answer 2

There are a number of different readability tests, depending on what you think is the best measure for your purposes. Most of them are structured around length of sentences, density of text and the use/choice of vocabulary. You probably heard of the Flesch Kincaid readability test used in Microsoft Word if you have used spell check in the old versions.

There are two aspects of readability to consider, the first being the amount of time it takes to read through the material, and the second aspect being the comprehension of that material (which may or may not be related to the amount of time taken to read).

You can measure the first by defining a start and end point for the reading task (e.g. by asking them to do something after they finish reading) and compare the difference in time between the two blocks of text.

And you can measure the second by creating a task that can be completed successfully if they have understood all the materials that they have been asked to read. Once again you would compare between the two blocks of text but not with the same person.

Once you have collected enough data from enough users to make you feel confident enough about the significance of the results, you can summarize and put a figure on the amount of improvement on time and task completion rate to show the impact of the change suggested.