3

I am looking into some software development metrics. Does anyone have a clear tried and tested strategy to measure the Wow-factor of a web-application?

A set of properties belonging to an object that pleasantly surprise a watcher. From commercials to cool electronics, the wow factor is an important thing to consider when designing it.

5
  • 1
    What, as in "This financial calendar software is officially 63% wow"?
    – JonW
    May 9 '16 at 13:32
  • Yes, but maybe more something like "63% of the audience grouped was wowed, when showed the new financial calendar". Maybe a bit more context. We are coming up with different metrics. I am now trying to figure out if it is at all possible to measure them. This one seems very hard to me. I hoped someone has an idea. May 9 '16 at 13:42
  • 1
    Sounds like you are trying to define some qualitative metrics which will be grouped by whatever cohorts you use.
    – SteveD
    May 9 '16 at 14:13
  • 1
    Let me guess... This is probably a case of Marketing wanting you to take a qualitative aspect and quantify it. I can't imagine it will quantify in any accurate or meaningful way. May 9 '16 at 18:07
  • No, its not marketing. ;-) Its us the development team brainstorming about possible quality metrics for our software. Now I am figuring out if our idea's make sense and what goal the metric could be used for. Then we pick a couple. This one sounded fun, but not so easy to quantify. May 10 '16 at 7:30
3

Well, co-incidentally, I was thinking on the same lines couple years ago. I happened to design a sort of metric that pointed to the line of thoughts at that point of time - it being of qualitatively defining a user interface. Here is the complete blog post.

Few excerpts from that blog post -


enter image description here

Lets call it FGF Score.

1) F for Focus – Very important, and can be easily judged by seeing the site/interface. If the site is supposed to sell oranges, it should first focus on getting that done and then try baby diapers. Weight of Focus being – 4 out of 10.

2) G for Good Arrangement – This leads to clarity. If there is lot of clutter then a user gets disturbed and probably does not ‘feel nice’. Weight being – 4 out of 10.

3) F for Fonts – This I suppose is very crucial, although might not place in many UX books/texts. Font selection, I believe can make or break your design. Hence this needs good importance, and is one of the first things you see on a website. Weight being 2 out of 10.

So, FGF Score(10 points) made out of –Focus(4 points) + Good Arrangement(4 points) + Fonts(2 points)


You could call it a little naive, but at that time it was an attempt to structure some thoughts around this bubbling question. Hope it serves as some food for thought.

2
  • +1 for coming up with something novel and interesting. Any plans on trying to expand and elaborate on this further?
    – Michael Lai
    May 10 '16 at 1:50
  • Thanks @MichaelLai. Well, now that I see even others reciprocating the same thoughts - I am motivated to pursue this further. With some more experience in the kitty and such a great community of fellow designers, I am sure something usable can be architected.
    – Amit Jain
    May 10 '16 at 5:43
3

I assume that you are referring to the "Delighters" in the Kano Model.

There are a couple of articles that go into a lot more details, such as the Folding Burritos website that you can read about to get more details.

You can also refer to previous questions on the subject:

Is the Kano model adaptable for measuring user experience?

How do I get KPIs within an enterprise environment?

Does anyone have statistics on usage of the Kano Model?

1

I can understand the thinking behind this. Looking for a way to qualify something as visually impressive is important to a lot of design professionals and teams. I would caution, however, to not let that cloud your judgement here. How visually impressive your site is makes no difference if its usability is trash. Smashing magazine has a great article from a few years back highlighting some sites that were visually stunning while being completely unusable.

If form truly follows function on your site, not only should it be visually impressive, but those visual cues should lead the user to efficient and pleasant experience. This means you can rely on traditional testing methods to get an idea of how your new "wow factor" site performs compared to its predecessor. Unlike the subjective wow factors, these type of metrics are objective, both in user feedback and hard analytics.

Scripted, in-person user testing can be helpful here. Do users find your site visually appealing? Can they easily perform given tasks from a scripted user test? What are the pain points in user workflows, and does the new visually impressive design help to address these pieces? What has the UI and UX made intuitive?

Additionally, you can look at some analytics to see what your site's visual changes have impacted. What's the change in average time spent on the site, or a given page, between designs? Is there a positive change in bounce rate? What about changes to the number of targeted interactions on your call to action?

Heat mapping might be helpful here as well. Do your new graphics and design draw user's eyes to the most important parts of the page, or are they a distraction from the actual content and navigation?

This all assumes, of course, that you have a previous version of a site to test against. If this is your first iteration, putting your analytics and scripted user testing up against a competitor may the most useful data you can gather.

The bottom line is that the usability of your site is not only a reflection of good UI/UX patterns, but also that "wow factor" that engages users. By gathering and interpreting traditional metrics, you should be able to get a good feel for whether or not your "wow factor" is a positive improvement to your site (the question you really should be asking in the first place).

1

To accomplish your goal you'll need an operational definition of the wow-factor. In simple words, something that can be measured and acts as a good proxy for this, rather abstract, concept.

One such definition could be the visual appeal or immediate aesthetic perception of tested websites. If you're happy with this definition then there's a clear path how to measure it. In a nutshell you expose participants to different websites for a very short time (500ms, although even 50ms is sufficient!). And after each exposure you ask a participant to rate the visual appeal of the website. You can use a 7-point scale ranging from Very Unattractive to Very Attractive. You can find more details in this paper: Attention web designers: You have 50 milliseconds to make a good first impression!

To determine if the observed differences between tested websites are statistically significant you can use the Wilcoxon Signed-Rank test (since the subjective scores are non-parametric). But just looking at the results should given you an idea if there could potentially be an interesting difference.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.