User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It only takes a minute to sign up.
Sign up to join this community
I am looking into some software development metrics. Does anyone have a clear tried and tested strategy to measure the Wow-factor of a web-application?
A set of properties belonging to an object that pleasantly surprise a watcher. From commercials to cool electronics, the wow factor is an important thing to consider when designing it.
Well, co-incidentally, I was thinking on the same lines couple years ago. I happened to design a sort of metric that pointed to the line of thoughts at that point of time - it being of qualitatively defining a user interface. Here is the complete blog post.
Few excerpts from that blog post -
Lets call it FGF Score.
1) F for Focus – Very important, and can be easily judged by seeing the site/interface. If the site is supposed to sell oranges, it should first focus on getting that done and then try baby diapers. Weight of Focus being – 4 out of 10.
2) G for Good Arrangement – This leads to clarity. If there is lot of clutter then a user gets disturbed and probably does not ‘feel nice’. Weight being – 4 out of 10.
3) F for Fonts – This I suppose is very crucial, although might not place in many UX books/texts. Font selection, I believe can make or break your design. Hence this needs good importance, and is one of the first things you see on a website. Weight being 2 out of 10.
So, FGF Score(10 points) made out of –Focus(4 points) + Good Arrangement(4 points) + Fonts(2 points)
You could call it a little naive, but at that time it was an attempt to structure some thoughts around this bubbling question. Hope it serves as some food for thought.
I assume that you are referring to the "Delighters" in the Kano Model.
There are a couple of articles that go into a lot more details, such as the Folding Burritos website that you can read about to get more details.
You can also refer to previous questions on the subject:
Is the Kano model adaptable for measuring user experience?
I can understand the thinking behind this. Looking for a way to qualify something as visually impressive is important to a lot of design professionals and teams. I would caution, however, to not let that cloud your judgement here. How visually impressive your site is makes no difference if its usability is trash. Smashing magazine has a great article from a few years back highlighting some sites that were visually stunning while being completely unusable.
If form truly follows function on your site, not only should it be visually impressive, but those visual cues should lead the user to efficient and pleasant experience. This means you can rely on traditional testing methods to get an idea of how your new "wow factor" site performs compared to its predecessor. Unlike the subjective wow factors, these type of metrics are objective, both in user feedback and hard analytics.
Scripted, in-person user testing can be helpful here. Do users find your site visually appealing? Can they easily perform given tasks from a scripted user test? What are the pain points in user workflows, and does the new visually impressive design help to address these pieces? What has the UI and UX made intuitive?
Additionally, you can look at some analytics to see what your site's visual changes have impacted. What's the change in average time spent on the site, or a given page, between designs? Is there a positive change in bounce rate? What about changes to the number of targeted interactions on your call to action?
Heat mapping might be helpful here as well. Do your new graphics and design draw user's eyes to the most important parts of the page, or are they a distraction from the actual content and navigation?
This all assumes, of course, that you have a previous version of a site to test against. If this is your first iteration, putting your analytics and scripted user testing up against a competitor may the most useful data you can gather.
The bottom line is that the usability of your site is not only a reflection of good UI/UX patterns, but also that "wow factor" that engages users. By gathering and interpreting traditional metrics, you should be able to get a good feel for whether or not your "wow factor" is a positive improvement to your site (the question you really should be asking in the first place).
To accomplish your goal you'll need an operational definition of the wow-factor. In simple words, something that can be measured and acts as a good proxy for this, rather abstract, concept.
One such definition could be the visual appeal or immediate aesthetic perception of tested websites. If you're happy with this definition then there's a clear path how to measure it. In a nutshell you expose participants to different websites for a very short time (500ms, although even 50ms is sufficient!). And after each exposure you ask a participant to rate the visual appeal of the website. You can use a 7-point scale ranging from Very Unattractive to Very Attractive. You can find more details in this paper: Attention web designers: You have 50 milliseconds to make a good first impression!
To determine if the observed differences between tested websites are statistically significant you can use the Wilcoxon Signed-Rank test (since the subjective scores are non-parametric). But just looking at the results should given you an idea if there could potentially be an interesting difference.
