11

My company has recently started an initiative for objective quarterly evaluation of employee performance based on the Rockefeller Habit's "One Page Strategy Plan". One of the key sections within the plan are measure-able targets for teams & individuals.

For example:

  • Sales Person: a target of converting 100 leads into new customers in a quarter.
  • Support Staff: a target of 85% satisfaction rating on customer tickets

Our Product team is struggling to find concrete metrics for evaluating performance.

Are there any recommended metrics for evaluating competencies of a UX-focused Product team and for UX Designers?

A target like shipping X number of features a quarter does not focus on whether we're providing a sound solution to the customer problem, or whether we're even tackling an important customer problem. A customer satisfaction rating for a new feature can be subjected to response bias. A measurement of minimum X concept/user tests for new work, means we always have user feedback. However, we're missing the other side of how effectively we're utilizing the feedback.

We've also considered things like NPS and customer churn reduction. However, so many factors influence these items that it's not fair for the team to be basing performance of them.

I'm curious as to what experiences people have when it comes to performance reviews. Or is it truly a pipe dream for UX Designers to know whether we're doing a good enough job.

  • 7
    Performance Reviews are the best way to scare away talent. – DA01 Sep 23 '15 at 20:14
8

Great question although I pretty much go along with Khoi Vinh's thoughts that performance reviews for designers can be a ghastly affair and of little real use.

As he points out:

The New York Times, to its credit, was a place where designers received reviews from people who actually understood our work, who were conversant in the vocabulary of design and who comprehended the value we tried to bring to the business. That may not be uncommon, but it’s also certainly not the rule for the profession, especially for those who work in-house at companies whose business is not principally design.

I don't think any single measurement is going to give you want you want, but there might be a selection of quantitative and qualitative KPIs and metrics that are relevant to your organisation that could be 'factored' in some way to give a usable result.

CXPartners has a Big List of KPIs and metrics. It's a seriously good resource! Perhaps it will provide some inspiration.

But there's no real substitute for real reviews from real people who know what you do - back to Khoi Vinh.

Getting a review from someone who has an inaccurate understanding of what you do is a waste of everyone’s time. Unfortunately, yearly salary increases are often tied into these reviews, further inflating the value of bad data.

And with that in mind, perhaps the use of 360 degree feedback could be utilised (in conjunction with traditional appraisals) somehow as a larger component of an accumulated metric.


Knowledge disclaimer: I have no direct personal experience of any of this. I merely wanted to provide food for thought. I'd be surprised if someone came up with an answer recommending a system that works, but would love to see it if there was one!

| improve this answer | |
  • Lots of great insight on Khoi Vinh's blog post and the comments there. Continuous review and check-ins seems like the way to go. Being the sole UX Designer on my team is a bit of an issue though when it comes to getting meaningful feedback. Yes, I suspect there's no magic bullet, but it doesn't hurt to ask. – nightning Sep 23 '15 at 20:19
3

As much as I hate trying to "quantify" success in areas of programming or UX, there are some possible key performance indicators that have been suggested:

From Designmodo:

Quantitative UX KPIs

Task success rate
Time on task
Use of search vs. navigation
User error rate
System Usability Scale (SUS)

Qualitative UX KPIs

Reported expectations and performance
Overall satisfaction

From uxmatters: ux metrics table

| improve this answer | |
  • I suppose we can try tying usability of new work as a part of the evaluation processes. The Designmodo article is a great reference for that. A part of me still feels it doesn't encapsulate the true vision of the UX Designer where we're in the forefront of tackling new problem as oppose to tinkers who only tweak existing workflows. – nightning Sep 23 '15 at 20:03
1

The problem with nearly all performance review systems is they mainly track one's ability to measure up to the performance review system rather than measuring actual work.

The television series The Wire summed this up succinctly with the phrase juking the stats.

The problem with measuring objective output of the UX team is that the output of the UX team is rarely the actual product output. In larger companies, the product (often software) will have a UX that is only partially controlled by the UX team. You have management, IT, development, consultants, outsourcing firms, etc all getting their hands on it and a lot of the final UX will come down to all those parties--not just the UX team.

| improve this answer | |
  • I'm curious, are there ways to measure the actual work UX does? We have the opportunity right now to define processes. – nightning Sep 23 '15 at 22:01
  • @nightning I'd have to ponder that, but I'd say 'no' as my gut reaction. I suppose you could do A/B testing...test current state, and then test it against UX Wireframes, but that's apples and oranges. – DA01 Sep 23 '15 at 22:05
  • Yeah, that makes sense. We do concept tests right now with users. It really only captures a small portion of what we do. There's also learning curve involved in changing a complex workflow that aren't easy to test. – nightning Sep 23 '15 at 22:13
0

I would say you would use AB testing to determine the outcome of the UX work being done.

From my point of view, the only way to measure UX changes is to measure how the new UX performs against the old UX, and can only be done objectively with AB testing.

I would measure Funnel completeness and conversion. At the end, the main objective of UX is to increase conversion.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.