4

We've been asked to justify a potential internal app by projecting its benefits quantitatively.

When I research measuring user experience, there's usually the opportunity to utilize conversion rates, engagement, etc. However, in building an internal app, it isn't so easy, as this process will happen whether it is easy or not.

Below you will see what metrics we're already projecting, but I want to do better than that. To put it crassly, we're going to be making an internal process "suck a lot less". How can we measure the benefit of that "intangible"? This app will be mainly targeted at the highest levels of leadership, so we really doubt that employee retention will be impacted as these people have too much invested for an annoying process to make them quit. But it will make that process much, much easier.

Description of internal app

Our highest level of leaders have a LOT of approvals to do. Current estimates put that number at over 600,000 a year. On top of that, the approvals happen in as many as 5 distinct systems, and cover topics such as client invoice approvals, AP approvals, pricing model approvals, and various compliance approvals.

What makes the problem REALLY frustrating is that the 5 distinct systems can only be accessed via a laptop, so you always have to be somewhere with WiFi and space to open up your computer.

We will be building a mobile app that consolidates the 5 streams of approvals into one place. When completed, you'll be able to approve while walking between meetings, in a cab, on an underground subway, or just before you go to bed.

Here's what we're already doing:

  • Measuring projected time savings per approval (2 min with a laptop minus 30 seconds with the phone = 90 seconds of savings. 90 seconds across 600k approvals = 15,000 hrs saved * $250 rate for leaders = $3.75M.
  • Attempting to measure the value of getting approvals done quicker by each type of approval (e.g. Client Invoices. 34% take >0 days to approve. speeding up that process means we can collect faster).

Here's what we've ruled out:

  • Measuring impact on employee retention. We don't think this is annoying enough that director-level and higher would quit over it.

  • We have very little access to detailed metrics on current state activities, as the 5 disparate systems don't all collect or have access to the same data.

  • We have only limited opportunity to interview approvers across all types as it takes a long time to find the right people with the right information.

So...

Given the above context, what ideas do you have on how can we measure the benefit of undertaking an effort that improves the user experience on a very frustrating internal process? What's the value of making an employee's life a little bit easier?

1

I think that the list you have provided is already very good. There is one thing that I would add: Error Reduction.

When working in 5 different systems, is there any data that would need to be copy/pasted or transcribed between the two systems? Are there any examples where clients have been presented with incorrect information as a result of the cumbersome system that is currently in place?

If you can show that these errors could be reduced or prevented with a one-stop application, then it could be possible to quantify this as a value of the app.

0

Based on what you stated its the "annoying process to make them quit" is your challenge. Also, how would you be certain that the person will do approvals while walking between meetings, in a cab, on an underground subway, or just before you go to bed. Would it be more sensible such action is more appropriate when their in focus 100% and not too busy with other errands?

Regardless when using the laptop or building new mobile app for completing pending approvals but the underlying activities executing these processes remain the same your not hitting the root of the problem. Your best route is to justify the value on the TO-BE process vs the CURRENT process. Change your focus rather than building your UX imaginary numbers. Look for bottlenecks, reducing steps, delegation and efficiency.

0

The thing that comes to mind is will this app replace the 5 systems or be an add on? In my experience IT consolidations can have a huge effect on maintenance costs of servers, necessary people, skillsets, etc. Which could be a huge motivator.

The other is do you have any idea of the number of errors or how long it takes to learn the 5 different systems? Like 5 hours of training for the 5 systems will be reduced to 2 for a single system. Or 50 invoices never went to colllections...

It sounds like the user is the leadership you are presenting to, so maybe appealing on the emotional level or even having the qualitative metrics like observations and surveys could help appeal to the emotional trials and tribulations of the 5 systems...

And finally, as far as the other commenter about on the go... maybe the appeal is in the time it takes to find system, log in, complete tasks, find next system repeat.... by having a single system, even if it is a laptop or dedicated time, there's still a time saver aspect.

And one more thing - try to figure out why they want the metrics. If you know their concerns ahead of time you'll be able to address those concerns more quickly. And build in metrics for the new app!!

0

Use recaptcha + A/B test

justify a potential internal app by projecting its benefits quantitatively.

To do that you will need real data. If you are committed to collect real data here is one option. Implement the new way of doing things in a tiny bit of the UX slice as in the photo. Now when a user needs to complete that particular slice of work we are A/B testing offer two buttons upfront:

  1. First time

        [NEW WAY]   [OLD WAY]
            |           |
            |           |
            |           |
            |           |
            |           |
            |           |
      [COMPLETION]   [COMPLETION]
    

2nd time

        [NEW WAY]   [OLD WAY]
            |           |
            |           |
            |           |
            |           |
       [RECAPTCHA]      |
            |           |
      [COMPLETION]   [COMPLETION]

3rd time

        [NEW WAY]   [OLD WAY]
            |           |
            |           |
            |           |
            |           |
     [2x RECAPTCHA]     |
            |           |
      [COMPLETION]   [COMPLETION]

Then, see how many recaptchas is a user willing to put up with before going back to "OLD WAY" the next time. Then you plot a histogram where X axis = iteration number of new way flow, Y = counts of completed tasks in new flow. Everybody can relate to recaptcha pain. If you have bins filled for X>2 that shows there is a quantifiable pain that users are willing to forgo in favour of your new way X>3 means more pain willing to forgo and so on. Recaptcha is a great measurement unit because it is time and mental energy.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.