How to measure complexity of a Business Software Builder?

Question

With my company we successfully build and run business applications for more than 225 clients wordwide and more than 420'000 end users. It's mostly used to support complex business processes such as client onboarding processes in a private bank.

The product is divided into two parts: (1) The part which end users see: Software running in a browser which supports the end user in his day to day job. (2) The part which is used to build part 1, which mostly happens through modelling.

To be more concrete: Part 1 of gets built with dragging and dropping screen components (form fields, texts, buttons, layout elements, ...) onto a screen builder. Additionally the right variables get bound to the elements and the screens get linked to a corresponding process.

You can imagine this to be very straight forward for simple applications, let's say a longer form of three steps. But the amount of elements increase when you have to cover many additional cases, when multiple roles with multiple permissions are involved or when the form has 25 steps instead of 3 because the compliance process requires it.

Bottom line, building client software with our product is amazingly straight forward but can also get painfully complex! I envision the product and its UI to be more simple but first, I want to have some evidence, that it's too complex. User testing is difficult because new users just don't understand enough of the context and the software itself and experienced users are - well - experienced enough and "just live" with the complexity.

Is there any way to measure complexity or cognitive load of a software just with going through it by oneself? E.g. counting the number of steps to reach a certain point? Counting the elements user's need to remember at a certain step? ...

Answer 1

A comparison might help. New against current system design/architecture. The new idea can be formed to a theoretical approach, a kind of sort-study, prestudy. Design previews, user flow diagrams, user system interaction diagrams etc can be compared by team members initially. If comparison results are positive this can grow to a full study and then to a real new system.

Answer 2

The best way to get reliable answers is to do user research. Doing the tasks yourself will give you numbers and a first impression, but no experience and arguments from real users. As programmers of the software we definitely are biased (well, many tech people deny it, neglecting the fact that other people with different background and in a different context see the world different). I lately experienced it myself, when I tested software myself and then let a non-tech user do the job. The results were completely different.

Answer 3

Short answer: No

You say that experienced users are a poor test audience because of their familiarity. Where does that place you? You should know your product well enough to sit the "highly experienced" category.

What can you do?

• Test with everyone
• Look for patterns within user types
• Gauge your average over time

The most reliable, repeatable system I'm aware of is the SUS format. It has a long history (much longer than most of us have been around) so there are reasonable benchmarks. And it just happens to work pretty well. I prefer to use the Positive SUS variant for consistency sake. More great info on using the SUS questionnaire at MeasuringU.