Where does user-testing happen in LeanUX/Agile?

Question

I'm trying to reconcile what I feel is a bit of a contradiction in LeanUX.

• test, validate, and iterate early
• reduce the documentation load

Does your organization user/usability test in an agile process? If so, when and how?

In our org, we do a lot of unmoderated remote testing. This requires a relatively complex, shareable prototype be created (typically Axure or Invision, sometimes HTML).

This isn't entirely bad, as in the process of creating the prototype, we do do a lot of internal iterating and design tweaking. However, it seems counter to bullet point 2...as these prototypes tend to be rather large documents in terms of creating time and effort (not to mention maintenance if they are also being used as wireframes for dev).

Question: Are those two bullet points contradictory or is the issue that I am interpreting them incorrectly?

UPDATE:

An alternative question that may make more sense to answer: Does one normally unmoderated usability testing in an Lean UX process?

Answer 1

TLDR: We ask this question "What's the smallest increment of work that can be tested?

We go and test that using the quickest solution that'll get us answers. Which means products like Axure and even Invision are often considered as too time consuming to use unless we're dealing with a heavy animation/interaction-based solution that is difficult to convey to people using static UI.

---

Breakdown of our process

We built our process off ideas from the book Inspired: How to Create Products Customers Love.

There are 4 phases

1. Opportunity Assessment - Research on perceived user problem, go/no go decision
2. Product Discovery - Iterative ideation and concept testing
3. Product Execution - Outline stories and work with Devs for implementation
4. Release - Staged (beta) or incremental release, get post-release feedback

Majority of the testing happens during the product discovery phase. We try to go for broad stroke checks at the beginning. So test group sizes are super small... 3-5. We may start off with a quick test with internal staff (support, on-boarding and sales guys). Then we schedule remote screen sharing sessions with users. (We deal with a complex software, it's usually easiest to pick up nuisances when you can hear the user.) Since we're there with the user, we frequently do a combo session with a mental model interview followed by A/B type concept tests.

The concept tests are as low fidelity as we can get away with. Typically this is just an opened omnigraffle file. I read out the question, click through pages of screens and collect their feedback directly on said file. You save a lot of time if you don't have to export and upload images, then tweak stuff in a separate tool. (Not saying tools like Axure and Invision don't have their place in our workflow. They do for stuff with more complex screen interactions. They're often not necessary.)

The most time consuming aspect of this is probably user recruitment. We're pretty lucky in having a very active user base who are super happy to help with testing. We then use a tool like youcanbook.me to indicate free blocks for testing and have our users sign up for the blocks.

I believe another term for this is dual-track scrum if you want to do a deeper dive into the area.

Update: About your question on unmoderated tests. We tried it in the past and stopped. Unmoderated tests require you to know a lot more about the question at hand. If the questions asked are completely off mark, the unmoderated test data will be completely wasted. With a moderated test, you have much more flexibility to adjust on the fly. Flip side with a moderated, you have to be there to run the test, which limits the number you can do. But, numbers aren't necessary for early testing, just the user's gut feelings.

Answer 2

The magic with lean UX is that you can test everytime you need. It means everytime you have a doubt in your development process.

You don't need to organize long and complex users testing sessions with a full complex prototype done in Axure. It is the lean UX force.

You can test just a specific feature, or interface with a sketched prototype done with free and simple apps (like Pop for example).

Invision's app provides good tools also like the Liveshare or the online comments wich can help you to make fast & short user tests.

Lean UX is a method which allowed failure. You have to see your sprints like experiences, so as in a science lab, you make assumptions, try, test, failed or success and then you go one.