0

We are rebuilding a large web application from the ground up. We have several thousands of technical users currently managing digital products in a web portal with a complex account/user structure. It has security overhead and a legacy nature that complicates the process of installing third party analytics tools. The UI is a mess of bolted on, unscalable functionality, and so we're completely reconsidering almost every aspect of the platform.

I'm putting together a strategy for reaching out to existing users to validate our assumptions from the persona development process, and to attempt to work their actual goals into the process from even before the ideation phase. I need a communication process that provides optimal data capture, and that allows sufficient control of the conversation to keep it focused on their goals, outside the context of any UI.

The key points from a user interaction perspective are that I don't have anything visual to share about the new UI - this isn't user testing - and I need to ensure the conversation isn't hung on the hooks of the problems with the current system.

Does anyone have experience in managing this type of communication and implementing it in an agile development process? What type of communication tools did you use, and what are the lessons you've learned?

2
  • Is this just a case of a poorly timed question, or have you guys really got nothing for me on this?
    – dennislees
    Mar 30, 2014 at 19:03
  • Your question could be clearer. Are you asking how do you communicate the result of user research within an agile project? If so it's down to how the team shares information. Talking to them is the best agile way. Apr 3, 2014 at 14:03

3 Answers 3

1

You might find this article about interviewing users to be helpful.

Decide how much you can share with the users. Can you mention that you are reworking the interface? You might find it helpful to be blunt, that you are aware of usability/design issues with the current interface, and are reworking the system. To make sure you focus on the right tasks, would they mind answering some questions?

You may also have to listen to their existing frustrations with the software. But asking the right follow up questions should help validate your current/future efforts. Get them to tell you about their experience.

For example, if they complain about the existing interface:

  • ask what they were trying to do when they encountered the issue. "Tell me a time when you ran into that issue. What was your goal and how did that interfere with it?"
  • ask more whys, why was that task important? What was the intended outcome of the task?
  • how often did they encounter the issue, was it a task they performed daily?

You may not be able to avoid hearing about current problems. But listen, and dig deeper, and you should find out a lot without getting stuck.

Additionally, don't forget to ask what the system does well! You don't want to leave out existing appreciated functionality!

0

Since you mention that the system is very complex and difficult to add analytic tools, you will have to work with what you have, what do you have?

One thing you have is web server logs, start there:

  • Pages that are loaded the most, that indicates what is more useful in terms of content or indexes.
  • Time spent per page, that indicates problematic pages, content that is difficult to understand or quality content.
  • Response time, this gives you slow pages, which can be because database or programming problems.
  • Redirection or processing rules that consume time and resources, with that you gain the knowledge to improve folder structure, navigation and common typos.
  • Pages that are less visited, that gives you an idea of useless pages or hidden resources that should be brought to attention.
  • Referrer. That gives you an idea if users are typing addresses (or copy and paste) which indicates problems to find the pages or bookmarked pages.

Most of the options above have a good side and a bad side to interpret, deciding which one is the right one each time depends on your knowledge of the system itself, the users, their goals and the logs.

You should combine that with some kind of interviewing or online questioning to help you decide. If you can't, then just use the above information.

Another thing that you have is, most probably, server space, so once you have the basics of the new system, build an interface and make that a live test, ask users to go to that beta site and use it, add a simple feedback form on each page where they can, anonymously, tell you what they want and what they dislike. Combine again that input with the information from the logs of the old site, the logs of the new site and any analytics that you set on the beta site. If you do this, before asking users to test it, you have to be sure that there are no serious problems, if users find holes, closed doors, unexpected behaviours, cryptic error messages, etc, they won't like the new site and most of the input will be bad an inaccurate.

The most important aspect, I'd say, is to build the new system in a flexible way, so you can adapt it quickly after a short period of testing, either on a beta site or in a final, live site. Consider that users will give you input at some point and you have to use it, if the site is too rigid, then changes are going to be too difficult to implement and then user input will be, most probably, ignored.

3
  • Thanks for your input, but I think it misses the mark. Any data we're going to get will be about the current system, which we know is slow and inefficient, which is why we're completely rebuilding it. We'd have to go data mining to make inferences about goals. I'm more interested in capturing input directly from users, outside the context of UI and analytics.
    – dennislees
    Mar 31, 2014 at 12:50
  • Yes you know it's inefficient, but the analysis of patterns is what is going to give you the information you need. I've done it before and know people who also have done it with good results. Analyse patterns, look for the pages visited and the information contained on it, then you know what the user is looking for and what path it follows to get it; that, joined with your knowledge of the system, user needs and basics of UX and usability will take you far enough to have a good beta site. Apart, as I mentioned, if you can, you should combine with intervieweing.
    – PatomaS
    Mar 31, 2014 at 14:07
  • I hear you, I do, and agree with your points, but the modification of the current system, even in the light of the potential goals, and even for something as seemingly simple as adding as virtual pageview functionality, isn't something I'm going to get approval for at this stage. This option boils down to trying to change my companies approach to prioritizing business value. That's not going to happen before I need to deliver on this stuff.
    – dennislees
    Mar 31, 2014 at 18:24
-1

In a nut shell it's called user research. From user research you don't get data but information.

The most direct form of user testing are one to one interviews. Find out what they do, why the do it and what they need to do it. Never ask them what they want directly, don't lead the questions and play dumb if they ask you questions about what your company is doing in the future!

Do a search for contextual enquiry as that's the technical name for one to one user research sessions.

Also try to avoid 'assumptions' in any personas you use. The more made up stuff in a persona the more likely they are to lead to the wrong solution.

3
  • Wow! Talk about a borderline insulting answer. Perhaps I'm partially responsible because I didn't point out in the question that I was aware of the concept of "user research", and I that I figured that these things you call "interviews" would play a part, but I did explicitly ask about tools and lessons learned, and hoped for sophisticated answers from people who have some actual experience to share. Your answer suggests you didn't fully understand my question.
    – dennislees
    Mar 31, 2014 at 17:42
  • Sorry if you belived I was just stating the obvious in a patronising way, that wasn't my intention. Reading your question you asked "I need a communication process that provides optimal data capture, and that allows sufficient control of the conversation to keep it focused on their goals, outside the context of any UI." The solution IS to use interviews. I've been doing this a while and interviews done well are much better than any 'sophisticated' tool. The tools used to support the interview process you'll know about. I have worked on multiple agile projects, no two use the same 'tools'. Apr 3, 2014 at 13:58
  • Having re-read your question, which is not very clear, you appear to want to know how you get the results of your user research to the other team members. I think by tools you may mean deliverables or documents you use. Or do you mean the systems used to share the information such as a physical wall or wiki? Apr 3, 2014 at 15:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.