I'm a UXer in a little software agency and we are currently redesigning a large and quite complex software with lots of features etc. By now we have developed an MVP that covers a part of the entire software. So, currently, users can create new emergencies and search for bus and hotel suppliers.
Premise: the goal of our usability test is utilitarian and not experiential.
In our first usability test of our MVP, as expected, I've noticed that users struggled with the new interface and didn't even notice the new features that we purposefully designed to solve some pain points they had in their old software. At the end of the test we told them about these features and they actually found them useful to overcome those pain points. Besides, as the users were completing the tasks they were given they started to get more familiar with the interface and struggled less with the interface.
Since most complex software do have instructional overlays and such for new users, I was wondering if it would make sense to instruct users before they take the usability test or not. I'm asking this question because I'm having a hard time at classifying usability problems that rise from the tests.. Indeed some of these problems are due to the fact that the interface is completely new to them.
What do you suggest?
Is there any literature that deals with this problem? My best guess is that by instructing users with overlay marks we can get the best out of our usability test.