User tests are a more qualitative type of evaluation methodology.
Meaning that when you observe the user you find out why exactly things don't work. Not just higher task completion time or other numbers. With a sample size of 5-15 users it will be hard to achieve statistical significance, anyway. So there is no point in overdoing it with stats.
The point of the user test is to learn:
- Where users experience difficulties, and confusions
- How they overcome them without the researcher helping them
- Users mental model or how the user understands how the interface works
- What parts of the interface the user likes, and prefers to use.
Forget about calculating optimal path versus users one. It's just not worth the hassle. You will observe these slowdowns live, and understand much more than time to complete. It is even good to ask questions, about particular problems.
For your measures, you can add number of errors per task, and severity of the errors. It is standard to divide error severity on low, middle, and high, where high is the user cannot complete the task. Just to confirm, time completion, and success rate are good measures to take.
Furthermore, deciding for an appropriate statistical measures depends on the application and context of use. Consequently, you can utilize measures appropriate for your app and context of use.
Finally, make sure to use a think aloud protocol while doing the test, in order to get valuable information of what they think while using the interface. Just ask the participant to think aloud while doing the task.