We have a map based native applications for Android and iOS, with the goal to help the user finding certain spots/locations that we have put on our map (sadly I can't go to close into detail of what those spots are or with what problem the app exactly helps, it's a lot more complex than what I will talk about in this thread).
Since our inventory grew tremendously we had to split up our locations into different clusters. At least from the data side – the user still sees the whole set of spots when opening the map for the first time. Each location is categorized into 1 of 5 categories. Each spot can also be associated with 1 or more of 8 properties describing properties/features/services of the location. Additionally, each location can have up to 4 pricing models (hourly, weekly, etc.).
To enable the user to pick the right location despite the huge inventory, we decided to finally add filter functionality. We planned this in form of a drawer setup (my description is basically focused on the iOS app at this point) close to this solution from Foursquare (http://pttrns.com/applications/57?y=2015#5660) or Zomato (http://pttrns.com/applications/514#5653). Our drawer will have first a horizontally scrollable area, where we we position first the category icons next to each other, than a small (vertical) divider and then the icons for the properties (also next to each other. by tapping one of the icons the user adds or removes it from the active filters). After the horizontal scrolling part we have a list with with switches so the user can decide which pricing rates (more precise: the associated locations) he wants to see.
Now my boss wants me to set up a test for the filtering feature. Sadly my experience in user testing is mostly based on my studies, where I never came in touch with such a complex/variable feature. Additionally there are not many people here having big experience with this either. So I hoped to get a few good tips on how to set up the test scenarios, since I'm a bit worried that I might falsify the test by giving a too specific goal (like "find a location from category D with the properties 1, 2 and 5 in an hourly setting") or getting to incomparable results by keeping the test to open (e.g. "find a location close to Alexanderplatz, Berlin")
What I really need to find out:
Will the user use the filter function in order to minimize and focus the output to his needs (fulfilled when the user sets any filters and taps "apply" in the drawer)
Will the user set all filters necessary to get really explicit results to the scenario, or will the user only set the most important filters and then continue to manually browse the locations
Will the user get that he can scroll horizontally in the first section of the filters? (fulfilled when the user scrolls, requires a property that has to be set when applying the filters but is not visible when opening the filter drawer)
When the user opens up the map layer in a different area, will the user understand that the same filters are still active and hiding elsewise available locations? (Fulfilled when the user try deactivating the filters to see what's hidden or at least opens up the drawer and comments on it)
For now I think of preparing a setup where I have 2 to 3 locations for each category in an area that equals around 4 times of the original viewport. As well as a secondary setup in another city or district.
Task one: "You are in [City] and want to go to [Activity] close to [district or POI]. Find a [category] location, where you can [property descr.], [property descr.] or [property descr.]. Book the closest one to [district or POI], that fits all your needs"
Task two: "A week later you find yourself in [other city or district] in the need of […]. Find the location that offers you [catgory or property description]" Hereby the location would be hidden by the users previous settings.
Besides that I'm thinking of going for a think aloud test with an afterwards interview. I'm not sure how to realize the test itself, jet. I would say all the filters shown would need to trigger change to be understandable. But that would also mean that I would have to prepare a enormous set of filtered map views and connecting them to each other in some click through tool, considering all the filters. Alternatively we would possibly have to put some IT work in it, to make the test available during a test-release of the native app (of course not going live to the store).
So I hope you have some feedback, tips or even best practice approaches for me. Maybe there are also papers or articles about testing filtering setups but I wasn't able to retrieve anything closely related for our very specific needs.
Thank you very much in advance for helping.
EDIT:
question 1 Is there anything, I should do different from my proposed setup for this test, based on your experience?
question 2 Is there a best practice approach to test filter functionality, on map based applications, you know of?
question 3 Is there any tool or approach that you can recommend, where I can perform this test with a click through prototype (not in the actual app) without manually creating views for all the possible permutations of the filters. If not: Is there any good way of handling the situation, when a user sets his filters and I can not deliver any appropriate screen that fits the users filter settings?
Sidenote I'm currently using Antetype
