As I mentioned in this question, I've been working on proof-of-concept applications for social network privacy and running usability studies to evaluate them for publication. I'm a beginner to user studies at best, and I'm ashamed to admit that I don't think I've been asking the right questions. I've generally asked users to compare the simplicity and usefulness of the applications to the privacy controls provided by other social networks, such as Facebook, but I'm curious if there are even basic questions I should be asking.
My questions are:
- Are there any basic questions a user should be answering in any usability study?
- Specifically in a comparative usability study, are there additional base questions a user should answer?
- Can a user usefully "quantify" the usefulness or simplicity of something, even in a comparative case, or should questions be of a binary nature?
- In a study trying to quantify the usability of an application versus simply make improvements, is there any advantage to asking open-ended questions?