I need to evaluate my application's novel interface. However, due to IRB issues I can not involve users for this stage of the evaluation. Previously, I asked How to quantitatively compare two interfaces?, which gave me some useful feedback but most of the metrics include humans.
At this point I have to do some form of evaluation. I plan on doing a formal study but that will be at a later date. My tool is an IDE so the two ideas that I can think of are:
Count the number of actions to complete a task in a few IDEs and then compare the results. I would probably decide on 3-5 different common tasks that developer's perform and compare them for 2 or 3 IDEs.
Write out a few use cases/user stories that describe the process of a user trying to accomplish a given task in a few different IDEs.
How else can I evaluate my tool without users?