How about this:
- Display one question at a time, with options to answer or skip to the next question.
- If question is answered, the question/answer set stays on screen for reference and future editing, while the next question is displayed.
- Otherwise only the new question is displayed.

download bmml source – Wireframes created with Balsamiq Mockups
A variation on this:

download bmml source
(This is slow, though, as it forces your user to go through the questions linearly, but it also acts more as a guide/wizard for the tool. That's the impression I got from your question, but I'm unsure as to whether you actually want that kind of behavior.)
Another idea that might be less frustrating for power users who know which fields they want to hit:
- Display all the questions at once, and when a question is clicked, an answer field appears.
- To further avoid clutter, the answer field can stay on screen after an answer has been given, but if another questions is selected for editing without the first question being answered, the previous answer field disappears.

download bmml source
In essence I am trying to get around the 'clunkiness' you dislike by streamlining the appearance of questions that are not relevant at the moment, by collapsing or hiding them.