Firstly a general comment about study design:
As mentioned in a comment, both between-subject and within-subject design for studies are valid approaches for system comparisons. See NN Group explanation of these terms here, which basically says:
Summary: In user research, between-groups designs reduce learning effects; repeated-measures designs require fewer participants and minimize the random noise.
But apart from that you want to ask your own questions. Here it's important to use a system that has been proven to work (e.g. not make up your own rating mechanism without statistical proof of its validity).
Using a rating system based on the Likert Scale is a very common and valid approach. Quoting Wikipedia:
It is the most widely used approach to scaling responses in survey research, such that the term (or more accurately the Likert-type scale) is often used interchangeably with rating scale, although there are other types of rating scales.
The main point is basically to give the user 5 options to rate a statement, ranging from very negative to very positive. A neutral option in the middle is usually given too.
If more precision is needed, it can be 7 points. If a neutral response is undesired, it is also common practice to omit the middle option and only present positive & negative options.
Also, a common option to counteract "auto-pilot" ratings is alternating between positive and negative wordings, see more here.
An example question for your test could be:
Using the common action feature was effortless.
Now the user can rate how much he agrees with this statement.