To answer your first question: yes, you must change order. In psychology research this is known as counterbalancing:
A technique in experimental design that is used to avoid the
introduction of confounding variables. In usability testing, this
technique is most commonly used when establishing task order.
For example, a website user test might ask participants to complete
two tasks: 1. find the sitemap, 2. sign up for the email newsletter.
If the study’s 12 participants complete these tasks in the same order
each time, a researcher who observes high success rates on task #2 may
falsely conclude that finding the email newsletter is easier than it
really is. The reality may be that success rates on task #2 were
artificially inflated because participants had already learned
important things about the website as they completed task #1.
Counterbalancing the task order by having 6 participants begin with
task #1 and 6 participants begin with task #2 removes this potential
As for your second question, I have never asked how hard the task was in A/B testing, because that would be a whole different kind of test, it looks like you're trying to use a Likert Scale or similar in an A/B test.
It also sounds like the test is presential, which I'm not sure if it's forbidden in A/B, but I'm 100% sure that you'll need to take much more measuring controls to avoid bias and contamination of samples (counterbalancing is one of those controls, btw).
Don't know if this answers your question. If not, you'll need to describe and explain your methodology since it sounds like some kind of particular test