Well, as you say, the obvious answer is to randomize (there are other, more sophisticated ways to do this, but probably too much for what you're looking for).
If you use the same pool of participants and show 2 versions in a row, there is no way to avoid stimulus exposure biased results. for this, you need to use a technique called Counterbalancing
The following is a translation of a Psychology Research Methodology Manual in Spanish (couldn't find a similar one in English, it's possible counterweights is not the proper translation)
Counterweights or counterbalances
It is used mainly to control the effect of progressive order or error,
and is based on the existence of a linear relationship between
progressive error and the order each treatment occupies within the
experimental sequence. As the ordinal number increases, the level of
progressive error increases. Each subject is presented with the
experimental conditions in a different order, so that in the total set
of conditions in the experiment, each of them has been applied the
same number of times in the same order.
Intrasubjective counterbalancing
It controls for progressive error at the individual level by ensuring
that each subject receives the conditions or treatments in a
particular order first and in the reverse order second. If we have
three ABC conditions, they are ordered as follows: ABCCBA. The order
ABC is applied to the subjects first and then the order CBA.
The role control technique distributes the progressive error evenly
throughout the series of trials.
The intra-subject balancing technique has the disadvantage that each
subject receives each treatment more than once, which increases the
experiment time. To overcome this inconvenience, the following control
techniques can be used.
Intragroup counterbalancing
It consists of administering different treatment sequences to
different subgroups of subjects. With this technique, the effect of
progressive error is controlled in the group and not formed
individually as in intra-subject counterbalancing. Full Intragroup
Counterbalancing
In this case, all possible permutations of the treatment sequences
must be used. To calculate these, the factorial of the number of
conditions is found.
Subgroup 1 AB sequence
Subgroup 2 sequence BA
If we have three experimental conditions ABC, the number of sequences
is
n! = 3! = 3 · 2 · 1 = 6
ABC, BCA, CAB, ACB, BAC, CBA
But just in case you want to do it anyway: If you have some JS /PHP knowledge, you could just create a page and serve a random version and save the results to a DB. You could even record the response time. If you know Python you could also use PsychoPy, there are some templates to do this.
Otherwise, it's best if you separate the groups. One will see version A, the other version B, which is classic A/B testing. Because if you use the same pool, you have to use at least ANOVA, and that's a lot more work than doing a little manual work.
All in all, I would opt for splitting the groups, it's much less effort and the results will be more accurate
