We've just run an A/B test for one of our landing pages using Visual Website Optimizer. We are an online hotel booking reservation site.
The landing page shows information about a particular hotel (pictures, description, map, amenities) and a search an engine box to select the dates of the stay, room type, currency... with a button to "search" availability on that property...
The search box is now placed at the bottom of the page (you have to scroll down to see it). The variation was to put the search box on the top of the page (horizontally in both cases). In this sort of landing pages, all our competitors have the search box on the top of the page (usually on the left hand side). The stats for the test: Percentage Traffic to include in Test: 100% Total visitors: 250
CONTROL (127): Submit Search: 34.65% Submit Booking: 4.72%
VARIATION (123): Submit Search: 27.64% Submit Booking: 3.25%
Contrarily to our hypothesis, visitors prefer to scroll down. They submit more searches and bookings (conversions) than in the variation.
Would you trust these results? Maybe no enough visitors?