First to explain what A/B test is.
Say you have a website where the homepage is really important. While you design it, you wonder whether it would be better to write in the heading section ( h1 tag). You say it would be better to write "Welcome to our website" while your colleague argues that "Take a look at our services" will do a better job.
To answer which heading is better you design one version of the homepage with "Welcome to our website" and one version with "Take a look at our services". Lets name them version 1 and version 2.
To get the answer you present to 50% of the real visitors version 1 and 50% see version 2. If the goal of your website is someone to subscribe to a billing plan you start tracking for each version how many subscriptions there are. After one month you watch the results and see that version 1 had 1000 visits and 12 subscriptions while version 2 got 1000 visits and 26 subscriptions. The second version gave you more money so you delete version 1 and use only version 2.
1) Even the smallest organization can do A/B testing, it doesn't matter the size of the company, all that matters is how much real traffic you got. The more traffic you have the faster you will get the results from A/B test. If you have for example 1000 visitors per month and test two variations only you will need approximately 1 month to get some results. This is because you need to reach statistical significance of 95%. To put it simply, this means that you will be 95% sure that the results are valid and true.
2) It's absolutely necessary to do A/B test with real traffic. Otherwise the results won't be valid.
3) According to my experience the first thing you should test is the heading text of your web pages. Then you can test layout, and other stuff.
You will probably have other questions, don't be afraid to ask, I'll be happy to answer.