What are the questions you need to ask or the methods you perform when you're doing a competitive web site analysis from a UX perspective? How do you gather the facts? When in the project process do you do it? Before you have done a user test analysis or afterwards?
The System Usability Scale was made for this; though rough it gives you quantifiable measures to compare two systems (sites) by running users through your site vs other sites. Since your sites (presumably) are significantly similar in intended purpose your results should be readily comparable to the other sites.
More generally, A/B testing your/their site for simple tasks can help you find where users have trouble with your site as opposed to their site or vice versa.
As for what questions to ask, ask what's relevant to your site. For Mint.com the big question is "would you trust this site with your financial information". For Facebook (in it's early days) it might be "would you use this site daily" or "would you recommend this site to friends". If your users trust the other site more, find it easier to log in or are more willing to spend time on their site, you need to know that, and then you need to know why.
Do some qualitative analysis as well; have users use both sites. Ask what stands out to them; maybe the competitor has a way better log in process, maybe your sites ads are annoying. Getting users to compare first impressions for both sites can be very enlightening.
This should be done after you've done some simple user testing on your own site; catch the egregious usability/UX issues on your own site before you bother to compare to other sites.
You want to be comparing a viable product to the opposition otherwise your competitive analysis is just going to tell you what your user testing will tell you.
I think in order to answer your questions, you should first focus on what type of metrics are important to you. Also focus on the main tasks that users perform on your website. You can judge your website's performance by user experience metrics, including efficiency, effectiveness, and satisfaction.
Here is a good webinar that you can watch on competitive benchmark of 4 allergy medication sites. It will give you answers to some of your questions. http://www.userzoom.com/resources-on-onlineremote-ux-research/webinars/pharmaceutical-allergy-medication-industry-user-experience-benchmark-study
Disclaimer: I work for Userzoom.