I can understand the thinking behind this. Looking for a way to qualify something as visually impressive is important to a lot of design professionals and teams. I would caution, however, to not let that cloud your judgement here. How visually impressive your site is makes no difference if its usability is trash. Smashing magazine has a great article from a few years back highlighting some sites that were visually stunning while being completely unusable.
If form truly follows function on your site, not only should it be visually impressive, but those visual cues should lead the user to efficient and pleasant experience. This means you can rely on traditional testing methods to get an idea of how your new "wow factor" site performs compared to its predecessor. Unlike the subjective wow factors, these type of metrics are objective, both in user feedback and hard analytics.
Scripted, in-person user testing can be helpful here. Do users find your site visually appealing? Can they easily perform given tasks from a scripted user test? What are the pain points in user workflows, and does the new visually impressive design help to address these pieces? What has the UI and UX made intuitive?
Additionally, you can look at some analytics to see what your site's visual changes have impacted. What's the change in average time spent on the site, or a given page, between designs? Is there a positive change in bounce rate? What about changes to the number of targeted interactions on your call to action?
Heat mapping might be helpful here as well. Do your new graphics and design draw user's eyes to the most important parts of the page, or are they a distraction from the actual content and navigation?
This all assumes, of course, that you have a previous version of a site to test against. If this is your first iteration, putting your analytics and scripted user testing up against a competitor may the most useful data you can gather.
The bottom line is that the usability of your site is not only a reflection of good UI/UX patterns, but also that "wow factor" that engages users. By gathering and interpreting traditional metrics, you should be able to get a good feel for whether or not your "wow factor" is a positive improvement to your site (the question you really should be asking in the first place).