Is it ok to incorporate engine-calculation and user-improvements-votes into one relevancy grade?

Question

Our product provides search functionality as well as profile viewing. Each profile is assigned with engine-calculated tags, with a relevancy score. Example: Profile JohnDoe is assigned the tag UX with 70% relevancy rate, and tag Indie-rock with 30%.

In order to make these results better, and increase user engagement, we are adding user-voting per tag. Each tag gets a vote-up option, and I can vote up that Indie-rock describes JohnDoe correctly.

This leaves us with two relevancy score per user,tag - the engine-calculated, and the user-rating. Our plan is to incorporate users votes with the engine calculation and generate a unified tag relevancy calculation. Meaning, in the above example, JohnDoe's relevancy to Indie-rock will be higher - lets say 33%, and UX will remain the same and the next end-user won't know if the 33% is because someone voted or is just natural engine-calculated. Moreover, search results will be affected from this new relevancy assignment. Dividing the two calculations seems non-productive for the next user that is still seeing the same not-accurate engine-calculation the user before him 'improved'. In order to encourage voters, was thinking of adding a 'how many people you've helped' counting.

Two things I'm concerned about:
1. What is the right way to explain to the users what this incorporated calculation is. Does he need to know these are two different metrics?
2. I've not seen many such approaches. Is this wrong? Good examples?

Answer 1

1. It is never wrong to inform your users of what you're doing. Transparency may be a populist word, but represents respects for your users. You don't have to put these calculations in front of your metrics, but those users who want to know should easely find them. That means a placement on the about page or FAQ page which may be even a better choise.

2. You're not wrong implementing this. Actually I think this is a very good idea where there is only one metric to keep track of. Two could cloud the message you're trying to convey. But you could display a drill down of the metric in a dashboard page (or profile page) where users can see from where the score comes from. Users love being informed and at least get the feeling of transparency.