Our product provides search functionality as well as profile viewing. Each profile is assigned with engine-calculated tags, with a relevancy score. Example: Profile JohnDoe is assigned the tag UX with 70% relevancy rate, and tag Indie-rock with 30%.
In order to make these results better, and increase user engagement, we are adding user-voting per tag. Each tag gets a vote-up option, and I can vote up that Indie-rock describes JohnDoe correctly.
This leaves us with two relevancy score per user,tag - the engine-calculated, and the user-rating. Our plan is to incorporate users votes with the engine calculation and generate a unified tag relevancy calculation. Meaning, in the above example, JohnDoe's relevancy to Indie-rock will be higher - lets say 33%, and UX will remain the same and the next end-user won't know if the 33% is because someone voted or is just natural engine-calculated. Moreover, search results will be affected from this new relevancy assignment. Dividing the two calculations seems non-productive for the next user that is still seeing the same not-accurate engine-calculation the user before him 'improved'. In order to encourage voters, was thinking of adding a 'how many people you've helped' counting.
Two things I'm concerned about:
1. What is the right way to explain to the users what this incorporated calculation is. Does he need to know these are two different metrics?
2. I've not seen many such approaches. Is this wrong? Good examples?