There have been some comments asking for specific details on the system I am trying to build, which are now at the bottom of the question.
I am designing a system that allows users to enter content to describe certain physical objects. Objects can attributes such as a height, weight, a name and a description, and so on. Different objects can be stored, with different relationships between them, with up to 10 different attributes per 'item'. However, historically, users have been reluctant to enter values for all of these fields due to time constraints or perceived effort. There has been a strong organisation push to force these people to add these fields after the fact, but this has not been wildly successful.
While it would be possible to enforce certain fields, this isn't always possible as the system must allow for blank attributes to be stored to allow for incremental saves of work. One possible strategy is to use a gamification strategy with a progress bar that shows how "complete" an item is.
A possible solution I am working on is that each field could have rules that are assigned a weight, and the sum of these weights is how "complete" each item is. For example, the StackOverflow careers site does this, by doing some magic on your CV and telling you how "complete" it is:
The initial thought is that the system can rules to check each field against (specifically rules that can be processed by a computer):
Rule 1. If there is a field named "name" that has content, add weight 10
Rule 2. If there is a field named "description" that has content, add weight 5
Rule 3. If there is a field named "description" over 100 characters, add weight 5
So an item:
name: "cat" description: "a create of the genus feline"
would be given a completion score of 15/20 or 75%.
However, I am concerned about the organisation will get too focused on everything being 100% percent, rather than everything being "good" and likewise, people adding cruft to fields when they aren't needed.
Another possible solution I'm considering is a 'star' based approach where a 5-star ranking could be achieved with less than 100%, as seen in the above rules, description has 2 rules attached to it. So depending on how scores are scaled the
cat example could get 5-stars, however if there are improvements to be made, what could be done to encourage more work?
So my challenge is:
- How can I display a progress bar that encourages participation, but doesn't focus on 100%.
- How can I encourage users to input accurate information, without encouraging them to 'game the system' to get a better score?
- How can I encourage good behaviour when an item is already at the highest star rating - but with scope for improvement?
- Can a duel-progress system be implemented with one for "mandatory requirements" and another for "optional or extra credit rules"?
- We are ranking the quality of the content not the quality of the user.
- I'm less concerned with rules how to rank things, than I am with how to effectively show these.
About the system I am building:
I understand this was a bit of a tough question that doesn't give concrete examples - but that was for two reasons, a) I didn't want to look like I was promoting a product and b) it has some very technical depth that might obscure the problem. I'll provide a brief background below, and for more details you can read this post I wrote seeking feedback from my community.
I am the lead developer for an ISO/IEC 11179 open-source metadata registry (think column descriptions, not telephone numbers), that has wide use in academic and government fields for describing data columns with extreme specificity for governance purposes and to ensure that data that is returned meets certain committee guidelines. Writing this content is dry and tough.
For example, we all take the concept of "the age of a person" (This is a link to this example) as a given, however its not - for that to mean anything you need to first define the object class "person", then define property "age" then define what the concept "person-age" really means. Before you ask, yes its important that you define these three things, there are very strong domain reasons for this that aren't worth going into here.
Along with this every item can have fields such as:
- Relations (as mentioned)
- Status levels
- And then different item can have their own specialised fields as well.
Its been noted that previously people have been reluctant to fill some fields out, due to perceived workloads or lack of context, but have ultimately been required to go back an fill fields in under managerial direction - i.e. all stick, no carrot. Also, since this is a non-volunteer business system, ranking users is not an option. Similarly, giving points to the number of changes to items isn't applicable as this is non-optional work, they have to do it.
What I am however trying to work on is a way to 'grade' the 'quality' of each metadata item. Without getting to technical, managers will be able to specify validation rules that grade each field that can be summed to give an indication of how complete an item is. If we assume that a raw score can be given, eg. an Object Class might be ranked out of 75, while a Data Element Concept can be ranked out of 104, then I am thinking these might need to be normalised to some standard a figure.
An example of this is below where an item could be given a star rating or progress bar. The idea being that these would be there from the very first change on a brand new item, so its likely that many items would start at 0 then build up over time.
The main problem I am trying to ensure is that whatever ranking system is used needs to encourage users to fill in content that is applicable, understand that rankings aren't fixed, but also not strive for 'progress bar' completion if thats undesirable.
The example here is "Person-age" from above. The description for this might be quite short as its a relatively self explanatory idea, but if on the whole we rank a "Person-age" out of 104 where 2 of those points are given if the description is over 50 words long. So if the short description is good, how to we how do we help a user get to score 102, and stop them from writing more just to get 104 when that might not be optimal.
Below is a few more sample ideas I was thinking for displaying this "quality score", obviously only one would be shown to a user at any one time.