Rewards System: Points or Money?

Question

Scenario

We have a soon to launch app which includes a rewards system. These rewards are real money, so a direct Rewards Balance: $xxxx should suffice (in theory). Basically put, rewards are a real payment for some task completion

However, our first tests shows that there are 2 parts to the story. Since these rewards may decrease there are 3 scenarios: Balance grows, balance shrinks, Balance stays the same

In the first scenario, tests obviously shown users have a very good reception for the amount shown in $.

However, on the second scenario (balance shrinks) the friction went to the roof, based on loss aversion bias. In turn, this lead to test groups become static and stop doing any tasks at all.

For third scenario, we didn't find negative signs (at least nothing important) in relation to the $ amount shown, but we found a curious threshold pattern: after some sum, they froze, which means loss aversion started to play even before decreasing.

Proposed Solution

So we decided to go with points. So far, test users didn't react well or bad, they seemed quite uninterested. The task completion was more or less on the above mentioned threshold pattern, maybe slightly higher, but this happened for all 3 cases: growing, shrinking, stalling. On the brighter side, this allows us for a marginal profit

Current conclusions

While showing money amounts has a lot of friction, it has more tasks performed until that friction stage. Points have a lot less friction, but they're not enticing enough to make users perform more tasks.

Question(s)

Since we're close to pack this up, times are really tight for more testing, so my question is: is there any study showing that either points method or $ method is definitely better?

Assuming points is the way to go, is there a way that works better than others to avoid the above mentioned problems? For example: 1 point = 1$ OR 10 points = 1$ OR 1 point =$10

or ranges that may lead to less money for users but a higher completion rate since they will need to get to the next step for a higher sum. Example: 50 to 99 points achieved = $50 . 100 to 199 points achieved = $100

EDIT: Just in case it is of help for the answer, sums may go from a few dollars (well, another currency which has more figures, but for the sake of example) to 5 figures in extreme cases. Not likely, but possible. However, most amounts will be on the 3-4 figures range

EDIT 2: This is not a gamification model. This is real money for real tasks performed. Due to very strict NDA, I can't provide more info, but as a totally unrelated example: let's say the client wants users to find yellow, blue or red flowers with 5 to 8 petals. So many users will send flowers. The client will want only pne yellow flower with 5 petals, one with 6, one with 7, one with 8. Same for red and blue. Everything else will be rejected.

But then we'll find there are many users sending yellow flowers with 5 petals, many users sending yellow flowers with 6 petals and so on. We have an algorithm in place for this , but for the sake of example, let's say client will approve only the first flower of each type.

Due to this huge waiting time, we need to reflect tasks were performed and as long as the specs have been followed, temporarily approved, or users won't have any way to follow their progress and see the result of their efforts.

So we're using temporary figures, and clearly specify these figures are subject to final review, so we have no problem with the process itself, as most answers are pointing out, the problem is with the figures: money amount or points amount, with the problems we've found for each one

Answer 1

I previously designed/tested an application similar to FanDuel/DraftKings that included both points and cash as a reward. With the points system, the user could accumulate points - lose them or have them stay the same. At a later time, we also included cash as a reward system as well. (Users could choose cash or points as their reward) From a testing perspective, this allowed us to analyze which reward system drove user behavior in a more positive manner.

What we found was that in a direct comparison of user behavior, cash maintained a higher percentage of daily active users. With points, a high percentage of users would utilize the product for 1-2 weeks then abandon their account. The users appeared to be desensitized/unenthusiastic as it relates the point reward system. Even when outlining the value of a point as it relates to cash - it still wasn't compelling enough.

There are downsides to both approaches...which you've clearly found through testing. We also experienced that the loss of cash is a friction point and even sometimes an abandonment catalyst. A good way to solve for that is to pad the loss with some sort of incentive. Meaning...hey, looks like you lost some here - but here's a bonus of xyz amount. Or here's what you can do to get back in the positive.

From that experience, it became clear to me that providing value to the user through the cash reward system was the way to go. You just have to work through the friction points. Whereas with the point system, the users aren't enthusiastic from the beginning of the experience. From that standpoint takes a lot more work to show them where the value exists when using your application.

Hope this helps.

Answer 2

I don't think the response to a shrinking money balance can be attributed to loss aversion. Your users are not being faced with a choice between certain small losses and less likely large losses. Additionally, because the balance can shrink by any degree at any time they can't really compare the proportions of gains to the proportions of losses. They can only react to loss as it happens. I think the response you observed in your research is more intuitive than loss aversion. People simply have a visceral reaction to losing money. For example, spending a lot of money has been found to cause pain.

If users stopped performing tasks it means they effectively stopped using your product. I would speculate that users stopped using your product because they had a negative experience (losing money) and believed they would have more negative experiences in the future. If they hold such a belief it means the product hasn't done a good enough job of educating them on what will and won't happen. This means they form their own pessimistic beliefs about what the future holds i.e. more losses.

Consider creating a provisional balance that has secondary prominence to the main balance. The provisional balance would allow the user to keep track of potential earnings i.e. earnings that have the potential to be erased. If the user has a clear understanding of the fluctuations that can take place with a provisional balance, they may not have a strong negative reaction to losses of this balance. Since you've already tested that losses to a points based balance does not affect task performance I would consider representing this balance as points. Since money has been shown to produce higher task performance you want your main balance to be represented as money instead of points. Once the 'security period' has expired you can convert points in the provisional balance to money in the main balance.

I think you should also consider adjusting your 'security period'. The 60 day 'security period' means that users have to wait quite a long time before they will receive their rewards. Keep in mind that users will compare payouts from your site to other activities where they are being paid to perform like jobs and services rendered. Some people fare well at delaying gratification while some don't. I would consider shortening the 'security period' to a 30 day period at most, since most adults have been 'trained' to wait 30 days for a salary. This would help to mitigate users who may be defecting from the product because they can't stand the wait between completing a task and receiving a payout.

Answer 3

Your situation reminds me of BitCoin Mining pools. Users do "work" for "points" that can be exchanged for "cash". BitCoin has the same problem, where work can be rejected by the network.

Slushpool, a BitCoin Mining Pool, shows several values of your rewards.

• Confirmed Reward
• Unconfirmed Reward
• All Times Reward

This makes the current state of your rewards clear. All Time Rewards will never decrease. Confirmed Reward only goes down when a user spends/withdraws/etc points.

The Unconfirmed Rewards balance will fluctuate naturally. Either by being transferred to Confirmed Rewards or by being rejected. Since it is labeled "Unconfirmed" your users should be more understanding when a dispute occurs and a reward is removed. (Your users know that the work needs to be accepted, not just submitted, for a reward right?)

Answer 4

It might be worth trying to combine both approaches. Users should have an incentive to do tasks, because the tasks will eventually lead to a provable, visible amount of money. On the other hand, you want them to experience a loss in rewards as less tragically.

So, let the users gain rewards in points, but allow converting points to money in a well-documented manner. (E.g., something like "You can convert all points above 1000 at any time to money at a rate of 10:1") Give them an incentive to work for the next cash-out.

This way users don't feel that reward earned or lost are directly touching their purse, but they still have the knowledge, that points will eventually translate to hard cash.

Answer 5

Personally, I would have said, and I feel, the same as what you discovered about interest in your app. Cash is king and I little care about "points". Cash creating higher interest might be what you really want: interest. If interest wanes as cash drops then maybe those are users you don't want.

So friction is what needs to be removed but I assume that's not what you can do. You might consider the cash drop to be a poke rather than a stab so they feel the nudge but not the pain.