MVP Proposal: User Trust and Reward System

The Problem

The site needs a system to know when to trust users with certain privileges, and the site needs a system to reward users when they do good things on the site. This proposal is based off of the reputation discussion thread and tries to take into account most opinions on the thread.

Proposed Solution:

To cover user trust:

  • We have a system (call it the “star system” for sake of concept) where users can earn stars for completing various activities around the site.
    • For example, there could be five types of stars that cover questions, answers, editing, reviewing, and moderating.
  • Users can earn stars by completing tasks specific to the star category.
    • For example, users could earn a star in the answers category by posting an answer which was accepted by the owner of the question (theoretical situation).
  • The categories are subject to change but we should probably keep the number six or below.
  • Users are not ranked by one general reputation number but ranked in each category. This solves the problem of representation and the sense of elitism.
  • Users are awarded privileges in different categories based on their participation in each category. This lets people contribute more in the categories they like.

To cover user rewards:

  • We have an extensive badge system with hundreds of specific badges.
  • Users can earn badges by either completing tasks or achieving certain amounts of stars in categories.
  • Badges can be shown off on the users profile. If anything, the badge system would be better to “rank” users since badges cover a wide variety of participation.
  • Badges do not offer privileges but may unlock special features (TBD).

Let me know what you think of this system.

11 Likes

I very much like the idea of tying privileges to related activities. The idea that you can ask a couple of popular questions on SE and thus be granted privileges like closing or editing without review has always seemed off to me.

16 Likes

Exactly. This is why I want to grant privileges based on specific merits - so if you get some nice posts that are well-liked, the system might give you some extra privileges related to posts, but you don’t get extra rights in reviewing until you earn them by reviewing.

Sets up a better plan for more well-rounded users, and specialization of “duties” leads to less negative contributions.

6 Likes

I’m not keen on hundreds of badges. If the set is smaller, people can be aware of most of them and might even choose goals to work toward. When I see a site with more badges than I can easily browse, I stop paying attention to badges and just note new things when they pop up. Maybe if they’re categorized and you’re somehow earning “advancement” in categories, it’s more managable?

10 Likes

My problem with the SO badge system is that it is hard to navigate. Maybe I am just going about it wrong, but I find I have to jump around too much to see what badges are available, what I have to do to get them, etc. So whether it is a short list or a long list, it needs to be easy to find the information.

4 Likes

Badges have the unfortunate effect of concentrating gamification. Far too many users on SE cast exactly 600 votes indiscriminately, and make exactly 500 edits indiscriminately, and review exactly 1000 posts indiscriminately… I’d rather avoid these threshold effects. Having accomplished enough of a certain kind of task should give related privileges, but should not give a shiny.

11 Likes

Maybe instead of a permanent badge, there could be something like a temporary trophy, weighting the determining numbers with their age. That is, if you earned e.g. the tag trophy, you don’t just keep it indefinitely, but you have to continue writing upvoted answers to keep it. Or get so much extra that you then can afford some time without before you lose the trophy again.

To keep it from oscillating, the threshold for losing should be a bit lower than the threshold for earning.

3 Likes

There’s a psychological effect to losing stuff: it often feels worse than never having had it. “Trophies” that you don’t keep should be exceptional. Even if it amounts to the same thing, if you want to reward ongoing activity, you should present it as getting the same trophy again for the next year (or whatever period) or not, rather than keeping or losing the trophy.

10 Likes

The third trust level in this Discourse forum works that way however: if your activity levels decrease you’ll be put back down to trust level 2. I haven’t heard people on the forum I run complain or feel discouraged, probably because only a small proportion of them reach it, and because it doesn’t actually gain you much extra power. It’s also important to note that this only applies to trust level 3, all other trust levels are permanent.

I could see something like direct editing being a permanently granted power, while gold-tag closing powers should be lost after inactivity because the community could have changed their site standards.

https://blog.discourse.org/2018/06/understanding-discourse-trust-levels/

2 Likes

@bertieb I think we’ve reached a consensus here, so could we have auto-lock enabled?

Thanks for the heads up, @ozewski. Your proposed system is interesting! However, there’s a couple things that would be good to be clarified:

  • which parts of your proposal are MVP?
  • for the ‘doing things grants privileges’ aspect, how would you get past the chicken-and-egg aspect? How would someone get, say, review privileges to they can earn more review privileges?

There has been objection to hundreds of badges, and to losing privileges.

Once we establish which bits are MVP, we can move towards consensus :slight_smile:

2 Likes

If users are given reputation / points / whatever for questions and answers, this would just bring us back to the problems discussed in the reputation thread. I would, however, support a tiny (largely hidden) privilege to allow you to contribute on “protected” Q&A’s (something like just needing to have at least 3 positive-voted answers on the site).

I would propose limiting rewards to moderating (editing, closing, reviewing, etc.), but it’s pretty vital to have proper validation in place so users don’t just robo-edit or robo-review. Maybe the number of reviews for some user can be shown together with the quality of those reviews, or something like that. This can get very complicated, so maybe such rewards shouldn’t even be in the MVP at all.

I have been thinking about a bounty system to reward amazing answers (as the only way to get reputation). But where would the bounty come from? Perhaps we could use the moderation reputation, although being a trusted moderator doesn’t mean you know what a good answer looks like. Or perhaps each user could have a hidden pool of points obtained through upvotes that could be awarded to other users as bounties (similar to regular reputation on Stack Exchange, except separating reputation obtained from upvotes versus bounties). I say “hidden”, but it might be useful to tell users they have at least 10 points, for example, so they know they still have some to spend, but also to prevent it from turning into the thing to game instead. This might just circle back to the same reputation problems, although, if implemented correctly, it should be much harder to obtain, making it a much more special reward and harder to game. Just some random thoughts…

1 Like

Since we won’t have a one-size-misfits-all “rep” number, bounties in the Stack Exchange sense which transfer “currency” don’t really work. Besides bounties don’t make so much sense for rewarding an existing answer: why should rewarding an answer make me less reputable? Instead I propose handling exceptional rewards through a reaction mechanism. No cost involved (but an entry barrier and a rate limit), no direct bonus, but indirect bonuses from making the answer more visible (because the default sort order should take reactions into account and not just score) and having a “mark of approval” next to it.

2 Likes

This was always my problem with bounties on Stack Exchange, so that’s not quite what I had in mind. I was thinking you’d consume “bounty points”, not “reputation”, i.e. something which exists exclusively to be awarded as bounties and serves no other purpose. Of course users can make something serve some purpose we didn’t intend, which is why I propose keeping the number hidden.

Even if we were to use moderator reputation (which, again, doesn’t make you a topic expert, so may not be the best idea), I would still propose separating what’s used for bounties from what your reputation is. So, for example, 10 good edits might give you 1 mod reputation and 1 bounty point. Spending the bounty point doesn’t affect your mod reputation.

Although it feels like overkill to do all of this just for bounties, so perhaps it’s just something to keep in the back of our minds to potentially combine with other functionality.

1 Like

I, too, don’t like the idea that bounties somehow make you less reputable on SE. One of our top users on one of my sites shows up much lower in the list than warranted because of great generosity with bounties.

I wonder whether we should allow anybody to award public, attributed kudos, with the catch that we’ll also show the number you’ve given out. There are people from whom getting a kudo at all is very nifty (ooh, Jon Skeet liked my answer!), and if somebody who rarely gives kudos gave you one, that’s pretty spiffy too (no names in this example; fill in your own). We’d probably want a low daily limit so they continue to mean something, and so people don’t feel social pressure to sprinkle them all over the place.

They shouldn’t be anonymous, because the whole point of the (e.g.) Gilles seal of approval is that we know it came from Gilles. Use it on stuff you stand behind.

By making them available to everybody, we include the people who have “site-local” reknown but not necessarily in one or a few tags, and also to people who are new to the site but are known from outside. It can be hard to get passing experts to stay and engage sometimes, but this would be something easy that would increase their visibility.

5 Likes

I like the idea of attributed kudos, but the downside of showing how many you’ve given out is that some people may try to game / maximise it, i.e. give the maximum number of kudos every day, even to posts they don’t feel deserve it.

Or people may judge you for not giving enough kudos (see also the whole acceptance rate SE fiasco) or for giving too many.


Kudos may fall under the reaction system Gilles mentioned above and might be comparable to my reply to that.

Hmm, yes. I’d forgotten about acceptance rate. So maybe we don’t show the number directly; site regulars will get to know who’s kudo-generous and whose are more rare, and visitors probably won’t care very much.

1 Like

The biggest flaw in the SE system by far, is that it mixes up domain knowledge with moderator suitability into a single mess called “user trust”. We need to get rid of that system.

A good domain expert doesn’t necessarily make a good moderator, or vice versa. Votes on answers should count towards domain knowledge reputation. Doing tasks like edits, close votes, flagging, reviews should count towards moderator reputation.

With this in mind, I think your proposal is quite sound, much better than what SE has currently. Maybe just three categories though?

  • Site experience. Everything you do counts for this - like the SE rep system.
  • Domain experience. Votes on answers count for this.
  • Moderator experience. Edits, reviews, flagging etc.

Maybe come up with some fancier names for the categories though.

10 Likes

The only real discussion about User Trust in this thread seems to be that it shouldn’t be linked to kudos/rep. The majority of these other replies would fit best in MVP: Reputation. While I understand the mention of a reward system, we seem to have strayed off of that topic also.

On that note, I’d like to propose a set of User Trust levels.

Trust Level 0 - Newbie

This is the level of trust for brand new users. Contains a "new contributor" label.
  • May post 1 question and 1 answer per day.
  • May comment infinitely on their own questions/answers.

Trust Level 1 - Novice

A level of trust for new members.

Requirements:

  • At least 1 well received (wilson score above 0.5) question/answer.

New Perks:

  • May now post n questions/answers per day, where n is the number of well-received questions/answers they have + 1.
  • May now suggest edits and flag posts.
  • May now post up to 5 10 comments per day on other people’s questions/answers.

Trust Level 2 - Member

The standard level of trust for most users on the site.

Requirements:

  • At least 5 well-received questions/answers.
  • At least 2 hours spent on the site.

New Perks:

  • Question/answer/comment count restrictions are removed.
  • May now access the downvote button.

Beyond Trust Level 2, levels should become site specific.

Trust Level 3 - Constable

The first level of moderation, intended for those who have an interest in moderating the site but not much time to do so.

Requirements:

  • At least 12 hours spent on the site.
  • At least 250 150 accepted flags.
  • At least 100 50 accepted edits.
  • At least 20 15 well-received questions/answers.

New Perks:

  • Ability to vote-to-close directly, rather than simply flagging.
  • Ability to edit directly, without review.
  • Ability to review suggested edits and flags.
  • May protect posts from being answered by any TL0s.

Trust Level 4 - Deputy

This is the level of trust for those wishing to become moderators - trial moderators, if you will.

Requirements:

  • At least 50 hours spent on the site.
  • At least 500 successful reviews completed, with at least 100 in each category (e.g 120 edits, 380 flags reviewed).

New Perks:

  • May temporarily lock.
  • May temporarily freeze posts, holding vote counts in place.
  • May soft-delete posts.

Trust Level 5 - Moderator

Full moderators receive trust level 5, but may lose it upon becoming inactive.

Requirements:

  • Voted in by community. (can be voted out again by community)
  • Vetting by all current moderators.

New Perks:

  • May permanently lock posts.
  • May block users.
  • May suspend specific permissions for users.
  • Has access to all moderation tools that we come up with in the future.

Trust Level 6 - Council

This is a temporary trust level granted to review council members when dealing with disciplinary actions. This could also serve for the developer-tier of trust, assuming developers don't just have console access to each site.
  • May adjust Trust Levels of all TL2-5s.

Please leave any feedback you may have. I’m happy to make changes based on people’s opinions, as I’m not experienced in Q&A moderation, so I don’t know what might be appropriate points in which we promote/demote people.

6 Likes