Proposed changes to the trust level system

Codidact Platform Ideas
,
1

This post is talking about the currently “official” trust level system, which can be found here:

https://github.com/codidact/docs/wiki/User-Privileges

I’d like to suggest the following changes:

Trust Level 0

No changes

Trust Level 1

  • Make the minimum rate-limit for posts configureable per site (bold type)

  • May post up to 10 comments per day on other people’s questions/answers.

    Change this to a rolling rate limit (i.e. once every 60 seconds), instead of limiting it. What if a TL1 user and a TL3 user have a technical discussion around an answer (that is permitted by site rules). May the TL1 user not reply after 10 comments?

Question:

May post N questions per day, where N is the number of well-received questions or answers already posted, but at least 5. (Set a minimum so there’s an increase from the previous level.)

Does this mean: 5+n (5, 6, 7, 8, …) or MAX(5;n) (5, 5, 5, 5, 5, 5, 6, 7, 8, …)?

Trust Level 2

I’d propose to change the requirements too:

  • At least 5 well-received questions/answers, OR
  • At most 1 bad-received post and 15 accepted suggested edits

This is neccessary to give users, who constantly provide good edits and are also valuable, the chance to get TL2.

Question/answer/comment count restrictions are removed.

I am not sure, whether this is opposed by this text, but I’d say, that we still must have rate limits, such as:

  • 1 question per 20 minutes
  • 1 answer per 10 minutes
  • 1 comment per 15 seconds

That’s probably neccessary to prevent abuse and to prevent low-quality content, because high-quality takes often more time.

Trust Level 3

As for TL2, I’d have multiple ways to reach TL3. Some sides have few flagable content, which could prevent anyone from reaching TL3.

My suggestion:

  • At least 15 well-received questions/answers, AND ANY OF:
  • At least 50 accepted flags AND 5 accepted edits, OR
  • At least 50 accepted edits AND 5 accepted flags.

What might be even better would be to decouple editing from trust levels, by giving users, who are experienced edit suggestors, the ability to edit directly

Also the ability to review flags should be limited to some (closure, reopen, low-quality), but not “custom flags”.

Trust Level 4

Same applies to requirements here. My suggestion:

  • At least 25 well-received questions/answers, AND TWO OF:
  • At least 100 accepted flags/successful close votes AND 10 edits not rolled back AND 50 successful reviews of any type , OR
  • At least 100 edits not rolled back AND 10 accepted flags/successful close votes AND 50 successful reviews of any type, OR
  • At least 100 successful reviews AND 15 accepted flags/successful close votes AND 15 edits not rolled back

All numeric requirements must fulfill a ratio of > 80%, i.e. if you need 50 accepted flags, your flags must also be 80% successful in general.

I’d think that soft-deletion of posts is a vote-process (as on SE for 10k-rep users), otherwise I hereby propose it.

Trust Level 5

For the vote process, I’d (shameless plug) recommend this one:

And this one for removal:

However, I’d oppose requiring the other moderators to agree with a newly elected one. This kind of defeats the process of democratic elections, if some already elected can simply override the community vote.

I think moderators should also be able to temporarily suspend a user totally (and not just specific privileges).

Trust Level 6

I think we should remove this. I’d favor us having a special admin-role, which automatically grants all privileges plus admin tools. Besides that, they could simply have the normal trust level.

Example: I could have ADMIN role, but any trust level between 1 and 5, depending on my activity.

Additionally, I’d suggest, that any validated spam/offensive flag against your account resets you to trust level 0, so that you’ll need to re-start earning the trust. (technically solved, by only counting the activity since the last reset)

Per request, open the collider to see a consolidated version:

CONSOLIDATED VERSION

Codidact will have a system of trust levels for user actions. These trust levels are based on specific site activity, not achieving specific reputation scores.

Numbers shown in bold are the defaults for full, active sites. We will also support a scaling factor for “small” or “new” sites, so privileges are achievable.

All changes to trust levels are logged.

Trust Level 0

This is the level of trust for brand new users. Contains a “new contributor” label.

  • May post 3 questions per day, but no more often than one every 60 minutes.
  • May post any number of answers, but no more often than one every 15 minutes.
  • May comment infinitely on own questions/answers.
  • May upvote answers to own questions.

(See these suggestions on messaging about rate limits.)

Trust Level 1

A level of trust for new members.

Requirements:

  • At least 1 well-received (Wilson score above 0.5) question or answer.

New Perks:

  • May post N questions per day, where N is the number of well-received questions or answers already posted, but at least 5. (Set a minimum so there’s an increase from the previous level.)
  • May upvote.
  • May suggest edits.
  • May raise flags.
  • May post up comments on other people’s questions/answers, but only once every 60 seconds.

Trust Level 2

The standard level of trust for most users on the site.

Requirements:

  • At least 5 well-received questions/answers, OR
  • At most 1 bad-received post AND 15 accepted suggested edits

New Perks:

  • Question/answer/comment count restrictions are removed, however rate-limits still apply (1 Q/20 minutes, 1 A/10 minutes, 1 C/15 seconds).
  • May downvote.

Trust Level 3 - Constable

The first level of moderation, intended for those who have an interest in moderating the site but not much time to do so.

Requirements:

  • At least 15 well-received questions/answers, AND ANY OF:
  • At least 50 accepted flags AND 5 accepted edits, OR
  • At least 50 accepted edits AND 5 accepted flags.

New Perks:

  • Ability to vote-to-close directly, rather than simply flagging.
  • Ability to edit directly, without review.
  • Ability to review suggested edits and flags (closure, reopen, low-quality/not an answer).
  • May protect posts from being answered by any TL0s. (Note: protection is not yet MVP.)

Trust Level 4 - Deputy

This is the level of trust for those wishing to become moderators - trial moderators, if you will.

Requirements:

  • At least 25 well-received questions/answers, AND TWO OF:
  • At least 100 accepted flags/successful close votes AND 10 edits not rolled back AND 50 successful reviews of any type, OR
  • At least 100 edits not rolled back AND 10 accepted flags/successful close votes AND 50 successful reviews of any type, OR
  • At least 100 successful reviews AND 15 accepted flags/successful close votes AND 15 edits not rolled back

All numeric requirements must fulfill a ratio of > 80%, i.e. if you need 50 accepted flags, your flags must also be 80% successful in general.

New Perks:

  • May temporarily lock posts.
  • May vote to soft-delete posts.

Trust Level 5 - Moderator

Full moderators receive trust level 5, but may lose it upon becoming inactive.

Requirements:

New Perks:

  • May permanently lock posts.
  • May suspend specific permissions for users.
  • May send private messages to users and temporarily suspend them (not just single privileges).
  • May annotate users (to record information relevant for future moderation decisions).
  • May view private messages and annotations.
  • Has access to all moderation tools that we come up with in the future.

Administrators

We havie a special admin role, which automatically grants all privileges plus access to admin tools. Besides that, admins simply have a normal trust level, which depends on their community activity. The trust level has no technical relevance for administrators.

2

It’s a little hard to follow what all this ends up resulting in. Perhaps you can summarize the total rules instead of mentioning only the deltas from the existing document?

1 Like
3

I wouldn’t change to a rolling rate limit like every 60 seconds. But I would include an exception for comments on one’s own post (Q or A or A to one’s own Q).

That sounds extreme - and subject to abuse. If I (TL0 through TL4) don’t like you, I would just throw a flag and boom you’re offline for a while. Shouldn’t be that easy.

4

@Olin completed.

I am not sure, whether rolling rate limit is the proper term. I would however oppose a small, day-based rate limited (10/day) in favor of even smaller, minute/hour-based rate limits (1/60 seconds). This helps to mitigate abuse, but still allows legitimate use.

You missed a part of my quote, which is actually quite important: “validated”. This means:

  • a moderator (TL5/admin) reviewed the flag and approved it, OR
  • a moderator (TL5/admin) casted the flag, OR
  • some number of TL1 users flagged this.

It’s intended to be a safe-guard, as the -100 reputation penalty on Stack Exchange.

And I think it should (of course?) not apply to TL5 users.

5

I did miss that! Agree to that, with your explanation. That is how it should work. Thank you.

1 Like
6

As was mentioned in a previous thread, we shouldn’t be rate-limiting answers by default. Answers and questions are different. It would be a big turn-off to an expert that discovers the site, and jumps in and answers are few questions. Apply a rate limit only after a user has posted bad content.

These limits might be better as ratios than absolute limits, like:

  • At least 5 well-received posts and less than 10% negative.
  • At least one post, less than 10% negative, and 15 accepted edits.

This doesn’t cover the user who has learned the site by writing a lot of good answers. Add one more option to your OR list:

  • At least 100 well-received posts, with less than 10% negative.

Same issue here. We need to give trust to those that have spent a lot of effort on the site by providing good content. One could argue they are actually a better judge of what moderation might be required than those that spend their time as wannabe moderators. Add:

  • At least 500 well-received posts, with less than 10% negative.
1 Like
7

That doesn’t sound right to me. Someone who had 49 accepted flags and 49 accepted edits certainly is better qualified for the level 3 privileges than someone with 5 flags and 50 edits.

This part could be mitigated by making a rule like:

Sum of accepted edits and accepted flags at least 55, but at least 5 of each.

However, I also see another problem: You’ll be able to review flags with only 5 accepted flags, or to do pre-approved edits with only 5 accepted edits. Both seem too low.

I do support to decouple editing from trust levels. Indeed, I think the idea to force trust into levels is a bad idea altogether, with the possible exception of the lowest levels (which in turn should be network-wide, replacing the association bonus in SE). You should earn pre-approved editing/edit reviewing privileges from doing good edits, and you should earn flag reviewing privileges from useful flagging. You’ll still have specific privileges depend on others, but it should be a DAG instead of a linear order.

Loosely related, maybe it’s a good idea to not only consider the accepted edits, but also the rejected edits. As is, someone who has proposed 1000 edits, and got 950 of them rejected, would still get the editing privileges. Maybe "At least 50 accepted edits, and less than 20% of edits rejected”?

1 Like
8

I can see the point about spam, but this way too harsh for “offensive”.

Usually spam is pretty clear cut and deliberate, posted by someone or a bot never intending to become a real user. I don’t recall much controversy about spam.

However, “rude”, “offensive”, “not according to the code of conduct”, has a much softer line and requires more judgement to decide. Bouncing someone back to trust level 0 for a single disagreement about where the line is is way over the top. This is probably better dealt with as a ratio to total posts, or posts in the last week/month/whatever. Having the sanction be triggered by a fraction of recent posts also automatically handles what SE did with question bans and time-bans.

1 Like
9

I disagree with that. Answers are a paradise for spammers. We’ll definitely need some kind of rate-limits.

I think 15 minutes is a reasonable compromise. Good answers to good questions (which is, what we want to encourage) need some time to build, to write and to proofread. One could consider 15 minutes as some kind of implicit “quality check”

Agreed. Will update later.

Disagreeing. Constable isn’t about knowledge, but moderation abilities. Knowledge is not within the scope of this specific discussion.

Moderator tools should only be given to people, who have shown that they use them correctly. As I said in my first post here, we should consider giving editing rights to people, who use them often and correctly.

Disagreeing for the same reasons as above. However, I could agree to raising the well-received posts limit to enforce more topic knowledge.

Some users are very successful moderators on some Stack Exchange sites, without having much topic knowledge. (Example: Glorfindel on Apple.SE)

2 Likes
10

That’s a valid and good point. You’ll see me advokating separating editing privileges after TL2 multiple times in this topic.

My suggestion was (originally) based on the idea, that people would either do a lot of flagging or a lot of editing. I agree, that this is probably a bad idea. I like your proposal with adding “quality improvement actions” (needing a better label) and will incorporate it into the next iteration of this post.

As you can see, I added this for the higher privileges (TL4), however you’re right, that it should be added to TL3 (and possibly lower) too. I am not sure, what we should do about “forgiving and forgetting”, i.e. whether some bad posts should forever count against users.

11

What about triggering this for:

  • 1 validated spam post, OR
  • 3 validated recent offensive posts

For the definition of recent, I’d suggest something like 14 days, however configurable per site.

1 Like
12

You’re missing the point that it’s about having put in the effort and understanding the site, not about knowledge. Someone that has answered 100 questions is actually in a better position to judge good/bad questions, for example, than someone that has only raised flags on questions.

In general, we are trying to be too specific with privileges. Editing shouldn’t just be for those that have done lots of approved edits, flagging for those that have successfully flagged, etc. Privileges should come from two things:

  • Having put some effort into the site.
  • Having had enough activity so that you understand the site norms and customs.

There are various ways to achieve each of these. If someone contributes by editing a lot of badly worded questions, that’s great. After enough accepted edits, we trust them. In the same way, if someone contributes by writing a lot of good answers, that’s great. After enough well-received answers, we trust them.

Both of these users have put effort in and added value to the site. Saying that the person doing mostly the editing knows better what a good questions is, and therefore should be allowed to edit directly makes no sense. Reading questions to answer them gives the other user at least the same expertise.

1 Like
13

I like this a lot better. However, I’d still make the last one a ratio. If someone, for example, writes 200 good posts, and 3 of them are deemed crossing the line, that’s much less of a problem than the same 3 bad posts out of 6 total.

1 Like
14

I agree with you for automatic penalties. What about doing this?

Considering your last 14 days, when your offensive posts are less than 20% of your total participation, a priority moderator flag is raised, instead. Furthermore moderators are able to lift the penalty.

This allows moderators to evaluate the severity of your offensiveness and suspend you manually, if it is necessary.

1 Like
15

First, maybe it’s the sites I’ve been on, but I haven’t seen spam be that big a problem. Second, there are already other mechanisms to deal with spam quickly, like spam flags.

Maybe if a post gets two or more spam flags, then the account is put on hold until a moderator can look into it. Of course then there needs to be some serious downside to casting frivolous spam flags to avoid them being weaponized. That would be up to a moderator judgement call. I think we all know spam when we see it, and a frivolous spam flag would also be obvious.

Even if it means a slightly slower response to spam, I’d rather have that than turning away a volunteer that came to add good content to the site.

16

I think you meant to say “more than”, but otherwise, something like this makes sense. As with all these values, they would be tweakable per site.

17

I disagree with 15 minutes being too much. Anyone can write an answer in less than fifteen minutes, but will it be a good one? The following takes most certainly more time than 15 minutes:

  • reading the question thouroughly, understanding the problem
  • doing on-site research (other answers, should it be closed as duplicate instead?)
  • doing external research
  • write an answer good for, both, asker and others
  • proofreading that answer for mistakes

One could even see answer time as an indirect quality measurement. I don’t think good expert-volunteers will ever hit this threshold, and for all others: is this really bad?

Nope. I meant, that if your “bad” content is less than one-fifth of your overall content, the automatic system doesn’t kick in. Instead a moderator reviews your behavior and acts appropriately (inc. possible suspension)

For all people, where the bad contribution makes up a high amount of their content (more than 1/5), the automatic system kicks in. Possibly also raising moderator flags, but with lower priority.

Example:
3 times Offensive, 20 posts → 15% → moderator flag
3 times Offensive, 10 posts → 30% → downleveled
10 times Offensive, 20 posts → 50% → downleveled

All this only applies to offensive, not to spam.

1 Like
18

Most people aren’t going to make it to level 3, so it seems counterproductive to rate limit once they hit level 2.

I doubt that it has ever taken me more than 15 minutes to answer someone else’s question. For the times that I have taken more, its been self answers when I do all the research beforehand. That can take multiple hours and yet be posted in quick succession.

I write self answers as drafts during the weekends or after work and then post them early in the morning because when a question gets posted has a strong effect on the score. I have posted 2 answers and 2 questions in less than 10 minutes before even if it didn’t take me 10 minutes to write them up.

19

What about reducing the threshold to 5 minutes once you reach TL2? This should help against abuse (user going rogue, posting hundreds of bad/identical answers), and shouldn’t disturb legitimate uses. I think your case (preparing Q&A pairs) is not that common and its pattern looks quite similar to abuse (I don’t call it abuse!!), so the system will probably have a hard time detecting it.

2 Likes
20

We should also separate discussion of trust-level-gated limits – throttles that apply because you’re at a lower trust level – from the rate limits we apply to everybody, if we’re going to have the latter.

I think we will probably have the latter, even at a level that would usually be unrealistic – like, no matter who you are you probably can’t post more than once per minute or comment more than once per ten seconds or delete more than five posts per day or whatever. I believe SE has these throttles for the infrequent cases of compromised accounts and rage-quits.

I don’t know what rate-limiting we’ll have, but let’s posit that there will be some baseline. Discussion here should be about trust-based limits, not all limits.

1 Like