MVP Proposal: User Trust and Reward System

We shouldn’t ever hard-delete posts, unless required to do so by laws.

Developer access should be a separate “user type”/boolean field.

5 Likes

Yes, and that’s why it’s not given to Deputies.

The first time flagging is mentioned in your post at all is trust level 3, which also states they can VTC directly instead of flagging.

Does this mean that flagging would be a perk given at trust level 0?

Ah, I should add that to trust level 1. Knew I’d forgotten something.

This seems low to me, but I’m not against having a capped limit. Perhaps it could be 5 comments on a question/answer that is not theirs, but on their own question page they shouldn’t be limited because that may be frustrating if the user is trying to clarify something or answer questions that had been asked in the comments by other users.

1 Like

I assumed that was already implied. Too much experience on SE where we can always comment on our own posts. Will edit and update again.

1 Like

Maybe it was implied, but that doesn’t mean we shouldn’t spell it out for clarity and to make sure we are all on the same page :slight_smile:

1 Like

General points:

  • I like the idea of a trust level system - it’s granular enough that it works well for MVP, and it leaves room for expanding to multiple “tracks” at a later date for different types of privilege (posting vs. reviewing vs. moderating vs. editing, etc), if we want to do that.
  • Most restrictions should be removed for OPs on their own posts, whatever trust level they’re at. For example, the 5 comment limit at TL1 should exclude comments on your own posts.
  • Tracking time spent on the site is… well, not difficult to do, but it’s unusual and I’d consider it unnecessary extra work, especially for MVP. Restrictions should be set so that they by their nature require having spent time on the site - e.g. the “5 well-received questions” at TL2 necessitates spending a while on the site, because asking 5 questions takes at least 3 days based on the restrictions for previous levels.

This feels low. 10, maybe?

This is high, especially if you’re looking at these as AND requirements rather than OR. I’d suggest default values of 100-150, 50-75, and ~15. We need to strike a balance between ensuring people have the experience to use the site effectively, while also not making people work too hard to access extra tools, otherwise we don’t have enough people moderating to make the site work.

These values should probably all be made configurable items, rather than hard-coded, so that we and anyone else using Codidact can chop and change them as we want.

Temporary locks I can get behind, and perhaps we want to consider something akin to protection on SE as well, perhaps at TL3.

Blocking or suspending users should be left to full moderators - they’re the people who the community have voted for. Users at TL4 are, effectively, still “regular users”, and letting unvetted, unvoted-for users suspend even temporarily doesn’t sit well with me.

Why? What’s the use-case for this? We shouldn’t be granting exceptions to the requirements for the next trust level (i.e. no promoting up), and if users are being a problem we should be using suspensions or disabling specific feature access rather than changing trust levels (i.e. no demoting down).

As @luap42 said, we should not be hard-deleting posts. The only people with the ability to hard-delete should be those with database access (or possibly through a “legal/compliance/maintenance tooling” UI separated from the site). Speaking of which…

We don’t need this one. Rather than having a 6th trust level, we just need an “admin” flag on user accounts which serves as a total override - if the admin flag is set on your account, you can access everything and all restrictions are removed, no matter what trust level you’re at. This can be set on staff and developer accounts (i.e. those running the site), and temporarily set on review board members when acting in their enforcement capacity.

7 Likes

That was the intention for TL3+ - customizable to the website you want.

I’ll add these to TL5 - could you expand on them slightly?

That was sort of the point of TL6. I think integrating it into the TL system would be much easier than creating a full new indication of administration.

This is great! We can quibble over some of the specific numbers, and I share @ArtOfCode’s concerns about raw “time on site” being a factor, but those are details. Let’s do this.

Constants should be adjustable per-site, not just per-instance. I’ve been on sites where it would take months to be able to raise a hundred or more (legitimate) flags. We’ll want communities to be able to scale flag and review requirements with the activity of the site.

6 Likes

The only tool we have to prevent problematic behaviours on SE is suspension. It’s a very blunt tool. We can message users, but ultimately if they don’t comply we can’t just disable their ability to comment or review; we have to suspend them. I’d like us to be able to disable specific abilities or groups of related abilities in Codidact.

On the contrary - creating an extra trust level requires integrating into the requirements system, but the concept of a specific access list doesn’t fit very well into that system. Adding a single boolean field to the user table, on the other hand, is dead easy and can be integrated into permissions checks very easily:

public class User {
    public boolean HasPrivilege(string privilegeName) {
        return this.IsAdmin || 
            Array.Exists(this.Privileges.Select(priv => priv.Name),
                         n => n == privilegeName);
    }
}
1 Like

This seems overly restrictive on answers. Most bad content we want to avoid area bad questions. Yes, bad answers definitely happen, but not as much. They are also easier to deal with as they are rated by the community relative to other answers.

Throttling back what you can ask of the site until you’ve shown you can ask without damage makes sense. But, I’d be rather pissed if I came to the site, see a bunch of questions I know I can write good answers for, then have the system prevent me. Maybe throttle back answers only as long as there aren’t any “negative” (how ever that is rated), until you get the total number of answers up enough so that we can rate you otherwise.

Again, questions and answers don’t have same quality issues and potential to damage the site.

Once the total number of posts gets high enough, you should be measuring the average or ratio, not absolute numbers. 5 well-received questions plus 10 crappy ones is very different from 5 total questions, all well received.

3 Likes

This makes sense from the perspective of someone who just wants to answer, but in overall by the numbers sense bad answers are a dime a dozen while bad questions that are so irredeemably bad that they can’t be fixed by a little editing and need to be deleted are much more rare.

I mean if you look at the deleted answers underneath a protected question, there’s a lot of junk down there.

If we are worried about overall quality, there is as lot more bad answers than bad questions.

Voting can push bad answers out of immediate view, though. Questions, on the other hand, are there for everybody. (We’ve been avoiding votes and scores for questions and thus can’t use that to affect visibility. Maybe we’ll decide to do that later, but we haven’t felt we needed it yet, especially for MVP.)

I agree with applying the limit just to questions – what you can ask the site to do for you – and not also to answers. We also need to talk about rate limits in general to counter aggressive spambots, but the rate limits we use shouldn’t affect normal, constructive use of the site by actual humans.

4 Likes

That’s a very uneven comparison. The reason to rate-limit is to prevent damage to the site. Bad questions do this, especially when there is a lot of noise resulting from attempts to fix them.

Bad answers, on the other hand, get downvoted and pushed to the bottom. They cause relatively little damage to the site.

Then there’s the slap in the face of well-meaning volunteers that are trying give by answering questions. By all means deal with those that have a history of writing bad answers, but don’t start out by pushing the volunteers away.

Basically, to post questions the burden of proof is on you that you can post good ones. For answers, the presumption is you will post good ones until you show you don’t.

4 Likes

Of the questions Askers on Mi Yodeya, 64% having only asked a single question. 15% have asked more than 5. I just don’t see a flood of bad questions coming in from a single user.

2 Likes
  • Forget “hours”. There are technical & people reasons why I don’t think it means much (few hours can still be “a lot” and many hours can be “nearly meaningless”).

  • 150 flags, 50 edits, 15 Q/A

This seems to be a very high bar. That is especially the case for flags. 50 (or 100) edits isn’t that big a deal to get to, IMHO and 15 Answers also seems reasonable for the Constable. But 150 accepted flags? That’s a lot of flags. Arguably, there shouldn’t be that many times I need to submit a flag (let alone have it accepted) - perhaps more definition of that is needed. On the other hand, Edits are part of the improve the Q & A directly process.

I suggest having one more level between 2 & 3 (calling it 2.5 but obviously it would instead be 3 and change the other #s and get some fancy name). I know we are trying to keep it simple, but having a handful of levels (even adding 1 or 2 more) is still far fewer “things” than the many badges & privileges on SE. Getting back to my Editing ides:

  • Minor Edit Approval - Allows a user to review Minor Edits (queue and/or when viewing a Q/A) and make one of the required approvals. Level 2.5
  • Major Edit Approval - Allows a user to review Major Edits (queue and/or when viewing a Q/A) and make one of the required approvals. Level 3 Constable
  • Editor - Allows a user to review Minor Edits or Major Edits and approve immediately without any additional approvals needed. Also allows editing of a Q/A without requiring approval from anyone else. Level 4 Deputy
2 Likes

All levels should be community specific. A bunch of reasons why, won’t spell them out right now. But I could see bypassing it, in a sense, by using “Association Bonus”. Perhaps “Level 2 or above” on one community gives you automatic Level 1 on any additional communities.

2 Likes

To be fair, there were some persistent trolls whose questions are no longer visible. I don’t have stats on that; it hasn’t overwhelmed us or anything like that, but the rate limits built into SE (which I think are more like one question per hour, not per day) definitely helped us.

1 Like

What about having multiple triggers instead? So you can gain a trust level, by either having 20 high-scoring posts, or by having 4 high-scoring posts and 20 good edits, or…

High scoring means positively received, not necessarily high in comparison to others.

1 Like