Proposal: Reputation per tag

I think the discussion about how to handle reputation (points connected to an account) is not finished yet? I’m a fan of giving points, but not reputation, to questions - while giving points & reputation for answers. If we go with something like that, I propose the following:

Strongly relate reputation to tags.

On the account page
Show a score per tag.
I don’t consider this part that important, but showing “this person knows about X, Y, and a little bit about Z” is better than “this person knows about stuff”.

For an answer
Show a score calculated from the reputation for the question’s tags.
This way, we display a value for knowledge about the question subject, instead of just general knowledge. Calculation algorithm to be done.

Example
Using a simple average as calculation for demonstration
Question: Tags “C#” and “Database”

  • User A: 500 rep C#, 500 rep Database
    = 1000 rep total vs. 500 related rep average
  • User B: 200 rep C#, 100 rep Database, 1000 rep version control
    = 1300 rep total vs. 150 related rep average
  • User C: 3000 rep Ruby, 1000 rep jquery
    = 4000 rep total vs. 0 rep related average
4 Likes

We’ve elected not to have/display a single reputation number, but to use trust levels instead: see the wiki for details. This idea still has potential, though—we could use it as part of the trust level progression (think “at least score +50 in at least one tag”).

Or something else we could do is to show something like “python topic expert” below the name of an answerer, if they have a certain score in a tag.

3 Likes

Oh, I thought that was about moderator reputation
(probably because it mainly consists of moderation privileges).

But yeah, same idea, why should somebody be considered “trust level 3” on a Java question, if they have shown no Java knowledge whatsoever?

1 Like

I would consider the trust level not to be an indicator of expertise, but a level of trust regarding the responsible use of moderation tools, regardless of topic.

A user that has “proven” that he/she doesn’t create spam post would for example be allowed to create more questions / answers.

8 Likes

Yeah, exactly; it makes sense for moderation.
To decide if they have certain moderation privileges.

However, if there’s 2 different answers about a certain topic, with no votes yet…
Well, it’d be a shame if we make people go with the one who fixed more typos, when we have better metrics available.

I want to make sure I’m understanding this correctly. Your proposal is to track rep earned by each user per tag, then when that user writes an answer, show the rep of the answerer based only on the tags on that question?

That’s an interesting concept - in principle. It hinges on how well questions are tagged, which has always been a problem area on SE. Still this does sound workable. One way or another, there needs to be some “awesomeness metric” associated with answerers. This fosters competition and provides something to aspire to.

OK, but what are you going to provide to reward experts, foster some competition resulting in more and better answers, and provide something to aspire to?

1 Like

Not necessarily “track by tag”.
I thought an answer upvoted to 20 points means you have 20 points for each tag on the question.

Say you have

  • 20 points answer for a question tagged with “C#” and “Database”.
  • 20 points answer for a question tagged “C#” and “LINQ”

So your rep is

  • C# => 40
  • Database => 20
  • LINQ => 20

New question tagged “Java” => you have 0 rep
New question tagged “Database” => you have 20 rep
New question tagged “C#” => 40 rep

Let’s say we calculate rep with a simple average:
New question tagged “C#” and “Database” => 30 rep
New question tagged “C#” and “asp-net” => 20 rep
New question tagged “Java” and “Database” => 10 rep

7 Likes

OK, that’s what I thought you meant.

Seems like an interesting idea I hadn’t heard before. You don’t have a single rep number that follows you around everywhere, but instead a “context sensitive” rep number. I’m willing to give this a try.

One important point is that a single rep number is still displayed in any one instance. If you really want to see the breakdown of how rep was arrived at in a particular instance for some reason, you have to go to the user’s profile and do the math. This keeps it simple and still provides the incentives to produce good content.

The only downside I see is that now it’s more important that tags be used properly. On SE, I always thought tags were silly and they also seemed to be a never-ending source of trouble for those that actually took them seriously. With this system you can’t just blissfully ignore tags anymore, since now they actually matter somewhat. Still, I think this is a workable problem.

1 Like

If you stick a number next to a username, people are going to work to make that number larger. In this case, tying it to a tag is going to encourage way broader tags because that will make the number go up.

I also see it making re tagging much harder because you would be taking points away in some cases.

4 Likes

Sorry, but I don’t understand.
Say I’m a user and want to exploit this, how am I going to do that?

As long as the retagging was justified, the lost points were not, were they?

1 Like

Right. Those that want to be competitive will write more answers and try to make them better than the next one. That’s exactly what we want them doing.

I think you are implying that this might also lead to trying to game the tag system. That would be a fault of how tags are created and curated. Frankly, trying to game the tags seems like a pretty far fetched loophole. It sounds like a lot of work for little gain, and it’s all out in the open so hard to get away with much. You can’t mess with tags until you’ve earned some trust, and there will be a lot of eyeballs watching. I really don’t see this being anything more than a minor annoyance at most.

3 Likes

But it would make them go up for everyone, so no real gain for any one user.

Finding the right level of specific versus general is going to be more tricky than before, but this has always been an issue. There will probably be combinations of general and specific tags, and it is up to question to include all that apply. This does make editing tags on questions more important because it is often hard to know what tags are out there, and new users in particular won’t be good at knowing which ones to include. Again though, this problem is nothing new.

1 Like

…which is something the community on SO has been quite good with correcting. Adding, replacing and removing tags are things I often saw when watching a question revision history.

1 Like

We’ve had this discussion elsewhere already. I forget where, but I’ll have a dig later on and try to find the link. I believe the conclusion was that a reputation like concept was not necessary for this.

2 Likes

Yes, absolutely, but this introduces misaligned incentives. The specific user(s) losing points would like to not lose points, and this now conflicts with the general desire to have an accurate tagging system. What’s more, the specific user doing the retagging may have to overcome their own sympathy for the specific user(s) losing points. Worse yet, the rep-preserving incentives are more focused and concentrated than the accurate-tagging incentives, which makes them more likely to be satisfied. (Think “special interest group”, which has the same structural problem in politics.)

Unless you edit tags on questions that you didn’t answer to make them more specific.

No, this is not an efficient, sensible thing to do, and in fact probably wouldn’t be at all common, but that doesn’t mean some jealous malcontents might not think it’s happening and use it to raise a bigger stink about legitimate, unbiased tag editing that happens to reduce their rep.

People come up with all kinds of crazy conspiracy theories, and while we can’t stop all of them, we can at least refrain from giving them such golden opportunities.

2 Likes

I don’t really like the idea of reputation per tag any more than SE’s method of calculating reputation.

If we are going to use a metric to determine expertise on the site or something, I think trust levels is good. If we want something additional, then I think perhaps something like gold badges as a metric could be helpful, similar to how people running for moderator on SE had to have certain moderation badges.

When I first joined SE, I definitely “farmed” for reputation, and it was more about the rep than about helping people. I think making it about the “score” is a negative, personally. Even if it motivates people to ask, answer, etc., it doesn’t motivate them to be selfless, because it’s all about their reputation. If someone cares most of all about their reputation (used in a non-score context here), they don’t have the community’s best interests in mind.

In a way, that’s what SE’s problem is, with the leadership trying to save face (their reputation), but not in a way that is sincere or humble or focuses on putting the community first.

2 Likes

So what? There are probably many different motivations of particular users out there to provide particular answers. There is no reason we should care, as long as good answers are being provided. And conversely, if people are providing good answers, let those that are so inclined have some fun with trying to get a higher score than some peers they respect, who are all doing the same thing. As we analyzed in some other thread, for some sites this dynamic won’t matter. For other sites, particularly those where professionals in the topic want to be seen as experts, it is essential.

My “main” site was Electrical Engineering where this mattered. The top users were all professionals. Without high rep to aspire to and get just a little more than the other guy, the site would not have been as vibrant.

2 Likes

The more rep is emphasized, the more Goodhart’s and Campbell’s laws distort the value of the system. Rep cannot be perfectly coupled to actual value provided to the site, so to the extent it is pursued as an end in itself, the site necessarily suffers.

Tag-specific rep is at least a little less likely to be a problem than SE’s universal rep, but it’s still something to watch out for.

4 Likes

You’d only be taking away rep if the tag was pointlessly added. If users complain about that, tell them to get rid of the tag before they answer the question or wait until someone else does it.

2 Likes