MVP Discussion: Reputation

[cellio]

This wide-ranging thread has been dormant for a week, so I’d like to try to settle the MVP requirements. Much of the discussion here is longer-term and we should definitely come back to it, but for MVP, here’s what I’m getting from this thread. (See also this one and this one.)

MVP:

• We do not display a reputation number even if we compute one under the hood.
• We do display upvote/downvote counts on posts. Removed here because there’s a discussion elsewhere about scores and whether it should be raw up/down.
• We do display whatever “stats” we’re tracking for privileges on the user profile page. There should be a separate MVP thread for user profiles; this interacts with that.

That’s it. Obviously there are related issues, like how privileges are granted, that need to be settled, but it seems like we are in agreement that a single reputation number is not desirable, though voting is.

If there are no objections within 48 hours, I’ll update the requirements.

[manassehkatz]

I highly recommend that whatever “rules” we come up with for privileges/badges/etc., which could include such items as:

• Points for upvotes/downvotes from others
• Points for actions such as asking, answering, commenting, voting, editing, flagging, etc.
• Cumulative time using the site
• Actions completed within a specified amount of time
  etc.

that the specifics be available for any user to see. That is, if you get a “silver star special user status pin”, you should be able to find out exactly what combination of actions/votes/etc. are needed to qualify for that status. SE actually does a pretty good job (not perfect, as it can be hard to navigate, but the information is (I think) all available) and in the nature of being “open”, it should be available in Codidact too.

[ArtOfCode]

I can get on board with not displaying a reputation number, but I do think we need to display something, so that users can track their progress towards the next milestone, whatever it is. If we’re allocating privileges based on something, we should display that something to the user so they can see how far it is to the next privilege.

[cellio]

Agreed. I was thinking we could do that on the user profile page, not attached to every post the user makes, just for simplicity. (We’ll want to do something better for “usercards” or the like later, once we have a better idea of what stats we’re tracking.)

I’ll add tracking on the user page to the highlighted post.

[gilles]

When a user visits the documentation page for a privilege, they should see where they are on the way to gain the privilege. Maybe there should also be a tab on the user’s profile showing the user’s progress on every privilege, but I’m less sure about that. Would it encourage people to do good stuff or bad stuff? I’ve yet to see evidence that badge hunting does more good than harm on Stack Exchange.

[Corsaka]

I would like to acknowledge the presence of old, incredibly basic questions on SE.

My proposal for those would be that we use a “Community” user to post them all, preventing any one user from receiving massive amounts of kudos/reputation/whatever we're using but still providing the information for possible duplicates later. I think this is essential for an MVP purely because of how influential those older questions are on reputations.

[cellio]

I added the following to the requirements based on the comments here:

Reputation and privileges:

• We do not display a user reputation number even if we compute one under the hood.
• We do display whatever “stats” we’re tracking for privileges on the user profile page.
• User can see their progress toward a particular privilege on the help page for that privilege.
• (more about privileges TBD in other threads)

[Corsaka]

Porting interesting answers from https://forum.codidact.org/t/mvp-proposal-user-trust-and-reward-system/114/:

@Lundin

The biggest flaw in the SE system by far, is that it mixes up domain knowledge with moderator suitability into a single mess called “user trust”. We need to get rid of that system. With this in mind:

• Site experience. Everything you do counts for this - like the SE rep system.
• Domain experience. Votes on answers count for this.
• Moderator experience. Edits, reviews, flagging etc.

@Academic-Quantity

I would still propose separating what’s used for bounties from what your reputation is. So, for example, 10 good edits might give you 1 mod reputation and 1 bounty point. Spending the bounty point doesn’t affect your mod reputation.

Although it feels like overkill to do all of this just for bounties, so perhaps it’s just something to keep in the back of our minds to potentially combine with other functionality.

[klutt]

@gilles

This is a difficult question. If you think there’s an obvious answer, you don’t understand how human beings work — specifically, you underestimate how different people work differently.

Perfectly true.

@gilles

Reputation (or karma or EXP or whatever you call it) has positive and negative aspects. It’s a motivator for some people, because they like having bigger numbers next to their name. But that’s a motivation to produce content that gets upvotes , with an incentive to produce popular content in preference to useful content. There’s a reason why the term “rep whore” is (or was) common on Stack Exchange meta sites: people who seek reputation for reputation’s sake often do not make the site better.

What you’re saying here is also somewhat true. But the problem I see here it that you seem to imply that this is binary and mutually exclusive. That there are the good (who does not care about rep, and only tries to make the content as good as possible) and the bad (rep whores who do anything to get some reputation)

The reality is certainly not binary. Most people are somewhere in between. I do enjoy the gamification on SE. I don’t know why, but I do. It’s fun and makes me motivated. But that does not mean that I do anything to get some reputation. I have deleted upvoted answers several times. Sometimes because they were plain wrong, and sometimes because I thought they simply did not add anything to other answers.

I can also admit that I did some badge hunting. For instance, the first time i put a bounty on another persons question, it was primarily to get a badge. But at the same time, it also kind of showed me that this is something I can do, which made me put a bounty on several other questions later.

Same with badges for flags. Sure, I did some flagging with the partial purpose of getting the badge, but I have never flagged anything that I did not think deserved a flag. This made me better at flagging, both in frequency and quality, which persisted even after I got the badge.

So it’s not as simple as posting to make the world better xor posting to get rewards. It’s or and not xor.

@Marco13

I agree that it may appear to be “shallow” to care for internet-points. And it’s tempting to say “I’m here only to (altruistically) help others”. But there is a reason of why these persons still do not turn each and every answer into a “Community Wiki” answer, and not publishing each and every code snippet with CC0/PublicDomain license: Attribution and confirmation are important.

This!

I have seen many high rep users on SO complaining about people caring for rep. Still, they do not make their answers into community wikis…

[Corsaka]

If you spend a lot of time on an answer, you’re not gonna want it to be a community wiki when other people could alter it and make it incorrect.

[Olin]

@klutt

I do enjoy the gamification on SE. I don’t know why, but I do. It’s fun and makes me motivated.

Yup. There are several important benefits from having some number (let’s call it “rep” for simplicity) that you can make higher by contributing more site content:

1. It's fun, and that motivates some people to produce more content.
2. It can be competitive. That motivates some people to put in more effort to overtake the "other guy", or not let him overtake you.
3. It gives you something to aspire to when you're new to the site and see experts with a certain rep.
4. It provides public recognition for having provided a large amount of free content.
5. With really high rep, you get somewhat of an "expert" status.
6. It gives others at least some idea of the trustworthiness of your answers.

I admit to having reasons 1-5 motivate me at various times, and I know I’m not the only one.

The objections I see here are mostly:

1. People shouldn't care about rep. They should be helping for the sake of helping.
2. It motivates people to only do things that build rep, not help the site.
3. It's not meaningful because there are too many ways to game the system.

These are very poor arguments relative the significant benefits (at least on some sites, more on that in another post):

1 - This is largely a religious or philosophical argument. People will care about what they care about. The site isn’t going to change that. The only choice we get is to work with what people actually do out there, or not. We’ve heard here again and again from people that admit that rep on SE motivated them to contribute more. What some beliefs say the motivation should be isn’t relevant when we know what the motivations actually are.

2,3 - Yes, no matter what system you create, there will always be some corner cases where it can be “gamed” a little for a user to get the reward without the system getting the desired result. However, these are edge cases at best. You don’t accumulate real rep by gaming a few 100 points here and there or getting lucky with a HNQ hit. The only way to get real rep is to write many consistently good answers.

Also, these supposed “gaming” moves are still mostly doing what we want you to do. You might get proportionally more points for writing an answer on a popular topic than a deep answer on a niche topic. But, note that you still contributed and wrote a good answer. In the long run, these things balance out.

Let’s not throw out the system just because it isn’t perfect in all aspects.

That said, getting rep should be limited to votes on answers only. We don’t need to motivate people for asking questions. They are already doing it to get answers. The people providing their free and sometimes expert time providing answers need and deserve something in return.

[Olin]

Analisys of rep on different sites

I think that some of the objection to rep comes from different site cultures where rep wasn’t that important, or not much of a motivator for the top users. I’ve thought about this a great deal, and it seems that certain site characteristics make rep unimportant, while others make it vital.

My main site on SE was Electrical Engineering. There, rep was a strong motivator for most of the top contributors. The site simply wouldn’t have been anywhere near as successful without rep. However, on other sites that I participated in, rep didn’t seem to be all that relevant. So, what are the differences, and why?

It seems that sites where the top content contributors are professionals in the field are those where rep matters. These people are there because they are interested in the topic and probably at least somewhat enjoy teaching it. But, that by itself isn’t enough for more than the occasional answer here and there. Rep provides the public recognition as a reward for lots of good contributions. That kind of recognition is valuable to these professional contributors, and draws out a lot more participation than otherwise. I know this from personal experience, and I’ve talked to enough other EEs to know that I’m not the only one.

As evidence, take a look at the users on EE.SE with 100k rep or more. I just checked, and there are 8 of them, and every last one is a professional in the field. Also note that the top 6 users all give their real names, mostly right in their user names. The top 6 also have their real mug shots as avatars. Unlike most users on most sites, they are not trying to hide. In fact, they want you to know who they are.

The one possible exception to the rep is important to the professionals rule may be where those professions already provide other means to be known among their peers and potential “customers”. This is the case for academics, for example, with their scientific paper publishing network. A good example of this kind of site is Physics. The top contributors are academics in the field, but I don’t know whether rep is much of a motivating factor for them. I noticed a few refugees from Physics are here (@dmckee), so maybe they can chime in.

An example of a different type of site is Photography. There, the top users are amateurs with day jobs in different fields. I always thought it was disappointing that there weren’t more active professionals. It didn’t seem to me that rep was much of a motivator. I know it wasn’t for me. I was there just for fun. However, also note the significantly lower participation rate. The top user has only 150k rep, and there are only 3 over 100k. The top users are also less active in terms of answers/day, don’t show their real faces as much, and don’t use their real names as much. This all fits with being there for fun rather than viewing it as a professional activity. Despite that, we have one user (@mattdm) who stated that badges motivated him. This again fits with the non-professional. Badges are shiny things to collect for fun.

Conclusion

Rep doesn’t matter much on some sites, but is critically important to other types of sites. The software must recognize this and be built with the capability of computing and displaying rep. Sites that are really against rep can have the ability to turn off the display (there is no harm in the software still computing rep, so just turning off the display is less complex).

I suggest that rep here should be the sum of votes received on all answers by a user.

[klutt]

@Olin

1 - This is largely a religious or philosophical argument. People will care about what they care about. The site isn’t going to change that. The only choice we get is to work with what people actually do out there, or not. We’ve heard here again and again from people that admit that rep on SE motivated them to contribute more. What some beliefs say the motivation should be isn’t relevant when we know what the motivations actually are.

…

Also, these supposed “gaming” moves are still mostly doing what we want you to do. You might get proportionally more points for writing an answer on a popular topic than a deep answer on a niche topic. But, note that you still contributed and wrote a good answer. In the long run, these things balance out.

This is something I think is important to acknowledge. There have been plenty of situations where I have been thinking “Hmmm, what’s the ratio for the reasons to why I did this? Is it 50/50 rep/contribute? Is it 20/80? 70/30?” and often I come to the conclusion that it does not really matter, because no matter the reason, it was a contribution.

[luap42]

@Olin

Rep doesn’t matter much on some sites, but is critically important to other types of sites. The software must recognize this and be built with the capability of computing and displaying rep. Sites that are really against rep can have the ability to turn off the display (there is no harm in the software still computing rep, so just turning off the display is less complex).

As I understood the prior agreement, we do plan to have something like rep (whether community-wide or tag-wide, this probably also depends on the community), just not for the first version (MVP). The reason for that is, that taking something away is always harder than adding something (loss-aversion). If we introduce reputation now, it will be hard if not impossible (not technically, but from a community and consensus-building perspective) to remove or even change it later. If we add something later, possible with some smaller experiments marked as such, we’ll be able to build a better system, which people actually want, which is tested and which doesn’t need to be removed after a few weeks or months, because it causes problems.

[Olin]

@luap42

just not for the first version (MVP). The reason for that is, that taking something away is always harder than adding something

Sorry, I don’t buy that argument. You’re saying you’re not sure how it should be done, so don’t want to get it wrong, and that you want to “experiment” before taking it live.

Experimenting means different things might be tried, so at least some will eventually be taken away. People will have a lot more tolerance for things changing earlier than later. There is no reason rep should be different from any other feature that is initially brought out. They are all essentially experimental until we get some experience with them. There will be tweaks to various algorithms and settings. Rep computation is no different.

If you’re not sure how it should work, then it is even especially important to bring it out early and get as much experience with it as possible, while the users are still tolerant of changes. Changes to the rep system clearly aren’t going to take rep away. They might change the computation formula, or change what contributes to rep, or maybe how exactly it’s displayed, or something. There is no reason that should be any different to users than any other change.

On a personal note, I was hoping to contribute here by spinning up an awesome electrical engineering site. I’d work with you to beta test the software and feed back results from real experience. However, rep is very important to that kind of site. I know several experts that I was going to personally invite, but minimum required basics of the site need to be there first. I have no problem with some details changing, but it has to be there in some form.

The kind of site that needs rep is something that should be in the mix as early on in testing as possible, if for no other reason than it’s represents a different set of site tradeoffs than everyone else seems to be imagining.

I’m also worried that “later” never comes, or won’t come for a while. Implementing a basic rep system is really pretty simple in the scheme of things. You already will have votes on answers. Summing up those votes received per user and displaying that next to the answer author’s name sounds like (I’m not a web development expert) relatively trivial compared to many of the much more complicated things that need to be in the first beta release.

If it needs to change, we’ll change it. This will be something communicated to the users up front. That is no reason to avoid implementation.

[manassehkatz]

I will fully admit to being motivated by Gamification in general and Reputation on SE specifically. That is totally normal (well, I hope it is!) and I am sure many others are motivated similarly.

However, once I reached a certain point, the reputation became far less of a factor for me. Do I notice the points and get a little bit of a good feeling every time I see +10? Absolutely. Does that significantly affect my desire to answer questions? Not much any more. (I’m no longer answering questions because of the SE actions over the last several months - but if those problems went away and at the same time they said "over ‘x’ rep and nobody will see it any more just “top 10 users” or whatever, that would be good enough for me).

So while it is clear that you are (or were) a big participant in EE based on the reputation, I would not be at all surprised if many of the other high rep. users (on EE and other SE sites) simply do not care about rep once they reached a certain level.

That doesn’t mean rep. is not a motivator. I am just saying that I think the highest rep. users once they’ve reached a high level may well be the ones where rep. matters the least. Though of course getting there is still a real issue.

[Olin]

@manassehkatz

So while it is clear that you are (or were) a big participant in EE based on the reputation, I would not be at all surprised if many of the other high rep. users (on EE and other SE sites) simply do not care about rep once they reached a certain level.

I think this depends on the nature of the site and the type of core users. There can be some competition going on, even if unspoken. I probably wouldn’t have cared about rep once I got past 100k or so if it weren’t for a few other people close on my heels. I mean, geesh, we couldn’t have Andy or Spehro have the top rep on the site, now could we!?

But, whether you care past a certain level or not, the point is that rep is still very important for some types of sites, and certainly drew a lot more contributions from people on SE than there would have been without it.

[tuggyne]

@Olin

The only way to get real rep is to write many consistently good answers.

A small, but important, correction: the only way to get real rep is to post many consistently good answers.

It is not always important in practice to write all that many.

The margin between these is answering easily-recognized duplicate questions with reposts and simply plagiarizing what others have written. The first helps only the asker and the answerer at the expense of later searchers, and is therefore a net loss. The second is obviously unethical. Either or both together can be routinely exploited (on a sufficiently large site) to gain tens or hundreds of thousands of rep, so it’s dangerous to simply write them off as a rounding error, as you would for the occasional, unpredictable few hundred rep that weren’t quite justified.

[Helmar]

We should also consider that there’s no single community based site of relevant size out there without a working recognition/reputation mechanism. It’s the driving force behind Wikipedia, Reddit, SO/SE, Facebook, Instagram and all the others. Every platform has some kind of aggregated “value” of the user “contributions”. It doesn’t matter if it’s a direct one (reputation / karma) or an indirect one (follower) every platform has one and not having one on Codidact seems an unreasonable gamble.

Especially since a user page design “as copied from SE” as Codidact’s doesn’t hide rep. It just puts behind a “useless” click. So I would actually put reputation or Q/A Count + Averages (which is basically the same) on the user card / post and if we really find a better metric later we can still “move reputation” to the profile and put the “better thing” in the user card.

[Helmar]

@manassehkatz

However, once I reached a certain point, the reputation became far less of a factor for me. Do I notice the points and get a little bit of a good feeling every time I see +10? Absolutely. Does that significantly affect my desire to answer questions? Not much any more .

There have been studies about editing Wikipedia and how it compares to addictive behavior. Likely the same applies to Q&A once one has caught the bug.