As someone who spent a lot of time on the programmers stackexchange beta and watched that community get most of its original most valuable content eventually get wrecked as ‘subjective’ when moderation there went over-aggressive, I searched for discussions of how that might be handled here. I did not find much. Has anything been discussed or decided so far in this topic? If not is this a welcome topic of discussion at this point?
2 Likes
How to handle subjective questions, including how much subjectivity to allow, is something that each site needs to decide. That’s true on SE too, though SE’s history might bring more pressure to bear.
Speaking personally, I think the guidelines in the oft-linked “good subjective, bad subjective” blog post are good general guidance, but they’re a starting point, not the end state for all communities. Some subject areas inherently invite more subjectivity than others. We want to work together with communities to create sites that work for them, which is probably 20% software configuration and 80% “people/culture stuff”, including how we guide people in meeting each site’s norms.
6 Likes
What the article you referenced said, and what happened immediately afterwards were two very different things and in my opinion only the final point about “getting to know” posts being inappropriate was ever acted upon in the way the text attempted to lay out.
npr/programmers/softwareengineering had originally wanted “more subjective” in a lot of areas and as people were replaced a lot of good content was lost as soon as enough “no subjective” power users entered the arena.
The site deciding a thing and the users that come after deciding a different thing will always be a possibility, but some of the common patterns on SE could likely be avoided if a few specific things were made either commonly enforced or static at the point a site is created. Off the top of my head:
Definitions
- a question can have more than one valid answer and not be subjective
- a question can have conditional answers and not be subjective
- a question attracting subjective answers is not by definition subjective
- etc… etc…
Things that might be worth deciding statically at the founding of a site
- asking what the best [insert type] tool for a specific situation is is less subjective than asking what the best [insert type] tool in general is - does the community invite both, none, or just the first?
- does the community allow questions asking for advice in a particular set of circumstances? if they do what form is expected of the answers?
- does the community allow questions where they answer is an explanation of process, questioning of “best practice”, scientific theory etc (answers will come from different schools of thought)
- are questions of a more subjective nature separated from straightforward questions? do they have different voting policy or scoring weight?
- etc…
SE sites get all of these points wrong more often than they get them right.
Allowing what is acceptable to make a dramatic 180 in an established community usually ends the “community” aspect of it. I have answered 2 questions in the last decade since they reversed the point if the whole not programming related/programmers/softwareengineering exchange and I am still in the top 2% of users. They should have just pulled the thing as most of the questions allowed there now would have been allowed on the main site at this point anyway.
I by no means think that this is a majority of people, but there are a sizable minority that have left various exchange sites over this sort of thing over the years and it is an easily avoidable decision/design flaw in my opinion. Ignoring the small groups is how you end up where SE is today… 
I would encourage people to think of ways to avoid moments like this: https://softwareengineering.meta.stackexchange.com/questions/350/the-six-subjective-question-guidelines-enforcement-notice
https://softwareengineering.meta.stackexchange.com/questions/406/why-is-the-discussion-about-good-and-bad-questions-happening-on-meta-stackexchan
I was surprised by this answer – but it is a community-specific or site-specific decision, i.e. what kinds of questions they want to allow or prevent for what range of subject matter.
Perhaps your question is mostly about this …
… i.e. evolution or change in a site’s policy, or the balance of users’ interpretation of its policy.
One possibility, I don’t whether this might be a factor – when the number of users grows, does it become easier (and perhaps too easy) to find any 5 users who will vote to close a question?
From the discussion here: Area 52: Ideas for new sub-site topics - #4 by Lundin
I was on “Programmers” very early on and saw how it went out of hand, to the point where moderators had made the site useless. Then I had my account there deleted many years ago, since it was already a lost cause back then - way too many different expectations. If not even the diamond moderators can agree with each other what’s on topic, then the site is doomed.
It would seem that this has to be handled by the “Programmers” community making a much better job of specifying what’s on scope. Don’t look so much at what the technical communities do, but define an unique scope for the “Programmers” community alone.
For example, it doesn’t make much sense to close down posts on such a site as subjective/opinion-based. That shouldn’t even be a valid close reason, given that things like program design are fairly subjective by their nature. Answers should however be encouraged to be backed up with sources. So rather than looking at how the tech/science sites handle questions, look at the literature/fiction sites. The latter encourage answers backed with sources and discourage opinions - but this is mostly handled by voting, not by closing everything down.
I believe that such a community should rather focus on specifying the allowed topics. Particularly if it should allow topics such as career advise, project management, workplace issues etc that aren’t actually about programming. And if deemed on-topic, how should such questions be formulated, what’s the programmer-specific angle.
Overall, such a programming community needs to be much more relaxed than “software engineering”/SO or we will be better off without it.
6 Likes
As a new pro-tem moderator I reviewed everything posted on Meta, made a faq out of topics where there was a clear community consensus, and posted there again to ask on topics where I wasn’t sure. Being a bit of a rules lawyer, I wanted to know what my rules were. Some decisions were non-SE-standard, but did a have clear consensus, at least among the then Meta users who founded the site.
The main problem was that different users/moderators had different agendas and what was on scope changed over time, variably. That doesn’t work well when the site is already live and finding actual community consensus on SE meta is notoriously hard. Closing down a post and then waving some meta post around will likely end badly.
I think the key is to settle what’s in scope once and for all before the site goes live, then only make minor changes to that over the site’s life time. If that scope doesn’t sit well with a lot of people, then create another site with a different scope. This is essentially how sites like Code Review or Code Golf popped up, when the SO scope was too narrow.
1 Like
Yes.
Also I wonder whether meta-tags might help to solve the problem too, i.e. allow different types of post to coexist on one site, optionally ignored by users who don’t want to see it.
This sounds a lot like deciding site policy for them. Not our job. If you want to suggest policy for a site, become a moderator of it.
Though this does bring up an interesting point; for sites that want to move, are we going to give them a location to discuss their new policies?
1 Like
I think you are close when you say
and
but the bigger problem in the SE approach is that a relatively small number of accounts with the time to spend to get their scores up to the level necessary to moderate can force the evolution of policy in the first place. The good subjective/bad subjective nor even the forced move by the admins to greatly restrict the quality of the subjective posts forced the community to where it went, 10-15 power users with strong opinions about what they thought was appropriate vs what the boundaries had been set to did far more damage.
To me the ease at which the theme of one SE site bleeds into another has always been a defect in the way SE handles the individual communities and you really don’t have to look any further than reddit to see the large scale problems with allowing the few loudest voices in a community dictate a communities policy.
Six or seven years into my career stackoverflow started to be useless to me for anything other than a quick google for some bit of syntax I cannot remember because of the limitations put in place on the questions. I was no longer at the point where I needed advice on things that have a single unarguable answer. If stackoverflow is meant for newbies, so be it, but moderating every site down to the point that no senior person in an industry has much point in participating beyond answering the newbies questions is a large part of why their overall community has a lot of the problems it does. Most people are not Jon Skeet.
Having separate communities with seperate goals is one way to try and avoid that, but not having a mechanism to stop the people that either disagree with the communities goals or are not yet to that point in a topic from bleeding into other sites and altering their purpose has the same effect as a single sitewide rule set in practice.
This is a much more succinct way of putting what I was badly labeling as a static setting for the site. Many things went wrong after programmers started to attract more power users, but changing what was off topic and the close procedures is what really killed it for many.
I was part of several betas on SE, joining during or very soon after the private beta. Communities aspire to nail down scope early, but it doesn’t always work. Sometimes you only realize you need to adjust – or figure out how to adjust – after you’ve seen dozens of questions that exhibit a particular issue. The reason SE has a beta phase, at least in principle, is to allow room for that kind of exploration and experimentation. (The idea falls down when mature sites are “beta” for close to a decade, but the theory was sound.)
The more established a site is the more solid its scope should be, of course; the rules shouldn’t change willy-nilly. Sometimes even old, established sites need to make changes. The key is to first have open discussions in the community to examine what’s happening and what the community would like to do about it. That’s true for scope, requirements for answers (like “back it up”), or anything else.
Yes, each site will have a place for meta discussion, implemented as a category.
6 Likes
I can see this side as well depending on what is involved in creating a separate area/community for similar topics under different rules and if having two similar communities would be detrimental in some fashion. I am not sure it needs to be as complex as SE sites, but if it is not trivial, point taken.
Assuming it is not trivial: would it be better to possibly do something along the lines of increasing some required percentage of user-base agrees with changes as the community ages along with some manner of negative metric for administrative actions done by a user that are reversed by community votes or something? There was never an effective way to handle the conflict between what the community wanted and what the most active power users wanted in the SE communities I participated in once you had a few that gave technically excellent answers, but wanted to alter the community.
In the case of CodeGolf SE we had agreed and then we had to change our collective minds. It wasn’t easy, but most people at least assumed good will and it got done.
1 Like
Note that categories may also help with the issue of different people wanting different types of questions. If there’s one large group of people who want certain types of questions (say, more subjective ones), and another group of people who don’t want them, then one can just make a new category for those questions. Those who want those questions can then use that category, and those who don’t can just ignore the category.
Also, should they later decide to get a new site instead, having the questions already separated in a different category would help as well. It would give an indication on how active the new site would be, it would present a set of policies the new site could build on (by taking what worked and changing what didn’t), and it would give a set of questions to seed the new site with.
2 Likes
It may be that you need to experiment a bit in beta phase to get it right, but the point here is that the scope shouldn’t be allowed to drastically change when you have a mature, released site. That’s what happened to Programmers, over and over. And users/mods didn’t always have consensus over the scope, so everyone was moderating according to their individual believes of what was on-topic.
But this was just one community failing with their scope, Programmers stand out a bit compared to most other mature communities at SE. Changing the scope later is possible, I agree that the key is to discuss such things in detail with the community in advance, to make sure that there’s a big majority of users agreeing with the change.
“Electronics” fairly successfully narrowed their scope somewhat when they turned into “Electrical Engineering”. It is quite a small community and less formal that most SE tech sites when it comes to moderation, scope and rules. So that’s probably why they could change the scope without causing harm.
(The aim was to reduce questions of the nature “what is this electric thingie” and various DIY home electronic questions where the OP lacked the basic electronics knowledge needed to carry out the task they were asking about.)
It was partially because of this name/scope change at Electrical Engineering that “Programmers” decided to change name to “Software Engineering”, though how well that change went, I don’t know since I don’t use that site.
3 Likes
I wasn’t there the whole time, but reconstructing the timeline as best I can, I don’t think that’s what happened. The initial shift away from NPR began less than a month after the site did; the post you linked about enforcing subjective question guidelines is dated Sept 29th, and the first meta question is Sept 1st. You can’t call a one-month-old site “mature”. (For that matter, this more detailed timeline suggests that the basic pivot was more or less complete by the time the site left beta three months later, although a lot of tedious cleanup and reeducation was still being done for years after that.) The famous “toilet bowl” post on MSO (now Meta SE) was posted at about the same time as exiting beta.
The shift that I was there for, years later, from Programmers to Software Engineering, and the accompanying re-emphasis of site scope, wasn’t a drastic change. It was driven by a sustained meta campaign by quite a few users and lots of voters who wanted better clarity to reduce new user confusion and the accompanying extremely high close rates. But it was repeatedly emphasized that, although the site’s “branding” and explanations needed to change, the point wasn’t to change the actual scope all that much, just to explain it much better.
One thing that is worth considering, though, for major policy considerations (i.e., 0-2 times/year): rather than hanging everything on the rather fragile and mutable support of voting on meta policy proposal posts, how about running an actual election and presenting the proposals for the site’s userbase to formally choose between? The downside of this is that it would require either a good deal of coding to run a reliable election semi-automatically, or it would require a lot of handholding by unimpeachable volunteers.
2 Likes
while this was unwelcome by many, this was not where the big issues really started
About 18 months after the initial dictate to alter the site’s policy the “close as subjective” stopped being about question quality (bad subjective) and started toward anything subjective more and more.
After about six to twelve months of that condition, voting to reopen over and over, deleting edits that changed the nature of questions, deleting accepted answers because they no longer applied to over-edited questions participation fell off dramatically by anyone that was asking a question the original site was meant for.
After that point a great deal of things were simply deleted.
Losing questions like “what is the best anime for programmers” was no great loss, losing a lot of the questions about the pros and cons of development practices and team dynamics was a significant loss, especially the ones that had been around long enough people were referencing them in articles and slide decks on the subjects.
You are not a community vetted, authoritative source if the theme of your content can be altered that easily. This was not limited to programmers, it was just the easier to see in some of the smaller communities than the larger system overall. IMHO something to counteract this theme would be a distinctive feature in a new platform.
Wait…wasn’t the whole point of programmers.SE to ask the subjective questions that weren’t allowed on SO?
If my memory is correct on this, then that’s a pretty important thing to consider, I think. I know I dropped off pretty quickly after it started getting muddy about what, exactly the purpose of the community was (especially as a couple questions got closed as I was in the middle of answering them). Looking back, I see a number of my answers (and most of my most upvoted answers) are questions that have since been closed as “off topic” or “opinion-based.”
There definitely needs to be some bounds on what level of subjectivity is or isn’t allowed and how much that allowance can be changed over the course of a community’s life, I think.
Subjective questions are a matter of “are the people who are moderating and curating the site willing to moderate and curate the additional workload that open ended questions create.”
I encourage people to look at MathOverflow and the soft-questions tag which has open questions such as Your favorite surprising connections in Mathematics, Most Interesting mathematics mistake and Mathematical games interesting to both you and a 5±year-old child. The trick is that these questions and the rest of the site are moderated very strictly. New answers on there are not taking up much curation time (it helps that MathOverflow has a special relationship with Hot Network Questions).
The challenge is when the questions provide prompts for poor quality answers that consume more time than the people who are doing the moderation and curation of the site believe that those answers are worth. The tools that they have are flag for a moderator (inconsistent - and not really an exception that the community can’t handle), down vote and delete the answer (often needs a bit of vote coordination on some questions and three 20k users), or close the question as a pre-emptive approach.
Admittedly, those tools are poor for the job at hand of curating the popular content - especially in light of the 90 9 1 problem of participation inequity.
The best way to make subjective questions more accepted by a community is for the people who want to create and read that material (the 9%) to become more active curators of the material so that it doesn’t pose a drain on the time resource of the people who are most active on the site (and additionally become part of that group).
This isn’t something that software alone can do. It needs an active community that wants to read that material and curate it. Ultimately that was the failing of early Not Programming Related and the years in the wilderness for Programmers - those who wanted to create and read the subjective (and admittedly fun) content were not active enough to make sure that short of diamond mod intervention such material was handled by the community (infamous cubicle, cat names, patron saints (meta)).
Just as a public garden has gardeners so that people can enjoy it - and it is those gardeners who decide what to plant and prune, so too is a community driven content site. If people want that site to welcome subjective content, those people should make sure to become gardeners and help pull the weeds when those start growing. The site follows from those who are committed to it.
3 Likes