Question closure in MVP

Continuing the discussion from Proposal: Closure is about "select all that apply" instead of "select one reason":


As part of quality control/moderation we will probably need a way to close questions (or however we call it). Closed questions cannot be answered. Closure is intended to prevent questions from being answered, when this would lead to some harm (such as: extended discussions, wild guesses, content duplication, …).

My prior post didn’t lead to either discussion or approval, maybe because it was too specific.

So here is my new question:

How should question closure in MVP work?

1 Like

Suggestion:

  • ‘x’ high-rep users agreeing on it
  • Moderator (does that need multiple? not sure how it works in SE right now as I’ve never been a moderator)
  • Maybe let a user close (as opposed to delete, which of course they can do) if there is already a VTC - e.g., if someone high-rep (but not moderator) VTC duplicate, let the OP close it without having to wait for more VTC.

Is that really MVP? At least @gilles said that voting can be added later. I think for now giving trustworthy users binding close powers is enough, if we have something to prevent close-wars (such as a rule as only one vote per question).

1 Like

Not voting per se. I mean the thing (don’t know exactly how it works, but I have participated now that I have sufficient rep - I could look up the details but not right now) where users with a certain rep can VTC. If ‘x’ (5? I am not sure of the number) VTC then it closes. Not a “vote” of “I want to close it or I don’t want to close it” - just an accumulation - i.e., a question could sit forever with 1 or 2 (unless they ‘age out’) but once it hits 5, it gets closed with no ‘higher moderator’ action needed. The frequent users will see the questions and if it really is a good reason (typically a clear duplicate or a very off topic question) then it gets closed pretty quickly. And if it is just one person’s “I don’t like it so I’ll VTC even though that really isn’t warranted” then it has no effect.

We talk about the same thing. :smiley:

VTC == vote to close

I’m distinguishing between Vote To Close which is more of a “when there are enough, just do it” and “vote for someone/something” like “voting for new moderators” (in SE) or “voting for a Tech Lead or Team Lead” (in Codidact) etc. There are two big differences:

  • VTC doesn’t have an alternate - you can’t say “No”, you either stay quiet or vote “Yes”
  • VTC doesn’t get broadcast to a group of people (“Come vote now!”). Anyone who sees it (and has the right to vote, however that is defined) casts a VTC if they agree. Or can be in a review queue, etc. But it isn’t sent out to everyone in the voting group, it isn’t put on a banner on the site page, etc.
1 Like

I think VTC is a term coined by Stack Exchange users, however we definitely talk about the same thing when we talk about voting/VTC.

This talks about giving trusted users binding close powers (similar to moderators on SE) instead of using a voting algorithm (VTC)l.

For the MVP

  • Some questions cannot or should not be answered. So we need a way to close questions.
  • Closed questions may be edited into something answerable. Questions may be closed by mistake. So we need a way to reopen questions.
  • There’s no way to salvage a question or to learn how to write better questions if you don’t know why your question was rejected. Therefore closing a question must state a reason which is visible to everyone.
  • We need to have some initial policies around closing. This can be a predefined set of reasons, or some more general guidelines allowing closers to write their own reasons.
  • Refering to the existing answers when someone asks a question that’s been answered before is essential to distinguish a knowledge repository from a discussion forum. Therefore “duplicate of [link]” must be a close reason.
  • Who can close? We can start with a co-opted set, or with some simple measure of participation (tag-based?). I don’t think we can let everybody close, even with a voting system.

Not in MVP, but later

  • I don’t think voting is part of the MVP, but it should come soon after. If we already have voting, that’s fine, but it’s a non-trivial feature that shouldn’t block the first deployment.
  • Some protection against close/reopen wars beyond flagging.
  • Fine-tune who can vote to close and how many votes it takes. I think we can go further than SE in taking tag competence into account. The ideal settings probably depend on the community.
  • Have a process around the selection of close reasons. Something I’d like to try is that all close reasons should link to a meta thread that explains why questions of a certain type get closed, and what askers can do (add details, ask elsewhere, etc.). (Exception: closing as duplicate links to the duplicate, not to meta.) To be determined: what the process is to make a meta thread a close-reason meta thread.
  • Review queues (for closure and other things). Of all the features that Stack Exchange has implemented, I think that’s one of the most useful.
3 Likes

Honestly interested in what’s the harm in having a discussion as long as it sticks somewhat close to the OPs question?

1 Like

The primary harm of discussions is that they distract from answers. If there’s discussion in addition to answers, that’s ok — and that’s why i advocate a hybrid system between SE’s comments and chat. The problem is when the answer gets buried in discussion. Surely you must have seen these forum threads that go on for hundreds of pages, and there’s a plausible answer on page 3 but in fact it became obsolete a couple of years ago and the current answer is on page 176, but good luck finding it. Q&A sites are much better than discussion forums at being a repository of knowledge because there are no answers to answers: replies to answers are second class, only replies to the original question get prominence.

If a topic calls for back-and-forth discussion rather than a straight answer, it isn’t suitable for the Q&A format and needs to be closed.

2 Likes

Liking it, sounds fair! :+1:

1 Like

I disagree with this.

In DIY (and I’m sure in some others as well), there are often either fairly complex questions where OP starts off not quite knowing what exactly they should be asking and X-Y problems where they ask something in a convoluted way and it turns out there is really a better question that they should be asking that triggered the question they did ask.

In both of these cases, and I am sure some others as well, quite a bit of discussion via comments is often beneficial to figure out the real goals of OP, which will often result in significant revision (generally a lot more detail) to the Question and much better Answers than would otherwise be provided.

Maybe the difference is between “discussion about the question” and “discussion inspired by the question”. Kind of hard to come up with a really good specific case right now, but it most definitely happens. Simply asking for “add more detail” or “post pictures” (which are almost automatic comments for DIY questions) doesn’t always result in the level of detail that a real discussion (via comments) generates. When that happens and someone jumps to “move comments to chat”, it is actually counter-productive because those valuable details get lost - at least in DIY, chat just doesn’t get used much, and new users have no idea about chat.

1 Like

Isn’t this a case where you say: hey, we need to work out what you’re really asking here – let’s put this on hold so people don’t spend time on answers that miss the mark, and we’ll sort it out in this comment discussion, you’ll edit the question, and we’ll reopen?

4 Likes

In a sense, yes. But “On hold” as “unclear” or “too broad” or similar sends the absolute wrong message. It gets interpreted as “You don’t know how to ask a question, go away.” That may not be the intention, but that is the result.

I (an experienced SE user in general, but new to DBA where I asked a question) got “On hold shopping” and revised to make it better…and while I did get one nice welcoming comment after that, I can’t even tell if how many, if any, of the 5 people who put my Q on hold have reconsidered, I didn’t get any response from any of them saying “good, we’ll reconsider” or “not nearly enough of a change” VERY UNWELCOMING and that’s to someone who already knows how the system is supposed to work!

A much better system would be if instead the OP gets a message saying “Hey, here is the problem we see with your question as it is currently written. Can you come up with more details/rephrase xyz/give more history about how you got to this point/etc. and then we’ll be able to help you by giving a more useful answer.”

And to be honest, I think some SE communities handle this better than others. But “On hold” (except for clearly off-topic - like asking about “social networks” on a site for “ethernet networks” or vice versa) really is too harsh and, IMHO, counter-productive.

(And of course except for actual spammers, trolls, etc. - those deserve harsh. But not newbies who just need some help asking the right question in the right place.)

1 Like

Whether we call it “on hold” or not, there must be some sort of mechanism to prevent people answering a question while its details are still being hammered out, or we will end up with lots of invalid answers because the question changed after they were posted.

Maybe we phrase it differently - “answers paused while we clarify details” or something along those lines - or maybe we don’t display it to the OP at all, though that does seem kind of counter-productive.

3 Likes

The current SE “On hold” is just too harsh. Plain and simple. And I am a “be blunt” type of person, so that is saying something!

It may be as simple as two different statuses:

  • On Hold - for the big problems - e.g., so off-topic that it is very unlikely that the question can be “fixed”, or so clearly opinion based as to be a bad match (even though the topic area is correct) like “What brand of battery powered drill should I buy?” - but where an immediate deletion (e.g., spam/troll) is not appropriate. Some of those questions might be fixable (e.g., replace “what brand…” with “Here is exactly what I want to do, what specs should I look for in a drill?”) and some might not.
  • Improvements Needed - This would be for “too broad”, “unclear”, and the less severe “off topic” like “shopping” (unless blatantly inappropriately shopping). Then we (the high-rep users or moderators) can provide specific suggestions and/or ask questions of OP to clarify, etc. in comments (not chat), with the goal of turning it back to a regular Question.

Both categories would block new Answers. But the message would be very different - “Improvements Needed” would be much more welcoming and make it clear that “we want to get this to be a good question” instead of “what do you think you’re doing asking a bad question? go away!”.

2 Likes

The way SE handles putting questions on hold and closing them is not great. We can definitely improve on the presentation. The message we want to send is “we want to help you get your answer; to do that we need a little more info (or for you to narrow it down, or whatever)”. In the meantime, we want to prevent answers – but give the OP a positive message, not a negative one.

5 Likes

It seems like there are two unexamined premises here: that answers of underdeveloped questions won’t want to follow up to make sure the answer isn’t invalidated, and that answerers aren’t equipped to decide for themselves if a question is developed enough to answer.

Both of those premises often do seem to be true. Most answerers, especially with limited experience with high-quality Q&A, are unable, or at times unwilling, to figure out if a question has show-stopper problems. Some of these problems can be fairly subtle. While many answerers probably would want to make some effort to match answers to changed questions, that opens up further issues: how, exactly, are they to do so, and what are they to do about chameleon questions? You can’t expect someone to babysit their answers by manually checking on them for the next few days, so that means you’re looking at notifying them if the question is edited (or if the edits reach a certain threshold of significance). That’s inefficient and obnoxious, and the natural response to getting an irritating stream of notifications that require more work out of you is to try to cut them off as quickly as possible. So answerers have an incentive to stop caring about the question, and stop getting pinged about it, as soon as they can expect that further votes on their answer are unlikely. Also, many answerers are drive-by users, that may not be interested in any further site interactions, and leaving low-quality questions open to attract borderline spam, chattiness, and other moderation drains is not helpful.

Finally, as I see it, the basic point of closure is to forcibly align the incentives of the asker with the incentives of the site. They want answers with as little effort as possible, especially including at least one that they see as good, but extras don’t hurt much; the site wants verifiably high-quality answers only, and whatever effort is needed to make the question good enough to attract those. Closure ensures that the asker wants to satisfy the site’s expectations about question quality (and therefore answer quality and conciseness) in order to get anything. (This does not, of course, mean that they need to single-handedly fix the question. It just means they are no longer trying to skirt quality requirements they don’t understand and don’t care about.)

This still leaves the problem that askers may see bargaining with close-voters as an easier way to get answers than actually fixing the question, or may try re-asking it. The second can be managed with rate limits, dupe detection (AI flagging?), and so forth; the first is partly handled by emphasizing next steps in an objective, detailed, but straightforward close message, while de-emphasizing individual voters.

3 Likes

Maybe if instead of “on hold” those questions could be marked as ”under discussion”.

I also don’t think it should be a temporary label for closed questions, but a separate state of the question. A question is under discussion if there’s hope that it can/will be improved. A question can be under discussion for any amount of time (if you only have time on weekends, then having your answer automatically closed after two days is discouraging). Closing a question should need a second explicit action. It might also make sense that the ability to close a question is a higher privilege than the ability of putting it under discussion.