They are too often used to get around the answer-ban on closed questions.
They clutter up the site.
I see your point about attributing something not intended for the same level of formalness in a more formal place. However, this is easy for users to avoid or deal with. The rules would clearly state that answers (giving the OP information relevant to the question, as apposed to giving feedback on writing a better question) are not allowed in comments. And, if you do it anyway, they will be converted to answers with all the resulting exposure and peer-review. If you don’t want that happening to you, you can easily avoid it by simply not answering in a comment, which of course was the point of all this in the first place.
When a comment is converted to an answer, the system should notify the commenter. That allows modifying the answer, or deleting it altogether.
Whether something is an answer is not going to be much of a gray area. I expect it will be quite rare that someone complains “But that wasn’t really an answer.”.
Again though, if you want to avoid all of this, don’t answer in a comment. It’s really a simple and easy rule to follow.
That depends a lot on which communities you’re looking at. It’s probably pretty clear on SO. On sites where frame challenges are permitted, it’s less clear. On sites like Workplace or IPS, there is a lot of disagreement about where the minimum bar for answers falls.
We are going to limit their visibility. Anybody who asks to see them all anyway opted in. And comments that aren’t serving the comment function can be deleted if they’re in the way.
If someone is putting useful content in the wrong textbox, we should be punishing ourselves for having bad UI, not the user for making a positive contribution.
I’ve found that if you have non-malicious users behaving in ways you don’t expect or want, you can’t go wrong with treating it as a human factors issue. (Disclaimer: I currently work on a UX team at a company you’ve heard of if you live in the US, but I would have said the same thing in my last assignment.)
This is what I’m talking about. In both of these cases, there are serious UX issues at play, and answering in comments is just a symptom. Treating symptoms is rarely an effective way to solve a problem, and negative reenforcement is not a particularly effective way to get people to stop doing something, so it’s not surprising that…
…a punitive approach doesn’t work. Good design focuses on the needs and motivations of all the non-malicious users providing value and finds ways for them to work together. Don’t punish someone for doing a good thing just because it isn’t perfect.
The root of the problem is actually that SE has high expectations on answers. I think these are the most common reasons why people answer in comments :
I don’t have time to flesh out a full answer right now.
Qualified guess. The OP knows the likely cause but can’t prove or verify it.
The question is so simple to answer that one sentence does it.
The answer can easily be found in RTFM or at an external site.
In all of the above cases, the root cause preventing an answer is high quality expectations on answers from the community. This isn’t necessarily a bad thing and I think it’s about community culture more so than site design.
The only common case for answers in comments that don’t fit in with the above is: “That problem again. I can’t be bothered to find a duplicate.” This one is a site design problem, the root cause being SE’s broken duplicate system. A working, user-maintained FAQ list with canonical duplicates integrated into the site itself would solve this.
A thought presents itself: is it practical to have a dedicated type of post for a “shared answer draft” for cases where someone wants to contribute to answering, per se, but doesn’t believe their answer meets the minimum quality for a regular post? In such a case, it seems like we could have some sort of vaguely community-wiki-like answer, which might then switch to more or less a regular answer if someone contributes enough to take over ownership (with any corresponding benefits or penalties from voting, etc).
I’ve been thinking about this too. We need to be intolerant of answers in comments, as experience on SE has shown this is a serious problem. However, since the correct behavior is to post an answer instead, we need to be a little more tolerant of completeness of answers.
Some comment-answers simply shouldn’t be. But, it should also be acceptable to provide relevant hints, general guidance, or related background information without specifically answering the question. These questions should not be downvoted for being incomplete, or whatever. I can see that they might not get many upvotes, but that’s OK. They should end up lower in the answer order than “real” answers, but the answerer shouldn’t be penalized as long as the content is somewhat relevant to the question, is not outright incorrect, and is well enough written. It would help if such answers are labeled as not intended to be full answers.
In the end, I think this is a social issue, not a technological one. The guidelines for what to upvote and downvote should mention these cases. There is really no excuse at all for answering in a comment when you could supply the same information in a answer without getting dinged for it.
Actually this is already the case on SE in my experience, just that most people don’t take the care to do it properly. I have on occasion written such partial answers, starting with something like “This isn’t a full answer, but briefly …”. In my experience, you don’t get downvoted for this. As long as you make it clear you are not pretending to supply a full answer, people seem to judge it according.
So the right mechanisms for such answers already exist. The only thing lacking is a description of it in the guidelines.
In such a case, it seems like we could have some sort of vaguely community-wiki-like answer, which might then switch to more or less a regular answer if someone contributes enough to take over ownership
That’s not a bad idea at all. Not necessarily shared community wiki, but some sort of “take it as is” draft, not to be voted on or counting towards rep, but containing a “draft answer”. Then at a later point the answerer can flesh it out to a real answer. Or when someone posts an answer that gets upvoted enough, we can have “draft answers” fade out or sink to the bottom of the answer list.
Well I know where you are coming from The EE site always had lots of debates about this. I think it is a community culture thing there specifically, more than anything else.
On other sites, it is regarded as less of a problem. For example, it is very common on SO too, but there you can always always count on someone hijacking the comment and fleshing it out to a full answer.
The network-wide stance is something along the lines of “comments are not important, just ignore them”. Not so sure I like that stance, personally. But well, it is bit of a waste of moderator resources to have them clean up comments all day.
Though of course it often helps to nudge the poster of the comment “you should post that as the answer of the question” and then they do so. So maybe something like “convert comment to answer” could be implemented, just to not only ensure that an answer is posted, but also that the comment gets deleted.
I have been thinking about going back to all my posts with discussions in comments and incorporating (archiving) those comments into the answers. But then again, that is too much effort. And, is it really needed? I consider those comments to something similar as the talk pages on Wikipedia. For some people it is usefull meta information (which may not really fit in an answer/article) that should not be deleted (except when there is too much of it).
If those comments would be easy to convert into an answer then it would happen, but often it is not that easy.
I get that Jeff Atwood and Joel Spolsky wanted to make a more clean platform without the lengthy, noisy, often useless, and sometimes flamy discussions (and therefore pushed away, but also underestimated, comments and chat). However, the feedback on a post may be very useful to read (some feedback can be incorporated into the posts but other feedback would be bad for the integrity of the original version which caries value/thoughts/opinion/ideas on it’s own). In that respect I find it a pity that the journal format where the letters from reviewers are published as well does not exist anymore.
I remember having posted drafts and then deleted them because I did not consider them complete answers or sufficiently fleshed out. (It is a pity that SE only allows to safe one single draft, I have lost drafts in this way).
Posting hints or partial results in comments which I do a lot is much easier.
It would certainly be nice to be able to post an answer with a status of being a draft. Although this could also be done in the text. This option I do as well. If I consider some comment usefull then I post it as an answer (on math.SE and Math Overflow it is very common to provide partial answers, taking small steps that’s how they do maths).
The problem here is the heterogeneity of the community. Not everybody is gonna do it that way, some will comment some will post partial answers, and maybe we should just consider that as fine. The problems with comments is mostly that there can become a lot of them. If you place the comments in a tree like structure then this mess is mostly gone (except some special cases).
If we had the ability to create a “partial answer”, my recommendation here could fit into that pretty well, if a comment is flagged as “answer in comments”, it could be converted to a “partial answer” and tagged as such.
In this case, the site design needs to deprioritize or even hide the comments (like Wikipedia’s “talk” pages). Otherwise, the comments outshine the actual answers even if they’re of lower quality.
This makes me think if we should perhaps introduce a new post type “supplement”. It would basically be a flag on the answer box, marking the text as not exactly being an answer, but still contributing valuable information. Such supplements would show below the real answers, and probably upvotes on them should get you less points than upvotes on answers (giving an incentive to answer), but downvotes should definitely count as much as downvotes for answers (otherwise it would be an invitation to abuse).
Supplements would be understood as having lower quality requirements than actual answers. A supplement may contain hints, speculations, related information, partial answers, basically anything that is helpful either for the asker or others who will come across that question later.
Since partial answers may grow into full answers, the poster should be allowed to convert a supplement into an answer later; however only if the supplement has positive score. When doing so, all upvotes probably should be lost (because a good supplement is not necessarily a good answer; the answer score should be earned on that), but the points earned on them should remain (you should not be penalized for completing your incomplete answers).
The comments may normally be collapsed (a post with comments has only a line “there are comments on this post”), and require explicit action to be shown. But I think when shown, they definitely should be shown below the post, because the refer to that specific post (not to the page in general, like Wikipedia discussion pages), and also because when reading comments, you may want to look at the post to see what the comment refers to (which is made much harder if the comment is on a separate web page).
Yes, I think “supplement” captures the idea nicely, and makes the intent of such answers clear.
Upvotes should earn no points at all. We really want proper answers. As you say, downvotes still count, else we’d see a lot of abuse, and even supplements need peer review. Upvotes still effect the sort order, and the overall quality rating for that post.
This also gets around the problem in your last paragraph, which is what to do with upvotes if you convert the supplement to a full answer. Answers are only eligible to earn points from upvotes when they are full answers, not supplements.
With such a system in place, we can clamp down hard on answers in comments. There is really no legitimate excuse at all now for attempting to answer in a comment.
Unfortunately, I think it would be nearly impossible to define this line in a way that everybody would understand. In practice, you’re saying that answers in comments should be allowed unless they are too good.
Can you specify this? I can agree that it is an annoyance in some situations, but calling it a “serious problem” is a big exaggeration in my opinion.
No, I’m saying answering in comments should never be allowed.
The part “acceptable to provide relevant hints” was part of the argument for having a different type of answer, called a “supplement” in later posts. Supplemental answers provide some relevant info but are specifically for not directly answering the question. They still collect up and down votes for the purpose of the sort order and to be peer-reviewed, but don’t earn rep or whatever for the author. Their purpose is primarily to allow for quick and dirty comments relevant to the question. This eliminates any remaining excuse for answering in comments, and therefore allows us to clamp down heavily on comment abuse.
I expect this varies from site to site. This abuse happened a lot on the electrical engineering site. Some even admitted they used comments deliberately to “answer” closed questions. For others it was a lazy way to answer that they knew wouldn’t be subject to peer review, and therefore couldn’t lose rep for getting something wrong and downvoted.
All around, comments that attempt to answer the question subvert the quality mechanisms, noise up the site, and add a lot of clutter, some of it wrong, to questions.
Comments on questions should only be for building a better question. I think of them as scaffolding or construction lines on a drawing. When done, they can and should be removed without loss of content.
The EE site always had lots of debates about this. I think it is a community culture thing there specifically, more than anything else.