I feel like there are two different, but equally important, issues we need to figure out fully before we can even begin to think about building something:
Command and control
In brief, who makes the ultimate decisions, and how they are kept accountable by the community. This site will require a management/leadership framework that simply does not allow the Stack Exchange BS of unilaterally firing a moderator for questioning a policy.
This also ties into the procuring and imposition of new policy decisions: if the community as a whole disagrees with a new policy, it should not be able to be forced upon users.
This is more Stack Overflow-specific, but what is killing that site is the continual relaxation of quality control. It makes finding good questions and answers difficult, which in turn discourages them, which causes the users posting those questions - literally the cream of the crop - to stop contributing, which leads to fewer good questions and answers, which leads to… a vicious cycle of continually losing the best users.
Any site that wishes to compete with SO (and I use SO here specifically, because it’s the 900-pound gorilla in the SE network) has to do something different, and what I hope to see from this site is an absolute emphasis on quality. Some examples of what I mean by this:
The number of question close reasons are kept short and to the point. Some variant of “user did not read site rules” is first on the close reason list.
Questions that do not conform to the site’s rules are not given any second chances, and are closed mercilessly. If a user can’t be arsed to read and follow the rules, we can’t be arsed to attempt to panel-beat their question into shape.
Closed questions (including dupes) are by default not visible to anyone except the asker, and admins or mods. Any other user hitting a URL to a closed question will instead be sent to the home page, unless they have chosen to see the content of closed questions in their profile. This prevents clutter on question pages.
Answers that do not answer the question are deleted mercilessly. No “this could maybe potentially in a parallel universe be an answer to this question” apples/oranges nonsense.
Answers must support delete flags from ordinary users.
Answers must be able to be promoted over other answers (including accepted ones) by tag experts and/or moderators. This prevents the case whereby the accepted answer from half a decade ago gets continually upvoted, whereas a newer, more elegant, more correct answer languishes because nobody ever sees it.
In relation to the above, there should be an answer-sorting algorithm (that is also the default sort) that is able to “promote” newer answers with relatively high upvotes over older answers that are accepted or have most upvotes. This could be as simple as (number of upvotes / time period).
Reputation farming results in instant suspension. This includes spurious question edits and sockpuppetry.
Functionality from third-party SO bots, like Natty and SmokeDetector, must be built into the site and operate by default. Moderation should be something that just happens, and only requires human intervention as a last resort.
In brief, we cannot hope to compete with SO on quantity of users or questions. But we can, and should, attempt to adhere to SO’s original mission byline of being a site for professional programmers. (Note that in this context, “professional” has nothing to do with experience or job title, and everything to do with the amount of effort you put into your contributions. We also need a way to rephrase this “professional” bit so it doesn’t discourage talented amateurs - “professional and enthusiast” doesn’t work for me either because it’s too unwieldy, maybe just “enthusiast”?)
In order to support this, moderation tools to curate questions and answers must be best-in-class - take the top-voted-but-not-implemented mod feature requests on meta SO and implement them here. I’d strongly suggest that someone with moderation experience, like @Makoto, drives this part of the site initially.
All the above may appear to create a very hostile environment for question askers. This is untrue: it creates a hostile environment only for those who are not interested in contributing to this knowledge repository. Help vampires, rep farmers, and the generally lazy are what’s killing SO - if we make it very clear to those sorts of users that they are not welcome here, they won’t come here, which means they won’t get entrenched, which means that any complaints about niceness will actually be around users being assholes to people, not about help vampires who are pissed off that their useless crap was nuked from orbit.
Finally, regarding inclusivity:
This should be a community for adults, not a safe space, which means sarcasm will happen and admins/mods must allow it It also means that the ability to understand sarcasm is required.
As a programming website, it shouldn’t matter whether you’re cis, LGBTI, a dog, or a space alien: your code matters. If you are not comfortable with being called him or her, let us know your preferred pronoun and we will try to remember to use it - but do not expect us to remember to do so at all times. Assume good faith.
That said, sexism must be dealt with harshly.
Anyone should have the right to criticise others’ beliefs or lack thereof, but not the right to denigrate or otherwise belittle others if their beliefs are not relevant to interacting with them. In other words, if you are going to treat a user differently because they’re not a cis white male, you should not be not welcome here. Your beliefs, or lack thereof, do not trump others’ rights to be treated with basic dignity.
So i was thinking over what i think would make a good MVP. These are ideas taken from a variety of people. Remember - these are just concepts/ideas.
The Q&A Community Hub.
The most successful societies are those with a strong mix of diversity, opinions and people from different backgrounds. The most amazing advancements in human history have come from people asking questions, pooling our knowledge and learning together.
Codidact aims to bring both of these ideas together in a digital environment where people can be free to challenge, grow and learn through the familiar Q&A format whilst keeping respect and human-ness at the centre of every interaction.
Codidact aims to keep the Q&A format simple and to-the-point. But we also understand that people are different and need different types of input on a question. Sometimes a discussion (without a direct answer) can be exactly what someone needs on a question or topic and other times it can just be the answer. The simple Q & A format with no BS is at the heart of Codidact though.
Questions and answers are organized into various categories.
Users have profiles where they can use settings to customize what subjects/topics they want to view. These settings should be accessible easily - Something like a dashboard could display all activity for the user and a general topic dashboard could also be implemented.
The core aspect of Codidact is Q&A. The question is given primary concern. A question is made up of a title and a body. This question can be categorized into different subjects, topics and/or sub-topics.
Users read the question and can add an answer.
The User Experience
The user experience is made up of three aspects. Kudos, privilege and badges.
Users can give and receive Kudos on an answer, question, comment or useful comment thread. Kudos is a signal of how active and beneficial that member is to the community - kudos could be used similarly to upvotes where a new user can be given small amounts of kudos by going through a tutorial and learning the different aspects of the community but is distinct to privilege. Kudos mostly matters to newer members. It provides a way to gate basic interaction until a user has completed the tutorial or basic easy to complete interaction in the community. Gamification is an important part of kudos and at certain levels could unlock perks (this is optional). A high kudos does not denote a high privilege.
Privilege is a separate aspect that is given to a member of the community by the community via votes. Votes would be held regularly and would be at varying levels. Privilege can be seen as moderator privilege. Privilege is earned and given to users who will not abuse their moderator rights and is given at varying levels. For example a low-level privilege could be to have the power to open up or accept comment threads on definitive questions. A higher level privilege could be to close questions or move questions into different areas etc.
Badges are given when a user receives a respectable amount of kudos on a given topic. Badges could also be represented in levels of accomplishment. Badges allow a user to be seen as a domain expert on a given topic. Badges allow a user a certain amount of privilege-like interactions but only on questions/answers within their domain knowledge. Most of these badge privileges will be specific to understanding the subject. Badges provide an aspect of gamification but with actual value.
There are two types of questions, definitive and subjective.
A definitive question always requires a definitive answer. A definitive answer directly answers the question and should result in the question being resolved.
A question can be re-answered if the given answer is no longer correct due to a change in circumstances. This new answer must be upvoted or acknowledged by privileged/ relevant badge owned users.
Definitive questions and answers can receive comments. These are known as comment threads. A comment thread is a single thread of comments unified around the thread title. These act in a secondary role to the answers. Comment threads are not visible by default but can be toggled. Comment threads are given a title at creation. These comment threads (not individual comments) can be upvoted as useful. The highest upvote count will cause the comment thread title to be visible in the useful comment thread section, distinct from the ‘view other comment threads’.
Comments within the comment thread must be direct and pertinent to the question or answer. Rather than keeping a string of individual comments in the open, a comment thread can be opened by a privileged user or requested to be opened by a non-privileged user which must be approved.
A thread title (character limited) can be chosen. It is reasonable to challenge a comment or add additional information to an answer, which is why comment threads can be useful but should be used sparingly and added only when they bring value to an answer/question. Any irrelevant or conversational comments must be closed/moderated.
The definitive question is the heart of the Codidact Q&A hub and is most similar to that of SO.
A question might also require a subjective answer. A question requiring a subjective answer will require more opinionated discussion. A subjective answer should be well reasoned and most of all respectful. Disagreeing with another poster does not warrant as disrepesctful but any kind of immature name calling, or offensive behaviour is
A big question around subjective questions is whether they are open and require no community moderation to be created or whether a new subjective question must be vetted by a privileged user before being opened to prevent spamming. The privilege to open subjective questions could be earned via a badge or voted privilege
Subjective questions and answers can receive kudos that count towards badges. Comments/discussions within a subjective question or answer can receive kudos but does not count towards badges.
Subjective questions can also be linked to topics as well as definitive questions. This relationship treats the definitive answer of a question as canonical and allows further linked subjective discussion around the definitive question. There can be a one-to-many relationship between definitive and subjective questions.
A subjective question operates very similarly to definitive question with some key differences.
Users can choose to not view subjective questions by toggling visibility.
Comment threads are treated differently they could be called discussion threads. They are open by default and don’t need privilege to open. Discussion threads can also be nested past one level. Comments within a discussion thread can receive kudos but don’t count towards badges because of the subjective nature of the topic.
Answers in subjective questions don’t get accepted. The asker of a subjective question isn’t responsible for accepting an answer. Answers receive kudos and the answers with the highest kudos are promoted to the top in a more visible manner.
Because of the nature of questions and their ability to get off topic. It is possible for users to flag comments and answers as unhelpful or off-topic. A post or comment with multiple flags will grey out the comment for moderation and can be moved or removed.
Posting off topic won’t remove kudos but repeating off topic offenses will cause kudos to drop and possible privilege/badge cool down period. This is non permanent and will not be stored on the users profile.
A cool down will solve most situations.
Rude comments or unhelpful or unconstructive behaviour can result in a user getting ‘benched’. This is a longer cool down period. This will remove privileges until a moderator is able to chat through the reasoning and ‘unbench’ the user, restoring privileges. Any user getting benched will show on their profile. It should be (unfortunately) expected that the average user will get benched as people can get passionate about things they like but it should be few and far between.
Repeated benching will result in more serious action been taken.
Any egregious behaviour will obviously result in the appropriate action. This idea of levelled moderation will prevent insta-ban culture from overzealous moderation and will help users learn from their mistakes whilst the cool down will help curtail the buildup of unhelpful comments and spam on answers. The design of comment threads will also help alleviate comment spam.
Not to disagree, but as an addition: even on the giant SO itself, Dunbar’s law holds. There are, essentially, “tag communities”, regulars within areas of expertise who recognize and respect each other. We “run into” each other doing things on the site, and probably interact with each other more like neighbors, or at least co-workers, than we do the anonymous userNNNs. There is definitely a shared experience within platforms/languages. (I speak from my experience in Apple-tech tags; the Python chat is another witness; also the (infamous) C++ chat.)
I couldn’t say which tag communities would be ripe for “poaching”, but I think it would be worth trying to identify one group at least to target as a start. I’m already seeing a fair number of SO-primary folks here…
with the possibility that, as a bonus, it’ll also be helpful to other people. (*) to be clear: I think short “hello” and “thank you” notes in posts are _fine_.
In answer #18, @gilles shares a reasonable opinion: the world doesn’t need another discussion platform. I would just go one step further and state, with almost certainty, it also doesn’t need another collaborative (impersonal) encyclopedia, particularly not one with less strict citation requirements. And, in any case, that does look like a recipe for failure, to me, personally (in terms of quality control).
I don’t think the competition for the highest score is as serious as this makes it sound like. Besides, the “careful editing” culture has been in place (afaik) for almost as long as the platform itself.
For a while now, we’ve had (among others) the following reason for rejecting suggested edits:
clearly conflicts with author’s intent
This edit deviates from the original intent of the post. Even edits that must make drastic changes should strive to preserve the goals of the post’s owner.
(…) how do you encourage groups to do what’s best for the world rather than their own specific, selfish needs?
When I looked at this problem, I felt I knew the answer. But there wasn’t a word for it in 2008. Now there is: Gamification.
Gamification is the use of game design techniques and mechanics to solve problems and engage audiences. […] Gamification works by … taking advantage of humans’ psychological predisposition to engage in gaming. The technique can encourage people to perform chores that they ordinarily consider boring, such as completing surveys, shopping, or reading web sites.
I had no idea this Wikipedia article even existed until a few months ago, but we are featured prominently in it. It is true that all our stolen ideas about reputation systems, achievements, identity, and vote scoring are in place specifically to encourage the adoption of the brave new no-nonsense, all-signal Stack Exchange Q&A model. Without those incentive systems, when left to their own devices, what you get is … well, every forum ever created. Broken by design.
Yes, we have ulterior motives, but let me explain why I think gaming elements are not tacked on to the Stack Exchange Q&A engine, but a natural and essential element of the design from day one.
(…) I had to think for a minute to realize that Stack Overflow has “gamification” too. Not a ton. Maybe a dusting of gamification, most of it around reputation.
Stack Overflow reputation started as a very simple score. The original idea was just that you would get 10 points when your answers were upvoted. Upvotes do two things. They get the most useful answers to the top, signaling that other developers who saw this answer thought it was good. They also send the person who wrote the answer a real signal that their efforts helped someone. This can be incredibly motivating.
Now, this wasn’t an original idea. It was originally inspired by Reddit Karma, which started out as an integer that appeared in parentheses after your handle. If you posted something that got upvoted, your karma went up as a “reward.” That was it. Karma didn’t do a single thing but still served as a system for reward and punishment.
What reputation and karma do is send a message that this is a community with norms, it’s not just a place to type words onto the internet. (…) Our goal is to get the best answers to questions. All the voting makes it clear that we have standards, that some posts are better than others, and that the community itself has some norms about what’s good and bad that they express through the vote.
By the way, Alexis Ohanian and Steve Huffman, the creators of Reddit, were themselves inspired by a more primitive karma system, on Slashdot. This system had real-world implications. You didn’t get karma so that other people could see your karma; you got karma so that the system knew you weren’t a spammer. If a lot of your posts had been flagged for abuse, your karma would go down and you might lose posting or moderation privileges. But you weren’t really supposed to show off your high karma. “Don’t worry too much about it; it’s just an integer in a database,” Slashdot told us.
This article goes then makes some excellent remarks such as “Stack Overflow’s reputation system serves to recognize that you’re a human being and we are super thankful for your contribution”, and then discusses a few of the downsides of the reputation system; and the end conclusion is not entirely clear.
However, a careful analysis will reveal that these problems are not inherent to reputation in a broad sense - rather, they are mostly specific to the implementation used throughout SE and can be solved in a number of effective ways, each very minor and incremental.
Jeff’s article from 2018 goes to great lenghts to draw similarities between SE and a wiki. The similarities are there, but limited. For instance, a wiki doesn’t have this:
Maybe competition just isn’t your jam. Could there be a less competitive Q&A system, a system without downvotes, a system without close votes, where there was never any anxiety about posting anything, just a network of super supportive folks who believe in you and want you to succeed no matter what? Absolutely! I think many alternative sites should exist on the internet so people can choose an experience that matches their personal preferences and goals. Should Stack build that alternative? Has it already been built? (…).
Now, is that what we’re trying to build? If you ask me, I maintain my position: no.
I was satisfied with Stack Exchange. Well, not a 100% of course. The duplicate proposal system was unfriendly and inneficient, comments are excessively ephemeral, people still get away by answering duplicate questions, deleted posts are way too secret. And what bothers me the most (at least on Stack Overflow, where I participate more): beginners are repelled - even when their questions are totally conformant to all rules and guidelines, they’re often swiftly downvoted and removed just own grounds of being “too basic”; and the same happens to questions that are arbitrarily deemed “too localized”.
Regarding the first point, even Jeff seems to agree with me on that in the 2018 post:
perhaps it’s time for a more beginner focused Stack Overflow where duplicates are less frowned upon, and conversational rules are a bit more lenient?
– Yes! Well, maybe. I’m not a proponent of two (or more) separate sites. The simplest approach would simply be to have a beginner meta-tag that can easily be ignored by the most grumpy seasoned members. I’m sure there might be more sound strategies.
In conclusion, reputation is a great mechanism for community self-moderation, a filter for spam and vandalism, a motivator for self improvement, and a reward for contributing your time and knowledge to helping others. It was a design consideration from day one in Stack Overflow, the first Q&A platform to *actually* succeed (to the best of my knowledge!). The SE model as currently is has its shortcomings, but ultimately, it has been demonstrated to work. I don’t think we want to forgo that lightly.
Implementing reputation when every post is a Wiki can’t possibly work without massive efforts - and even then, I remain skeptical it can work as well as the model I’m defending.
Has it been done before? I think so, more or less - thankfully. Remember Stack Overflow Documentation? That ended up discontinued and archived? (kudos, riptutorial)
Did I like it? Yes. Contribute to it? Yes. But how did it go? Did it fail? That depends on how you look at it, but… It never came even close to replacing “classic” Q&A, and never intended to, anyway (despite some expectations).
The reputation system, however, took a long while to be implemented, a couple iterations to be adjusted to a decent level, but even then, was never entirely satisfactory as far as I know.
Can we do better? And most importantly, should we?
Hence, when it comes to this project, I vouch for sticking to what works, improving (carefully) only where necessary.
In conclusion: Joel and Jeff started Stack Overflow because, at the time, the alternatives frankly sucked. The platform took off, helped millions of people and we were satisfied. But now, ten years later, it’s got its own sucking too. We have waited, hopeful that it would get back on the rails. With time, we progressively lost hope. More recently, it became so much clear just how unlikely it is that things will ever be fine with the platform once again, that we decided that enough is enough and realized that an alternative is direly needed. “If you want a thing done well, do it yourself”. Usually, we can settle for something that is just good enough; but, when the best thing you got actually sucks, it is time to take a stand.
But do you want to?
I know, you aren’t talking about yourself, but rather, of the largest proportion of SE users, who aren’t as concerned with any of the pressing problems that are driving us away. That is certainly true about Stack Overflow, and much less so about the smaller communities - though, of course, actual data would be needed. However, the members who are being pushed away are those who actually care about quality, about the community, about fairness and I’d even risk saying, about making the world a better place. These are the pearls. They deserve better.
Now, granted, if by some miracle SE actually manages to fix all of these ***k-ups and regain trust and engagement from their most valuable contributors, then it’s quite possible that I’ll happily go back to their platform and this project will not be as essential for me as it is right now.
Wow, this is a sobering voice. Thank you for this.
This can’t be true enough—radical changes like removing the reputation system, or removing answer accepting, etc will only stray us further from whats proven to work. We need to make our best effort to recreate these systems, and only change things when its obvious that it will be better.
Now that’s a horse I wouldn’t bet my money on, lol
I would argue that the answer accepting and scoring system does not work except for gamification. Yes StackExchange works, but we do not know whether this is ‘thanks to’ or ‘despite’ the votes and accepting.
(It also depends on what you want. Do you want some helpdesk type of continuous answering of questions, which requires votes to fuel motivation of answerers. Or do you want to make a solid voting system that helps future visitors to quickly differentiate quality in some type of community generated guide based on q&a’s)
The scores often make little sense… The majority of scores is 0 and 1. That means there’s little differentiation. When there is differentiation then it may be wrong, and scores reflect only the volume of votes (which is highly variable and not neccesarily related with quality of the post) instead of something more precise like a user rating (e.g. like movie ratings).
Many questions are without accepted answers. And among questions with accepted answers, several have equally good other answers and the accepted answer is an arbitrary choice by the OP, which may not need to reflect the idea of other’s. To me it makes little sense that only one person decides on the accepted answer. Also multiple answers can be acceptable, why only one?
The point is that yeah, while we may not know whether its ‘thanks to’ or ‘despite’, SE as a whole works. So when in doubt, assume its ‘thanks to’ until we know better. After all, we’ve got a working sample size of one to analyze. I don’t want this project to be a guinea pig to some strange mix of ideas that may be good independently but work poorly in unison.
I assume (but correct me if I’m wrong, as I haven’t done any formal analysis) that’s referring mostly to questions. I don’t care if Questions don’t get many votes. I would even be OK if questions couldn’t even be voted upon, though I am also fine with votes for Questions if lower than for Answers (e.g., 5 vs. 10 points) as the Answers within a question are what really need ranking (based on voting). I generally find that if there are several answers to a question then there also are plenty of votes for the “best” answers.
Many questions are without accepted answers
This is definitely true. Occasionally there are questions where there simply isn’t a “good enough answer” or where the OP doesn’t like any answers. But far more often, I think that it comes down to one of:
New user who reads the answer and leaves, not having really learned about (or care about) accepting an answer
New user who doesn’t actually realize that they should accept an answer. I know this sounds stupid, but I think that the SE page layout just doesn’t make it totally obvious. And people “read” the Tour but they don’t really READ the Tour, which also isn’t really so clear.
So I think the “Low Votes” problem is not necessarily a problem. And the “Accepted Answer” problem really is a problem, but some better design (I have some specific ideas) should be able to improve that considerably but never 100%.
Part of the reason I think SE worked so well is that its gamified around SEO, people who write high hit questions tend to get better rewards. This is actually a really good thing for SE as a site as it drives a lot of traffic to sites. Good is useful and useful is searchable, and people who write searchable questions should be rewarded.
I’m going to propose that we lock this thread soon just for the sake of organization. This thread was created before we had the other MVP discussion threads and I think any new ideas should go in one of those threads or another thread entirely.
SE isn’t monolithic; things that work well on SO don’t always work well on other sites, and vice-versa. It sounds like answer acceptance is important or even essential on SO; on the other hand, some other sites have asked to disable the feature entirely. On the sites where I was active, acceptance wasn’t nearly as important as community voting; on other sites, a snarky answer that gets there first gets disproportionate votes even though it’s not a good answer.
For the most part I think what SE built has been shown to, roughly, work. It’s not 100% by any means and we should feel free to adjust things, but we should make sure we consider more than one use case (and consider making some things adjustable per-site). We can also defer many things; we’re not trying to build an SE clone on day 1, but, rather, identifying the essential core and then expanding from there. IMO.
Just brainstorming here, but I’m wondering if we could fix some or all of the major voting problems with SO? Gilles pointed out the issues with voting - users can inadvertently vote for a well-written but incorrect answer. And it can sometimes be difficult to tell if an answer is correct. If it’s a programming question, working code can usually be relied on to prove something works, but as we all know, there’s some situations/bugs/conditions/configurations where code behaves differently.
I’m not sure if we’re really trying to rewrite SO - jeez, I hope not - but there’s definitely things that has made SO successful for the last 10 years. What if we somehow changed the concept of SO reputation into more of a concept of technical prowess/ability? For example, if users like Stephen Cleary, Steven Toub or Eric Vogel provide a comment or answer on asynchronous operations, it’s most likely correct, regardless of their reputation number. If you provide working code, that’s a pretty good indicator too, but not always right. SO uses that, and it’s mostly reliable, but there’s a lot of times, better ways of writing code and some answers are just better than others based on developer ability. Those typically get more upvotes at SO, unless they’re a more recent answer.
Github uses a “Pro” indicator for users like Marc Gravell or Nick Craver, the main supporters of Dapper. I’m somewhat new to Github so I have no idea (without looking it up) how these users are designated as Pros, but it seems to be a pretty good indicator that they’re at a different level than the general development community, and I’d trust pretty much anything they say on a site like SO or Codidact. Interestingly enough, Stephen Cleary’s github account has no “Pro” designation - https://github.com/StephenCleary.
How can we identify users like this, that whenever they “vote” we can assign the utmost confidence that they’re probably right? One way might be to piggy-back off github or other sites, using them as a kind of “gravatar” or reference link to establish either identity or technical ability. Users can of course opt out of identifying themselves but opting in provides another level of swag that helps pad their ego - lol - and establishes them as a real expert. What if we had something like Pluralsight’s skill assessment - https://app.pluralsight.com/score/skill-assessment/angular/intro?context=paths&path_type=skill&path_url=angular#/v2/landing - that could establish ability? Or maybe a way to link to that skill assessment for the Pluralsight account?
And - it just dawned on me - why have the OP mark an answer as the best answer? They might not be the best one to establish the answer. Should we base the “answer” on number of votes? Or maybe the fact that a user like Marc Gravell or Stephen Cleary said that’s the answer and so, that’s the answer?
Again, just brainstorming, but I’m not sure how I feel about SO’s voting mechanism. It’s really more of an opinion poll but it serves to involve the user and at the same time makes them feel included and a part of the site, that they’ve contributed somehow. If we’re looking for the best professional programmer super-wiki that we can build, establishing correct answers is key IMO. Quality over quantity.
Like some of the folks above are saying, our goal is building a better community without all the problems and frustrations that are an inherent at SO. And because they’re a corporation with values that don’t match the community’s vision. It seems SO is so anti-community and that’s the very reason we’re all here. SO is out of touch and out of reach for the community and it’s going to be their downfall. Let’s make sure we don’t build anything like that. Great discussion so far.
One thing we might do is to weight topic experts’ votes in rankings. (Topic experts per their contributions in this topic before). Might be also good for reputation, however I am not a big fan of that, as it makes the system less transparent.