[Feature Idea] Epistemic Status
Epistemic Status: My best guess (but, epistemic effort was “talked to like 2-3 people about it and it felt good to each of us”)
I’ve been thinking about a number of issues relating to “what are the standards supposed to be on LessWrong, anyway?”, with an understanding that there are different stages of idea development. I’d like the site to have a (valid) reputation for being epistemically sound. I also think that often in the early stages of idea-formation, it’s important to be in a looser, playful brainstormy mindset.
A possibility is to actually make Epistemic Status a formal field in the Post submission form. In theory people can just do this fine without special fields, but having it be an official part of the site helps steer the culture of the site.
I’m currently imagining two fields: an freeform text field that people can write whatever they want into, and a multiple-choice field with a few specific options, most likely being:
Empty (both for backwards compatibility and for posts that for whatever reason don’t neatly fit into the epistemic status framework)
Exploratory (for brainstorming, off-the-cuff or other early stage idea generation. Correspondingly, critiques of these posts should lend themselves to a more playful/brainstormy atmosphere)
My Best Guess (for when you’re pretty sure the idea is actually good and are now ready for serious critique)
Authoritative (for when you are making a strong or clear-cut claim that you are confident is well backed by legible evidence)
The main cost of this is that every additional UI element comes with some complexity, and small bits of complexity can add up over time. But I think this would be good both for shaping how people approach writing on LessWrong, as well as commenting.
- (My understanding of) What Everyone in Technical Alignment is Doing and Why by (29 Aug 2022 1:23 UTC; 413 points)
- My Overview of the AI Alignment Landscape: A Bird’s Eye View by (15 Dec 2021 23:44 UTC; 127 points)
- Say Wrong Things by (24 May 2019 22:11 UTC; 115 points)
- Clarifying “What failure looks like” by (20 Sep 2020 20:40 UTC; 97 points)
- Novum Organum: Introduction by (19 Sep 2019 22:34 UTC; 86 points)
- Alignment Org Cheat Sheet by (20 Sep 2022 17:36 UTC; 70 points)
- Possible miracles by (9 Oct 2022 18:17 UTC; 64 points)
- Forecasting thread: How does AI risk level vary based on timelines? by (EA Forum; 14 Sep 2022 23:56 UTC; 47 points)
- My Overview of the AI Alignment Landscape: A Bird’s Eye View by (EA Forum; 15 Dec 2021 23:46 UTC; 45 points)
- How concerned are you about LW reputation management? by (17 May 2021 2:11 UTC; 42 points)
- Possible miracles by (EA Forum; 9 Oct 2022 18:17 UTC; 38 points)
- Forecasting thread: How does AI risk level vary based on timelines? by (14 Sep 2022 23:56 UTC; 34 points)
- Evidence Sets: Towards Inductive-Biases based Analysis of Prosaic AGI by (16 Dec 2021 22:41 UTC; 22 points)
- Continental Philosophy as Undergraduate Mathematics by (26 Apr 2022 8:05 UTC; 15 points)
- 's comment on The Hard Work of Translation (Buddhism) by (10 Apr 2019 0:43 UTC; 9 points)
- 's comment on Why defensive writing is bad for community epistemics by (EA Forum; 10 Oct 2022 23:16 UTC; 8 points)
- Intuitions by ML researchers may get progressively worse concerning likely candidates for transformative AI by (25 Nov 2022 15:49 UTC; 7 points)
- 's comment on Staring into the abyss as a core life skill by (2 Aug 2023 4:57 UTC; 2 points)
- Measuring pre-peer-review epistemic status by (8 Feb 2024 5:09 UTC; 1 point)
- 's comment on Estimating the Current and Future Number of AI Safety Researchers by (5 Apr 2023 2:53 UTC; 1 point)
I would support this initiative on the condition that list of epistemic statuses is not freeform text. A list of confidence levels, plus options like “log” or “review” for opinion pieces would serve the original purpose of epistemic status—allowing an author to write confidently and forthrightly about conclusions while still indicating that they’re not 100% sure about those conclusions.
Allowing freeform text, however, would lead to epistemic statuses like:
“Casual”
“political, opinionated, personal, all the typical caveats for controversial posts”
Whatever on earth this is
We must remember that the original purpose of epistemic status was to increase the legibility of posts. The hope was that by introducing a standard set of words indicating confidence, one could quickly and easily share and compare the confidence that one had in their conclusions, facilitating the process of reaching agreement. Epistemic statuses, as they’re actually used in the community, are the opposite of that. Most of the time, I ignore them, because they don’t add to the content of the post, and just serve as a bonus field for the author to signal how witty and subversive they are.
The introduction of an epistemic standard field for LessWrong posts represents a chance to reclaim the concept and reimplement it in the manner that it was originally meant to be. We should not squander this opportunity.
Edit (2018-08-22): it turns out that one of my friends has an entire tumblr of non-epistemic epistemic statuses.
The intent was that the list of statuses was _not_ freeform text, but that the freeform text was in addition. So, basically all posts would choose one of the “exploratory”, “my best guess” or “authoritative” fields, and then if desired could add additional comments.
I think things like “political, controversial and opinionated” is a reasonable thing to tack onto a “exploratory” post since it adds actual (if context-heavy) information about how to relate epistemically to the post. I also think optional things like “epistemic effort” are reasonable to add, and it wouldn’t make sense to limit them to preset options.
I have noticed “witty signaling” epistemic statuses on the uptick over the past year and think it’d be good to push back against that (esp. since they’re only actually funny if they’re not the majority of posts). But I think forcing all epistemic statuses into a single list would be too costly a solution.
I would be okay with a limited (say, 140 characters) explanatory text if it were in addition to an already chosen epistemic status. That is, in the implementation I’m visualizing, selecting an epistemic status would enable the free-form text box, but the free-form text would remain read-only otherwise.
Yep, that was roughly what we were thinking about.
Also, worth noting there’s a separate feature we’re thinking about which is to introduce “claims”, which come with actual probabilities attached (and which other users can add their own probabilities too), influenced by arbital.com’s implementation. Which is where I think it makes more sense to actually convey explicit confidence.
I actually really like the claims idea. One of the limitations I’ve found with PredictionBook is that I can make a prediction, but I can’t lay out my evidence or reasoning (nor can others make long-form responses to tell me why prediction is wrong). Allowing a post to make formal claims with probabilities attached would make it easier to use LessWrong to practice rationality as well as theorize about it.
I disagree with the idea that freeform text leads to bad epistemic statuses. I sense that a big reason why I disagree is because I think you can read into the connotation of a lot of the “bad” epistemic statuses, even if the denotation isn’t particularly informative.
“Casual” tells me that there wasn’t too much effort spent doing research, thinking hard about the ideas, getting feedback on them, or iterating, which implies that the author isn’t too confident in the ideas. It’s more of a brain dump. I think that the absence of elaboration is also very telling. If the author was more confident, she would have (most likely) said so. And if the author was particularly skeptical of the ideas, she also (most likely) would have said so. So the absence of elaboration makes me feel like it’s somewhere in the middle.
“political, opinionated, personal, and all the typical caveats for controversial posts” interpreted literally is more of a trigger warning than an epistemic status, but I think the implication is that the author realizes that she may have some biases, and is thus not as confident as she otherwise would be. More importantly to me is what wasn’t said. The fact that no claim was made that she is or isn’t particularly confident says a lot to me.
Yeah, I have a pretty hard time understanding that third epistemic status too.
We’ve just been having a whole big discussion that boils down to “communication is hard, you can’t be sure what anyone means when they say even very simple and straightforward things, and if you have to interpret ambiguous things? forget it”. In light of this, I think that “you can read into the connotation” is… not the most strategically sound… reason to support freeform epistemic statuses.
The other problem, of course, is that reading into connotations requires shared cultural context. In many cases, when it comes to rational-sphere writing, it requires quite a lot of shared cultural context—so much that I, with distressing frequency, find myself lacking it, despite having been reading Less Wrong since before there was a Less Wrong. (Sometimes, when reading the writing of rationalists, I find myself thinking: “who, exactly, are you talking to?” It often seems like the answer is “just me local circle of friends; if that’s not you, go away”—and this, with writing that purports to be intended as a contribution to the public, global conversation about rationality and related issues and causes.)
In short, the problem with the “connotations” approach, and with freeform epistemic statuses in general, is that they are illegible. This is fine, of course—even desirable—if what you intend to do is to build, and reinforce, a tight-knit, insular community. Is it? If not, then standardization and discipline would do better.
I think that communication can be hard, but isn’t necessarily hard. And I also believe that connotations can be confusing, but aren’t necessarily confusing.
As for connotations requiring shared cultural contexts, I agree, but I think that often times the shared cultural context that they require is pretty wide, as opposed to a narrow group of friends. Eg. if you send someone a “how are you?” text message and they reply “good” and don’t respond afterwards, it’s a pretty good signal that they don’t want to continue the conversation. It’s not a perfect signal—they could have intended to follow up with a second reply and then forgot—but I’d argue that it’s a pretty strong one. More to the point, I think that the shared cultural context that it requires is pretty broad.
Still, I agree that it is often wise to err on the side of being explicit. My opinion is that it would be good to give authors a sort of prompt like “be careful about assuming that readers understand the connotation of what you’re saying” rather than completely taking the (semi)sharp knives away from them.
I also think it’d be really cool to have a sort of meta-comments section where you could offer critiques of the general writing style, as opposed to the meat of the post. That way authors can learn what approaches do and don’t work. I suspect that the quality of writing in the community will improve if that feature existed and was used.
I am very sympathetic to your view that it is unfortunate to remove often-useful tools from our toolbox. But we can observe what the “rationalist diaspora” has done with these “semi-sharp knives” thus far. I do not think the track record is good. A moratorium seems wise. My view on this is similar to my view on the use of fictional examples and analogies: one day, we may once more trust ourselves with these powerful and versatile tools. One day—but not today, and not for some time. There have been abuses, serious enough that erring on the side of discipline and structure is warranted, now.
I think that this is an excellent idea!
But do you think that authors will be in favor of this? Recent discussions seem to suggest otherwise…
Fair. When I query my brain, I don’t get the impression that the abuses are so bad, but my brain isn’t giving me back concrete examples, so it’s hard to support my impression. I don’t feel too, too confident in my impression though, and I wouldn’t be surprised if you were right.
Hm, I’m not sure. But that question you pose just lead to a thought—perhaps it could be opt in! That way, if you’re looking for feedback on your writing you can opt in to have that section enabled, whereas if you just are interested in discussion of the meat of the post, you can avoid opting in.
I suppose a downside to that could be that it makes those who don’t want to opt in feel uncomfortable. It could lead to a feeling of, “If I don’t opt in, won’t people see me as thinking I’m so good that I don’t need any writing advice?”.
That is, unfortunately, not the most disturbing concern I can think of, about your suggestion.
What is problematic, in my view, is not the perception that declining to opt in to the meta-comments is a signal of confidence in the quality of your presentation. Far more problematic is the perception that declining to opt in is a signal that you don’t care about presentation, combined with a norm that not caring about presentation is acceptable.
Such a norm is already prevalent on platforms like Tumblr, and it has made inroads into Less Wrong. (I will refrain, in public, from naming names and linking posts, but feel free to ask privately, and I will provide examples.) If “meta-comments” (i.e., critique of presentation) were opt-in, then I predict that this norm would entrench itself here. (Why? Because if you have to opt into having your presentation critiqued, then, naturally, if you don’t opt in, then that must mean that readers are not allowed to critique your presentation—right? Well, actually, that logic doesn’t stand up to scrutiny; but it seems to me that it would be naive not to expect exactly this understanding, fallacious though it may be, to become accepted local truth.)
And then Less Wrong will have taken a giant step toward becoming nothing more than “Tumblr for cultural rationalists”. Instead of improving, the quality of writing in the community would degrade dramatically.
(There are other reasons why opt-in “presentation critique” / “meta-comments” would be detrimental to quality of writing; but the one I gave above is the central one, and the others are even more unpleasant to think about, and distasteful to discuss. I hope, therefore, that what I said above is sufficient to give you pause.)
Have you checked that these labels are good matches for a sample of existing posts?
Haven’t explicitly done that, and yeah that seems like a good idea.
The main thing I can imagine people wanting is a medium-confidence “I’ve thought about this a bit but ‘best guess’ doesn’t really convey how I feel about.” (And some posts that aren’t really making claims at all, although I think in most of those cases I think either listing “exploratory” or just leaving it blank would be fine).
I was slightly wary of too fine-grained an approach, since I thought that would be more paralysis inducing than useful.
It seems (Epistemic) Status refers to post headers with a variety of purposes:
Idea Status (I believe this/I don’t, but I’m open to more information/someone else might find this useful/ this is completely theory/brainstorming etc.)
Knowledge Status (I’m pretty sure I understand general relativity/the history of science/This is what I do for a living.)
Confidence with which post is written. (I find it easier to write with confidence than not / I wrote a fictional story to explain this thing in philosophy/science.)
Science/Current Events (I’m pretty sure general relativity is right / this field of science is growing and more up to date information will always supersede it / This is psychology, (link to replication crisis), so...)
Legal disclaimers (I am not a nutritionist/neuroscientist/do not try this at home, do not sue me if you do)
Subject Metadata/Instructions (Don’t read this if you don’t want to read politics/heavy topology math post/This is an experiment, if you name starts with the letter J click here(link), write down your answer to questions before reading onward, etc.)
Which brings us to another feature: Updates. (Sometimes their own post, sometimes added to the bottom or top of the post, possibly with a link to important comments.)
These can be updates on
-events (“This drug...looked really good, but it didn’t make it out of trials.” )
-predictions (“My prediction was correct/falsified/is now being tested!”)
-beliefs/knowledge (“I have changed my mind, thank you for engaging with this idea/we had an expert weigh in (link to comments)”)
-prior work (“I reinvented the wheel, here’s what it’s called (link to wikipedia page).” )
-later work (“I’ve written a better introduction to this subject here (link)”, etc.)
I would use this feature because there is a trade-off between using an epistemic status and losing valuable preview real estate to encourage people to read your posts. Currently I’ve taken to inserting notes after the first paragraph.
It’s a cost. If it’s not freeform, it will lose nuance, which I would find annoying, like those multiple-choice tests where every option is at least slightly wrong, and you are being forced to affirm one of them. If it’s freeform, the only benefit is a reminder. A freeform field that is optional won’t cost much, but a reminder doesn’t need to be a field on the submission form.
Losing nuance is a cost to the writer, but a benefit to the reader. The really nice thing about having fixed categories is that they form a set of fixed reference points allowing us to compare articles. This then allows the user to more quickly determine whether an article is or is not relevant to their interests. It also allows the LessWrong (or GreaterWrong) software to implement filters, which allow this sorting to happen automatically.
Nod.
Realizing this’d be dissatisfying to quanticle, but does this issue remain for you if both the multiple-choice and the freeform are independently optional? (i.e. if the choices don’t feel right you can just include the freeform thing?)
The issues you’re pointing at are real, but I think there’s some important benefit to communicating more clearly what the norms of the site are. (I think this may be valuable both because newcomers won’t understand it at all, and because there’s enough variation in how people currently use epistemic status that it’s not even that clear how to parse the most common cases for oldtimers. Creating default standards makes it so most people don’t reinvent wheels with slightly different wording that people have to parse)
I suppose a major consideration for this is that it wouldn’t necessarily interface well with authors who primarily post on other sites (Zvi, Scott Alexander, Sarah Constantin and Benquo and Jacob come easily to mind), since they wouldn’t have the epistemic status in their original post, and would have to edit it in on LW.
I dislike the idea, although it may be because I find the whole notion of “epistemic status” as front-matter a little off-putting because it’s weird and often feels to me like copying a think Scott does. That said, I’ve been known to start my writing with an apology that I use to set expectations in general and that might include setting expectations about my confidence of the enclosed ideas.
So now that I say all that I think a fair read of my opinion is “just be a better writer and address those things within the body of the text instead of adding metadata”. That may not be, of course, be in line with the direction you want to take LW.
Scott took the idea from gwern, who, in turn, took the idea from muflax.
Muflax’s system is a set of belief tags, “strongly believed”, “partially believed”, and “not believed”, which indicate how strongly he believes in a post. In addition to the belief tags, he has other tags, like “fiction” or “log”, which indicate that a post doesn’t contain any real claims, but is commentary or opinion.
Gwern took muflax’s system and formalized it further by using a variant of Kesselmann’s estimative words, a list of words from National Intelligence Estimates that are used by analysts to indicate how probable they believe a particular event is likely to be. To the list of estimative words, he added “log”, which indicates that a particular piece of writing is intended to document an existing text or event, and is not intended to create predictions.
Scott, in turn, took gwern’s version and turned it into a more freeform text, which, so far as I can tell, he really only uses as a disclaimer on posts that are wildly speculative. Other people in the rationality community took Scott’s version of free-form epistemic status and took it as a license to engage in witticism and signalling.
Of the three implementations above, the implementation described in OP most resembles Muflax’s version—a set of coarse-grained categories that range from “I’m totally sure of this, and it would rock my world to be proven wrong,” to “This is interesting, but I’m not at all sure that it’s actually true.” While I would prefer gwern’s version, with a rigorous set of epistemic words which are standardized across posts, these coarse grained categories are certainly better than the chaos that we have today.
One of the motivators here was actually something in a recent Sarah Constantin post (I think the monopoly one), where the initial post was “written confidently”, but where her actual level of confidence was much lower. Some people complained about this, and she noted that she found it harder to think when regulating her words through a “what will people find sufficiently modest?” lens.
And I think this is a fairly common thing in the rationalsphere – people who are doing the “Strong Opinions Weakly Held” thing which helps them build out models concrete enough to be wrong but which come across sounding as if they think they’ve found the One True Way.
And one thing you could do is ask everyone to get way better at writing, but another thing you can do is separate out the expression of confidence from the writing.
Man, mind-space is big: having to model how other people will perceive what I write is the thing that helps me think and the exercise of trying to give epistemically appropriate words to my thoughts is what helps me figure out better what I really mean rather than what I just vaguely believe might be true.
1. I really like the idea! I think epistemic statuses are really useful to readers, and I think that they can often times make authors feel comfortable writing things that they otherwise wouldn’t write. Ie. I think a lot of people are hesitant to post things that aren’t authoritative. But if there was an explicit field for “exploratory” or “my best guess”, it is obvious that “oh, ok, I’m allowed to write something like that”.
Another thing I like is that it can make the writing smoother. Without epistemic statuses, it can be tempting to have a lot of “I have a reasonably strong impression”-like qualifiers throughout a post. Whereas with an epistemic status of My Best Guess, you could probably leave out most of those qualifiers while still communicating that your statements are reasonably strong impressions, rather than statements of truth.
2. Although I like the idea, I don’t think it’s completely obvious that it’s a good feature to include. However, I feel pretty confident in saying that it’s worth running an experiment on. Maybe try it out for a month and see how it goes. If people aren’t liking it you leave users slightly frustrated for a month. But if it does work, I think the upside is pretty large, and it is multiplied across a much larger period of time. I guess the same arguments could be made for most potential features—and I think that there are many things worth experimenting with! - but I think this one is particularly high upside, in contrast to something like a UI change I guess.
3. I think that there should be an option for a freeform epistemic status, eg. one that isn’t Empty, Exploratory, My Best Guess or Authoritative. Sometimes the epistemic status doesn’t quite fit in to one of those buckets. In general, I think it’s pretty hard to predict in advance what all of the possible buckets for things are. And so I feel especially strongly about this as a ways to start off. Perhaps after a few months of use, if it is clear that Empty, Exploratory, My Best Guess and Authoritative are the buckets, it would be appropriate to get rid of the freeform option, but it seems prudent to test it out before making that assumption.
I suppose the big downside to this is that, with a freeform field, authors may say confusing things, whereas if there wasn’t a freeform field, they would have to choose something (and then perhaps add additional text to their choice), and that would be easier for readers. I don’t see this as a big downside because I think that authors on LessWrong are pretty capable as writers. Of course, they’re not perfect and there will be moments of confusion, but overall I don’t think there will be too much. And I think that issue can be addressed in other ways.
4. I agree that there are times when epistemic statuses are used to be witty and clever rather than clearly communicate the epistemic status, but I haven’t found this to be a big problem at all. And when it is a problem, you can easily skim/skip through it.
Given the rise of AI-generated content, would it be appropriate to add a “Synthetic” or “Machine-Generated” status? Or, I guess to be fair to (future) machines that can make their own assessments of epistemic status, we should just list machines as contributors.
Reading random posts I saw a pattern of these epistemic statuses but they seemed very confusing and illeigble for me. It was not obvoius how they should be taken and how one would go about leaning what they mean.
I came across as “I don’t know what these people are doing, so I can’t really participate”.
Reading some of the older comments here it seems there was an idea of providing some sort of clarity and it seems what is going on now really doesn’t provide what was sought after (maybe a bit seprate question whether providing that is a good idea in the first place).
I’d be interested in a few examples of the posts you came across. Some of them use it in a weirder, more poetic way (which was discussed in this thread, and which I think made it more confusing than it needed to be)