You move from ‘very difficult to believe’ to ‘seems like a very exaggerated version of true events,’ which is almost the same thing but from that you directly jump to ‘This story is credible’ without any middle ground. I am sorry but this seems like a really poor questionnaire design.
Once the poll is made, the answers cannot be edited. The difference was supposed to mean approximately a) I don’t believe this could ever happen, b) I believe it can happen exceptionally, but not all the time as the author claims, c) I believe this can be the way system works.
In other words, the second option is like: “I believe that with so many grant proposals, once in a while a crazy thing passes unnoticed; but I don’t believe that it happens all the time, not even half of the time—you have probably seen one or two bad cases, and now you exaggerate to make your case more appealing”.
I found myself wanting to say “I think this sort of grant proposal thing happens maybe 25% of the time, but not all the time, the way the post implies.”
I also wished there was some kind of gradation for “school insider/outsider”. I’m an outsider, but I talk a lot with a friend who teaches full time. I showed this article to her and she said “yes, yes, yes. This is basically how it is.”
I actually DO still assign substantial probability to this being a hoax, despite it matching my understanding—we know the this is sockpuppet account, created ostensibly for NDA anonymity. But can think of some people here who might have created this explicitly as a test of rationality, who are sort of annoyed that the politics involved here get less scrutiny and want to demonstrate that.
But can think of some people here who might have created this explicitly as a test of rationality, who are sort of annoyed that the politics involved here get less scrutiny and want to demonstrate that.
Spreading false data as a “test of rationality” would be actively harmful. But I can imagine people misunderstanding that.
Rationality is a method of working with the data you have. You should update on evidence correctly, instead of updating incorrectly. You should be able to recognize that this specific piece of evidence contradicts the model based on all other evidence, which makes this specific piece of evidence suspicious. But also should estimate your degree of certainty in a given model.
It is proper to say “I defy the data” when one’s model is based on a lot of reliable evidence. Saying it more often would be overconfidence, not rationality.
I agree that this would be the most likely course of action if the essay is a hoax, but I think it would still risk being harmful overall, since retractions generally don’t result in an appropriate corresponding decrease in confidence in the material that was originally presented. I’d expect Less Wrong members in general to be better at reducing their confidence in a retracted claim than most people, but better is not necessarily good enough.
But can think of some people here who might have created this explicitly as a test of rationality, who are sort of annoyed that the politics involved here get less scrutiny and want to demonstrate that.
Which people? Stylometry might us allow to work out whether someone’s writting style matches this post. Anonymity is hard.
Yes, we could probably de-anonymize OP; I have some passing familiarity with stylometrics, so I considered trying that myself. I decided not to because so far no one has produced any smoking guns that this is false (school districts vary massively in quality across the USA & I have already mentioned an existing and far more shocking instance of school failure/corruption), if it is true then I approve of whistleblowing and have no interest in attacking the OP*, de-anonymizing probably would not set a good precedent, and if it were false—well, I do not disapprove of red team tests of LW (and would be hypocritical to disapprove) and so far this seems to be limited to LW.
* Similarly, it’s been suggested to me by a few people that it would be an interesting project to try to de-anonymize Satoshi Nakamoto or La Griffe du Lion. I am not sure I could, and even if I could, I would choose not to since I either approve of their work or find their material interesting.
If this post ever looked like it was both false and not a limited-scale test, like it was something else (an entrapment of an off-site person? an attempt to discredit LW entirely with a Sokal-style attack?), then I might change my mind. But so far, that does not seem to be the case since I see nothing indicating this post has been picked up by the Drudge Report, Hacker News, Fox News, Breitbart, etc.
Yeah, I also generally consider that posts like this have around 5% chances of being a hoax followed by a gloating “they swallowed it” update (here or somewhere else), though this post doesn’t have any huge red flags (there doesn’t seem to be any huge gloating potential, I mean it’s basically just venting).
So, you are asking me to condition on belief that the author of this post isn’t a troll who crafted this story in such a way to be especially appealing to the LW community. Am I right? If I am, you probably should edit your original post to make your idea more clear.
b) I believe it can happen exceptionally, but not all the time as the author claims
That sounds very different from “This story seems like a very exaggerated version of true events”. One is about frequency (how often do things in the author’s intended reference class occur?) while the other is about severity (how bad are the events that actually happened to the author?)
You move from ‘very difficult to believe’ to ‘seems like a very exaggerated version of true events,’ which is almost the same thing but from that you directly jump to ‘This story is credible’ without any middle ground. I am sorry but this seems like a really poor questionnaire design.
Once the poll is made, the answers cannot be edited. The difference was supposed to mean approximately a) I don’t believe this could ever happen, b) I believe it can happen exceptionally, but not all the time as the author claims, c) I believe this can be the way system works.
In other words, the second option is like: “I believe that with so many grant proposals, once in a while a crazy thing passes unnoticed; but I don’t believe that it happens all the time, not even half of the time—you have probably seen one or two bad cases, and now you exaggerate to make your case more appealing”.
I found myself wanting to say “I think this sort of grant proposal thing happens maybe 25% of the time, but not all the time, the way the post implies.”
I also wished there was some kind of gradation for “school insider/outsider”. I’m an outsider, but I talk a lot with a friend who teaches full time. I showed this article to her and she said “yes, yes, yes. This is basically how it is.”
I actually DO still assign substantial probability to this being a hoax, despite it matching my understanding—we know the this is sockpuppet account, created ostensibly for NDA anonymity. But can think of some people here who might have created this explicitly as a test of rationality, who are sort of annoyed that the politics involved here get less scrutiny and want to demonstrate that.
Spreading false data as a “test of rationality” would be actively harmful. But I can imagine people misunderstanding that.
Rationality is a method of working with the data you have. You should update on evidence correctly, instead of updating incorrectly. You should be able to recognize that this specific piece of evidence contradicts the model based on all other evidence, which makes this specific piece of evidence suspicious. But also should estimate your degree of certainty in a given model.
It is proper to say “I defy the data” when one’s model is based on a lot of reliable evidence. Saying it more often would be overconfidence, not rationality.
I was assuming that if it were a hoax, they’d let people know in another few days with a gloating update.
I agree that this would be the most likely course of action if the essay is a hoax, but I think it would still risk being harmful overall, since retractions generally don’t result in an appropriate corresponding decrease in confidence in the material that was originally presented. I’d expect Less Wrong members in general to be better at reducing their confidence in a retracted claim than most people, but better is not necessarily good enough.
Which people? Stylometry might us allow to work out whether someone’s writting style matches this post. Anonymity is hard.
Yes, we could probably de-anonymize OP; I have some passing familiarity with stylometrics, so I considered trying that myself. I decided not to because so far no one has produced any smoking guns that this is false (school districts vary massively in quality across the USA & I have already mentioned an existing and far more shocking instance of school failure/corruption), if it is true then I approve of whistleblowing and have no interest in attacking the OP*, de-anonymizing probably would not set a good precedent, and if it were false—well, I do not disapprove of red team tests of LW (and would be hypocritical to disapprove) and so far this seems to be limited to LW.
* Similarly, it’s been suggested to me by a few people that it would be an interesting project to try to de-anonymize Satoshi Nakamoto or La Griffe du Lion. I am not sure I could, and even if I could, I would choose not to since I either approve of their work or find their material interesting.
If this post ever looked like it was both false and not a limited-scale test, like it was something else (an entrapment of an off-site person? an attempt to discredit LW entirely with a Sokal-style attack?), then I might change my mind. But so far, that does not seem to be the case since I see nothing indicating this post has been picked up by the Drudge Report, Hacker News, Fox News, Breitbart, etc.
Yeah, I also generally consider that posts like this have around 5% chances of being a hoax followed by a gloating “they swallowed it” update (here or somewhere else), though this post doesn’t have any huge red flags (there doesn’t seem to be any huge gloating potential, I mean it’s basically just venting).
So, you are asking me to condition on belief that the author of this post isn’t a troll who crafted this story in such a way to be especially appealing to the LW community. Am I right? If I am, you probably should edit your original post to make your idea more clear.
That sounds very different from “This story seems like a very exaggerated version of true events”. One is about frequency (how often do things in the author’s intended reference class occur?) while the other is about severity (how bad are the events that actually happened to the author?)