The outside view isn’t magic. Finding the right reference class to step back into, in particular, can be tricky, and the experiments the technique is drawn from deal almost exclusively with time forecasting; it’s hard to say how well it generalizes outside that domain.
Okay, you’re doing precisely the thing I hate and which I am criticizing about Less Wrong. Allow me to illustrate:
LW1: Guys, it seems to me that Less Wrong is not very rational. What do you think? LW2: What makes you think Less Wrong isn’t rational? LW1: Well if you take a step back and use the outside view, Less Wrong seems to be optimizing for being clever rather than optimizing for being rational. That’s a pretty decent indicator. LW3: Well, the outside view has theoretical limitations, you know. Eliezer wrote a post about how it is possible to misuse the outside point of view as a conversation stopper. LW1: Uh, well unless I actually made a mistake in applying the outside view I don’t see why that’s relevant? And if I did make a mistake in applying it it would be more helpful to say what it was I specifically did wrong in my inference. LW4: You are misusing the term inference! Here, someone wrote a post about this at some point. LW5: Yea but that post has theoretical limitations. LW1: I don’t care about any of that, I want to know whether or not Less Wrong is succeeding at being rational. Stop making needlessly theoretical abstract arguments and talk about the actual thing we were actually talking about. LW6: I agree, people here use LW jargon as as a form of applause light! LW1: Uh... LW7: You know, accusing others of using applause lights is a fully generalized counter argument! LW6: Oh yea? Well fully generalized counter arguments are fully generalized counter arguments themselves, so there!
We’re only at LW3 right now so maybe this conversation can still be saved from becoming typical Less Wrong-style meta screwery. Or to make my point more politely: Please tell me whether or not you think Less Wrong is rational and whether or not something should be done, because that’s the thing we’re actually talking about.
Dude, my post was precisely about how you’re making a mistake in applying the outside view. Was I being too vague, too referential? Okay, here’s the long version, stripped of jargon because I’m cool like that.
The point of the planning fallacy experiments is that we’re bad at estimating the time we’re going to spend on stuff, mainly because we tend to ignore time sinks that aren’t explicitly part of our model. My boss asks me how long I’m going to spend on a task: I can either look at all the subtasks involved and add up the time they’ll take (the inside view), or I can look at similar tasks I’ve done in the past and report how long they took me (the outside view). The latter is going to be larger, and it’s usually going to be more accurate.
That’s a pretty powerful practical rationality technique, but its domain is limited. We have no idea how far it generalizes, because no one (as far as I know) has rigorously tried to generalize it to things that don’t have to do with time estimation. Using the outside view in its LW-jargon sense, to describe any old thing, therefore is almost completely meaningless; it’s equivalent to saying “this looks to me like a $SCENARIO1”. As long as there also exists a $SCENARIO2, invoking the outside view gives us no way to distinguish between them. Underfitting is a problem. Overfitting is also a problem. Which one’s going to be more of a problem in a particular reference class? There are ways of figuring that out, like Yvain’s centrality heuristic, but crying “outside view” is not one of them.
As to whether LW is rational, I got bored of that kind of hand-wringing years ago. If all you’re really looking for is an up/down vote on that, I suggest a poll, which I will probably ignore because it’s a boring question.
Ok, I guess I could have inferred your meaning from your original post, so sorry if my reply was too snarky. But seriously, if that’s your point I would have just made it like this:
“Dude you’re only supposed to use the phrase ‘outside view’ with regards to the planning fallacy, because we don’t know if the technique generalizes well.”
And then I’d go back and change “take a step back and look at it from the outside view” into “take a step back and look at it from an objective point of view” to prevent confusion, and upvote you for taking the time to correct my usage of the phrase.
My guess is that the site is “probably helping people who are trying to improve”, because I would expect some of the materials here to help. I have certainly found a number of materials useful.
But a personal judgement probably helping” isn’t the kind of thing you’d want. It’d be much better to find some way to measure the size of the effect. Not tracking your progress is a bad, bad sign.
The outside view isn’t magic. Finding the right reference class to step back into, in particular, can be tricky, and the experiments the technique is drawn from deal almost exclusively with time forecasting; it’s hard to say how well it generalizes outside that domain.
Don’t take this as quoting scripture, but this has been discussed before, in some detail.
Okay, you’re doing precisely the thing I hate and which I am criticizing about Less Wrong. Allow me to illustrate:
LW1: Guys, it seems to me that Less Wrong is not very rational. What do you think?
LW2: What makes you think Less Wrong isn’t rational?
LW1: Well if you take a step back and use the outside view, Less Wrong seems to be optimizing for being clever rather than optimizing for being rational. That’s a pretty decent indicator.
LW3: Well, the outside view has theoretical limitations, you know. Eliezer wrote a post about how it is possible to misuse the outside point of view as a conversation stopper.
LW1: Uh, well unless I actually made a mistake in applying the outside view I don’t see why that’s relevant? And if I did make a mistake in applying it it would be more helpful to say what it was I specifically did wrong in my inference.
LW4: You are misusing the term inference! Here, someone wrote a post about this at some point.
LW5: Yea but that post has theoretical limitations.
LW1: I don’t care about any of that, I want to know whether or not Less Wrong is succeeding at being rational. Stop making needlessly theoretical abstract arguments and talk about the actual thing we were actually talking about.
LW6: I agree, people here use LW jargon as as a form of applause light!
LW1: Uh...
LW7: You know, accusing others of using applause lights is a fully generalized counter argument!
LW6: Oh yea? Well fully generalized counter arguments are fully generalized counter arguments themselves, so there!
We’re only at LW3 right now so maybe this conversation can still be saved from becoming typical Less Wrong-style meta screwery. Or to make my point more politely: Please tell me whether or not you think Less Wrong is rational and whether or not something should be done, because that’s the thing we’re actually talking about.
Dude, my post was precisely about how you’re making a mistake in applying the outside view. Was I being too vague, too referential? Okay, here’s the long version, stripped of jargon because I’m cool like that.
The point of the planning fallacy experiments is that we’re bad at estimating the time we’re going to spend on stuff, mainly because we tend to ignore time sinks that aren’t explicitly part of our model. My boss asks me how long I’m going to spend on a task: I can either look at all the subtasks involved and add up the time they’ll take (the inside view), or I can look at similar tasks I’ve done in the past and report how long they took me (the outside view). The latter is going to be larger, and it’s usually going to be more accurate.
That’s a pretty powerful practical rationality technique, but its domain is limited. We have no idea how far it generalizes, because no one (as far as I know) has rigorously tried to generalize it to things that don’t have to do with time estimation. Using the outside view in its LW-jargon sense, to describe any old thing, therefore is almost completely meaningless; it’s equivalent to saying “this looks to me like a $SCENARIO1”. As long as there also exists a $SCENARIO2, invoking the outside view gives us no way to distinguish between them. Underfitting is a problem. Overfitting is also a problem. Which one’s going to be more of a problem in a particular reference class? There are ways of figuring that out, like Yvain’s centrality heuristic, but crying “outside view” is not one of them.
As to whether LW is rational, I got bored of that kind of hand-wringing years ago. If all you’re really looking for is an up/down vote on that, I suggest a poll, which I will probably ignore because it’s a boring question.
Ok, I guess I could have inferred your meaning from your original post, so sorry if my reply was too snarky. But seriously, if that’s your point I would have just made it like this:
“Dude you’re only supposed to use the phrase ‘outside view’ with regards to the planning fallacy, because we don’t know if the technique generalizes well.”
And then I’d go back and change “take a step back and look at it from the outside view” into “take a step back and look at it from an objective point of view” to prevent confusion, and upvote you for taking the time to correct my usage of the phrase.
My guess is that the site is “probably helping people who are trying to improve”, because I would expect some of the materials here to help. I have certainly found a number of materials useful.
But a personal judgement probably helping” isn’t the kind of thing you’d want. It’d be much better to find some way to measure the size of the effect. Not tracking your progress is a bad, bad sign.
LW8...rationality is more than one thing