I promoted this to Featured for being a useful frame for rationality, and also for concluding a marathon of excellent posts. I both want to thank you for writing down so many excellent ideas, and also want everyone else to know IT’S FINISHED NOW YOU CAN READ ALL OF THEM IN ONE GO. We’ll try and get these into a sequence of their own soon.
I think I have almost the exact opposite intuition. It seems to me that most of the coin flips barely matter at all and some small number of them matter hugely, so the game isn’t about getting a ton of heads in a row so much as it is about clearly identifying which heads it really matters to get and then really getting them (and then there’s something to say about giving yourself more opportunities to flip important coins but maybe we’re stretching the metaphor past its intended domain of application at this point).
Otherwise it seems difficult to explain how people who aren’t strong rationalists (i.e. almost everyone including most of us) can lead okay lives and some of them can even be wildly successful.
This idea has interested me a lot and I really want to see data on outcomes. I know CFAR is trying to do this but is anyone else doing this? Anyone who visits this site, can they see how the decisions they’re making now are leading to improved outcomes?
Are the better outcomes that happen in my life due to the fact that I really am making better decisions or are they mostly because my parents never got divorced, etc....
I don’t think I can offer info on how rationality affects later life outcomes. There’s a bunch of confounding / lurking variables in the space, and I’m unsure what to say substantially (Scott’s recent posts on IQ have also shifted me towards more uncertainty, but I’m also not really following all of what’s been researched.)
Here’s what I think we do seem to know:
Debiasing people in the short-term is possible.
People who get calibration training end up more calibrated / make better predictions.
Lots of goal-setting/getting techniques have been shown to be effective in allowing people to achieve their short-term goals, e.g. go on diets, exercise more, floss more, etc.
What I personally seem to have gained from CFAR-esque material is sort of in the way of “stuff I can’t easily verabalize and thus is also perhaps suspicious from the outside view because all I can do is mumble things like ‘ontological upgrade’”.
Concretely, though, I can point to basically 3 TAPs which I have as concrete habits which I counterfactually wouldn’t have had, had I not been exposed to the CFAR content:
A TAP for putting down to-dos into my phone. (Paired w/ switching the ring on my finger to the other hand, in a “string on the wrist” sort of a way.)
A TAP for asking people for examples during conversations.
A TAP for not immediately getting angry when I feel frustrated in conversations.
In the spirit of the OP, it definitely seems like these little TAPs do add up over time. So even if all the other stuff isn’t there (which I emphatically claim that it subjectively feels like it is), I think I’ve personally still benefited.
I’m not sure if this is possible, but I think I’m looking for something like “I used this TAP and got this result where past me would not have used a TAP and gotten this other result which is demonstrably worse”.
This might be a good test of the calibration idea and making better predictions. Can we accurately predict what would happen without the extra thinking tools?
(Slight nitpick is that I meant ‘calibration’ in the Tetlock sense, like being able to make informed judgments about how global events will play out, but I agree there’s certainly an analogous component that maps onto ‘how well you can predict your own life’.)
I currently don’t think that I have good answers to them, so this is sort a placeholder reply until me (or someone else) puts in more thoughts into this line of inquiry.
And I definitely agree with this view of rationality, I think the idea of incremental successes enphasizes the need to track successes and failures over time so that you can see where you did well and where you did poorly and plan to make the coin come up heads more often in the future.
If you forget that the data collection happens through selective action and the data’s meaning is seen through a flawed lens, though, then your ‘objective view’ can wind up more sharply skewed than your vague gut feels.
I promoted this to Featured for being a useful frame for rationality, and also for concluding a marathon of excellent posts. I both want to thank you for writing down so many excellent ideas, and also want everyone else to know IT’S FINISHED NOW YOU CAN READ ALL OF THEM IN ONE GO. We’ll try and get these into a sequence of their own soon.
I hope people enjoy binging.
Now *that’s* how you end a post & a sequence! Well done.
I think I have almost the exact opposite intuition. It seems to me that most of the coin flips barely matter at all and some small number of them matter hugely, so the game isn’t about getting a ton of heads in a row so much as it is about clearly identifying which heads it really matters to get and then really getting them (and then there’s something to say about giving yourself more opportunities to flip important coins but maybe we’re stretching the metaphor past its intended domain of application at this point).
Otherwise it seems difficult to explain how people who aren’t strong rationalists (i.e. almost everyone including most of us) can lead okay lives and some of them can even be wildly successful.
This idea has interested me a lot and I really want to see data on outcomes. I know CFAR is trying to do this but is anyone else doing this? Anyone who visits this site, can they see how the decisions they’re making now are leading to improved outcomes?
Are the better outcomes that happen in my life due to the fact that I really am making better decisions or are they mostly because my parents never got divorced, etc....
Anyone have any thoughts on how to evaluate this?
I don’t think I can offer info on how rationality affects later life outcomes. There’s a bunch of confounding / lurking variables in the space, and I’m unsure what to say substantially (Scott’s recent posts on IQ have also shifted me towards more uncertainty, but I’m also not really following all of what’s been researched.)
Here’s what I think we do seem to know:
Debiasing people in the short-term is possible.
People who get calibration training end up more calibrated / make better predictions.
Lots of goal-setting/getting techniques have been shown to be effective in allowing people to achieve their short-term goals, e.g. go on diets, exercise more, floss more, etc.
What I personally seem to have gained from CFAR-esque material is sort of in the way of “stuff I can’t easily verabalize and thus is also perhaps suspicious from the outside view because all I can do is mumble things like ‘ontological upgrade’”.
Concretely, though, I can point to basically 3 TAPs which I have as concrete habits which I counterfactually wouldn’t have had, had I not been exposed to the CFAR content:
A TAP for putting down to-dos into my phone. (Paired w/ switching the ring on my finger to the other hand, in a “string on the wrist” sort of a way.)
A TAP for asking people for examples during conversations.
A TAP for not immediately getting angry when I feel frustrated in conversations.
In the spirit of the OP, it definitely seems like these little TAPs do add up over time. So even if all the other stuff isn’t there (which I emphatically claim that it subjectively feels like it is), I think I’ve personally still benefited.
I’m not sure if this is possible, but I think I’m looking for something like “I used this TAP and got this result where past me would not have used a TAP and gotten this other result which is demonstrably worse”.
This might be a good test of the calibration idea and making better predictions. Can we accurately predict what would happen without the extra thinking tools?
These are all very good questions.
(Slight nitpick is that I meant ‘calibration’ in the Tetlock sense, like being able to make informed judgments about how global events will play out, but I agree there’s certainly an analogous component that maps onto ‘how well you can predict your own life’.)
I currently don’t think that I have good answers to them, so this is sort a placeholder reply until me (or someone else) puts in more thoughts into this line of inquiry.
Great sequence, I’ve really enjoyed it.
And I definitely agree with this view of rationality, I think the idea of incremental successes enphasizes the need to track successes and failures over time so that you can see where you did well and where you did poorly and plan to make the coin come up heads more often in the future.
Tracking helps avoid some bias.
If you forget that the data collection happens through selective action and the data’s meaning is seen through a flawed lens, though, then your ‘objective view’ can wind up more sharply skewed than your vague gut feels.