After they encounter a few smart players who defect in non-public rounds, they should learn this.
Unless the smart players didn’t defect in non-public rounds, in which case the dumb players who can only look at their behavior wouldn’t become prejudiced against smart players, and everyone is happy.
But if some of the smart players are still causal decision theorists, and the dumb players can’t distinguish a TDT from a CDT but can distinguish a TDT from a dumb player, then your reward will be based on other people’s assumption that your decision is correlated with something that it really isn’t. Which brings us back to “the mistaken belief that smart players will defect”.
Unless the smart players didn’t defect in non-public rounds, in which case the dumb players who can only look at their behavior wouldn’t become prejudiced against smart players, and everyone is happy.
But notice that this isn’t evolutionarily stable. If a mutation causes a smart player to start defecting in non-public rounds, then it would have an advantage. On the other hand, smart players defecting in non-public rounds is evolutionarily stable. So either TDT also implies that smart players should play defect in non-public rounds, or TDT could never have arisen in the first place by evolution. (I’m not sure which is the case yet, but the disjunction must be true.) I conclude that “the mistaken belief that smart players will defect” isn’t really mistaken.
But notice that this isn’t evolutionarily stable. If a mutation causes a smart player to start defecting in non-public rounds, then it would have an advantage.
Evolutionary stability isn’t about TDT because organisms don’t simulate each other. You, however, are running a very small and simple computation in your own mind when you conclude “smart players should defect on non-public rounds”. But this is assuming the smart player is calculating in a way that doesn’t take into account your simple simulation of them, and your corresponding reaction. So you are not using TDT in your own head here, you are simulating a “smart” CDT decision agent—and CDT agents can indeed be harmed by increased knowledge or intelligence, like being told on which rounds an Omega is filling a Newcomb box “after” rather than “before” their decision. TDT agents, however, win—unless you have mistaken beliefs about them that don’t depend on their real actions, but that’s a genuine fault in you rather than anything dependent on the TDT decision process; and you’ll also suffer when the TDT agents calculate that you are not correctly computing what a TDT agent does, meaning your action is not in fact dependent on the output of their computation.
TDT could never have arisen in the first place by evolution
It didn’t.
Evolutionary biology built humans to have a sense of honor, which isn’t the same thing, but reflects our ancestral inability to calculate the unobserved rounds with exactitude.
TDT can arise in many ways—e.g. a CDT agent who believes they will in the future face Newcomblike problems will self-modify to use TDT for all Newcomblike problems dependent on decisions made after the instant of CDT self-modification, i.e., “use TDT for problems dependent on my decision after 9am on Tuesday and CDT for all problems dependent on decisions before then”. This is inelegant, and a simple application of the unknown meta-decision-theory that wakes up and realizes this is stupid, says “Just use TDT throughout”. A truepure CDT agent would never realize this and would just end up with an ugly and awkward decision theory in descendants, which points up the importance of the meta-problem.
But evolutionary dynamics simply are not decision-theory dynamics. You might as well point out that no interstellar travel could arise by evolutionary biology because there’s no incremental advantage to getting halfway to another solar system.
I think my earlier comments may not have been as clear as they could be. Let me back off and try again. We should distinguish between two different questions:
Is my article correct and relevant within the context of the past evolution of intelligence?
What happens from now on?
I don’t think you’ve given any arguments against 1. Since TDT didn’t arise from evolution, and it wasn’t invented until recently, clearly TDT-related arguments aren’t relevant as far as question 1 is concerned. So again, I see no reason to retract the article.
As for 2, I have some doubts about this:
“Just use TDT throughout”
I’m trying to explore it using this puzzle. Do you have any thoughts on it?
Woah, it took me a long time to parse “Smart Losers”. The technical parts of the article seem to be correct, but as for its evolutional relevance… In your scenario, being smart doesn’t hurt you, being known to be smart does; so it’s most advantageous to be “secretly smart”. So if your conclusions were correct, we’d probably see many adaptations aimed at concealing our intelligence from people we interact with.
So if your conclusions were correct, we’d probably see many adaptations aimed at concealing our intelligence from people we interact with.
Not if the cost of concealing intelligence was too high. Our ancestors lived in tribes with a lot of gossip. Trying to conceal intelligence would have entailed pretending to be dumb at virtually all times, which implies giving up most of the benefits of being intelligent.
Trying to conceal intelligence would have entailed pretending to be dumb at virtually all times, which implies giving up most of the benefits of being intelligent.
There would still be benefits if your model is at all accurate and there are ‘secret rounds’ in ordinary human life. Just pretend to be stupid in public and then be smart in private rounds. To frustrate this, one would need to assume that the additional smartness costs too much. (It is so expensive that it outweighs the gains, or the gains are minimal so any cost outweighs them.)
It seems reasonable to me that there are private rounds in real life and that smartness is a net win.
Unless the smart players didn’t defect in non-public rounds, in which case the dumb players who can only look at their behavior wouldn’t become prejudiced against smart players, and everyone is happy.
But if some of the smart players are still causal decision theorists, and the dumb players can’t distinguish a TDT from a CDT but can distinguish a TDT from a dumb player, then your reward will be based on other people’s assumption that your decision is correlated with something that it really isn’t. Which brings us back to “the mistaken belief that smart players will defect”.
But notice that this isn’t evolutionarily stable. If a mutation causes a smart player to start defecting in non-public rounds, then it would have an advantage. On the other hand, smart players defecting in non-public rounds is evolutionarily stable. So either TDT also implies that smart players should play defect in non-public rounds, or TDT could never have arisen in the first place by evolution. (I’m not sure which is the case yet, but the disjunction must be true.) I conclude that “the mistaken belief that smart players will defect” isn’t really mistaken.
Evolutionary stability isn’t about TDT because organisms don’t simulate each other. You, however, are running a very small and simple computation in your own mind when you conclude “smart players should defect on non-public rounds”. But this is assuming the smart player is calculating in a way that doesn’t take into account your simple simulation of them, and your corresponding reaction. So you are not using TDT in your own head here, you are simulating a “smart” CDT decision agent—and CDT agents can indeed be harmed by increased knowledge or intelligence, like being told on which rounds an Omega is filling a Newcomb box “after” rather than “before” their decision. TDT agents, however, win—unless you have mistaken beliefs about them that don’t depend on their real actions, but that’s a genuine fault in you rather than anything dependent on the TDT decision process; and you’ll also suffer when the TDT agents calculate that you are not correctly computing what a TDT agent does, meaning your action is not in fact dependent on the output of their computation.
It didn’t.
Evolutionary biology built humans to have a sense of honor, which isn’t the same thing, but reflects our ancestral inability to calculate the unobserved rounds with exactitude.
TDT can arise in many ways—e.g. a CDT agent who believes they will in the future face Newcomblike problems will self-modify to use TDT for all Newcomblike problems dependent on decisions made after the instant of CDT self-modification, i.e., “use TDT for problems dependent on my decision after 9am on Tuesday and CDT for all problems dependent on decisions before then”. This is inelegant, and a simple application of the unknown meta-decision-theory that wakes up and realizes this is stupid, says “Just use TDT throughout”. A true pure CDT agent would never realize this and would just end up with an ugly and awkward decision theory in descendants, which points up the importance of the meta-problem.
But evolutionary dynamics simply are not decision-theory dynamics. You might as well point out that no interstellar travel could arise by evolutionary biology because there’s no incremental advantage to getting halfway to another solar system.
I think my earlier comments may not have been as clear as they could be. Let me back off and try again. We should distinguish between two different questions:
Is my article correct and relevant within the context of the past evolution of intelligence?
What happens from now on?
I don’t think you’ve given any arguments against 1. Since TDT didn’t arise from evolution, and it wasn’t invented until recently, clearly TDT-related arguments aren’t relevant as far as question 1 is concerned. So again, I see no reason to retract the article.
As for 2, I have some doubts about this:
I’m trying to explore it using this puzzle. Do you have any thoughts on it?
Woah, it took me a long time to parse “Smart Losers”. The technical parts of the article seem to be correct, but as for its evolutional relevance… In your scenario, being smart doesn’t hurt you, being known to be smart does; so it’s most advantageous to be “secretly smart”. So if your conclusions were correct, we’d probably see many adaptations aimed at concealing our intelligence from people we interact with.
Not if the cost of concealing intelligence was too high. Our ancestors lived in tribes with a lot of gossip. Trying to conceal intelligence would have entailed pretending to be dumb at virtually all times, which implies giving up most of the benefits of being intelligent.
There would still be benefits if your model is at all accurate and there are ‘secret rounds’ in ordinary human life. Just pretend to be stupid in public and then be smart in private rounds. To frustrate this, one would need to assume that the additional smartness costs too much. (It is so expensive that it outweighs the gains, or the gains are minimal so any cost outweighs them.)
It seems reasonable to me that there are private rounds in real life and that smartness is a net win.