Well, maybe. Or maybe not. The “irrational” behavior might be optimal as it is. (Probably not, but we can’t know until we look to see, right?) One hopes that successful, but irrational-looking behaviors have rational cores. But it might be that the benefit (the rational part) only comes with the associated costs (the irrational-looking part).
The probability of random irrational behavior being optimal is extremely low. The space of behaviors is super huge.
Also, If you can show me even one case where the irrationality is necessary, I will be very surprised. My current model of the mind has no room for such dependencies.
If, as I believe, no such dependency is possible, then observing that some behavior has irrational components is pretty strong evidence for non-optimality.
Interesting. What kind of possibility do you have in mind when you say that no such dependency is possible? I can definitely imagine simplistic, abstract worlds in which there is a cost that necessarily has to be paid in order to get a larger benefit.
Example. Consider a world in which there are two kinds of game that you play against Omega. Games of type A have huge payouts. Games of type B have modest payouts. Both kinds of game come up with roughly equal regularity. Omega has rigged the games so that you win at type A games iff you implement an algorithm that loses at type B games. From the perspective of any type B game, it will look like you are behaving irrationally. And in some sense, you are. But nonetheless, given the way the world is, it is rational to lose at type B games.
Anyway, I can see why you might want to ignore such fanciful cases. But I would really like to know where you confidence comes from that nothing like this happens in the actual world.
What kind of possibility do you have in mind when you say that no such dependency is possible
This universe, things you might actually run into (including omega’s usual tricks, tho ve could certainly come up with something to break my assumption). I know of no reason that there are gains that are lost once you become a rationalist. And I have no reason to believe that there might be.
I can’t explain why. I don’t have introspective access to my algorithms, sorry.
I can definitely imagine simplistic, abstract worlds in which there is a cost that necessarily has to be paid in order to get a larger benefit.
But a cost that can’t be analyzed rationally and paid by a rationalist who knows what they are doing? I don’t buy it.
Game A, Game B, Omega
you may be behaving irrationally in game B, but that’s ok, because Game B isn’t the game you are winning.
You can take almost any rationally planned behavior out of context such that it looks irrational. The proof is that locally optimal/greedy algorithms are not always globally optimal.
If you look at the context where your strategy is winning, it looks rational, so this example does not apply.
If you look at the context where your strategy is winning, it looks rational, so this example does not apply.
I think maybe we’re talking past each other, then. I thought the idea was to imagine cases where the algorithm or collection of behaviors generated by the algorithm is rational even though it has sub-parts that do not look rational. You are absolutely right when you say that in-context, the play on Game B is rational. But that’s the whole point I was making. It is possible to have games where optimal play globally requires sub-optimal play locally.
That is why I put “irrational” in those scare quotes in my first comment. If a behavior really is optimal, then any appearance of irrationality that it has must come from a failure to see the right context.
ok good point. It’s still not optimal tho. And if it does work (not optimal, just works), that means it’s got some rational core to it that we can discover and purify.
Yup. I imagine that there are some cases where you can get the benefits without the costs but I’d doubt that it’s always the case. It seems perfectly plausible that sometimes happiness/success relies on genuinely not knowing or not thinking about the truth.
Well, maybe. Or maybe not. The “irrational” behavior might be optimal as it is. (Probably not, but we can’t know until we look to see, right?) One hopes that successful, but irrational-looking behaviors have rational cores. But it might be that the benefit (the rational part) only comes with the associated costs (the irrational-looking part).
The probability of random irrational behavior being optimal is extremely low. The space of behaviors is super huge.
Also, If you can show me even one case where the irrationality is necessary, I will be very surprised. My current model of the mind has no room for such dependencies.
If, as I believe, no such dependency is possible, then observing that some behavior has irrational components is pretty strong evidence for non-optimality.
Interesting. What kind of possibility do you have in mind when you say that no such dependency is possible? I can definitely imagine simplistic, abstract worlds in which there is a cost that necessarily has to be paid in order to get a larger benefit.
Example. Consider a world in which there are two kinds of game that you play against Omega. Games of type A have huge payouts. Games of type B have modest payouts. Both kinds of game come up with roughly equal regularity. Omega has rigged the games so that you win at type A games iff you implement an algorithm that loses at type B games. From the perspective of any type B game, it will look like you are behaving irrationally. And in some sense, you are. But nonetheless, given the way the world is, it is rational to lose at type B games.
Anyway, I can see why you might want to ignore such fanciful cases. But I would really like to know where you confidence comes from that nothing like this happens in the actual world.
This universe, things you might actually run into (including omega’s usual tricks, tho ve could certainly come up with something to break my assumption). I know of no reason that there are gains that are lost once you become a rationalist. And I have no reason to believe that there might be.
I can’t explain why. I don’t have introspective access to my algorithms, sorry.
But a cost that can’t be analyzed rationally and paid by a rationalist who knows what they are doing? I don’t buy it.
you may be behaving irrationally in game B, but that’s ok, because Game B isn’t the game you are winning.
You can take almost any rationally planned behavior out of context such that it looks irrational. The proof is that locally optimal/greedy algorithms are not always globally optimal.
If you look at the context where your strategy is winning, it looks rational, so this example does not apply.
I think maybe we’re talking past each other, then. I thought the idea was to imagine cases where the algorithm or collection of behaviors generated by the algorithm is rational even though it has sub-parts that do not look rational. You are absolutely right when you say that in-context, the play on Game B is rational. But that’s the whole point I was making. It is possible to have games where optimal play globally requires sub-optimal play locally.
That is why I put “irrational” in those scare quotes in my first comment. If a behavior really is optimal, then any appearance of irrationality that it has must come from a failure to see the right context.
Our behaviors based on system 1 judgments aren’t “random”; they are likely psychological adaptions that were ESS’s in the ancestral environment.
ok good point. It’s still not optimal tho. And if it does work (not optimal, just works), that means it’s got some rational core to it that we can discover and purify.
Is the space of surviving behaviors super huge?
Yup. I imagine that there are some cases where you can get the benefits without the costs but I’d doubt that it’s always the case. It seems perfectly plausible that sometimes happiness/success relies on genuinely not knowing or not thinking about the truth.