Can you explain this suspicion? I’m not saying that “Rationalists always win”: I am saying that they win more often than average.
Say you are in society X, which maximizes potential values [1, 2, 7] though mechanism P and minimzies potential values [4, 9, 13] through mechanism Q.
A rationalist (A) who values [1, 4, 9] will likely not do as well as a random agent (B) that values [1, 2, 7] under X, because the rationalist will only get limited help from P while having to counteract Q, while the other agent (rationalist or not) will recieve full benefit from P and no harm from Q. So it’s trivially true that a rationalist does not always do better than other agents: sometimes the game is set against them.
A rationalist (A) will do better than a non-rationalist (C) with values [1, 4, 9] if having an accurate perception of P allows you to maximize P for 1 or having an accurate perception of Q allows you to minimize Q for [4, 9]. In the world we live in, at least, this usually proves true.
But A will also do better than B in any society that isn’t X, unless B is also a rationalist. They will have a more accurate perception of the reality of the society they are in and thus be better able to maximize the mechanisms that aid their values while minimizing the mechanisms that countermand them.
That’s what I meant by “more likely to win over more of those societies than average.”
I haven’t thought about this carefully, so this may be a howler, but here is what I was thinking:
“Winning” is an optimization problem, so you can conceive of the problem of finding the winning strategy in terms of efficiently minimizing some cost function. Different sets of values—different utility functions—will correspond to different cost functions. Rationalism is a particular algorithm for searching for the minimum. Here I associate “rationalism” with the set of concrete epistemic tools recommended by LW; you could, of course, define “rationalism” so that whichever strategy most conduces to winning in a particular context is the rational one, but then your claim would be tautological.
The No Free Lunch Theorem for search and optimization says that all algorithms that search for the minimum of a cost function perform equally well when you average over all possible cost functions. So if you’re really allowing the possibility of any set of values, then the rationalism algorithm is no more likely to win on average than any other search algorithm.
Again, this is a pretty hasty argument, so I’m sure there are holes.
I suspect you are right if we are talking about epistemic rationality, but not instrumental rationality.
In practice, when attempting to maximize a value, once you know what sort of system you are in, most of your energy has to go into gaming the system: finding the cost of minimizing the costs and looking for exploits. This is more true the more times a game is iterated: if a game literally went on forever, any finite cost becomes justifiable for this sort of gaming of the system: you can spend any bounded amount of bits. (Conversely, if a game is unique, you are less justified in spending your bits on finding solutions: your budget roughly becomes what you can afford to spare.)
If we apply LW techniques of rationalism (as you’ve defined it) what we get is general methods, heuristics, and proofs on ways to find these exploits, a summation of this method being something like “know the rules of the world you are in” because your knowledge of a game directly affects your ability to manipulate its rules and scoring.
In other words, I suspect you are right if what we are talking about is simply finding the best situation for your algorithm: choosing the best restaurant in the available solution space. But when we are in a situation where the rules can be manipulated, used, or applied more effectively I believe this dissolves. You could probably convince me pretty quickly with a more formal argument, however.
Can you explain this suspicion? I’m not saying that “Rationalists always win”: I am saying that they win more often than average.
Say you are in society X, which maximizes potential values [1, 2, 7] though mechanism P and minimzies potential values [4, 9, 13] through mechanism Q.
A rationalist (A) who values [1, 4, 9] will likely not do as well as a random agent (B) that values [1, 2, 7] under X, because the rationalist will only get limited help from P while having to counteract Q, while the other agent (rationalist or not) will recieve full benefit from P and no harm from Q. So it’s trivially true that a rationalist does not always do better than other agents: sometimes the game is set against them.
A rationalist (A) will do better than a non-rationalist (C) with values [1, 4, 9] if having an accurate perception of P allows you to maximize P for 1 or having an accurate perception of Q allows you to minimize Q for [4, 9]. In the world we live in, at least, this usually proves true.
But A will also do better than B in any society that isn’t X, unless B is also a rationalist. They will have a more accurate perception of the reality of the society they are in and thus be better able to maximize the mechanisms that aid their values while minimizing the mechanisms that countermand them.
That’s what I meant by “more likely to win over more of those societies than average.”
I haven’t thought about this carefully, so this may be a howler, but here is what I was thinking:
“Winning” is an optimization problem, so you can conceive of the problem of finding the winning strategy in terms of efficiently minimizing some cost function. Different sets of values—different utility functions—will correspond to different cost functions. Rationalism is a particular algorithm for searching for the minimum. Here I associate “rationalism” with the set of concrete epistemic tools recommended by LW; you could, of course, define “rationalism” so that whichever strategy most conduces to winning in a particular context is the rational one, but then your claim would be tautological.
The No Free Lunch Theorem for search and optimization says that all algorithms that search for the minimum of a cost function perform equally well when you average over all possible cost functions. So if you’re really allowing the possibility of any set of values, then the rationalism algorithm is no more likely to win on average than any other search algorithm.
Again, this is a pretty hasty argument, so I’m sure there are holes.
I suspect you are right if we are talking about epistemic rationality, but not instrumental rationality.
In practice, when attempting to maximize a value, once you know what sort of system you are in, most of your energy has to go into gaming the system: finding the cost of minimizing the costs and looking for exploits. This is more true the more times a game is iterated: if a game literally went on forever, any finite cost becomes justifiable for this sort of gaming of the system: you can spend any bounded amount of bits. (Conversely, if a game is unique, you are less justified in spending your bits on finding solutions: your budget roughly becomes what you can afford to spare.)
If we apply LW techniques of rationalism (as you’ve defined it) what we get is general methods, heuristics, and proofs on ways to find these exploits, a summation of this method being something like “know the rules of the world you are in” because your knowledge of a game directly affects your ability to manipulate its rules and scoring.
In other words, I suspect you are right if what we are talking about is simply finding the best situation for your algorithm: choosing the best restaurant in the available solution space. But when we are in a situation where the rules can be manipulated, used, or applied more effectively I believe this dissolves. You could probably convince me pretty quickly with a more formal argument, however.