I generally liked this post, but have some caveats and questions.
Why does local validity work as well as it does in math? This post explains why it will lead you only to the truth, but why does it often lead to mathematical truths that we care about in a reasonable amount of time? In other words, why aren’t most interesting math questions like P=NP, or how to win a game of chess?
Relying purely on local validity won’t get you very far in playing chess, or life in general, and we instead have to frequently use intuitions. As Jan_Kulveit pointed out, it’s generally not introspectively accessible why we have the intuitions that we do, so we sometimes make up bad arguments to explain them. So if someone points out the badness of an argument, it’s important to adjust your credence downwards only as much as you considered the bad argument to be additional evidence on top of your intuitions.
Lastly, I can see the analogy between proofs and more general arguments, but I’m not sure about the third thing, law/coordination. I mean I can see some surface similarities between them, but are there also supposed to be deep correspondences between their underlying mechanisms? If so, I’m afraid I didn’t get what they are.
Relying purely on local validity won’t get you very far in playing chess
The equivalent of local validity is just mechanically checking “okay, if I make this move, then they can make that move” for a bunch of cases. Which, first, is a major developmental milestone for kids learning chess. So we only think it “won’t get you very far” because all the high-level human play explicitly or implicitly takes it for granted.
And secondly, it’s pretty analogous to doing math; proving theorems is based on the ability to check the local validity of each step, but mathematicians aren’t just brute-forcing their way to proofs. They have to develop higher-level heuristics, some of which are really hard to express in language, to suggest avenues, and then check local validity once they have a skeleton of some part of the argument. But if mathematicians stopped doing that annoying bit, well, then after a while you’ll end up with another crisis of analysis when the brilliant intuitions are missing some tiny ingredient.
Local validity is an incredibly important part of any scientific discipline; the fact that it’s not a part of most political discourse is merely a reflection that our society is at about the developmental level of a seven-year-old when it comes to political reasoning.
I suspect there may be a miscommunication here. To elaborate on “Relying purely on local validity won’t get you very far in playing chess”, what I had in mind is that if you decided to play a move only if you can prove that’s it’s the optimal move, you won’t get very far, since we can’t produce proofs of this form (even by using higher-level heuristics to guide us). Was your comment meant as a response to this point, or to a different interpretation of what I wrote?
My comment was meant to explain what I understood Eliezer to be saying, because I think you had misinterpreted that. The OP is simply saying “don’t give weight to arguments that are locally invalid, regardless of what else you like about them”. Of course you need to use priors, heuristics, and intuitions in areas where you can’t find an argument that carries you from beginning to end. But being able to think “oh, if I move there, then they can take my queen, and I don’t see anything else good about that position, so let’s not do that then” is a fair bit easier than proving your move optimal.
Oh, I see. I think your understanding of Eliezer makes sense, and the sentence you responded to wasn’t really meant as an argument against the OP, but rather as setup for the point of that paragraph, which was “if someone points out the badness of an argument, it’s important to adjust your credence downwards only as much as you considered the bad argument to be additional evidence on top of your intuitions.”
To elaborate, I thought there was a possible mistake someone might make after reading the OP and wanted to warn against that. Specifically the mistake is that someone makes a bad argument to explain an intuition, the badness of the argument is pointed out, they accept that and then give up their intuition or adjust their credence downward too much. This is not an issue for most people who haven’t read the OP because they would just refuse to accept the badness of the argument.
(ETA: On second thought maybe I did initially misinterpret Eliezer and meant to argue against him, and am now forgetting that and giving a different motivation for what I wrote. In any case, I currently think your interpretation is correct, and what I wrote may still be valuable in that light. :)
Why does local validity work as well as it does in math? <...> In other words, why aren’t most interesting math questions like P=NP, or how to win a game of chess?
Why do you think that it works well? Are you sure most possible mathematical questions aren’t exactly like P=NP, or worse? The set of “interesting” questions isn’t representative of all questions, this set starts with “2+2=?” and grows slowly, new questions become “interesting” only after old ones are answered. There is also some intuition about which questions might be answerable and which might be too hard, that further guides the construction of this set of “interesting” questions.
I think the key difference between math and chess is that chess is a two player game with a simple goal. In math there is no competitive pressure to be right about statements fast. If you have an intuition that says P=NP, then nobody cares, you get no reward from being right, unless that intuition also leads to a proof (sometimes it does). But if you have an intuition that f3 is the best chess opening move, you win games and then people care. I’m suggesting that if there was a way to “win” math by finding true statements regardless of proof, then you’d see how powerless local validity is.
I generally liked this post, but have some caveats and questions.
Why does local validity work as well as it does in math? This post explains why it will lead you only to the truth, but why does it often lead to mathematical truths that we care about in a reasonable amount of time? In other words, why aren’t most interesting math questions like P=NP, or how to win a game of chess?
Relying purely on local validity won’t get you very far in playing chess, or life in general, and we instead have to frequently use intuitions. As Jan_Kulveit pointed out, it’s generally not introspectively accessible why we have the intuitions that we do, so we sometimes make up bad arguments to explain them. So if someone points out the badness of an argument, it’s important to adjust your credence downwards only as much as you considered the bad argument to be additional evidence on top of your intuitions.
Lastly, I can see the analogy between proofs and more general arguments, but I’m not sure about the third thing, law/coordination. I mean I can see some surface similarities between them, but are there also supposed to be deep correspondences between their underlying mechanisms? If so, I’m afraid I didn’t get what they are.
The equivalent of local validity is just mechanically checking “okay, if I make this move, then they can make that move” for a bunch of cases. Which, first, is a major developmental milestone for kids learning chess. So we only think it “won’t get you very far” because all the high-level human play explicitly or implicitly takes it for granted.
And secondly, it’s pretty analogous to doing math; proving theorems is based on the ability to check the local validity of each step, but mathematicians aren’t just brute-forcing their way to proofs. They have to develop higher-level heuristics, some of which are really hard to express in language, to suggest avenues, and then check local validity once they have a skeleton of some part of the argument. But if mathematicians stopped doing that annoying bit, well, then after a while you’ll end up with another crisis of analysis when the brilliant intuitions are missing some tiny ingredient.
Local validity is an incredibly important part of any scientific discipline; the fact that it’s not a part of most political discourse is merely a reflection that our society is at about the developmental level of a seven-year-old when it comes to political reasoning.
I suspect there may be a miscommunication here. To elaborate on “Relying purely on local validity won’t get you very far in playing chess”, what I had in mind is that if you decided to play a move only if you can prove that’s it’s the optimal move, you won’t get very far, since we can’t produce proofs of this form (even by using higher-level heuristics to guide us). Was your comment meant as a response to this point, or to a different interpretation of what I wrote?
My comment was meant to explain what I understood Eliezer to be saying, because I think you had misinterpreted that. The OP is simply saying “don’t give weight to arguments that are locally invalid, regardless of what else you like about them”. Of course you need to use priors, heuristics, and intuitions in areas where you can’t find an argument that carries you from beginning to end. But being able to think “oh, if I move there, then they can take my queen, and I don’t see anything else good about that position, so let’s not do that then” is a fair bit easier than proving your move optimal.
Oh, I see. I think your understanding of Eliezer makes sense, and the sentence you responded to wasn’t really meant as an argument against the OP, but rather as setup for the point of that paragraph, which was “if someone points out the badness of an argument, it’s important to adjust your credence downwards only as much as you considered the bad argument to be additional evidence on top of your intuitions.”
To elaborate, I thought there was a possible mistake someone might make after reading the OP and wanted to warn against that. Specifically the mistake is that someone makes a bad argument to explain an intuition, the badness of the argument is pointed out, they accept that and then give up their intuition or adjust their credence downward too much. This is not an issue for most people who haven’t read the OP because they would just refuse to accept the badness of the argument.
(ETA: On second thought maybe I did initially misinterpret Eliezer and meant to argue against him, and am now forgetting that and giving a different motivation for what I wrote. In any case, I currently think your interpretation is correct, and what I wrote may still be valuable in that light. :)
Why do you think that it works well? Are you sure most possible mathematical questions aren’t exactly like P=NP, or worse? The set of “interesting” questions isn’t representative of all questions, this set starts with “2+2=?” and grows slowly, new questions become “interesting” only after old ones are answered. There is also some intuition about which questions might be answerable and which might be too hard, that further guides the construction of this set of “interesting” questions.
I think the key difference between math and chess is that chess is a two player game with a simple goal. In math there is no competitive pressure to be right about statements fast. If you have an intuition that says P=NP, then nobody cares, you get no reward from being right, unless that intuition also leads to a proof (sometimes it does). But if you have an intuition that f3 is the best chess opening move, you win games and then people care. I’m suggesting that if there was a way to “win” math by finding true statements regardless of proof, then you’d see how powerless local validity is.
This is all a bit off topic though.