I agree that Bob is probably a weak analogy because who knows how stock-pickers actually pick stocks? I hope I didn’t construct a reverse intuition pump. Still, Eliezer’s job is either a research mathematician or a philosopher of epistemology, depending on what matters to you. Both those jobs depend quite a bit on getting the rules of probability right. I think disagreeing with Eliezer on the rules of probability isn’t something one can ignore.
You also hit an important point with the question: is Bob a quant researcher? Or more specifically, what’s his epistemology in each domain? A few years ago I met Robert Aumann, everyone’s favorite theist-mathematician. It’s clear to me that Aumann has separate epistemologies for claims about God, Israeli politics and game theory. I read his book about games with incomplete information, he actually goes to the trouble of writing out every single proof, it never says “God told me this is the upper bound”. If Aumann tells me “I’ve proven with rigorous math than an AI boxing system is possible”, I sit up and pay attention. If he tells me “AIs don’t have souls so humans will always be superior” I won’t take that argument seriously.
Eliezer claims and seems to have a single epistemology (EWOR) that he applies to all domains. Because of that, his accuracy in one domain directly reflects on the dependability of EWOR, and thus on his accuracy in other domains.
Upvoted for criticizing the actual point :)
I agree that Bob is probably a weak analogy because who knows how stock-pickers actually pick stocks? I hope I didn’t construct a reverse intuition pump. Still, Eliezer’s job is either a research mathematician or a philosopher of epistemology, depending on what matters to you. Both those jobs depend quite a bit on getting the rules of probability right. I think disagreeing with Eliezer on the rules of probability isn’t something one can ignore.
You also hit an important point with the question: is Bob a quant researcher? Or more specifically, what’s his epistemology in each domain? A few years ago I met Robert Aumann, everyone’s favorite theist-mathematician. It’s clear to me that Aumann has separate epistemologies for claims about God, Israeli politics and game theory. I read his book about games with incomplete information, he actually goes to the trouble of writing out every single proof, it never says “God told me this is the upper bound”. If Aumann tells me “I’ve proven with rigorous math than an AI boxing system is possible”, I sit up and pay attention. If he tells me “AIs don’t have souls so humans will always be superior” I won’t take that argument seriously.
Eliezer claims and seems to have a single epistemology (EWOR) that he applies to all domains. Because of that, his accuracy in one domain directly reflects on the dependability of EWOR, and thus on his accuracy in other domains.