...is not true. In the comments to Einstein’s Speed, Scott Aaronson explains the real story: Einstein spent over a year going down a blind alley, and was drawn back by—among other things—his inability to make his calculations fit the observation of Mercury’s perihelion motion. Einstein was able to reason his way from a large hypothesis space to a small one, but not to actually get the right answer.
(and of course, in physics you get a lot of experimental data for free. If you’re working on a theory of gravity and it predicts that things should fall away from each other, you can tell right away that you’ve gone wrong without having to do any new experiments. In AI safety we are not so blessed.)
That’s a really good point. I didn’t go into that debate in the post (because I tried to not criticize Yudkowky, and also because the post is already way too long), but my take on this is: Yudkowsky probably overstates the case, but that doesn’t mean he’s wrong about the relevance for Einstein’s work of the constrains and armchair reasoning (even if the armchair reasoning was building on more empirical evidence that Yudkowsky originally pointed out). As you say, Einstein apparently did reduce the search space significantly: he just failed to find exactly what he wanted in the reduced space directly.
My comment had an important typo, sorry: I meant to write that I hadn’t noticed this through-line before!
I mostly agree with you re: Einstein, but I do think that removing the overstatement changes the conclusion in an important way. Narrowing the search space from (say) thousands of candidate theories to just 4 is an great achievement, but you still need a method of choosing among them, not just to fulfill the persuasive social ritual of Science but because otherwise you have a 3 in 4 chance of being wrong. Even someone who trusts you can’t update that much on those odds. That’s really different from being able to narrow the search space down to just 1 theory; at that point, we can trust you—and better still, you can trust yourself! But the history of science doesn’t, so far as I can tell, contain any “called shots” of this type; Einstein might literally have set the bar.
I think we disagree on Yudkowsky’s conclusion: his point IMO is that Einstein was able to reduce the search space a lot. He overemphasize for effect (and because it’s more impressive to have someone who guesses right directly through these methods), but that doesn’t change that Einstein reduced the state space a lot (which you seem to agree with).
Many of the relevant posts I quoted talk about how the mechanism of Science are fundamentally incapable of doing that, because they don’t specify any constraint on hypothesis except that they must be falsifiable. Your point seems to be that in the end, Einstein still used the sort of experimental data and methods underlying traditional Science, and I tend to agree. But the mere fact that he was able to get the right answer out of millions of possible formulations by checking a couple of numbers should tell you that there was a massive hypothesis-space reducing step before.
Nah, we’re on the same page about the conclusion; my point was more about how we should expect Yudkowsky’s conclusion to generalize into lower-data domains like AI safety. But now that I look at it that point is somewhat OT for your post, sorry.
Thanks for the kind and thoughtful comment!
That’s a really good point. I didn’t go into that debate in the post (because I tried to not criticize Yudkowky, and also because the post is already way too long), but my take on this is: Yudkowsky probably overstates the case, but that doesn’t mean he’s wrong about the relevance for Einstein’s work of the constrains and armchair reasoning (even if the armchair reasoning was building on more empirical evidence that Yudkowsky originally pointed out). As you say, Einstein apparently did reduce the search space significantly: he just failed to find exactly what he wanted in the reduced space directly.
My comment had an important typo, sorry: I meant to write that I hadn’t noticed this through-line before!
I mostly agree with you re: Einstein, but I do think that removing the overstatement changes the conclusion in an important way. Narrowing the search space from (say) thousands of candidate theories to just 4 is an great achievement, but you still need a method of choosing among them, not just to fulfill the persuasive social ritual of Science but because otherwise you have a 3 in 4 chance of being wrong. Even someone who trusts you can’t update that much on those odds. That’s really different from being able to narrow the search space down to just 1 theory; at that point, we can trust you—and better still, you can trust yourself! But the history of science doesn’t, so far as I can tell, contain any “called shots” of this type; Einstein might literally have set the bar.
I think we disagree on Yudkowsky’s conclusion: his point IMO is that Einstein was able to reduce the search space a lot. He overemphasize for effect (and because it’s more impressive to have someone who guesses right directly through these methods), but that doesn’t change that Einstein reduced the state space a lot (which you seem to agree with).
Many of the relevant posts I quoted talk about how the mechanism of Science are fundamentally incapable of doing that, because they don’t specify any constraint on hypothesis except that they must be falsifiable. Your point seems to be that in the end, Einstein still used the sort of experimental data and methods underlying traditional Science, and I tend to agree. But the mere fact that he was able to get the right answer out of millions of possible formulations by checking a couple of numbers should tell you that there was a massive hypothesis-space reducing step before.
Nah, we’re on the same page about the conclusion; my point was more about how we should expect Yudkowsky’s conclusion to generalize into lower-data domains like AI safety. But now that I look at it that point is somewhat OT for your post, sorry.