Strong agree, I think EY overstates a bit not the capabilities of an ASI, but the capabilities of pure intellect in general—I don’t think you really can just get infinite leverage out of arbitrarily tiny amounts of information if only you are smart enough. It would require that information to be uniquely determined (which it obviously isn’t; trivially, the law of the universe could be a horribly hackneyed “these pixels appear in this sequence”, just a lookup table of PNGs), or at least that the “correct” explanation is invariably the highest entropy one (which rules out lookup tables, sure, but I’d say Newtonian gravity is higher entropy than General Relativity, all other things equal).
Though:
Why should gravitational force be proportional to mM/r^2
I assume here the point could be “because it’s a 3D world and an inverse square force keeps flux constant”. That is a highest entropy answer, but again, as you pointed out, the ASI could just be fed frames from a simulation in which gravity follows some weird ad hoc law. Or snapshots from a cellular automaton that just happens to produce patterns that evolve precisely like a picture of a falling apple.
I think an ASI would need some strong priors about what “kinds” of worlds it should look for to even reach the conclusion of 3D objects and Newtonian gravity (the latter I would say would be preferred by simple symmetry arguments if you know that space has three dimensions and prefer laws not to be special at any point in it; though there still is the possibility that you’re just watching a video from inside an accelerating spaceship). General relativity, can’t really see any way to reach it from just that, yeah. I suppose you could get there by deduction if you had Newtonian mechanics and electromagnetism, though, so it’s not that far.
Strong agree, I think EY overstates a bit not the capabilities of an ASI, but the capabilities of pure intellect in general—I don’t think you really can just get infinite leverage out of arbitrarily tiny amounts of information if only you are smart enough. It would require that information to be uniquely determined (which it obviously isn’t; trivially, the law of the universe could be a horribly hackneyed “these pixels appear in this sequence”, just a lookup table of PNGs), or at least that the “correct” explanation is invariably the highest entropy one (which rules out lookup tables, sure, but I’d say Newtonian gravity is higher entropy than General Relativity, all other things equal).
Though:
I assume here the point could be “because it’s a 3D world and an inverse square force keeps flux constant”. That is a highest entropy answer, but again, as you pointed out, the ASI could just be fed frames from a simulation in which gravity follows some weird ad hoc law. Or snapshots from a cellular automaton that just happens to produce patterns that evolve precisely like a picture of a falling apple.
I think an ASI would need some strong priors about what “kinds” of worlds it should look for to even reach the conclusion of 3D objects and Newtonian gravity (the latter I would say would be preferred by simple symmetry arguments if you know that space has three dimensions and prefer laws not to be special at any point in it; though there still is the possibility that you’re just watching a video from inside an accelerating spaceship). General relativity, can’t really see any way to reach it from just that, yeah. I suppose you could get there by deduction if you had Newtonian mechanics and electromagnetism, though, so it’s not that far.