Is this the passage you’re referring to that means I’m “fundamentally misunderstanding computation”?
suppose we actually wanted to use a waterfall to help us calculate chess moves. [...] I conjecture that, given any chess-playing algorithm A that accesses a “waterfall oracle” W, there is an equally-good chess-playing algorithm A0, with similar time and space requirements, that does not access W. If this conjecture holds, then it gives us a perfectly observer-independent way to formalize our intuition that the “semantics” of waterfalls have nothing to do with chess.
This boils down to the Chalmers response. He isn’t arguing that the waterfall couldn’t implement a single pass through of a chess game, but it couldn’t robustly play many different chess games. I discuss the Chalmers response in the appendix and why I think it doesn’t fix the issue.
Yes, and no, it does not boil down to Chalmer’s argument. (as Aaronson makes clear in the paragraph before the one you quote, where he cites the Chalmers argument!) The argument from complexity is about the nature and complexity of systems capable of playing chess—which is why I think you need to carefully read the entire piece and think about what it says.
But as a small rejoinder, if we’re talking about playing a single game, the entire argument is ridiculous; I can write the entire “algorithm” a kilobyte of specific instructions. So it’s not that an algorithm must be capable of playing multiple counterfactual games to qualify, or that counterfactuals are required for moral weight—it’s that the argument hinges on a misunderstanding of how complex different classes of system need to be to do the things they do.
PS. Apologies that the original response comes off as combative—I really think this discussion is important, and wanted to engage to correct an important point, but have very little time to do so at the moment!
Is this the passage you’re referring to that means I’m “fundamentally misunderstanding computation”?
This boils down to the Chalmers response. He isn’t arguing that the waterfall couldn’t implement a single pass through of a chess game, but it couldn’t robustly play many different chess games. I discuss the Chalmers response in the appendix and why I think it doesn’t fix the issue.
Yes, and no, it does not boil down to Chalmer’s argument. (as Aaronson makes clear in the paragraph before the one you quote, where he cites the Chalmers argument!) The argument from complexity is about the nature and complexity of systems capable of playing chess—which is why I think you need to carefully read the entire piece and think about what it says.
But as a small rejoinder, if we’re talking about playing a single game, the entire argument is ridiculous; I can write the entire “algorithm” a kilobyte of specific instructions. So it’s not that an algorithm must be capable of playing multiple counterfactual games to qualify, or that counterfactuals are required for moral weight—it’s that the argument hinges on a misunderstanding of how complex different classes of system need to be to do the things they do.
PS. Apologies that the original response comes off as combative—I really think this discussion is important, and wanted to engage to correct an important point, but have very little time to do so at the moment!