If you follow his strategy, then you can spend arbitrarily long trying to find a faithful concrete operationalization of a part of the problem that doesn’t exist.
I don’t think that’s how this works? The strategy I’m recommending explicitly contains two parts where we gain evidence about whether a part of the problem actually exists:
noticing an intuitive pattern in the failure-modes of some strategies
attempting to formalize (which presumably includes backpropagating our mathematics into our intuitions)
… so if a part of the problem doesn’t exist, then (a) we probably don’t notice a pattern in the first place, but even if our notoriously unreliable human pattern-matchers over-match, then (b) while we’re attempting to formalize we we have plenty of opportunity to notice that maybe the pattern doesn’t actually exist the way we thought it did.
It feels like you’re looking for a duality which does not exist. I mean, the duality between “look for concrete solutions” and “look for concrete problems” I buy (and that would indeed cause one side to be over-optimistic and the other over-pessimistic in exactly the pattern we actually see between Paul and Nate/Eliezer). But it feels like you’re also looking for a duality between how-Paul’s-recommended-search-order-just-fails and how-mine-just-fails. And the reason that duality does not exist is because my recommended search order is using strictly more evidence; Paul is basically advocating ignoring a whole class of very useful evidence, and that makes his strategy straightforwardly suboptimal. If we were both picking different points on a pareto frontier, then yeah, there’d be a trade-off. But Paul just isn’t on the pareto frontier.
I feel confused about the difference between your “attempt to formalize” step and Paul’s “attempt to concretize” step. It feels like you can view either as a step towards the other—if you successfully formalize, then presumably you’ll be able to concretize; but also one valuable step towards formalizing is by finding concrete examples and then generalizing from them. I think everyone agrees that it’d be great to end up with a formalism for the problem, and then disagrees on how much that process should involve “finding concrete examples of the problem”. My own view is that since it’s so incredibly easy for people to get lost in abstractions, people should try to concretize much more when talking about highly abstract domains. (Even when people are confident that they’re not lost in abstractions, like Eliezer and Nate are, that’s still really useful for conveying ideas to other people.)
I don’t think that’s how this works? The strategy I’m recommending explicitly contains two parts where we gain evidence about whether a part of the problem actually exists:
noticing an intuitive pattern in the failure-modes of some strategies
attempting to formalize (which presumably includes backpropagating our mathematics into our intuitions)
… so if a part of the problem doesn’t exist, then (a) we probably don’t notice a pattern in the first place, but even if our notoriously unreliable human pattern-matchers over-match, then (b) while we’re attempting to formalize we we have plenty of opportunity to notice that maybe the pattern doesn’t actually exist the way we thought it did.
It feels like you’re looking for a duality which does not exist. I mean, the duality between “look for concrete solutions” and “look for concrete problems” I buy (and that would indeed cause one side to be over-optimistic and the other over-pessimistic in exactly the pattern we actually see between Paul and Nate/Eliezer). But it feels like you’re also looking for a duality between how-Paul’s-recommended-search-order-just-fails and how-mine-just-fails. And the reason that duality does not exist is because my recommended search order is using strictly more evidence; Paul is basically advocating ignoring a whole class of very useful evidence, and that makes his strategy straightforwardly suboptimal. If we were both picking different points on a pareto frontier, then yeah, there’d be a trade-off. But Paul just isn’t on the pareto frontier.
I feel confused about the difference between your “attempt to formalize” step and Paul’s “attempt to concretize” step. It feels like you can view either as a step towards the other—if you successfully formalize, then presumably you’ll be able to concretize; but also one valuable step towards formalizing is by finding concrete examples and then generalizing from them. I think everyone agrees that it’d be great to end up with a formalism for the problem, and then disagrees on how much that process should involve “finding concrete examples of the problem”. My own view is that since it’s so incredibly easy for people to get lost in abstractions, people should try to concretize much more when talking about highly abstract domains. (Even when people are confident that they’re not lost in abstractions, like Eliezer and Nate are, that’s still really useful for conveying ideas to other people.)