Not Paul, but some possibilities why ARC’s work wouldn’t be relevant for self-driving cars:
The stuff Paul said about them aiming at understanding quite simple human values (don’t kill us all, maintain our decision-making power) rather than subtle things. It’s likely for self-driving cars we’re more concerned with high reliability and hence would need to be quite specific. E.g., maybe ARC’s approach could discern whether a car understands whether it’s driving on the road or not (seems like a fairly simple concept), but not whether it’s driving in a riskier way than humans in specific scenarios.
One of the problems that I think ARC is worried about is ontology identification, which seems like a meaningfully different problem for sub-human systems (whose ontologies are worse than ours, so in theory could be injected into ours) than for human-level or super-human systems (where that may not hold). Hence focusing on the super-human case would look weird and possibly not helpful for the subhuman case, although it would be great if they could solve all the cases in full generality.
Maybe once it works ARC’s approach could inform empirical work which helps with self-driving cars, but if you were focused on actually doing the thing for cars you’d just aim directly at that, whereas ARC’s approach would be a very roundabout and needlessly complex and theoretical way of solving the problem (this may or may not actually be the case, maybe solving this for self-driving cars is actually fundamentally difficult in the same way as for ASI, but it seems less likely).
Not Paul, but some possibilities why ARC’s work wouldn’t be relevant for self-driving cars:
The stuff Paul said about them aiming at understanding quite simple human values (don’t kill us all, maintain our decision-making power) rather than subtle things. It’s likely for self-driving cars we’re more concerned with high reliability and hence would need to be quite specific. E.g., maybe ARC’s approach could discern whether a car understands whether it’s driving on the road or not (seems like a fairly simple concept), but not whether it’s driving in a riskier way than humans in specific scenarios.
One of the problems that I think ARC is worried about is ontology identification, which seems like a meaningfully different problem for sub-human systems (whose ontologies are worse than ours, so in theory could be injected into ours) than for human-level or super-human systems (where that may not hold). Hence focusing on the super-human case would look weird and possibly not helpful for the subhuman case, although it would be great if they could solve all the cases in full generality.
Maybe once it works ARC’s approach could inform empirical work which helps with self-driving cars, but if you were focused on actually doing the thing for cars you’d just aim directly at that, whereas ARC’s approach would be a very roundabout and needlessly complex and theoretical way of solving the problem (this may or may not actually be the case, maybe solving this for self-driving cars is actually fundamentally difficult in the same way as for ASI, but it seems less likely).