On the topic of impact measures, I’ll repeat what I’ve said before: I think that it’s hard to satisfy the conjunction of three desiderata—objectivity (no dependence on human values), safety (preventing any catastrophic outcomes) and usefulness (the AI system is still able to do useful things). Impact measures are very clearly aiming for the first two criteria, but usually don’t have much to say about the third one. My expectation is that there is a strong tradeoff between the first two criteria and the third one, and impact measures have not dealt with this fact yet, but will have to at some point.
But it’s true that that quoted passage is the best summary of my current position. Daniel’s answer is a good example of an underlying intuition that drives this position.
Here’s a relevant passage by Rohin (from Alignment Newsletter #49, March 2019):
Other relevant writing of mine:
Comment on the AUP post
Comment on the desiderata post
But it’s true that that quoted passage is the best summary of my current position. Daniel’s answer is a good example of an underlying intuition that drives this position.