This isn’t a full response, but it seems to me that Vika is largely talking about problems she percieves with impact measures in general, as defined by “measures of how much impact things have on the world”, and is thinking of AUP as an element of this class (as would I, had I not read this comment). Reasons to think this include:
A perception of your research as primarily being the development of AUP, and of this post as being research for that development and exposition.
If AUP is not in fact about restricting an agent’s impact on the world (or, in other words, on the state of the world), then I would describe it as something other than an “impact measure”, since that term is primarily used by people using the way of thinking you denounce (and I believe was invented that way: it seems to have morphed from ‘side effects’, which strongly suggests effects on parts of the world, according to my quick looking-over of the relevant section of Concrete Problems in AI Safety). Perhaps “optimisation regularisation technique” would be better, although I don’t presume to understand your way of thinking about it.
If AUP is not in fact about restricting an agent’s impact on the world (or, in other words, on the state of the world)
So the end result is this, but it doesn’t do it by considering impact to be a thing that happens to the state primarily, but rather to agents; impact not in the sense of “how different is the state”, but “how big of a deal is this to me?”. The objective is to limit the agent’s impact on us, which I think is the more important thing. I think this still falls under normal colloquial use of ‘impact’, but I agree that this is different from the approaches so far. I’m going to talk about this distinction quite a bit in the future.
This isn’t a full response, but it seems to me that Vika is largely talking about problems she percieves with impact measures in general, as defined by “measures of how much impact things have on the world”, and is thinking of AUP as an element of this class (as would I, had I not read this comment). Reasons to think this include:
A perception of your research as primarily being the development of AUP, and of this post as being research for that development and exposition.
The introduction of AUP being in a post titled “Towards a New Impact Measure”.
If AUP is not in fact about restricting an agent’s impact on the world (or, in other words, on the state of the world), then I would describe it as something other than an “impact measure”, since that term is primarily used by people using the way of thinking you denounce (and I believe was invented that way: it seems to have morphed from ‘side effects’, which strongly suggests effects on parts of the world, according to my quick looking-over of the relevant section of Concrete Problems in AI Safety). Perhaps “optimisation regularisation technique” would be better, although I don’t presume to understand your way of thinking about it.
So the end result is this, but it doesn’t do it by considering impact to be a thing that happens to the state primarily, but rather to agents; impact not in the sense of “how different is the state”, but “how big of a deal is this to me?”. The objective is to limit the agent’s impact on us, which I think is the more important thing. I think this still falls under normal colloquial use of ‘impact’, but I agree that this is different from the approaches so far. I’m going to talk about this distinction quite a bit in the future.