This post introduces the concept of the _locality_ of a goal, that is, how “far” away the target of the goal is. For example, a thermometer’s “goal” is very local: it “wants” to regulate the temperature of this room, and doesn’t “care” about the temperature of the neighboring house. In contrast, a paperclip maximizer has extremely nonlocal goals, as it “cares” about paperclips anywhere in the universe. We can also consider whether the goal depends on the agent’s internals, its input, its output, and/or the environment.
The concept is useful because for extremely local goals (usually goals about the internals or the input) we would expect wireheading or tampering, whereas for extremely nonlocal goals, we would instead expect convergent instrumental subgoals like resource acquisition.
I ask myself if there’s anything in particular I want to say about the post / paper that the author(s) didn’t say, with an emphasis on ensuring that the opinion has content. If yes, then I write it.
(Sorry, that’s not very informative, but I don’t really have a system for it.)
Planned summary for the Alignment Newsletter:
Thanks for the summary! It’s representative of the idea.
Just by curiosity, how do you decide for which posts/paper you want to write an opinion?
I ask myself if there’s anything in particular I want to say about the post / paper that the author(s) didn’t say, with an emphasis on ensuring that the opinion has content. If yes, then I write it.
(Sorry, that’s not very informative, but I don’t really have a system for it.)
No worries, that’s a good answer. I was just curious, not expecting a full-fledged system. ;)