I really just want to know what Eliezer means by it. It seems to me like I have some notion of an optimisation process, that says “yep, that’s definitely an optimisation process” when I think about evolution and human minds and Clippy, and says “nope, that’s not really an optimisation process—at least, not one worth the name” about water rolling down a hill and thermodynamics. And I think this notion is sufficiently similar to the one that Eliezer is using. But my attempts to formalise this definition, to reduce it, have failed—I can’t find any combination of words that seems to capture the boundary that my intuition is drawing between Clippy and gravity.
Treating the boundary between Clippy and gravity as a quantitative one (optimization power over a wide range of targets, as above) rather than a qualitative one does violence to some of my intuitions as well, but on balance I don’t endorse those intuitions. Gravity is, technically speaking, an optimization process… though, as you say, it’s not worth the name in a colloquial sense.
I find this very unsatisfying, not least because the optimisation power over a wide range of targets is easily gamed just by dividing any given ‘target’ of a process into a whole lot of smaller targets and then saying “look at all these different targets that the process optimised for!”
Claiming that optimisation power is defined simply by a process’s ability to hit some target from a wide range of starting states, and/or has a wide range of targets that it can hit, both seem to be easily gameable by clever sophistry with your choice of how you choose the targets by which you measure its optimisation power. There must be some part of it that separates processes we feel genuinely are good at optimising (like Clippy) from processes that only come out as good at optimising if we select clever targets to measure them by.
We seem to have very different understandings of what constitutes a wide range. A narrow target does not suddenly become a wide range of targets because I choose to subdivide it, any more than I can achieve a diversified stock portfolio by separately investing each dollar into the same company’s stock.
So I’m still pretty comfortable with my original stance here: optimization is as optimization does.
That said, I certainly agree that clever sophistry can blur the meaning of our definitions. This seems like a good reason to eschew clever sophistry when analyzing systems I want to interact effectively with.
And I can appreciate finding it unsatisfying. Sometimes the result of careful thinking about a system is that we discover our initial intuitions were incorrect, rather than discovering a more precise or compelling way to express our initial intuitions.
There must be some part of it that separates processes we feel genuinely are good at optimising (like Clippy)
I’m not really sure what you mean by “part” here. But general-purpose optimizers are more interesting than narrow optimizers, and powerful optimizers are more interesting than less powerful optimizers, and if we want to get at what’s interesting about them we need more and better tools than just the definition of an “optimization process”.
(apologies for delayed reply)
I really just want to know what Eliezer means by it. It seems to me like I have some notion of an optimisation process, that says “yep, that’s definitely an optimisation process” when I think about evolution and human minds and Clippy, and says “nope, that’s not really an optimisation process—at least, not one worth the name” about water rolling down a hill and thermodynamics. And I think this notion is sufficiently similar to the one that Eliezer is using. But my attempts to formalise this definition, to reduce it, have failed—I can’t find any combination of words that seems to capture the boundary that my intuition is drawing between Clippy and gravity.
Treating the boundary between Clippy and gravity as a quantitative one (optimization power over a wide range of targets, as above) rather than a qualitative one does violence to some of my intuitions as well, but on balance I don’t endorse those intuitions. Gravity is, technically speaking, an optimization process… though, as you say, it’s not worth the name in a colloquial sense.
I find this very unsatisfying, not least because the optimisation power over a wide range of targets is easily gamed just by dividing any given ‘target’ of a process into a whole lot of smaller targets and then saying “look at all these different targets that the process optimised for!”
Claiming that optimisation power is defined simply by a process’s ability to hit some target from a wide range of starting states, and/or has a wide range of targets that it can hit, both seem to be easily gameable by clever sophistry with your choice of how you choose the targets by which you measure its optimisation power. There must be some part of it that separates processes we feel genuinely are good at optimising (like Clippy) from processes that only come out as good at optimising if we select clever targets to measure them by.
We seem to have very different understandings of what constitutes a wide range. A narrow target does not suddenly become a wide range of targets because I choose to subdivide it, any more than I can achieve a diversified stock portfolio by separately investing each dollar into the same company’s stock.
So I’m still pretty comfortable with my original stance here: optimization is as optimization does.
That said, I certainly agree that clever sophistry can blur the meaning of our definitions. This seems like a good reason to eschew clever sophistry when analyzing systems I want to interact effectively with.
And I can appreciate finding it unsatisfying. Sometimes the result of careful thinking about a system is that we discover our initial intuitions were incorrect, rather than discovering a more precise or compelling way to express our initial intuitions.
I’m not really sure what you mean by “part” here. But general-purpose optimizers are more interesting than narrow optimizers, and powerful optimizers are more interesting than less powerful optimizers, and if we want to get at what’s interesting about them we need more and better tools than just the definition of an “optimization process”.