I think that breaks down into what is subjective specialness, and what is objective specialness.
Clippy wants there to be as many paper clips as possible.
Which is to implicitly treat them as special or valuable in some way.
Clippy’s stopping to care about paper clips is arguably not conducive there being more paperclips, so from Clippy’s caring about paper clips, it follows that Clippy doesn’t want to be altered so that it doesn’t care about paper clips anymore.
Which leaves Clippy in a quandary. Clippy can’t predict which self modifications might lead to Clippy ceasing to care about clips, so if Clippy takes a conservative approach and never self-modifies, Clippy remains inefficient and no threat to anyone.
I think that breaks down into what is subjective specialness, and what is objective specialness.
What kind of answer is that?
Which is to implicitly treat them as special or valuable in some way.
Well, then we have it: they are special. Clippy does not want them because they are special. Clippy wants them, period. Brute fact. If that makes them special, well, you have all the more problem.
Clippy can’t predict which self modifications might lead to Clippy ceasing to care about clips
I think that breaks down into what is subjective specialness, and what is objective specialness.
Which is to implicitly treat them as special or valuable in some way.
Which leaves Clippy in a quandary. Clippy can’t predict which self modifications might lead to Clippy ceasing to care about clips, so if Clippy takes a conservative approach and never self-modifies, Clippy remains inefficient and no threat to anyone.
What kind of answer is that?
Well, then we have it: they are special. Clippy does not want them because they are special. Clippy wants them, period. Brute fact. If that makes them special, well, you have all the more problem.
Says who?
Subjectively, but not objectively.
Whoever failed to equip Clippy with the appropriate oracle when stipulating Clippy.