I don’t think it’s really a necessary distinction; the idea of an unselfish utility maximizer doesn’t quite make sense, because utility is defined so nebulously that pretty much everyone has to seek maximizing their utility.
the idea of an unselfish utility maximizer doesn’t quite make sense
You’re right that it doesn’t make sense, which is why some people assume I mean something else when I say “selfish”. But a lot of commenters do seem to believe in unselfish utility maximizers, which is why I keep using the word.
(I deleted that paragraph.)
Do you have an idea for something else to try?
I don’t think it’s really a necessary distinction; the idea of an unselfish utility maximizer doesn’t quite make sense, because utility is defined so nebulously that pretty much everyone has to seek maximizing their utility.
You’re right that it doesn’t make sense, which is why some people assume I mean something else when I say “selfish”. But a lot of commenters do seem to believe in unselfish utility maximizers, which is why I keep using the word.
Avoiding morally charged words. If possible shy far far away from ANY pattern that people can automatically match against with system 2 so that system 1 stays engaged.
My article here http://www.forbes.com/2009/06/22/singularity-robots-computers-opinions-contributors-artificial-intelligence-09_land.html is an attempt to do this.
Do you mean “system 1 … system 2”?