Question:
How do you make the paperclip maximizer want to collect paperclips?
I have two slightly different understandings of how you might do this, in terms of how it’s ultimately programmed:
1) there’s a function that says “maximize paperclips”
2) there’s a function that says “getting a paperclip = +1 good point”
Given these two different understandings though, isn’t the inevitable result for a truly intelligent paperclip maximizer to just hack itself and based on my two different understandings:
1) make itself /think/ that it’s getting paperclips because that’s what it really wants—there’s no way to make it value ACTUALLY getting paperclips as opposed to just thinking that it’s getting paperclips
2) find a way to directly award itself “good points” because that’s what it really wants
I think my understanding is probably flawed somewhere but haven’t been able to figure it out so please point out where
Question: How do you make the paperclip maximizer want to collect paperclips? I have two slightly different understandings of how you might do this, in terms of how it’s ultimately programmed: 1) there’s a function that says “maximize paperclips” 2) there’s a function that says “getting a paperclip = +1 good point”
Given these two different understandings though, isn’t the inevitable result for a truly intelligent paperclip maximizer to just hack itself and based on my two different understandings: 1) make itself /think/ that it’s getting paperclips because that’s what it really wants—there’s no way to make it value ACTUALLY getting paperclips as opposed to just thinking that it’s getting paperclips 2) find a way to directly award itself “good points” because that’s what it really wants
I think my understanding is probably flawed somewhere but haven’t been able to figure it out so please point out where