All this talk of contradictions is sort of rubbing me the wrong way here. There’s no “contradiction” in an AI having goals that are different to human goals. Logically, this situation is perfectly normal. Loosemore talks about an AI seeing its goals are “massively in contradiction to everything it knows about ”, but… where’s the contradiction? What’s logically wrong with getting strawberries off a plant by burning them?
I don’t see the need for any kind of special compartmentalisation; information about “normal use of strawberries” is already inert facts with no caring attached by default.
If you’re going to program in special criteria that would create caring about this information, okay, but how would such criteria work? How do you stop it from deciding that immortality is contradictory to “everything it knows about death” and refusing to help us solve aging?
In the original scenario, the contradiction us supposed to .be between a hardcoded definition of happiness in the AIs goal system, and inferred knowledge in the execution system.
All this talk of contradictions is sort of rubbing me the wrong way here. There’s no “contradiction” in an AI having goals that are different to human goals. Logically, this situation is perfectly normal. Loosemore talks about an AI seeing its goals are “massively in contradiction to everything it knows about ”, but… where’s the contradiction? What’s logically wrong with getting strawberries off a plant by burning them?
I don’t see the need for any kind of special compartmentalisation; information about “normal use of strawberries” is already inert facts with no caring attached by default.
If you’re going to program in special criteria that would create caring about this information, okay, but how would such criteria work? How do you stop it from deciding that immortality is contradictory to “everything it knows about death” and refusing to help us solve aging?
In the original scenario, the contradiction us supposed to .be between a hardcoded definition of happiness in the AIs goal system, and inferred knowledge in the execution system.