Problem is, “Fucking up when presented with surprising new situations” is actually a chronic human behavior.
It’s why purse snatchers are so effective—by the time someone
registers Wait, did somebody just yank my purse off my shoulder?,
the snatcher is long gone.
That’s true of most frequently referenced elements of human nature, if not all of them.
Even Love.
~The Homo Sapiens Class has a trusted computing override that enables it to lock itself into a state of heightened agreeability towards a particular target unit. More to the point: it can signal this shift in modes in a way that is both recognizable to other units, and which the implementation makes very difficult for it to forge. The Love feature then provides HS units on either side of a reciprocated Love signalling a means of safely cooperating in extremely high-stakes PD scenarios without violating their superrationality circumvention architecture.
Hmm.. On reflection, one would hope that most effective designs for time-constrained intelligent(decentralized, replication-obsessed) agents would not override superrationality(“override”: Is it reasonable to talk about it like a natural consequence of intelligence?), and that, then, the love override may not occur.
-- Ferrett Steinmetz
But is it only a human behavior? I’d think anything with cached thoughts/results/computations would be similarly vulnerable.
That’s true of most frequently referenced elements of human nature, if not all of them.
Even Love.
~The Homo Sapiens Class has a trusted computing override that enables it to lock itself into a state of heightened agreeability towards a particular target unit. More to the point: it can signal this shift in modes in a way that is both recognizable to other units, and which the implementation makes very difficult for it to forge. The Love feature then provides HS units on either side of a reciprocated Love signalling a means of safely cooperating in extremely high-stakes PD scenarios without violating their superrationality circumvention architecture.
Hmm.. On reflection, one would hope that most effective designs for time-constrained intelligent(decentralized, replication-obsessed) agents would not override superrationality(“override”: Is it reasonable to talk about it like a natural consequence of intelligence?), and that, then, the love override may not occur.
Hard to say.