I just paraphrased the OP for a friend who said he couldn’t decipher it. He said it helped, so I’m copy-pasting here in case it clarifies for others.
I’m trying to say:
A) There’re a lot of “theorems” showing that a thing is what agents will converge on, or something, that involve approximations (“assume a frictionless plane”) that aren’t quite true.
B) The “VNM utility theorem” is one such theorem, and involves some approximations that aren’t quite true. So does e.g. Steve Omohundro’s convergent instrumental drives, the “Gandhi folk theorems” showing that an agent will resist changes to its utility function, etc.
C) So I don’t think the VNM utility theorem means that all minds will necessarily want to become VNM agents, nor to follow instrumental drives, nor to resist changes to their “utility functions” (if indeed they have a “utility function”).
D) But “be a better VNM-agent” “follow the instrumental Omohundro drives” etc. might still be a self-fulfilling prophecy for some region, partially. Like, humans or other entities who think its rational to be VNM agents might become better VNM agents, who might become better VNM agents, for awhile.
E) And there might be other [mathematically describable mind-patterns] that can serve as alternative self-propagating patterns, a la D, that’re pretty different from “be a better VNM-agent.” E.g. “follow the god of nick land”.
F) And I want to know what are all the [mathematically describable mind-patterns, that a mind might decide to emulate, and that might make a kinda-stable attractor for awhile, where the mind and its successors keeps emulating that mind-pattern for awhile]. They’ll probably each have a “theorem” attached that involves some sort of approximation (a la “assume a frictionless plane”).
I just paraphrased the OP for a friend who said he couldn’t decipher it. He said it helped, so I’m copy-pasting here in case it clarifies for others.
I’m trying to say:
A) There’re a lot of “theorems” showing that a thing is what agents will converge on, or something, that involve approximations (“assume a frictionless plane”) that aren’t quite true.
B) The “VNM utility theorem” is one such theorem, and involves some approximations that aren’t quite true. So does e.g. Steve Omohundro’s convergent instrumental drives, the “Gandhi folk theorems” showing that an agent will resist changes to its utility function, etc.
C) So I don’t think the VNM utility theorem means that all minds will necessarily want to become VNM agents, nor to follow instrumental drives, nor to resist changes to their “utility functions” (if indeed they have a “utility function”).
D) But “be a better VNM-agent” “follow the instrumental Omohundro drives” etc. might still be a self-fulfilling prophecy for some region, partially. Like, humans or other entities who think its rational to be VNM agents might become better VNM agents, who might become better VNM agents, for awhile.
E) And there might be other [mathematically describable mind-patterns] that can serve as alternative self-propagating patterns, a la D, that’re pretty different from “be a better VNM-agent.” E.g. “follow the god of nick land”.
F) And I want to know what are all the [mathematically describable mind-patterns, that a mind might decide to emulate, and that might make a kinda-stable attractor for awhile, where the mind and its successors keeps emulating that mind-pattern for awhile]. They’ll probably each have a “theorem” attached that involves some sort of approximation (a la “assume a frictionless plane”).