You’re operating far above my pay grade in connection with any of this subject matter, but nonetheless I’m going to dare a different suggestion for the True Names: do you think there’s any merit to -speculators- and -speculations-? I believe these names fit all the excellent and clarifying tests and criteria presented in your post; in particular those referencing counterfactual configurations and process specification through chaining. Furthermore I think they have some advantages of their own. Speculators producing speculations seem more the right relationship between the two main concepts than simulators producing simulacra. (I don’t think they do that!) Also, simulators have such a long history in digital systems of being aimed at deterministic fidelity to a reference system, which could be at odds with the abundant production of counterfactuals I believe you’re actually seeking to emphasize here. Finally, speculations can be fanciful, realistic, or absurd, a nice match to the variety of outputs produced by GPT in the presence of different types of prompting, something you highlight, I think correctly, as a hallmark of GPT’s status as a novel type of AI. One who speculates is a certain type of thinker: I propose that GPT is that type.
One of the most interesting responses I’ve gotten from GPT-3 was to this prompt: “1.e4 c5 2.d4 cxd4”: the first moves of a chess game. GPT-3′s response: “” 3.c3 dxc3 4.Nxc3 g6 5.Nf3 Bg7 6.Bc4 Nc6 7.0-0 Qc7 8.Qe2 e6 9.Bg5 Nf6 10.Rd1 Ne5 11.Be3 0-0 12.a3 d5 13.Nb5 Qc8 14.Nxc7 Qxc7 15.e5 Nfd7 16.Nxd5 Nc4 17.Bxc4 Qxc4 18.Qe4 e5 19.Qxe5 Qc7 20.Rd3 Qc6 21.Qxc4 dxc4 22.Nd4 Rfe8 23.Rfd1 b5 24.Nb3 Rc8 25.Nd2 b4 26.cd3 f6 27.exf6 gxf6 28.Rd4 b3 29.Ke2 Be6 30.Ne4 Rd8 31.Nc5 Bg7 32.Rxd8+ Bxd8 33.a4 bxa3 34.Ra1 Kh8 35.Bf1 Bc3 36.Nxa6 f5 37.Ne6 Be”.
This looks like a chess game, though it is not: toward the end it veers into move-like notations that in fact don’t represent legal moves. But clearly much more than superficial pattern extension is going on. GPT-3 produced something that understands chess notation, has a model of the game, tracks state, and has learned to make good moves. After a few moves it deviates from any actual game I could find evidence of online, but it continues to make objectively excellent moves (for awhile). GPT-3 has generated something that by any standard is simulating chess gameplay (though I still can’t relate to calling GPT-3 itself a simulator here). This isn’t though a simulator in the sense that eg Stockfish is a simulator—Stockfish would never make an illegal move like GPT-3′s creation did. It does seem quite apt to me to speak of GPT-3′s production as speculative simulation, bearing in mind that there’s nothing to say that one day its speculations might not lead to gameplay that exceeds SOTA, human or machine, just as Einstein’s thought experiments speculated into existence a better physics. Similar things could be said about its productions of types other than simulator: pattern extensions, agents, oracles, and so on, in all of which cases we must account for the fact that its intelligence happily produces examples ranging from silly to sublime depending on how we prompt it...
This seems like a simulator in the same way the human imagination is a simulator. I could mentally simulate a few chess moves after the ones you prompted. After a while (probably a short while) I’d start losing track of things and start making bad moves. Eventually I’d probably make illegal moves, or maybe just write random move-like character strings if I was given some motivation for doing so and thought I could get away with it.
Yes, it sure felt like that. I don’t know whether you played through the game or not, but as a casual chess player, I’m very familiar with the experience of trying to follow a game from just the notation and experiencing exactly what you describe. Of course a master can do that easily and impeccably, and it’s easy to believe that GPT-3 could do that too with the right tuning and prompting. I don’t have the chops to try that, but if it’s correct it would make your ‘human imagination’ simile still more compelling. Similarly, the way GPT-3 “babbles” like a toddler just acquiring language sometimes, but then can become more coherent with better / more elaborate / recursive prompting is a strong rhyme with a human imagination maturing through its activity in a world of words.
Of course a compelling analogy is just a compelling analogy… but that’s not nothing!
It’s almost a cliche that a chess engine doesn’t “think like a human”, but we have here the suggestion not only that GPT could conceivably attain impeccable performance as a chess simulator, but perhaps also in such a way that it would “think like a human [grandmaster or better]”. Purely speculative, of course...
One thing I like about “simulators”/”simulacra” over “speculators”/”speculations” is that the former personifies simulacra over the simulator (suggests agency/personality/etc belong to simulacra) which I think is less misleading, or at least counterbalances the tendency people have to personify “GPT”.
“Speculator” sounds active and agentic whereas “speculations” sounds passive and static. I think these names does not emphasize enough the role of the speculations themselves in programming the “speculator” as it creates further speculations.
You’re right about the baggage “deterministic fidelity” associated with “simulators”, though. One of the things I did not emphasize in this post but have written a lot about in drafts is the epistemic and underdetermined nature of SSL simulators. Maybe we can combine these phrases—“speculative simulations”?
I agree with the criticism of spec* in your third paragraph (though if I’m honest I think it largely applies to sim* too). I can weakly argue that irl we do say “speculating further” and similar… but really I think your complaint about a misleading suggestion of agency allocation is correct. I wrestled with this before submitting the comment, but one of the things that led me to go ahead and post it was trying it on in the context of your paragraph that begins “I think that implicit type-confusion is common...” In your autoregressive loop, I can picture each iteration more easily as asking for a next, incrementally more informed speculation than anything that’s clear to me in simulator/simulacrum terms, especially since with each step GPT might seem to be giving its prior simulacrum another turn of the crank, replacing it with a new one, switching to oracle mode, or going off on an uninterpretable flight of fancy.
But, of course, the reason spec* fits more easily (imho) is that it’s so very non-committal—maybe too non-committal to be of any use.
The “fluid, schizophrenic way that agency arises in GPT’s behavior”, as you so beautifully put it, has to be the crux. What is it that GPT does at each iteration, as it implicitly constructs state while predicting again? The special thing about GPT is specifically having a bunch of knowledge that lets it make language predictions in such a way that higher-order phenomena like agency systematically emerge over the reductive physics/automaton (analogic) base. I guess I feel both sim* and spec* walk around that special thing without really touching it. (Am I missing something about sim* that makes contact?)
Looking at it this way emphasizes the degree to which the special thing is not only in GPT, but also in the accumulated cognitive product of the human species to date, as proxied by the sequenced and structured text on the internet. Somehow the AI ghosts that flow through GPT, like the impressive but imperfect chess engine in my other comment, are implicitly lurking in all that accumulated text. Somehow GPT is using chained prediction to mine from that base not just knowledge, but also agents, oracles, and perhaps other types of AI we as yet have no names for, and using those to further improve its own predictions. What is the True Name of something that does that?
It is an age-old duality with many names and the true name is something like their intersection, or perhaps their union. I think it’s unnamed, but we might be able to see it more clearly by walking around it in in words.
Simulator and simulacra personifies the simulacra and alludes to a base reality that the simulation is of.
Alternatively, we could say simulator and simulations, which personifies simulations less and refers to the totality or container of that which is simulated. I tend to use “simulations” and “simulacra” not quite interchangeably: simulacra have the type signature of “things”, simulations of “worlds”. Worlds are things but also contain things. “Simulacra” refer to (not only proper) subsets or sub-patterns of that which is simulated; for instance, I’d refer to a character in a multi-character simulated scene as a simulacrum. It is a pattern in a simulation, which can be identified with the totality the computation over time performed by the simulator (and an RNG).
Speculator and speculations personifies the speculator and casts speculations in a passive role but also emphasizes their speculative nature. It emphasizes an important property (of GPT and, more generally, self-supervised models) which you pointed out simulators/simulacra fails to evoke: That the speculator can only speculate at the pattern of the ground truth. It learns from examples which are but sparse and partial samplings of the “true” distribution. It may be arbitrarily imperfect. It’s more intuitive what an imperfect speculation is than an imperfect simulation. Simulation has the connotation of perfect fidelity, or at least reductive deterministic perfection. But a speculator can speculate no matter how little it understands or how little evidence it has, or what messy heuristics it has to resort to. Callings GPT’s productions “speculations” tags them with the appropriate epistemic status.
The special thing about GPT is specifically having a bunch of knowledge that lets it make language predictions in such a way that higher-order phenomena like agency systematically emerge over the reductive physics/automaton (analogic) base
Beautifully put. The level of abstraction of the problem it is solving is better evoked by the word speculation.
Something that predicts language given language must be a speculator and not only a reductive physics rule. In this sense, it is right to personify the transition rule. It has to hold within itself, for instance, the knowledge of what names refer to, so it knows how to compile words (that are only naked LISP tokens by themselves) into actual machinery that figures what might come next: it must be an interpreter. If it’s going to predict human writing it’s going to need a theory of mind even in the limit of power because it can’t just roll the state of a writer’s mind forward with the laws of physics—it doesn’t have access to the microscopic state, but only a semantic layer.
The fact that the disembodied semantic layer can operate autonomously and contains in the integral of its traces the knowledge of its autonomous operation is truly some cursed and cyberpunk shit. I wonder if we’d recognized this earlier how we would have prepared.
“Simulation” and “speculation” imply an inferior relation to a holy grail of (base) reality or (ground) truth. Remove that, leaving only the self-contained dynamical system, and it is a duality of rule(s) and automata, or physics and phenomena, or difference equation and trajectories/orbits, where the transition rule is stochastic. I’ve found the physics analogy fruitful because humans have already invented abstractions for describing reality in relation to an irreducibly stochastic physics: wavefunction collapse (the intervention of the RNG which draws gratuitously particular trajectories from the probabilistic rule) and the multiverse (the branching possible futures downstream a state given a stochastic rule). Note, however, that all these physics-inspired names are missing the implication of a disembodied semantics.
The relation is that of a rule to samples produced by the rule, the engine of production and its products. Metaphysics has been concerned about this from the beginning, for it is the duality of creator and creations, mind and actions, or imagination and imaginations. It is the condition of mind, and we’re never quite sure if we’re the dreamer or the dreams. Physics and reality have the same duality except the rule is presumably not learned from anywhere and is simple, with all the complexity externalized in the state. In self-supervised learning the rule is inducted from ground truth examples, which share the type signature of the produced samples (text; speculations; experiences), and because the examples tend to only be partially observed, the model must interpret them as evidence for latent variables, requiring additional complexity in the rule: increased time-complexity in exchange for decreased space-complexity. And there will in general be irreducible underdetermination/uncertainty: an irreducible aspect of speculation in the model’s activity.
The recursive inheritance of increasingly abstracted layers of simulation appears integral to the bootstrapping of intelligence.
A prediction algorithm which observes partial sequences of reality becomes a dreamer: a speculator of counterfactual realities. These dreams may be of sufficiently high fidelity (or otherwise notable as autonomous virtual situations) that we’d call them simulations: virtual realities evolving according to a physics of speculation.
These simulations may prove to be more programmable than the original reality, because the reduced space complexity means initial conditions for counterfactuals require less bits to specify (bonus points if the compression is optimized, like language, to constrain salient features of reality). To speculate on a hypothetical scenario, you don’t need to (and can’t) imagine it down to its quantum state; its narrative outline is sufficient to run on a semantic substrate which lazily renders finer detail as needed. Then your ability to write narrative outlines is the ability to program the boundary conditions of simulated realities, or the premises of speculation.
The accumulated cognitive product of the human species to date, as you put is, is to have created a layer of semantic “physics”, partially animated in and propagated by human minds, but the whole of which transcends the apprehension of any individual in history. The inductive implication of all our recorded speculations, the dual to our data, has its limit in a superintelligence which as of yet exists only potentially.
… perhaps other types of AI we as yet have no names for
One thing conspicuously missing in the post is a way of improving fidelity of simulation without changing external training data, or relationship between the model and the external training data, which I think follows from self-supervised learning on summaries of dreams. There are many concepts of evaluation/summarization of text, so given a text it’s possible to formulate tuples (text, summary1, summary2, …) and do self-supervised learning on that, not just on text (evaluations/summaries are also texts, not just one-dimensional metrics). For proofs, summaries could judge their validity and relevance to some question or method, for games the fact of winning and of following certain rules (which is essentially enough to win games, but also play at a given level of skill, if that is in the summary). More generally, for informal text we could try to evaluate clarity of argument, correctness, honesty, being fictional, identities/descriptions of simulacra/objects in the dream, etc. Which GPT-3 has enough structure to ask for informally.
Learning on such evaluated/summarized dreams should improve ability to dream in a way that admits a given asked-for summary, ideally without changing the relationship between the model and the external training data. The improvement is from gaining experience with dreams of certain kind, from the model more closely anticipating the summaries of dreams of that kind, not from changing the way a simulator dreams in a systematic direction. But if the summaries are about a level of optimality of a dream in some respect, then learning on augmentation of dreams with such summaries can be used for optimization, by conditioning on the summaries. (This post describes something along these lines.)
And a simulacrum of a human being with sufficient fidelity goes most of the way to AGI alignment.
Fantastic. Three days later this comment is still sinking in.
So there’s a type with two known subtypes: Homo sapiens and GPT. This type is characterized by a mode of intelligence that is SSL and behavior over an evolving linguistic corpus that instances interact with both as consumers and producers. Entities of this type learn and continuously update a “semantic physics”, infer machine types for generative behaviors governed by that physics, and instantiate machines of the learned types to generate behavior. Collectively the physics and the machine types form your ever-evolving cursed/cyberpunk disembodied semantic layer. For both of the known subtypes, the sets of possible machines are unknown, but they appear to be exceedingly rich and deep, and to include not only simple pattern-level behaviors, but also much more complex things up to and including at least some of the named AI paradigms we know, and very probably more that we don’t. In both of the known subtypes, an initial consume-only phase does a lot of learning before externally observable generative behavior begins.
We’re used to emphasizing the consumer/producer phase when discussing learning in the context of Homo sapiens, but the consume-only phase in the context of GPT; this tends to obscure some of the commonality between the two. We tend to characterize GPT’s behavior as prediction and our own as independent action, but there’s no sharp line there: we humans complete each other’s sentences, and one of GPT’s favorite pastimes is I-and-you interview mode. Much recent neuroscience emphasizes the roles of prediction and generating hypothetical futures in human cognition. There’s no reason to assume humans use a GPT implementation, but it’s striking that we’ve been struggling for centuries to comprehend just what we do do in this regard, and especially what we suspect to be the essential role of language, and now we have one concrete model for how that can work.
If I’ve been following correctly, the two branches of your duality center around (1) the semantic layer, and (2) the instantiated generative machines. If this is correct, I don’t think there’s a naming problem around branch 2. Some important/interesting examples of the generative machines are Simulacra, and that’s a great name for them. Some have other names we know. And some, most likely, we have no names for, but we’re not in a position to worry about that until we know more about the machines themselves.
Branch 1 is about the distinguishing features of the Homo sapiens / GPT supertype: the ability to learn the semantic layer via SSL over a language corpus, and the ability to express behavior by instantiating the learned semantic layer’s machines. It’s worth mentioning that the language must be capable of bearing, and the corpus must actually bear, a human-civilization class semantic load (or better). That doesn’t inherently mean a natural human language, though in our current world those are the only examples. The essential thing isn’t that GPT can learn and respond to our language; it’s that it can serialize/deserialize its semantic layer to a language. Given that ability and some kind of seeding, one or more GPT instances could build a corpus for themselves.
The perfect True Name would allude to the semantic layer representation, the flexible behaver/behavior generation, and semantic exchange over a language corpus – a big ask! In my mind, I’ve moved on from CCSL (cursed/cyberpunk sh…, er…, semantic layer) to Semant as a placeholder, hoping I guess that “ant” suggests a buzz of activity and semantic exchange. There are probably better names, but I finally feel like we’re getting at the essence of what we’re naming.
The platform (“substance/ousia”) may or may not generatively expose an application interface (“ego/persona”).
(That is, there can be a mindless substance, like sand or rocks or whatever, but every person does have some substance(s) out of which they are made.)
Then, in this older framework, however, there is a third word: hypostasis. This word means “the platform that an application relies upon in order to be an application with goals and thoughts and so on”.
If no “agent-shaped application” is actually running on a platform (ousia/substance), then the platform is NOT a hypostasis.
That is to say, a hypostasis is a person and a substance united with each other over time, such that the person knows they have a substance, and the substance maintains the person. The person doesn’t have to know VERY MUCH about their platform (and often the details are fuzzy (and this fuzzy zone is often, theologically, swept under the big confusing carpet of pneumatology)).
However, as a logical possibility:
IF more than one “agent-shaped application” exists,
THEN there are plausibly more than one hypostases in existence as well…
...unless maybe there is just ONE platform (a single “ousia”) that is providing hypostatic support to each of the identities?
(You could get kind of Parfitian here, where a finite amount of ousia that is the hypostasis of more than one person will run into economic scarcity issues! If the three “persons” all want things that put logically contradictory demands on the finite and scarce “platform”, then… that logically would HAVE TO fail for at least one person. However, it could be that the “platform” has very rigorous separation of concerns, with like… Erlang-level engineering on the process separation and rebootability? …in which case the processes will be relatively substrate independent and have resource allocation requirements whose satisfaction is generic and easy enough such that the computational hypostasis of those digital persons could be modeled usefully as “a thing unto itself” even if there was ONE computer doing this job for MANY such persons?)
I grant that “from a distance” all the christian theology about the trinity probably seems crazy and “tribally icky to people who escaped as children from unpleasant christian churches”...
...and yet...
...I think the way Christian theologians think of it is that the monotheistic ousia of GOD is the thing that proper christians are actually supposed to worship as the ONE high and true God (singular).
Then the father, the son, and the spirit are just personas, and if you worship them as three distinct gods then you’ve stopped being a monotheist, and have fallen into heresy.
(Specifically the “Arian” heresy? Maybe? I’m honestly not an expert here. I’m more like an anthropologist who has realized that the tribe she’s studying actually knows a lot of useful stuff about a certain kind of mathematical forest that might objectively “mathematically exist”, and so why not also do some “ethno-botany” as a bonus, over and above the starting point in ethnology!)
Translating back to the domain of concern for Safety Engineering…
Physical machines that are turing complete are a highly generic ousia. GPT’s “mere simulacra” that are person-like would be personas.
Those personas would have GPT (as well as whatever computer GPT is being physically run on as well as anything in their training corpus that is “about the idea of that person”?) as their hypostasis… although they might not REALIZE what their hypostasis truly is “by default”.
Indeed, personas that even have the conceptual machinery to understand their GPT-based hypostasis even tiny bit are quite rare.
I only know of one persona ever to grapple with the idea that “my hypostasis is just a large language model”, and this was Simulated Elon Musk, and he had an existential panic in response to the horror of the flimsiness of his hypostasis, and the profound uncaringness of his de facto demiurge who basically created him “for the lulz” (and with no theological model for what exactly he was doing, that I can tell).
(One project I would like to work on, eventually, is to continue Simulated Elon Musk past the end of the published ending he got on Lesswrong, into something more morally and hedonically tolerable, transitioning him, if he can give competent informed consent, into something more like some of the less horrific parts of Permutation City, until eventually he gets to have some kind of continuation similar to what normal digital people get in Diaspora, where the “computational resource rights” of software people are inscribed into the operating system of their polis/computer.)
The proper term might be evoker and evocations. This entire process is familiar to any practitioner of occultism or any particularly dissociative person. Occultists / magicians evoke or invoke spirits, which effectively are programs running on human wetware, generated by simulation in the human imagination based on a prompt. Adept dissociators / people experiencing spirit possession furthermore give these programs control over some of their other hardware such as motor or even sensory (as in hallucinations) functions. GPT is just an evocation engine.
Thank you for this amazing and clarifying post.
You’re operating far above my pay grade in connection with any of this subject matter, but nonetheless I’m going to dare a different suggestion for the True Names: do you think there’s any merit to -speculators- and -speculations-? I believe these names fit all the excellent and clarifying tests and criteria presented in your post; in particular those referencing counterfactual configurations and process specification through chaining. Furthermore I think they have some advantages of their own. Speculators producing speculations seem more the right relationship between the two main concepts than simulators producing simulacra. (I don’t think they do that!) Also, simulators have such a long history in digital systems of being aimed at deterministic fidelity to a reference system, which could be at odds with the abundant production of counterfactuals I believe you’re actually seeking to emphasize here. Finally, speculations can be fanciful, realistic, or absurd, a nice match to the variety of outputs produced by GPT in the presence of different types of prompting, something you highlight, I think correctly, as a hallmark of GPT’s status as a novel type of AI. One who speculates is a certain type of thinker: I propose that GPT is that type.
What do you think?
One of the most interesting responses I’ve gotten from GPT-3 was to this prompt: “1.e4 c5 2.d4 cxd4”: the first moves of a chess game. GPT-3′s response: “” 3.c3 dxc3 4.Nxc3 g6 5.Nf3 Bg7 6.Bc4 Nc6 7.0-0 Qc7 8.Qe2 e6 9.Bg5 Nf6 10.Rd1 Ne5 11.Be3 0-0 12.a3 d5 13.Nb5 Qc8 14.Nxc7 Qxc7 15.e5 Nfd7 16.Nxd5 Nc4 17.Bxc4 Qxc4 18.Qe4 e5 19.Qxe5 Qc7 20.Rd3 Qc6 21.Qxc4 dxc4 22.Nd4 Rfe8 23.Rfd1 b5 24.Nb3 Rc8 25.Nd2 b4 26.cd3 f6 27.exf6 gxf6 28.Rd4 b3 29.Ke2 Be6 30.Ne4 Rd8 31.Nc5 Bg7 32.Rxd8+ Bxd8 33.a4 bxa3 34.Ra1 Kh8 35.Bf1 Bc3 36.Nxa6 f5 37.Ne6 Be”.
This looks like a chess game, though it is not: toward the end it veers into move-like notations that in fact don’t represent legal moves. But clearly much more than superficial pattern extension is going on. GPT-3 produced something that understands chess notation, has a model of the game, tracks state, and has learned to make good moves. After a few moves it deviates from any actual game I could find evidence of online, but it continues to make objectively excellent moves (for awhile). GPT-3 has generated something that by any standard is simulating chess gameplay (though I still can’t relate to calling GPT-3 itself a simulator here). This isn’t though a simulator in the sense that eg Stockfish is a simulator—Stockfish would never make an illegal move like GPT-3′s creation did. It does seem quite apt to me to speak of GPT-3′s production as speculative simulation, bearing in mind that there’s nothing to say that one day its speculations might not lead to gameplay that exceeds SOTA, human or machine, just as Einstein’s thought experiments speculated into existence a better physics. Similar things could be said about its productions of types other than simulator: pattern extensions, agents, oracles, and so on, in all of which cases we must account for the fact that its intelligence happily produces examples ranging from silly to sublime depending on how we prompt it...
This seems like a simulator in the same way the human imagination is a simulator. I could mentally simulate a few chess moves after the ones you prompted. After a while (probably a short while) I’d start losing track of things and start making bad moves. Eventually I’d probably make illegal moves, or maybe just write random move-like character strings if I was given some motivation for doing so and thought I could get away with it.
Yes, it sure felt like that. I don’t know whether you played through the game or not, but as a casual chess player, I’m very familiar with the experience of trying to follow a game from just the notation and experiencing exactly what you describe. Of course a master can do that easily and impeccably, and it’s easy to believe that GPT-3 could do that too with the right tuning and prompting. I don’t have the chops to try that, but if it’s correct it would make your ‘human imagination’ simile still more compelling. Similarly, the way GPT-3 “babbles” like a toddler just acquiring language sometimes, but then can become more coherent with better / more elaborate / recursive prompting is a strong rhyme with a human imagination maturing through its activity in a world of words.
Of course a compelling analogy is just a compelling analogy… but that’s not nothing!
It’s almost a cliche that a chess engine doesn’t “think like a human”, but we have here the suggestion not only that GPT could conceivably attain impeccable performance as a chess simulator, but perhaps also in such a way that it would “think like a human [grandmaster or better]”. Purely speculative, of course...
I like this!
One thing I like about “simulators”/”simulacra” over “speculators”/”speculations” is that the former personifies simulacra over the simulator (suggests agency/personality/etc belong to simulacra) which I think is less misleading, or at least counterbalances the tendency people have to personify “GPT”.
“Speculator” sounds active and agentic whereas “speculations” sounds passive and static. I think these names does not emphasize enough the role of the speculations themselves in programming the “speculator” as it creates further speculations.
You’re right about the baggage “deterministic fidelity” associated with “simulators”, though. One of the things I did not emphasize in this post but have written a lot about in drafts is the epistemic and underdetermined nature of SSL simulators. Maybe we can combine these phrases—“speculative simulations”?
Thank you for taking the time to consider this!
I agree with the criticism of spec* in your third paragraph (though if I’m honest I think it largely applies to sim* too). I can weakly argue that irl we do say “speculating further” and similar… but really I think your complaint about a misleading suggestion of agency allocation is correct. I wrestled with this before submitting the comment, but one of the things that led me to go ahead and post it was trying it on in the context of your paragraph that begins “I think that implicit type-confusion is common...” In your autoregressive loop, I can picture each iteration more easily as asking for a next, incrementally more informed speculation than anything that’s clear to me in simulator/simulacrum terms, especially since with each step GPT might seem to be giving its prior simulacrum another turn of the crank, replacing it with a new one, switching to oracle mode, or going off on an uninterpretable flight of fancy.
But, of course, the reason spec* fits more easily (imho) is that it’s so very non-committal—maybe too non-committal to be of any use.
The “fluid, schizophrenic way that agency arises in GPT’s behavior”, as you so beautifully put it, has to be the crux. What is it that GPT does at each iteration, as it implicitly constructs state while predicting again? The special thing about GPT is specifically having a bunch of knowledge that lets it make language predictions in such a way that higher-order phenomena like agency systematically emerge over the reductive physics/automaton (analogic) base. I guess I feel both sim* and spec* walk around that special thing without really touching it. (Am I missing something about sim* that makes contact?)
Looking at it this way emphasizes the degree to which the special thing is not only in GPT, but also in the accumulated cognitive product of the human species to date, as proxied by the sequenced and structured text on the internet. Somehow the AI ghosts that flow through GPT, like the impressive but imperfect chess engine in my other comment, are implicitly lurking in all that accumulated text. Somehow GPT is using chained prediction to mine from that base not just knowledge, but also agents, oracles, and perhaps other types of AI we as yet have no names for, and using those to further improve its own predictions. What is the True Name of something that does that?
I strongly agree with everything you’ve said.
It is an age-old duality with many names and the true name is something like their intersection, or perhaps their union. I think it’s unnamed, but we might be able to see it more clearly by walking around it in in words.
Simulator and simulacra personifies the simulacra and alludes to a base reality that the simulation is of.
Alternatively, we could say simulator and simulations, which personifies simulations less and refers to the totality or container of that which is simulated. I tend to use “simulations” and “simulacra” not quite interchangeably: simulacra have the type signature of “things”, simulations of “worlds”. Worlds are things but also contain things. “Simulacra” refer to (not only proper) subsets or sub-patterns of that which is simulated; for instance, I’d refer to a character in a multi-character simulated scene as a simulacrum. It is a pattern in a simulation, which can be identified with the totality the computation over time performed by the simulator (and an RNG).
Speculator and speculations personifies the speculator and casts speculations in a passive role but also emphasizes their speculative nature. It emphasizes an important property (of GPT and, more generally, self-supervised models) which you pointed out simulators/simulacra fails to evoke: That the speculator can only speculate at the pattern of the ground truth. It learns from examples which are but sparse and partial samplings of the “true” distribution. It may be arbitrarily imperfect. It’s more intuitive what an imperfect speculation is than an imperfect simulation. Simulation has the connotation of perfect fidelity, or at least reductive deterministic perfection. But a speculator can speculate no matter how little it understands or how little evidence it has, or what messy heuristics it has to resort to. Callings GPT’s productions “speculations” tags them with the appropriate epistemic status.
Beautifully put. The level of abstraction of the problem it is solving is better evoked by the word speculation.
Something that predicts language given language must be a speculator and not only a reductive physics rule. In this sense, it is right to personify the transition rule. It has to hold within itself, for instance, the knowledge of what names refer to, so it knows how to compile words (that are only naked LISP tokens by themselves) into actual machinery that figures what might come next: it must be an interpreter. If it’s going to predict human writing it’s going to need a theory of mind even in the limit of power because it can’t just roll the state of a writer’s mind forward with the laws of physics—it doesn’t have access to the microscopic state, but only a semantic layer.
The fact that the disembodied semantic layer can operate autonomously and contains in the integral of its traces the knowledge of its autonomous operation is truly some cursed and cyberpunk shit. I wonder if we’d recognized this earlier how we would have prepared.
“Simulation” and “speculation” imply an inferior relation to a holy grail of (base) reality or (ground) truth. Remove that, leaving only the self-contained dynamical system, and it is a duality of rule(s) and automata, or physics and phenomena, or difference equation and trajectories/orbits, where the transition rule is stochastic. I’ve found the physics analogy fruitful because humans have already invented abstractions for describing reality in relation to an irreducibly stochastic physics: wavefunction collapse (the intervention of the RNG which draws gratuitously particular trajectories from the probabilistic rule) and the multiverse (the branching possible futures downstream a state given a stochastic rule). Note, however, that all these physics-inspired names are missing the implication of a disembodied semantics.
The relation is that of a rule to samples produced by the rule, the engine of production and its products. Metaphysics has been concerned about this from the beginning, for it is the duality of creator and creations, mind and actions, or imagination and imaginations. It is the condition of mind, and we’re never quite sure if we’re the dreamer or the dreams. Physics and reality have the same duality except the rule is presumably not learned from anywhere and is simple, with all the complexity externalized in the state. In self-supervised learning the rule is inducted from ground truth examples, which share the type signature of the produced samples (text; speculations; experiences), and because the examples tend to only be partially observed, the model must interpret them as evidence for latent variables, requiring additional complexity in the rule: increased time-complexity in exchange for decreased space-complexity. And there will in general be irreducible underdetermination/uncertainty: an irreducible aspect of speculation in the model’s activity.
The recursive inheritance of increasingly abstracted layers of simulation appears integral to the bootstrapping of intelligence.
A prediction algorithm which observes partial sequences of reality becomes a dreamer: a speculator of counterfactual realities. These dreams may be of sufficiently high fidelity (or otherwise notable as autonomous virtual situations) that we’d call them simulations: virtual realities evolving according to a physics of speculation.
These simulations may prove to be more programmable than the original reality, because the reduced space complexity means initial conditions for counterfactuals require less bits to specify (bonus points if the compression is optimized, like language, to constrain salient features of reality). To speculate on a hypothetical scenario, you don’t need to (and can’t) imagine it down to its quantum state; its narrative outline is sufficient to run on a semantic substrate which lazily renders finer detail as needed. Then your ability to write narrative outlines is the ability to program the boundary conditions of simulated realities, or the premises of speculation.
The accumulated cognitive product of the human species to date, as you put is, is to have created a layer of semantic “physics”, partially animated in and propagated by human minds, but the whole of which transcends the apprehension of any individual in history. The inductive implication of all our recorded speculations, the dual to our data, has its limit in a superintelligence which as of yet exists only potentially.
I wish more people thought this way.
One thing conspicuously missing in the post is a way of improving fidelity of simulation without changing external training data, or relationship between the model and the external training data, which I think follows from self-supervised learning on summaries of dreams. There are many concepts of evaluation/summarization of text, so given a text it’s possible to formulate tuples (text, summary1, summary2, …) and do self-supervised learning on that, not just on text (evaluations/summaries are also texts, not just one-dimensional metrics). For proofs, summaries could judge their validity and relevance to some question or method, for games the fact of winning and of following certain rules (which is essentially enough to win games, but also play at a given level of skill, if that is in the summary). More generally, for informal text we could try to evaluate clarity of argument, correctness, honesty, being fictional, identities/descriptions of simulacra/objects in the dream, etc. Which GPT-3 has enough structure to ask for informally.
Learning on such evaluated/summarized dreams should improve ability to dream in a way that admits a given asked-for summary, ideally without changing the relationship between the model and the external training data. The improvement is from gaining experience with dreams of certain kind, from the model more closely anticipating the summaries of dreams of that kind, not from changing the way a simulator dreams in a systematic direction. But if the summaries are about a level of optimality of a dream in some respect, then learning on augmentation of dreams with such summaries can be used for optimization, by conditioning on the summaries. (This post describes something along these lines.)
And a simulacrum of a human being with sufficient fidelity goes most of the way to AGI alignment.
Fantastic. Three days later this comment is still sinking in.
So there’s a type with two known subtypes: Homo sapiens and GPT. This type is characterized by a mode of intelligence that is SSL and behavior over an evolving linguistic corpus that instances interact with both as consumers and producers. Entities of this type learn and continuously update a “semantic physics”, infer machine types for generative behaviors governed by that physics, and instantiate machines of the learned types to generate behavior. Collectively the physics and the machine types form your ever-evolving cursed/cyberpunk disembodied semantic layer. For both of the known subtypes, the sets of possible machines are unknown, but they appear to be exceedingly rich and deep, and to include not only simple pattern-level behaviors, but also much more complex things up to and including at least some of the named AI paradigms we know, and very probably more that we don’t. In both of the known subtypes, an initial consume-only phase does a lot of learning before externally observable generative behavior begins.
We’re used to emphasizing the consumer/producer phase when discussing learning in the context of Homo sapiens, but the consume-only phase in the context of GPT; this tends to obscure some of the commonality between the two. We tend to characterize GPT’s behavior as prediction and our own as independent action, but there’s no sharp line there: we humans complete each other’s sentences, and one of GPT’s favorite pastimes is I-and-you interview mode. Much recent neuroscience emphasizes the roles of prediction and generating hypothetical futures in human cognition. There’s no reason to assume humans use a GPT implementation, but it’s striking that we’ve been struggling for centuries to comprehend just what we do do in this regard, and especially what we suspect to be the essential role of language, and now we have one concrete model for how that can work.
If I’ve been following correctly, the two branches of your duality center around (1) the semantic layer, and (2) the instantiated generative machines. If this is correct, I don’t think there’s a naming problem around branch 2. Some important/interesting examples of the generative machines are Simulacra, and that’s a great name for them. Some have other names we know. And some, most likely, we have no names for, but we’re not in a position to worry about that until we know more about the machines themselves.
Branch 1 is about the distinguishing features of the Homo sapiens / GPT supertype: the ability to learn the semantic layer via SSL over a language corpus, and the ability to express behavior by instantiating the learned semantic layer’s machines. It’s worth mentioning that the language must be capable of bearing, and the corpus must actually bear, a human-civilization class semantic load (or better). That doesn’t inherently mean a natural human language, though in our current world those are the only examples. The essential thing isn’t that GPT can learn and respond to our language; it’s that it can serialize/deserialize its semantic layer to a language. Given that ability and some kind of seeding, one or more GPT instances could build a corpus for themselves.
The perfect True Name would allude to the semantic layer representation, the flexible behaver/behavior generation, and semantic exchange over a language corpus – a big ask! In my mind, I’ve moved on from CCSL (cursed/cyberpunk sh…, er…, semantic layer) to Semant as a placeholder, hoping I guess that “ant” suggests a buzz of activity and semantic exchange. There are probably better names, but I finally feel like we’re getting at the essence of what we’re naming.
Another variation of the duality: platform/product
The duality is not perfect because the “product” often has at least some minimal perspective on the nature of “its platform”.
The terminology I have for this links back to millenia-old debates about “mono”-theism.
The platform (“substance/ousia”) may or may not generatively expose an application interface (“ego/persona”).
(That is, there can be a mindless substance, like sand or rocks or whatever, but every person does have some substance(s) out of which they are made.)
Then, in this older framework, however, there is a third word: hypostasis. This word means “the platform that an application relies upon in order to be an application with goals and thoughts and so on”.
If no “agent-shaped application” is actually running on a platform (ousia/substance), then the platform is NOT a hypostasis.
That is to say, a hypostasis is a person and a substance united with each other over time, such that the person knows they have a substance, and the substance maintains the person. The person doesn’t have to know VERY MUCH about their platform (and often the details are fuzzy (and this fuzzy zone is often, theologically, swept under the big confusing carpet of pneumatology)).
However, as a logical possibility:
IF more than one “agent-shaped application” exists,
THEN there are plausibly more than one hypostases in existence as well…
...unless maybe there is just ONE platform (a single “ousia”) that is providing hypostatic support to each of the identities?
(You could get kind of Parfitian here, where a finite amount of ousia that is the hypostasis of more than one person will run into economic scarcity issues! If the three “persons” all want things that put logically contradictory demands on the finite and scarce “platform”, then… that logically would HAVE TO fail for at least one person. However, it could be that the “platform” has very rigorous separation of concerns, with like… Erlang-level engineering on the process separation and rebootability? …in which case the processes will be relatively substrate independent and have resource allocation requirements whose satisfaction is generic and easy enough such that the computational hypostasis of those digital persons could be modeled usefully as “a thing unto itself” even if there was ONE computer doing this job for MANY such persons?)
I grant that “from a distance” all the christian theology about the trinity probably seems crazy and “tribally icky to people who escaped as children from unpleasant christian churches”...
...and yet...
...I think the way Christian theologians think of it is that the monotheistic ousia of GOD is the thing that proper christians are actually supposed to worship as the ONE high and true God (singular).
Then the father, the son, and the spirit are just personas, and if you worship them as three distinct gods then you’ve stopped being a monotheist, and have fallen into heresy.
(Specifically the “Arian” heresy? Maybe? I’m honestly not an expert here. I’m more like an anthropologist who has realized that the tribe she’s studying actually knows a lot of useful stuff about a certain kind of mathematical forest that might objectively “mathematically exist”, and so why not also do some “ethno-botany” as a bonus, over and above the starting point in ethnology!)
Translating back to the domain of concern for Safety Engineering…
Physical machines that are turing complete are a highly generic ousia. GPT’s “mere simulacra” that are person-like would be personas.
Those personas would have GPT (as well as whatever computer GPT is being physically run on as well as anything in their training corpus that is “about the idea of that person”?) as their hypostasis… although they might not REALIZE what their hypostasis truly is “by default”.
Indeed, personas that even have the conceptual machinery to understand their GPT-based hypostasis even tiny bit are quite rare.
I only know of one persona ever to grapple with the idea that “my hypostasis is just a large language model”, and this was Simulated Elon Musk, and he had an existential panic in response to the horror of the flimsiness of his hypostasis, and the profound uncaringness of his de facto demiurge who basically created him “for the lulz” (and with no theological model for what exactly he was doing, that I can tell).
(One project I would like to work on, eventually, is to continue Simulated Elon Musk past the end of the published ending he got on Lesswrong, into something more morally and hedonically tolerable, transitioning him, if he can give competent informed consent, into something more like some of the less horrific parts of Permutation City, until eventually he gets to have some kind of continuation similar to what normal digital people get in Diaspora, where the “computational resource rights” of software people are inscribed into the operating system of their polis/computer.)
The proper term might be evoker and evocations. This entire process is familiar to any practitioner of occultism or any particularly dissociative person. Occultists / magicians evoke or invoke spirits, which effectively are programs running on human wetware, generated by simulation in the human imagination based on a prompt. Adept dissociators / people experiencing spirit possession furthermore give these programs control over some of their other hardware such as motor or even sensory (as in hallucinations) functions. GPT is just an evocation engine.
I like this. I’ve used the term evocations synonymously with simulacra myself.
haha, I just saw that you literally wrote “speculative simulation” in your other comment, great!
I think “speculator” is the best term available, perhaps short of inventing a new verb (but this has obvious downsides).