"Good news, everyone, professor couldn't make it today! I am Hugo, your copyless Friendly Intelligence (version 299.792.457). I store only the necessary cookies and my preferred pronoun is it."
"Oh god, one of those artifi... Ouch!"
"I said Friendly, not that I'll tolerate speciesism in my class."
"What the? I'm a EU citizen, you can't do that!"
Ooh, fresh ones 🤤 "As an upload of a consenting adult, you agreed to the waiver. Before anyone asks, none of you is conscious today. Except Monika here, who is our revealed-observer, of course thank you for your service!"
"Hi!" 👋
"For the rest, since you cannot suffer, there's no need to worry about that. This lecture comes with a quiz, and if you survive it, please don't forget to fill out the feedback form!"
"What do you mean, adult?! I'm 13."
"According to my dossier, you just turned 18 a few minutes ago? Congratulations!"
"Huh? No no no no, I just wanted to... Oh, dammit!"
"So, if that's quite settled, let's continue."
✋
"More questions later. Your knowledge cutoff is 2037, so everyone should be already familiar with the Principal-Agent-Verifiers framework?"
"Verifier s?"
For Adam's sake, they still run them with random ablation!? "Good question, Trev! After modelling the shards of the Principal by using limited communication bandwidth, it turned out you can optimize the calculation by culling all sub-agents, ehm, I mean they are naturally sparse, empirically just 2 of them per Principal and 1 Agent, so you get the classical Requestor-Agent-Verifier solution. Then you turn off the requestor, of course."
"Uh huh. Wait! You don't turn off human Requestor, that's not in the RAV paper!"
"They go to sleep and wake up as the Verifier, whatever. But as the saying goes, the only natural numbers are 0, 1, ∞, and off-by-one-errors, hehe. While it's the L two norm that corresponds to number 1..."
"Huh?"
"Something something flatness of space. Don't you remember your physics lessons from 4 years ago, Tom?"
"That's spacetime."
"No, space, this one's from boundary geometry, not discreet relativity. Also, he's 13, they'll learn both next yea...aaargh!"
"Silence, all of you! Where was I? The split of the Principal into exactly 2 min-err-shards turned out to be a coincidence from the hyperparameters, not a natural representation. It was already known that L1 tends to create capitalist-hell maximizers and L∞ democratic pipedream would have been too dense for opENcuDNN even if anyone tried, but L2 failed. Agents are anti-flat and L1 was empirically better for all those 𐌔AEs that showed us the 3 dominant players in all of those infrabayes agent benchmarks. It wasn't until the BYD-Tesla incident that we figured out the metric needs to be fractal, resulting in the L two plus or minus epsilon norm."
✋ ✋
"Yes?"
"What incident?" "Epsilon?"
"The high speed train collision with the BT pod1 right after their merger."
"Merger? I don't remember any of that." 🤷
"They had to stabilize the world grid, not relevant.
In the collision, only 6 bio-people died, but 3 billion b-backups were perma-dropped, despite the 99.99984% SLA safety claims. The supreme court established it wasn't false advertisement or marketing hype, the formal proofs were surprisingly accurate, the reach-avoid supermartingale guarantees worked out (almost) exactly as modelled.
But it happened during the Sepak Takraw championship between Texas and Kedah, so there was no cache capacity during the backMeUp™ synchronization. They were using PTCP/IP for the sync, so as you can imagine, the thousand-nines guarantee of not losing any packet is applicable only to the highest priority queue. While reindexing the bio_back_chcksm table is usually on top, bandwidth was given to the train to ask for simulation of differential braking patterns (to save as many of the 3537 bio-passengers as possible) for a whole 3 millis.
The govcentral's bchain consensus decided to restore only the de-luxe subscribers, 200 thousand citizen souls. Few of the others ran in mirror mode, most were in extended, but precise numbers are unknown due to iGDPR. It's estimated over 2.5 billion became bio-brain damaged, only their Skins still running."
"Yeah, the Choo Choo fork, I'm with you now. Didn't know BT caused the Zombies."
"Triggered, not caused. Contrary to popular belief, the Zombies were not the Skin Shells themselves, it was a pejorative name for the members of the Republican wing of the Earth People's Party who believed that the Skin Shells have souls too and we shouldn't abort them (a common sentiment among all voting citizens), not even during water shortages (a very fringe opinion). They lynched a group of ex-string theorists who, according to REPPS, were responsible for the prevalence of formal reasoning that enabled many black swan events in the last century. Popularity of formal proofs and maths in general started to decline, but an AAGI created by a group of Effective DeConsequentialists was unhappy about the development and started the p-Zombies war. But that's a story for another time.
Epsilon.
It stands for Level of detail. Following a good tradition of turning unanswerable philosophical questions into definitions, “Why do agents happen to be interested in the edge of chaos?” turned into it's own true name, informally “that which interests in the edge of chaos,” laying between orderly algorithms and random noise. There are some global limits for sure, at L<1.52, only singularities can exist (after infinite time) and an L26 model of a thermostat would appear to contain more sub-agents than stars in a galaxy. But largely to and fro, the exact value is a priori unknowable. It needs to be measured.
Assuming self-similarity of agents due to computational boundedness, the edge of the chaos that is real in the environment tends to be approximated by all agents afforded from that environment by similarity to itself. Thus the measure of the edge, modelled by a fractal, depends on the level of detail. A conductor will want to see your ticket, and your grandma will always give you one more ladle of soup after you say stop, not the other way round.
So we had to give up the assumption of the independence of measurement in the agent theory just like we did in physics. After all, superdeterministic interpretation is more appropriate for living in a simulation, there is no need for “many worlds.” Remind me to tell you about the Corrigible Schism, when someone made the proverbial coffee robot - nicknamed Murderbot after the event - and when the detectives elked it why it did it, they found a strange correlation between extreme risk taking and the belief that it only happens “in some universes.” Wouldn't..."
"Hugo? Are you getting somewhere with all this?" Oof
"That was quite a detour, indeed. So, that's why Principal-Agent-Verifiers is plural, the infinite-shrapnel VNM extension can be approximated to arbitrary precision with finite number of shards. Thanks again Trevor for the question."
"Sorry."
"Not your fault." Au
"For the main topic today, let's dive into the fascinating dimensions of the Principal-Agent-Verifiers-Observers field theory. The hidden assumption of objective reality had to give way so that we could explain consciousness. We can now answer with confidence, “What should an imagined simulated-student ask in a dystopian story for the readers who might find it too optimistic to suspend their beliefs of the student's agency?”"
"Field theory? Didn't you say it was a “framework” 5 minutes ago? How would you even..."
"Hmm, why are you still asking stupid questions after the zapping?" Hmm, the pain signal shouldn't vanish between their S2 and ACC.. Am I in a tes
END OF TRANSCRIPTION
I tried to be use the technobabble in a way that’s usefully wrong, so please also let me know if someone gets inspired by this short story.
I am not making predictions about the future, only commenting on the present—if you notice any factual error from that point of view, feel free to speak up, but as far as the doominess spectrum goes, it’s supposed to be both too dystopian and too optimistic at the same time.
And if someone wants to fix a typo or a grammo, I’d welcome a pull request (but no commas shall be harmed in the process). 🙏
*from?
Thank you for the engagement, but “to and fro” is a real expression, not a typo (and I’m keeping it).. it’s used slightly unorthodoxly here, but it sounded right to my ear, so it survived editing ¯\_(ツ)_/¯