I have a medianworld. It’s called Byway. It also solves alignment. In many ways Byway looks nothing like dath ilan. I want to write about how Byway solves alignment while looking nothing like dath ilan, in case it’s a helpful second data point for anyone else who, like me, finds it nontrivial to sort the parts of the dath ilani success scenario that are critical, from the parts that are just dath ilan being the medianworld of Eliezer.
Byway is an ancap medianworld, so it has no central will whatsoever, except the social consensus that actively emerges from smaller social consensuses. This is ultimately grounded in group projects with membership under Dunbar’s number, called “cults” [because that’s what they are]. Bywayeans have work—which is always engineering, never “white-collar”, and where they are always paid by commissions and/or profit shares, never wages or salaries. They have family life—they’re serial monogamists, but 1950s Earthlings would see most Bywayean families as “broken” or “blended”. They also have something Earthlings don’t: cults, or socially-valorized third spaces, where they can accomplish meaningful not-yet-profitable pursuits outside the family.
Bywayeans generally save up enough to move out of their parents’ houses around age 9. If they get sick or injured, Bywayeans have no one they can rely on but themselves, the market, and their personal connections. It’s “every man for himself”. But Bywayeans older than the age of 15 rarely experience Earthling-level financial hardship, because they love working and innovating and the labor market can afford to pay them a lot. Byway is more technologically advanced than Earth, so it is richer. For example, they “get around” having no central authority to maintain car-roads by using evtols for off-rail travel. i.e. they’ve had flying cars for a while.
I have talked to a lot of people who are horrified by descriptions of Byway, and while I think they’re mostly just not seeing the picture I’m trying to paint, over time I’ve come to realize that maybe people are different, such that for some people the social “lockstep” of dath ilan may feel saner and more desirable. My purpose here isn’t to prosyletize about how you should want to live on Byway [even though I instinctively want to!], but, like I said, to try and illustrate a promising alternate Earth that isn’t made of eliezera, to shake “promising” and “eliezera” loose from each other.
Bywayean science [like Bywayean aesthetics and philosophy] advances inside cults. There is always work the labor market doesn’t “know it wants to” pay for, yet—activity that would be “labor”, except the market hasn’t “priced it in”. Bywayeans don’t need any venture capitalists at all; they get together with 2-20 like-minded people and fund themselves. And you don’t need much money, to valuably pursue a very young hypothesis in paleontology or experimental art form, because in the baby stages of a hypothesis, where all the low-hanging fruit hasn’t already been picked, marginal intellectual return-on-labor-hour is very high.
“The baby stages” have high intellectual return-on-labor, low capital cost-of-labor; but also low capital return-on-labor. On Byway, like on Earth, the extra-hard problem of AI alignment stays in the baby stages for an anomalously long time. So it stays a cult activity.
How can a hard problem be solved without dath ilani lockstep, if the largest allowed simultaneous collective project is 3-20 people strong? Realistically, even though Bywayeans are smarter than Earthlings, even if the planet lucked into their very smartest 3-20 people all being in the same cult and having the right idea at the right time, those people alone wouldn’t have the attention span or the will to handle it correctly.
Byway can solve alignment because the social consensus that emerges via the network of Bywayean cults is very strong, and, by a form of memetic selection, quasi-”aligned” to human interests. “Aligned” enough to work.
When, for the very first time, a Bywayean cult has the idea that it should build a Vingean superintelligence, they run up against the Dartmouth project. Several members get bored and leave; several members are already heavily time-invested in other cults [Bywayeans have a lot of energy and are usually in more than one cult at one time]. These members drop hints in meetings of other cults; the other-cult-members, who are jealous of the AGI cultists’ time, poke holes in the idea that such a project would ever work in the first place. Some of these comments are just pointless sniping, but some are to the tune of “what will the robot even do when you turn it on?”. And those comments make the AGI cultists think. They bring them back to their AGI cults and the AGI cult asks itself what it’s even doing, and either comes up with a coherent answer, or disbands, its members saving the good ideas for better projects.
This happens again and again, every time Byway invents AGI, until gradually “everyone knows” that you have to have decided what the robot will do after you turn it on, or you are not going to be building a very beneficial Vingean superintelligence. The smartest, best-connected people, of course, know the most about alignment—but these are also the only people who know anything about capabilities. Bywayeans are pretty censorious and scrupulous about violations of the NAP, which is what an unaimed Vingean superintelligence is. So they check themselves, and check each other, and capabilities doesn’t get ahead of alignment.
Toward the end, there is a more intensively coalesced project—a “meta-cult” [no hereditary relation to the alt-right concept handle]. Byway is no stranger to these; it’s how they write their Code of ethics [a secret wiki]. But most technical projects don’t warrant the alignment meta-cult’s scope and size. It’s just a hard problem. Heads of subsections of the AI alignment meta-cult have physical meetings in secret workshops where they share ideas from their co-cultists and build prototypes. There’s an intelligence-based hierarchy—they are open about the fact that they are passing information up to the secret workshops, but not back down. No one breaks into the secret workshops and repurposes the machines because
a] Very few people know about the secret workshops, and the people who do, are almost all in awe of the people who run them, and would lose most of their social standing if they sabotaged the project
b] Each secret workshop has bespoke experimental hardware and encrypted [often on paper] instructions
c] Toward the end, they bother with a decent guard, just in case.
Like the dath ilani meta-project, there are multiple Creations, building towards a single final superintelligence at the end. I don’t know how many actually get built, what the earliest prototypes are aligned to do, or to what extent the earlier Creations are distributed across multiple component cults. I also have a wide uncertainty over how much internal technical jargon they invent, what it’s about, and how widely it’s shared.
A superficially plausible promising alternate Earth without lockstep
[ Context re dath ilan:
- [Keltham reflects on the concept of medianworlds]
- [dath ilani AI alignment] ]
re Byway:
- [the Bywayean Orbital Shade] ]
Dath ilan is Eliezer’s medianworld.
I have a medianworld. It’s called Byway. It also solves alignment. In many ways Byway looks nothing like dath ilan. I want to write about how Byway solves alignment while looking nothing like dath ilan, in case it’s a helpful second data point for anyone else who, like me, finds it nontrivial to sort the parts of the dath ilani success scenario that are critical, from the parts that are just dath ilan being the medianworld of Eliezer.
Byway is an ancap medianworld, so it has no central will whatsoever, except the social consensus that actively emerges from smaller social consensuses. This is ultimately grounded in group projects with membership under Dunbar’s number, called “cults” [because that’s what they are]. Bywayeans have work—which is always engineering, never “white-collar”, and where they are always paid by commissions and/or profit shares, never wages or salaries. They have family life—they’re serial monogamists, but 1950s Earthlings would see most Bywayean families as “broken” or “blended”. They also have something Earthlings don’t: cults, or socially-valorized third spaces, where they can accomplish meaningful not-yet-profitable pursuits outside the family.
Bywayeans generally save up enough to move out of their parents’ houses around age 9. If they get sick or injured, Bywayeans have no one they can rely on but themselves, the market, and their personal connections. It’s “every man for himself”. But Bywayeans older than the age of 15 rarely experience Earthling-level financial hardship, because they love working and innovating and the labor market can afford to pay them a lot. Byway is more technologically advanced than Earth, so it is richer. For example, they “get around” having no central authority to maintain car-roads by using evtols for off-rail travel. i.e. they’ve had flying cars for a while.
I have talked to a lot of people who are horrified by descriptions of Byway, and while I think they’re mostly just not seeing the picture I’m trying to paint, over time I’ve come to realize that maybe people are different, such that for some people the social “lockstep” of dath ilan may feel saner and more desirable. My purpose here isn’t to prosyletize about how you should want to live on Byway [even though I instinctively want to!], but, like I said, to try and illustrate a promising alternate Earth that isn’t made of eliezera, to shake “promising” and “eliezera” loose from each other.
Bywayean science [like Bywayean aesthetics and philosophy] advances inside cults. There is always work the labor market doesn’t “know it wants to” pay for, yet—activity that would be “labor”, except the market hasn’t “priced it in”. Bywayeans don’t need any venture capitalists at all; they get together with 2-20 like-minded people and fund themselves. And you don’t need much money, to valuably pursue a very young hypothesis in paleontology or experimental art form, because in the baby stages of a hypothesis, where all the low-hanging fruit hasn’t already been picked, marginal intellectual return-on-labor-hour is very high.
“The baby stages” have high intellectual return-on-labor, low capital cost-of-labor; but also low capital return-on-labor. On Byway, like on Earth, the extra-hard problem of AI alignment stays in the baby stages for an anomalously long time. So it stays a cult activity.
How can a hard problem be solved without dath ilani lockstep, if the largest allowed simultaneous collective project is 3-20 people strong? Realistically, even though Bywayeans are smarter than Earthlings, even if the planet lucked into their very smartest 3-20 people all being in the same cult and having the right idea at the right time, those people alone wouldn’t have the attention span or the will to handle it correctly.
Byway can solve alignment because the social consensus that emerges via the network of Bywayean cults is very strong, and, by a form of memetic selection, quasi-”aligned” to human interests. “Aligned” enough to work.
When, for the very first time, a Bywayean cult has the idea that it should build a Vingean superintelligence, they run up against the Dartmouth project. Several members get bored and leave; several members are already heavily time-invested in other cults [Bywayeans have a lot of energy and are usually in more than one cult at one time]. These members drop hints in meetings of other cults; the other-cult-members, who are jealous of the AGI cultists’ time, poke holes in the idea that such a project would ever work in the first place. Some of these comments are just pointless sniping, but some are to the tune of “what will the robot even do when you turn it on?”. And those comments make the AGI cultists think. They bring them back to their AGI cults and the AGI cult asks itself what it’s even doing, and either comes up with a coherent answer, or disbands, its members saving the good ideas for better projects.
This happens again and again, every time Byway invents AGI, until gradually “everyone knows” that you have to have decided what the robot will do after you turn it on, or you are not going to be building a very beneficial Vingean superintelligence. The smartest, best-connected people, of course, know the most about alignment—but these are also the only people who know anything about capabilities. Bywayeans are pretty censorious and scrupulous about violations of the NAP, which is what an unaimed Vingean superintelligence is. So they check themselves, and check each other, and capabilities doesn’t get ahead of alignment.
Toward the end, there is a more intensively coalesced project—a “meta-cult” [no hereditary relation to the alt-right concept handle]. Byway is no stranger to these; it’s how they write their Code of ethics [a secret wiki]. But most technical projects don’t warrant the alignment meta-cult’s scope and size. It’s just a hard problem. Heads of subsections of the AI alignment meta-cult have physical meetings in secret workshops where they share ideas from their co-cultists and build prototypes. There’s an intelligence-based hierarchy—they are open about the fact that they are passing information up to the secret workshops, but not back down. No one breaks into the secret workshops and repurposes the machines because
a] Very few people know about the secret workshops, and the people who do, are almost all in awe of the people who run them, and would lose most of their social standing if they sabotaged the project
b] Each secret workshop has bespoke experimental hardware and encrypted [often on paper] instructions
c] Toward the end, they bother with a decent guard, just in case.
Like the dath ilani meta-project, there are multiple Creations, building towards a single final superintelligence at the end. I don’t know how many actually get built, what the earliest prototypes are aligned to do, or to what extent the earlier Creations are distributed across multiple component cults. I also have a wide uncertainty over how much internal technical jargon they invent, what it’s about, and how widely it’s shared.