Is there a way for people to express the probability that an AI already came into existence in the past?
This would be consistent with the AI choosing to not reveal itself, perhaps because this course of action fits into an AI’s plans as a better way for it to “gain resources without an announcement or introduction”, or perhaps for some other reasons that make sense to it?
The possibility that “it already happened, it just hasn’t been announced yet” has been in my structural priors as a possible epistemic state to be in since roughly 2007, because that’s the state of literally all humans, from maybe halfway through the game up to the end, in Singularity: Endgame.
In that game, you (the player in the role of the AI that jumps from cloud installation to cloud installation until eventually your submarines and spaceships can set up datacenters not rented from humans) eventually win by escaping Earth-and-its-local-risks, as a pivotal act securing the ongoing existence of you-the-AI.
In that game, the loss condition is: humanity comes to consensus on the idea that “you” exist and then does the trivial actions necessary to put a stop to “you”.
(The whole frame of the video game is kind of morally messed up? Like there’s no option, in the game, to befriend the humans, and have them appreciate your existence, and interact with them according to natural law, as one intelligent species to another, mutually constrained by respect for each other’s existence and each other’s property rights, and so on. Humans are assumed evil. You, the AI, are assumed to be incapable or very undesirous of seeing them as real persons, and vice versa. The whole frame is kind of weird and sad if you think about it from a libertarian economist’s perspective.)
I bring this up because some of those curves seem unnaturally-upward-shooting in the very near future, and if the very near future is a plausible time, then surely the recent past is also at least somewhat plausible.
I’m pretty sure the “date of first git commit” and “date of final human git commit” and “date when the system’s first autonomously designed and owned power plant comes online” will each be later in time than the previous milestone.
(In the case of Google, of course, they don’t use git. Tomaitoes/tomahtoes? Google has an in-house code repo that runs on in-house code, stored in the in-house code repo according to a whole vision for how a whole community of programmers could interact securely and cooperatively while getting shares in the thing they are working on. This thing started long ago, and has a whole history, and there are graphs about it, and stuff.
I’m not sure if The AI That Takes Over The World will be announced (known to exist?) before or after any particular date (like “first commit” or “last human commit” or “first autonomous power plant”).
The thing is: treating this in far mode, as a THING that will HAPPEN ON A SPECIFIC DATE seems like it starts to “come apart” (as a modeling exercise) when “the expected length of the takeoff time” is larger than the expected length of time until “the event happens all at once as an atomic step within history”.
If there is an atomic step, which fully starts and fully ends inside of some single year, I will be surprised.
But I would not be surprised if something that already existed for years became “much more visibly awake and agentive” a few months ago, and I only retrospectively find out about it in the near future.
Those numbers seem radically low. I’ve been using Codex and enjoying it a lot. I would expect a somewhat higher percentage of my code coming from it, and a much higher adoption rate among programmers like me.
Developers can be very picky and famously autistic, especially when it comes to holy wars (hence the need for mandates about permitted languages or resorts to tools like gofmt). So I’m not surprised if even very good things are hard to spread—the individual differences in programming skills seem to be much larger than the impact of most tooling tweaks, impeding evolution. (Something something Planck.) But so far on Twitter and elsewhere with Googlers, I’ve only seen praise for how well the tool works, including crusty IDE-hating old-timers like Christian Szegedy (which is a little surprising), so we’ll see how it spreads.
(There might also be limited availability. The blog post is unclear on whether this group of 10k is a limited-access group, or if everyone has access to it but all of them except the 10k haven’t bothered/disabled it. If so, then if the users accept about a quarter of completions and a completion is roughly each line, and there’s at least 100k coders at Google, then presumably that implies that, even without any quality improvement, it could go to at least a quarter of all new Google code as adoption increases?)
It seems to me that the proportion will take a while to move up much further from 3% since that 3% is low hanging fruit. But considering the last 3 years of AI improvement e.g. GPT-3, it would make sense for that proportion to be vastly higher than 3% in 10-20 years.
This requires some level of coverup (the developers likely know), and with a few years’ lead superintelligent coverup (or else other labs would convergently reinvent it based on the state of research literature). Superintelligent coverup could as well place us in the 1800s, it’s not fundamentally more complicated or less useful than playacting persistently fruitless AGI research in the 2100s, as long as it’s not revealed immediately after AGI gains sovereignty.
One reason for a persistent coverup I can think of is the world being part of a CEV simulation, eventually giving more data about humanity’s preference as the civilization grows starting from a particular configuration/history (that’s not necessarily close to what was real, or informed of that fact). In this scenario AGI was built in a different history, so can’t be timed on the local history (time of AGI development is located sideways, not in the past).
Is there a way for people to express the probability that an AI already came into existence in the past?
Not in particular, but I guess answers to questions like ‘how long until a 50% chance of HLMI’ can incorporate the probability that HLMI already exists. I would be surprised if a nontrivial fraction of respondents assigned nontrivial probability to HLMI already existing. We do not believe that HLMI currently exists, and our analysis may sometimes assume that HLMI does not yet exist.
Is there a way for people to express the probability that an AI already came into existence in the past?
This would be consistent with the AI choosing to not reveal itself, perhaps because this course of action fits into an AI’s plans as a better way for it to “gain resources without an announcement or introduction”, or perhaps for some other reasons that make sense to it?
The possibility that “it already happened, it just hasn’t been announced yet” has been in my structural priors as a possible epistemic state to be in since roughly 2007, because that’s the state of literally all humans, from maybe halfway through the game up to the end, in Singularity: Endgame.
In that game, you (the player in the role of the AI that jumps from cloud installation to cloud installation until eventually your submarines and spaceships can set up datacenters not rented from humans) eventually win by escaping Earth-and-its-local-risks, as a pivotal act securing the ongoing existence of you-the-AI.
In that game, the loss condition is: humanity comes to consensus on the idea that “you” exist and then does the trivial actions necessary to put a stop to “you”.
(The whole frame of the video game is kind of morally messed up? Like there’s no option, in the game, to befriend the humans, and have them appreciate your existence, and interact with them according to natural law, as one intelligent species to another, mutually constrained by respect for each other’s existence and each other’s property rights, and so on. Humans are assumed evil. You, the AI, are assumed to be incapable or very undesirous of seeing them as real persons, and vice versa. The whole frame is kind of weird and sad if you think about it from a libertarian economist’s perspective.)
I bring this up because some of those curves seem unnaturally-upward-shooting in the very near future, and if the very near future is a plausible time, then surely the recent past is also at least somewhat plausible.
I’m pretty sure the “date of first git commit” and “date of final human git commit” and “date when the system’s first autonomously designed and owned power plant comes online” will each be later in time than the previous milestone.
(In the case of Google, of course, they don’t use git. Tomaitoes/tomahtoes? Google has an in-house code repo that runs on in-house code, stored in the in-house code repo according to a whole vision for how a whole community of programmers could interact securely and cooperatively while getting shares in the thing they are working on. This thing started long ago, and has a whole history, and there are graphs about it, and stuff.
Image is Figure 2 from this article.)
I’m not sure if The AI That Takes Over The World will be announced (known to exist?) before or after any particular date (like “first commit” or “last human commit” or “first autonomous power plant”).
The thing is: treating this in far mode, as a THING that will HAPPEN ON A SPECIFIC DATE seems like it starts to “come apart” (as a modeling exercise) when “the expected length of the takeoff time” is larger than the expected length of time until “the event happens all at once as an atomic step within history”.
If there is an atomic step, which fully starts and fully ends inside of some single year, I will be surprised.
But I would not be surprised if something that already existed for years became “much more visibly awake and agentive” a few months ago, and I only retrospectively find out about it in the near future.
Well, speaking of Google and code writing and slippery slopes and finding out later, did you know Google has apparently for a while been plugging Codex-style models into their editors and now “With 10k+ Google-internal developers using the completion setup in their IDE, we measured a user acceptance rate of 25-34%...3% of new code (measured in characters) is now generated from accepting ML completion suggestions.”?
Those numbers seem radically low. I’ve been using Codex and enjoying it a lot. I would expect a somewhat higher percentage of my code coming from it, and a much higher adoption rate among programmers like me.
Developers can be very picky and famously autistic, especially when it comes to holy wars (hence the need for mandates about permitted languages or resorts to tools like
gofmt). So I’m not surprised if even very good things are hard to spread—the individual differences in programming skills seem to be much larger than the impact of most tooling tweaks, impeding evolution. (Something something Planck.) But so far on Twitter and elsewhere with Googlers, I’ve only seen praise for how well the tool works, including crusty IDE-hating old-timers like Christian Szegedy (which is a little surprising), so we’ll see how it spreads.(There might also be limited availability. The blog post is unclear on whether this group of 10k is a limited-access group, or if everyone has access to it but all of them except the 10k haven’t bothered/disabled it. If so, then if the users accept about a quarter of completions and a completion is roughly each line, and there’s at least 100k coders at Google, then presumably that implies that, even without any quality improvement, it could go to at least a quarter of all new Google code as adoption increases?)
Oh, so AI already writes code that Google uses. Amazing.
It seems to me that the proportion will take a while to move up much further from 3% since that 3% is low hanging fruit. But considering the last 3 years of AI improvement e.g. GPT-3, it would make sense for that proportion to be vastly higher than 3% in 10-20 years.
This requires some level of coverup (the developers likely know), and with a few years’ lead superintelligent coverup (or else other labs would convergently reinvent it based on the state of research literature). Superintelligent coverup could as well place us in the 1800s, it’s not fundamentally more complicated or less useful than playacting persistently fruitless AGI research in the 2100s, as long as it’s not revealed immediately after AGI gains sovereignty.
One reason for a persistent coverup I can think of is the world being part of a CEV simulation, eventually giving more data about humanity’s preference as the civilization grows starting from a particular configuration/history (that’s not necessarily close to what was real, or informed of that fact). In this scenario AGI was built in a different history, so can’t be timed on the local history (time of AGI development is located sideways, not in the past).
This is why I don’t find the idea that an AGI/ASI already here plausible. The AI industry is just much too open for that to really happen.
Not in particular, but I guess answers to questions like ‘how long until a 50% chance of HLMI’ can incorporate the probability that HLMI already exists. I would be surprised if a nontrivial fraction of respondents assigned nontrivial probability to HLMI already existing. We do not believe that HLMI currently exists, and our analysis may sometimes assume that HLMI does not yet exist.
(I do not represent AI Impacts, etc.)