Right now it doesn’t make sense; it is better to let the current owners keep improving their AIs.
Only if alignment progress keeps up with or exceeds AI progress, and you thus expect a controllable AI you can take over to do your bidding. But isn’t all the evidence pointing towards AI progress >> alignment progress?
A lot of things could happen, but something that has already happened is that official US AI policy is now that not racing towards AGI is bad, and that impeding AI progress is bad. Doesn’t that policy imply that AI lab nationalization is now less likely, rather than more likely, than it would’ve been under a D president?
Conversely, your scenario assumes that the Trump administration can do whatever it wants to do, but this ability is partially dependent on it staying popular with the general public. The public may not care about AI for now, but it very much does care about economics and inflation, and once Trump’s policies worsen those (e.g. via tariffs), then that severely restricts his ability to take arbitrary actions in other domains.
AI assistants are weird. Here’s a Perplexity Pro search I did for an EY tweet about finding the sweet spot between utilitarianism & deontology. Perplexity Pro immediately found the correct tweet:
Eliezer Yudkowsky, a prominent figure in the rationalist community, has indeed expressed a view that suggests finding a balance between utilitarianism and deontology. In a tweet, he stated: “Go three-quarters of the way from deontology to utilitarianism and then stop. You are now in the right place. Stay there at least until you...”
But I wondered why it didn’t provide the full quote (which is just a few more words, namely “Stay there at least until you have become a god.”), and I just couldn’t get it to do so, even with requests like “Just quote the full tweet from here: <URL>”. Instead, it invented alternative versions like this:
Go three-quarters of the way from deontology to utilitarianism and then stop. You are now in the right place. Stay there at least until you understand why.
or this:
Go three-quarters of the way from deontology to utilitarianism and then stop. You are now in the correct place. Stay there at least until you understand why you shouldn’t go any further.
I finally provided the full quote and asked it directly:
Does the following quote represent Yudkowsky’s tweet with 100% accuracy?
“Go three-quarters of the way from deontology to utilitarianism and then stop. You are now in the right place. Stay there at least until you have become a god.”
And it still doubled down on the wrong version.
“Copy link to highlight” is not available in Firefox. And while e.g. Bing search seems to automatically generate these “#:~:text=” links, I find they don’t work with any degree of consistency. And they’re even more affected by link rot than usual, since any change to the initial text (like a typo fix) will break that part of the link.
I don’t like it. Among various issues, people already muddy the waters by erroneously calling climate change an existential risk (rather than what it was, a merely catastrophic one, before AI timelines made any worries about climate change in the year 2100 entirely irrelevant), and it’s extremely partisan-coded. And you’re likely to hear that any mention of AI x-risk is a distraction from the real issues, which are whatever the people cared about previously.
I prefer an analogy to gain-of-function research. As in, scientists grow viruses/AIs in the lab, with promises of societal benefits, but without any commensurate acknowledgment of the risks. And you can’t trust the bio/AI labs to manage these risks, e.g. even high biosafety levels can’t entirely prevent outbreaks.
This seems way overdetermined. For example, AI labs have proven extremely successful at spending arbitrary amounts of money to increase capabilities (<-> scaling laws), and there’s been no similar ability to convert arbitrary amounts of money into progress on alignment.
The question is not whether alignment is impossible (though I would be astonished if it was), but rather whether it’s vastly easier to increase capabilities to AGI/ASI than it is to align AGI/ASI, and ~all evidence points to yes. And so the first AGI/ASI will not be aligned.
i am much more worried
Why? I figure all the AI labs worry mostly about how to get the loot, without ensuring that there’s going to be any loot in the first place. Thus there won’t be any loot, and we’ll go extinct without any human getting to play god-emperor. It seems to me like trying to build an AGI tyranny is an alignment-complete challenge, and since we’re not remotely on track to solving alignment, I don’t worry about that particular bad ending.
The German federal election is tomorrow. I looked up whether any party was against AI or for a global AI ban or similar. From what I can tell, the answer is unfortunately no, they all want more AI and just disagree about how much. I don’t want to vote for any party that wants more AI, so this is rather disappointing.
(Trollish reply. I’m not in favor of the death penalty.) I take your point re: that a death penalty cannot be implemented if individual executioners need to kill individual people via guns. But I’ll counter that it also can’t be implemented in the 21st century via gas chambers, because your executioners will realize the parallel to Nazi Germany. (“Are we the baddies?”) To split the difference, how about having executioners execute people via drones armed with bullets?
I assume the idea of “lens” as a term is that it’s one specific person’s opinionated view of a topic. As in, “here’s the concept seen through EY’s lens”. So terms like “variant” or “alternative” are too imprecise, but e.g. “perspective” might also work.
Based on AI organisations frequently achieving the opposite of their chosen name (OpenAI, Safe Superintelligence, etc.), UNBIASED would be the most biased model, INTELLECT would be the dumbest model, JUSTICE would be particularly unjust, MAGA would in effect be MAWA, etc.
I’m skeptical to which extent the latter can be done. That’s like saying an AI lab should suddenly care about AI safety. One can’t really bolt a security mandate onto an existing institution and expect a competent result.
or stopped disclosing its advancements publicly
Does this matter all that much, given lack of opsec, relationships between or poaching of employees of other labs, corporate espionage, etc.?
A more cynical perspective is that much of this arms race, especially the international one against China (quote from above: “If we don’t build fast enough, then the authoritarian countries could win.”), is entirely manufactured by the US AI labs.
Thanks! I haven’t had time to skim either report yet, but I thought it might be instructive to ask the same questions to Perplexity Pro’s DR mode (it has the same “Deep Research” as OpenAI, but otherwise has no relation to it, in particular it’s free with their regular monthly subscription, so it can’t be particularly powerful): see here. (This is the second report I generated, as the first one froze indefinitely while the app generated the conclusion, and thus the report couldn’t be shared.)
How about “analyze the implications for risk of AI extinction, based on how OpenAI’s safety page has changed over time”? Inspired by this comment (+ follow-up w/ Internet archive link).
apparently China as a state has devoted $1 trillion to AI
Source? I only found this article about 1 trillion Yuan, which is $137 billion.
If this risk is in the ballpark of a 5% chance in the next couple of years, then it seems to me entirely dominated by AI doom.
So much +1 on this.
Also, I’ve played a ton of games, and in the last few years started helping a bit with playtesting them etc. And I found it striking how games aren’t inherently intuitive, but are rather made so via strong economic incentives, endless playtests to stop players from getting stuck, etc. Games are intuitive for humans because humans spend a ton of effort to make them that way. If AIs were the primary target audience, games would be made intuitive for them.
And as a separate note, I’m not sure what the appropriate human reference class for game-playing AIs is, but I challenge the assumption that it should be people who are familiar with games. Rather than, say, people picked at random from anywhere on earth.