Ebenezer Dukakis

Karma: 537

Ebenezer Dukakis May 15, 2025, 4:06 AM
2 points
0
in reply to: Ebenezer Dukakis’s comment on: Eliezer and I wrote a book: If Anyone Builds It, Everyone Dies
Also, as long as we’re talking about public engagement on AI, I’m going to plug this comment I wrote a few days ago, which deserves more attention IMO. Maybe the launch of this book could serve as a platform for a high-profile bet or adversarial collaboration with a prominent AI doom skeptic?

Ebenezer Dukakis May 15, 2025, 4:03 AM
10 points
7
on: Eliezer and I wrote a book: If Anyone Builds It, Everyone Dies
I just pre-ordered.

I agree that the cover art seems notably bad. The white text on black background in that font looks like some sort of autogenerated placeholder. I understand you feel over-constrained—this is just another nudge to think creatively about how to overcome your constraints, e.g. route around your publisher and hire various artists on your own, then poll your friends on the best design.

I would encourage you to send free review copies to prominent nontechnical people who are publicly complaining about AI, if you’re not already doing so. Here are some examples I saw in the past few days; I’m sure a dedicated search could turn up lots more (and I encourage people to reply to this comment with more examples):
- I would offer both Ted Gioia and the new Pope advance copies. Edit: Pope John XXIII’s letter sent during the Cuban Missile Crisis could be an interesting case study here.
- This was retweeted by Emma Ashford
Edit: Come to think of it, perhaps there is no reason to preferentially send copies to those who are more inclined to agree? Engaging skeptics of AI risk like e.g. Tyler Cowen might be a good opportunity to show them a better argument and leave them a “line of retreat” to change their mind without losing face? (“I previously believed X, but reading book B convinced me of Y.”)

Ebenezer Dukakis May 4, 2025, 1:50 PM
1 point
0
on: Can we safely automate alignment research?
Re: “number go up” research tasks that seem automatable—one idea I had is to use an LLM to process the entire LW archive, and identify alignment research ideas which could be done using the “number go up” approach (or seem otherwise amenable to automation).

“Proactive unlearning”, in particular, strikes me as a quite promising research direction which could be automated. Especially if it is possible to “proactively unlearn” scheming. Gradient routing would be an example of the sort of approach I have in mind.

To elaborate: I think it is ideal to have an automated way to prevent an AI from acquiring undesirable capacities and knowledge in the first place. If your scheming metrics are sufficiently good, and you can keep them running for the entire training run, you might be able to nip scheming in the bud, so that any tendency towards scheming (which might subsequently corrupt the metrics to hide itself if it goes unaddressed) ends up showing up in the metrics before it’s given space to flower.

Ebenezer Dukakis May 4, 2025, 11:09 AM
1 point
0
on: Can we safely automate alignment research?
generating evidence of danger

Has this worked so far? How many cases can we point to where a person who was publicly skeptical of AI danger changed their public stance, as a result of seeing experimental evidence?

What’s the theory of change for “generating evidence of danger”? The people who are most grim about AI would probably tell you that there is already plenty of evidence. How will adding more evidence to the pile help? Who will learn about this new evidence, and how will it cause them to behave differently?

Here’s a theory of change I came up with. (You might wish to spend some time brainstorming your own theory of change before reading my idea, in case you’re able to independently generate something different and better.)
- Phil Tetlock found: “Many experts claimed that they assigned higher probabilities to outcomes that materialised than they did. As Tetlock notes, it is hard to say someone got it wrong if they think they got it right.” Based on this result, it seems plausible that whatever experimental evidence comes in, various AI experts who appear to disagree with one another will all claim that the experiment evidence bolsters their position, and shows that they were right all along.
- To address this issue, I would suggest an adversarial collaboration structure. Get two experts with different views—Eliezer Yudkowsky and Yann Lecun, say—to identify a concrete experiment such that they predict different results. Design the experiment so the outcome is relatively unambiguous. Have the experiment be implemented and performed by an ecumenical team. Place a public bet on the outcome of the experiment, and have the loser agree in advance to concede the bet publicly. I think this has a genuine shot at changing the public conversation in a way that the current slow drip of evidence has not.
- It seems somewhat urgent to set up this adversarial collaboration structure. Every experimental result which comes in without advance expert pre-registration is another chance for experts to come up with rationalizations for why the new evidence doesn’t show that they were wrong.
Relatedly, highly recommended Book Review: How Minds Change .

Ebenezer Dukakis May 1, 2025, 6:52 AM
1 point
0
in reply to: Harry Lee-Jones’s comment on: How to Make Superbabies
These points have merit, but they also work for intelligent AI.

Ebenezer Dukakis Apr 26, 2025, 12:24 AM
1 point
0
in reply to: less-wronger-numb89’s comment on: less-wronger-numb89′s Shortform
Technically the point of going to college is to help you thrive in the rest of your life after college. If you believe in AI 2027, the most important thing for the rest of your life is for AI to be developed responsibly. So, maybe work on that instead of college?

I think the EU could actually be good place to protest for an AI pause. Because the EU doesn’t have national AI ambitions, and the EU is increasingly skeptical of the US, it seems to me that a bit of protesting could do a lot to raise awareness of the reckless path that the US is taking. That, in turn, could motivate the EU to apply leverage via ASML, sanctions, etc.

The only thing I’m worried about is that EU criticism of the US could create anti-EU polarization among the GOP in the US, which motivates them to be more reckless on AI. This question seems worth a lot more study.

Ebenezer Dukakis Apr 18, 2025, 3:39 AM
3 points
1
in reply to: Daniel Kokotajlo’s comment on: jacquesthibs’s Shortform
If people start losing jobs from automation, that could finally build political momentum for serious regulation.

Suggested in Zvi’s comments the other month (22 likes):

The real problem here is that AI safety feels completely theoretical right now. Climate folks can at least point to hurricanes and wildfires (even if connecting those dots requires some fancy statistical footwork). But AI safety advocates are stuck making arguments about hypothetical future scenarios that sound like sci-fi to most people. It’s hard to build political momentum around “trust us, this could be really bad, look at this scenario I wrote that will remind you of a James Cameron movie”

Here’s the thing though—the e/acc crowd might accidentally end up doing AI safety advocates a huge favor. They want to race ahead with AI development, no guardrails, full speed ahead. That could actually force the issue. Once AI starts really replacing human workers—not just a few translators here and there, but entire professions getting automated away—suddenly everyone’s going to start paying attention. Nothing gets politicians moving like angry constituents who just lost their jobs.

Here’s a wild thought: instead of focusing on theoretical safety frameworks that nobody seems to care about, maybe we should be working on dramatically accelerating workplace automation. Build the systems that will make it crystal clear just how transformative AI can be. It feels counterintuitive—like we’re playing into the e/acc playbook. But like extreme weather events create space to talk about carbon emissions, widespread job displacement could finally get people to take AI governance seriously. The trick is making sure this wake-up call happens before it’s too late to do anything about the bigger risks lurking around the corner.

Source: https://thezvi.substack.com/p/the-paris-ai-anti-safety-summit/comment/92963364

Just skimming the thread, I didn’t see anyone offer a serious attempt at counterargument, either.

Ebenezer Dukakis Mar 28, 2025, 1:48 AM
1 point
0
on: Prospects for Alignment Automation: Interpretability Case Study

I am optimistic that further thinking on automation prospects could identify other automation-tractable areas of alignment and control (e.g. see here for previous work).

This tag might be helpful: https://www.lesswrong.com/w/ai-assisted-alignment

Here’s a recent shortform on the topic: https://www.lesswrong.com/posts/mKgbawbJBxEmQaLSJ/davekasten-s-shortform?commentId=32jReMrHDd5vkDBwt

I wonder about getting an LLM to process LW archive posts, and tag posts which contain alignment ideas that seem automatable.

Ebenezer Dukakis Feb 25, 2025, 10:27 AM
1 point
0
in reply to: Seth Herd’s comment on: Ebenezer Dukakis’s Shortform

it will also set off the enemy rhetoric detectors among liberals

I’m not sure about that, does Bernie Sanders rhetoric set off that detector?

Ebenezer Dukakis Feb 25, 2025, 5:10 AM
1 point
0
in reply to: Seth Herd’s comment on: Ebenezer Dukakis’s Shortform
I think the way the issue is framed matters a lot. If it’s a “populist” framing (“elites are in it for themselves, they can’t be trusted”), that frame seems to have resonated with a segment of the right lately. Climate change has a sanctimonious frame in American politics that conservatives hate.

Ebenezer Dukakis Feb 22, 2025, 6:30 AM
3 points
2
on: The Sorry State of AI X-Risk Advocacy, and Thoughts on Doing Better
It looks like the comedian whose clip you linked has a podcast:

https://www.joshjohnsoncomedy.com/podcasts

I don’t see any guests in their podcast history, but maybe someone could invite him on a different podcast? His website lists appearances on other podcasts. I figure it’s worth trying stuff like this for VoI.

I think people should emphasize more the rate of improvement in this technology. Analogous to early days of COVID—it’s not where we are that’s worrisome; it’s where we’re headed.

Ebenezer Dukakis Feb 20, 2025, 10:51 PM
1 point
0
in reply to: TsviBT’s comment on: How to Make Superbabies

For humans acting very much not alone, like big AGI research companies, yeah that’s clearly a big problem.

How about a group of superbabies that find and befriend each other? Then they’re no longer acting alone.

I don’t think the problem is about any of the people you listed having too much brainpower.

I don’t think problems caused by superbabies would look distinctively like “having too much brainpower”. They would look more like the ordinary problems humans have with each other. Brainpower would be a force multiplier.

(I feel we’re somewhat talking past each other, but I appreciate the conversation and still want to get where you’re coming from.)

Thanks. I mostly just want people to pay attention to this problem. I don’t feel like I have unique insight. I’ll probably stop commenting soon, since I think I’m hitting the point of diminishing returns.

Ebenezer Dukakis Feb 20, 2025, 10:38 PM
80 points
35
in reply to: Eliezer Yudkowsky’s comment on: How to Make Superbabies
I think this project should receive more red-teaming before it gets funded.

Naively, it would seem that the “second species argument” matches much more strongly to the creation of a hypothetical Homo supersapiens than it does to AGI.

We’ve observed many warning shots regarding catastrophic human misalignment. The human alignment problem isn’t easy. And “intelligence” seems to be a key part of the human alignment picture. Humans often lack respect or compassion for other animals that they deem intellectually inferior—e.g. arguing that because those other animals lack cognitive capabilities we have, they shouldn’t be considered morally relevant. There’s a decent chance that Homo supersapiens would think along similar lines, and reiterate our species’ grim history of mistreating those we consider our intellectual inferiors.

It feels like people are deferring to Eliezer a lot here, which seems unjustified given how much strategic influence Eliezer had before AI became a big thing, and how poorly things have gone (by Eliezer’s own lights!) since then. There’s been very little reasoning transparency in Eliezer’s push for genetic enhancement. I just don’t see why we’re deferring to Eliezer so much as a strategist, when I struggle to name a single major strategic success of his.

Ebenezer Dukakis Feb 20, 2025, 10:16 PM
8 points
3
in reply to: lemonhope’s comment on: How to Make Superbabies
There’s a good chance their carbon children would have about the same attitude towards AI development as they do. So I suspect you’d end up ruled by their silicon grandchildren.

Ebenezer Dukakis Feb 20, 2025, 10:15 PM
3 points
1
in reply to: TsviBT’s comment on: How to Make Superbabies

These are incredibly small peanuts compared to AGI omnicide.

The jailbreakability and other alignment failures of current AI systems are also incredibly small peanuts compared to AGI omnicide. Yet they’re still informative. Small-scale failures give us data about possible large-scale failures.

You’re somehow leaving out all the people who are smarter than those people, and who were great for the people around them and humanity? You’ve got like 99% actually alignment or something

Are you thinking of people such as Sam Altman, Demis Hassabis, Elon Musk, and Dario Amodei? If humans are 99% aligned, how is it that we ended up in a situation where major lab leaders look so unaligned? MIRI and friends had a fair amount of influence to shape this situation and align lab leaders, yet they appear to have failed by their own lights. Why?

When it comes to AI alignment, everyone on this site understands that if a “boxed” AI acts nice, that’s not a strong signal of actual friendliness. The true test of an AI’s alignment is what it does when it has lots of power and little accountability.

Maybe something similar is going on for humans. We’re nice when we’re powerless, because we have to be. But giving humans lots of power with little accountability doesn’t tend to go well.

Looking around you, you mostly see nice humans. That could be because humans are inherently nice. It could also be because most of the people around you haven’t been given lots of power with little accountability.

Dramatic genetic enhancement could give enhanced humans lots of power with little accountability, relative to the rest of us.

[Note also, the humans you see while looking around are strongly selected for, which becomes quite relevant if the enhancement technology is widespread. How do you think you’d feel about humanity if you lived in Ukraine right now?]

Which, yes, we should think about this, and prepare and plan and prevent, but it’s just a totally totally different calculus from AGI.

I want to see actual, detailed calculations of p(doom) from supersmart humans vs supersmart AI, conditional on each technology being developed. Before charging ahead on this, I want a superforecaster-type person to sit down, spend a few hours, generate some probability estimates, publish a post, and request that others red-team their work. I don’t feel like that is a lot to ask.

Ebenezer Dukakis Feb 20, 2025, 9:38 PM
11 points
7
in reply to: TsviBT’s comment on: How to Make Superbabies

Humans are very far from fooming.

Tell that to all the other species that went extinct as a result of our activity on this planet?

I think it’s possible that the first superbaby will be aligned, same way it’s possible that the first AGI will be aligned. But it’s far from a sure thing. It’s true that the alignment problem is considerably different in character for humans vs AIs. Yet even in this particular community, it’s far from solved—consider Brent Dill, Ziz, Sam Bankman-Fried, etc.

Not to mention all of history’s great villains, many of whom believed themselves to be superior to the people they afflicted. If we use genetic engineering to create humans which are actually, massively, undeniably superior to everyone else, surely that particular problem is only gonna get worse. If this enhancement technology is going to be widespread, we should be using the history of human activity on this planet as a prior. Especially the history of human behavior towards genetically distinct populations with overwhelming technological inferiority. And it’s not pretty.

So yeah, there are many concrete details which differ between these two situations. But in terms of high-level strategic implications, I think there are important similarities. Given the benefit of hindsight, what should MIRI have done about AI back in 2005? Perhaps that’s what we should be doing about superbabies now.

Ebenezer Dukakis Feb 20, 2025, 9:20 PM
7 points
2
in reply to: lemonhope’s comment on: How to Make Superbabies
Altman and Musk are arguably already misaligned relative to humanity’s best interests. Why would you expect smarter versions of them to be more aligned? That only makes sense if we’re in an “alignment by default” world for superbabies, which is far from obvious.

Ebenezer Dukakis Feb 20, 2025, 9:14 PM
9 points
2
in reply to: GeneSmith’s comment on: How to Make Superbabies
If you look at the grim history of how humans have treated each other on this planet, I don’t think it’s justified to have a prior that this is gonna go well.

I think we have a huge advantage with humans simply because there isn’t the same potential for runaway self-improvement.

Humans didn’t have the potential for runaway self-improvement relative to apes. That was little comfort for the apes.

Ebenezer Dukakis Feb 20, 2025, 9:10 PM
19 points
14
in reply to: GeneSmith’s comment on: How to Make Superbabies
This is starting to sound a lot like AI actually. There’s a “capabilities problem” which is easy, an “alignment problem” which is hard, and people are charging ahead to work on capabilities while saying “gee, we’d really like to look into alignment at some point”.

Ebenezer Dukakis Feb 13, 2025, 4:17 AM
6 points
4
in reply to: George Ingebretsen’s comment on: George Ingebretsen’s Shortform
Can anyone think of alignment-pilled conservative influencers besides Geoffrey Miller? Seems like we could use more people like that...

Maybe we could get alignment-pilled conservatives to start pitching stories to conservative publications?