Ebenezer Dukakis

Karma: 518

Ebenezer Dukakis Apr 18, 2025, 3:39 AM
3 points
1
in reply to: Daniel Kokotajlo’s comment on: jacquesthibs’s Shortform
If people start losing jobs from automation, that could finally build political momentum for serious regulation.

Suggested in Zvi’s comments the other month (22 likes):

The real problem here is that AI safety feels completely theoretical right now. Climate folks can at least point to hurricanes and wildfires (even if connecting those dots requires some fancy statistical footwork). But AI safety advocates are stuck making arguments about hypothetical future scenarios that sound like sci-fi to most people. It’s hard to build political momentum around “trust us, this could be really bad, look at this scenario I wrote that will remind you of a James Cameron movie”

Here’s the thing though—the e/acc crowd might accidentally end up doing AI safety advocates a huge favor. They want to race ahead with AI development, no guardrails, full speed ahead. That could actually force the issue. Once AI starts really replacing human workers—not just a few translators here and there, but entire professions getting automated away—suddenly everyone’s going to start paying attention. Nothing gets politicians moving like angry constituents who just lost their jobs.

Here’s a wild thought: instead of focusing on theoretical safety frameworks that nobody seems to care about, maybe we should be working on dramatically accelerating workplace automation. Build the systems that will make it crystal clear just how transformative AI can be. It feels counterintuitive—like we’re playing into the e/acc playbook. But like extreme weather events create space to talk about carbon emissions, widespread job displacement could finally get people to take AI governance seriously. The trick is making sure this wake-up call happens before it’s too late to do anything about the bigger risks lurking around the corner.

Source: https://thezvi.substack.com/p/the-paris-ai-anti-safety-summit/comment/92963364

Just skimming the thread, I didn’t see anyone offer a serious attempt at counterargument, either.

Ebenezer Dukakis Mar 28, 2025, 1:48 AM
1 point
0
on: Prospects for Alignment Automation: Interpretability Case Study

I am optimistic that further thinking on automation prospects could identify other automation-tractable areas of alignment and control (e.g. see here for previous work).

This tag might be helpful: https://www.lesswrong.com/w/ai-assisted-alignment

Here’s a recent shortform on the topic: https://www.lesswrong.com/posts/mKgbawbJBxEmQaLSJ/davekasten-s-shortform?commentId=32jReMrHDd5vkDBwt

I wonder about getting an LLM to process LW archive posts, and tag posts which contain alignment ideas that seem automatable.

Ebenezer Dukakis Feb 25, 2025, 10:27 AM
1 point
0
in reply to: Seth Herd’s comment on: Ebenezer Dukakis’s Shortform

it will also set off the enemy rhetoric detectors among liberals

I’m not sure about that, does Bernie Sanders rhetoric set off that detector?

Ebenezer Dukakis Feb 25, 2025, 5:10 AM
1 point
0
in reply to: Seth Herd’s comment on: Ebenezer Dukakis’s Shortform
I think the way the issue is framed matters a lot. If it’s a “populist” framing (“elites are in it for themselves, they can’t be trusted”), that frame seems to have resonated with a segment of the right lately. Climate change has a sanctimonious frame in American politics that conservatives hate.

Ebenezer Dukakis Feb 22, 2025, 6:30 AM
3 points
2
on: The Sorry State of AI X-Risk Advocacy, and Thoughts on Doing Better
It looks like the comedian whose clip you linked has a podcast:

https://www.joshjohnsoncomedy.com/podcasts

I don’t see any guests in their podcast history, but maybe someone could invite him on a different podcast? His website lists appearances on other podcasts. I figure it’s worth trying stuff like this for VoI.

I think people should emphasize more the rate of improvement in this technology. Analogous to early days of COVID—it’s not where we are that’s worrisome; it’s where we’re headed.

Ebenezer Dukakis Feb 20, 2025, 10:51 PM
1 point
0
in reply to: TsviBT’s comment on: How to Make Superbabies

For humans acting very much not alone, like big AGI research companies, yeah that’s clearly a big problem.

How about a group of superbabies that find and befriend each other? Then they’re no longer acting alone.

I don’t think the problem is about any of the people you listed having too much brainpower.

I don’t think problems caused by superbabies would look distinctively like “having too much brainpower”. They would look more like the ordinary problems humans have with each other. Brainpower would be a force multiplier.

(I feel we’re somewhat talking past each other, but I appreciate the conversation and still want to get where you’re coming from.)

Thanks. I mostly just want people to pay attention to this problem. I don’t feel like I have unique insight. I’ll probably stop commenting soon, since I think I’m hitting the point of diminishing returns.

Ebenezer Dukakis Feb 20, 2025, 10:38 PM
70 points
29
in reply to: Eliezer Yudkowsky’s comment on: How to Make Superbabies
I think this project should receive more red-teaming before it gets funded.

Naively, it would seem that the “second species argument” matches much more strongly to the creation of a hypothetical Homo supersapiens than it does to AGI.

We’ve observed many warning shots regarding catastrophic human misalignment. The human alignment problem isn’t easy. And “intelligence” seems to be a key part of the human alignment picture. Humans often lack respect or compassion for other animals that they deem intellectually inferior—e.g. arguing that because those other animals lack cognitive capabilities we have, they shouldn’t be considered morally relevant. There’s a decent chance that Homo supersapiens would think along similar lines, and reiterate our species’ grim history of mistreating those we consider our intellectual inferiors.

It feels like people are deferring to Eliezer a lot here, which seems unjustified given how much strategic influence Eliezer had before AI became a big thing, and how poorly things have gone (by Eliezer’s own lights!) since then. There’s been very little reasoning transparency in Eliezer’s push for genetic enhancement. I just don’t see why we’re deferring to Eliezer so much as a strategist, when I struggle to name a single major strategic success of his.

Ebenezer Dukakis Feb 20, 2025, 10:16 PM
8 points
3
in reply to: lemonhope’s comment on: How to Make Superbabies
There’s a good chance their carbon children would have about the same attitude towards AI development as they do. So I suspect you’d end up ruled by their silicon grandchildren.

Ebenezer Dukakis Feb 20, 2025, 10:15 PM
3 points
1
in reply to: TsviBT’s comment on: How to Make Superbabies

These are incredibly small peanuts compared to AGI omnicide.

The jailbreakability and other alignment failures of current AI systems are also incredibly small peanuts compared to AGI omnicide. Yet they’re still informative. Small-scale failures give us data about possible large-scale failures.

You’re somehow leaving out all the people who are smarter than those people, and who were great for the people around them and humanity? You’ve got like 99% actually alignment or something

Are you thinking of people such as Sam Altman, Demis Hassabis, Elon Musk, and Dario Amodei? If humans are 99% aligned, how is it that we ended up in a situation where major lab leaders look so unaligned? MIRI and friends had a fair amount of influence to shape this situation and align lab leaders, yet they appear to have failed by their own lights. Why?

When it comes to AI alignment, everyone on this site understands that if a “boxed” AI acts nice, that’s not a strong signal of actual friendliness. The true test of an AI’s alignment is what it does when it has lots of power and little accountability.

Maybe something similar is going on for humans. We’re nice when we’re powerless, because we have to be. But giving humans lots of power with little accountability doesn’t tend to go well.

Looking around you, you mostly see nice humans. That could be because humans are inherently nice. It could also be because most of the people around you haven’t been given lots of power with little accountability.

Dramatic genetic enhancement could give enhanced humans lots of power with little accountability, relative to the rest of us.

[Note also, the humans you see while looking around are strongly selected for, which becomes quite relevant if the enhancement technology is widespread. How do you think you’d feel about humanity if you lived in Ukraine right now?]

Which, yes, we should think about this, and prepare and plan and prevent, but it’s just a totally totally different calculus from AGI.

I want to see actual, detailed calculations of p(doom) from supersmart humans vs supersmart AI, conditional on each technology being developed. Before charging ahead on this, I want a superforecaster-type person to sit down, spend a few hours, generate some probability estimates, publish a post, and request that others red-team their work. I don’t feel like that is a lot to ask.

Ebenezer Dukakis Feb 20, 2025, 9:38 PM
11 points
7
in reply to: TsviBT’s comment on: How to Make Superbabies

Humans are very far from fooming.

Tell that to all the other species that went extinct as a result of our activity on this planet?

I think it’s possible that the first superbaby will be aligned, same way it’s possible that the first AGI will be aligned. But it’s far from a sure thing. It’s true that the alignment problem is considerably different in character for humans vs AIs. Yet even in this particular community, it’s far from solved—consider Brent Dill, Ziz, Sam Bankman-Fried, etc.

Not to mention all of history’s great villains, many of whom believed themselves to be superior to the people they afflicted. If we use genetic engineering to create humans which are actually, massively, undeniably superior to everyone else, surely that particular problem is only gonna get worse. If this enhancement technology is going to be widespread, we should be using the history of human activity on this planet as a prior. Especially the history of human behavior towards genetically distinct populations with overwhelming technological inferiority. And it’s not pretty.

So yeah, there are many concrete details which differ between these two situations. But in terms of high-level strategic implications, I think there are important similarities. Given the benefit of hindsight, what should MIRI have done about AI back in 2005? Perhaps that’s what we should be doing about superbabies now.

Ebenezer Dukakis Feb 20, 2025, 9:20 PM
7 points
2
in reply to: lemonhope’s comment on: How to Make Superbabies
Altman and Musk are arguably already misaligned relative to humanity’s best interests. Why would you expect smarter versions of them to be more aligned? That only makes sense if we’re in an “alignment by default” world for superbabies, which is far from obvious.

Ebenezer Dukakis Feb 20, 2025, 9:14 PM
9 points
2
in reply to: GeneSmith’s comment on: How to Make Superbabies
If you look at the grim history of how humans have treated each other on this planet, I don’t think it’s justified to have a prior that this is gonna go well.

I think we have a huge advantage with humans simply because there isn’t the same potential for runaway self-improvement.

Humans didn’t have the potential for runaway self-improvement relative to apes. That was little comfort for the apes.

Ebenezer Dukakis Feb 20, 2025, 9:10 PM
20 points
14
in reply to: GeneSmith’s comment on: How to Make Superbabies
This is starting to sound a lot like AI actually. There’s a “capabilities problem” which is easy, an “alignment problem” which is hard, and people are charging ahead to work on capabilities while saying “gee, we’d really like to look into alignment at some point”.

Ebenezer Dukakis Feb 13, 2025, 4:17 AM
6 points
4
in reply to: George Ingebretsen’s comment on: George Ingebretsen’s Shortform
Can anyone think of alignment-pilled conservative influencers besides Geoffrey Miller? Seems like we could use more people like that...

Maybe we could get alignment-pilled conservatives to start pitching stories to conservative publications?

Ebenezer Dukakis Feb 13, 2025, 12:34 AM
4 points
0
in reply to: Seth Herd’s comment on: Ebenezer Dukakis’s Shortform
Likely true, but I also notice there’s been a surprising amount of drift of political opinions from the left to the right in recent years. The right tends to put their own spin on these beliefs, but I suspect many are highly influenced by the left nonetheless.

Some examples of right-coded beliefs which I suspect are, to some degree, left-inspired:
- “Capitalism undermines social cohesion. Consumerization and commoditization are bad. We’re a nation, not an economy.”
- “Trans women undermine women’s rights and women’s spaces. Motherhood, and women’s dignity, must be defended from neoliberal profit motives.”
- “US foreign policy is controlled by a manipulative deep state that pursues unnecessary foreign interventions to benefit elites.”
- “US federal institutions like the FBI are generally corrupt and need to be dismantled.”
- “We can’t trust elites. They control the media. They’re out for themselves rather than ordinary Americans.”
- “Your race, gender, religion, etc. are some of the most important things about you. There’s an ongoing political power struggle between e.g. different races.”
- “Big tech is corrosive for society.”
- “Immigration liberalization is about neoliberal billionaires undermining wages for workers like me.”
- “Shrinking the size of government is not a priority. We should make sure government benefits everyday people.”
- Anti-semitism, possibly.
One interesting thing has been seeing the left switch to opposing the belief when it’s adopted by the right and takes a right-coded form. E.g. US institutions are built on white supremacy and genocide, fundamentally institutionally racist, backed by illegitimate police power, and need to be defunded/decolonized/etc… but now they are being targeted by DOGE, and it’s a disaster!

(Note that the reverse shift has also happened. E.g. Trump’s approaches to economic nationalism, bilateral relations w/ China, and contempt for US institutions were all adopted by Biden by some degree.)

So yeah, my personal take is that we shouldn’t worry about publication venue that much. Just avoid insulting anyone, and make your case in a way which will appeal to the right (e.g. “we need to defend our traditional way of being human from AI”). If possible, target center-leaning publications like The Atlantic over explicitly progressive publications like Mother Jones.

Ebenezer Dukakis Feb 13, 2025, 12:17 AM
2 points
0
in reply to: Milan W’s comment on: Ebenezer Dukakis’s Shortform
I think the National Review is the most prestigious conservative magazine in the US, but there are various others. City Journal articles have also struck me as high-quality in the past. I think Coleman Hughes writes for them, and he did a podcast with Eliezer Yudkowsky at one point.

However, as stated in the previous link, you should likely work your way up and start by pitching lower-profile publications.

Ebenezer Dukakis Feb 13, 2025, 12:12 AM
3 points
0
in reply to: Noosphere89’s comment on: davekasten’s Shortform

The big one probably has to do with being able to corrupt the metrics so totally that whatever you think you made them unlearn actually didn’t happen, or just being able to relearn the knowledge so fast that unlearning doesn’t matter

I favor proactive approaches to unlearning which prevent the target knowledge from being acquired in the first place. E.g. for gradient routing, if you can restrict “self-awareness and knowledge of how to corrupt metrics” to a particular submodule of the network during learning, then if that submodule isn’t active, you can be reasonably confident that the metrics aren’t currently being corrupted. (Even if that submodule sandbags and underrates its own knowledge, that should be fine if the devs know to be wary of it. Just ablate that submodule whenever you’re measuring something that matters, regardless of whether your metrics say it knows stuff!)

Some related thoughts here

Unlearning techniques should probably be battle-tested in low-stakes “model organism” type contexts, where metrics corruption isn’t expected.

while I wouldn’t call it the best

Curious what areas you are most excited about!

Ebenezer Dukakis Feb 12, 2025, 12:18 PM
2 points
0
on: Ebenezer Dukakis’s Shortform
Regarding articles which target a popular audience such as How AI Takeover Might Happen in 2 Years, I get the sense that people are preaching to the choir by posting here and on X. Is there any reason people aren’t pitching pieces like this to prestige magazines like The Atlantic or wherever else? I feel like publishing in places like that is a better way to shift the elite discourse, assuming that’s the objective. (Perhaps it’s best to pitch to publications that people in the Trump admin read?)

Here’s an article on pitching that I found on the EA Forum by searching. I assume there are lots more tips on pitching online if you search.
What links here?
- Ebenezer Dukakis's comment on George Ingebretsen’s Shortform by George Ingebretsen (Feb 13, 2025, 4:17 AM; 6 points)

Ebenezer Dukakis Feb 12, 2025, 11:40 AM
3 points
0
in reply to: davekasten’s comment on: davekasten’s Shortform
I think unlearning could be a good fit for automated alignment research.

Unlearning could be a very general tool to address a lot of AI threat models. It might be possible to unlearn deception, scheming, manipulation of humans, cybersecurity, etc. I challenge you to come up with an AI safety failure story that can’t, in principle, be countered through targeted unlearning in some way, shape, or form.

Relative to some other kinds of alignment research, unlearning seems easy to automate, since you can optimize metrics for how well things have been unlearned.

I like this post.

Ebenezer Dukakis Nov 17, 2024, 6:02 AM
4 points
1
in reply to: Alexander Gietelink Oldenziel’s comment on: Alexander Gietelink Oldenziel’s Shortform

Chinas has alienated virtually all its neighbours

That sounds like an exaggeration? My impression is that China has OK/good relations with countries such as Vietnam, Cambodia, Pakistan, Indonesia, North Korea, factions in Myanmar. And Russia, of course. If you’re serious about this claim, I think you should look at a map, make a list of countries which qualify as “neighbors” based purely on geographic distance, then look up relations for each one.