Eliezer Yudkowsky

Karma: 145,926

Eliezer Yudkowsky 5 Nov 2024 19:24 UTC
27 points
3
on: Current safety training techniques do not fully transfer to the agent setting
Well-checked.

Eliezer Yudkowsky 19 Oct 2024 18:14 UTC
32 points
26
in reply to: Matthew Barnett’s comment on: The Hidden Complexity of Wishes
Your distinction between “outer alignment” and “inner alignment” is both ahistorical and unYudkowskian. It was invented years after this post was written, by someone who wasn’t me; and though I’ve sometimes used the terms in occasions where they seem to fit unambiguously, it’s not something I see as a clear ontological division, especially if you’re talking about questions like “If we own the following kind of blackbox, would alignment get any easier?” which on my view breaks that ontology. So I strongly reject your frame that this post was “clearly portraying an outer alignment problem” and can be criticized on those grounds by you; that is anachronistic.
You are now dragging in a very large number of further inferences about “what I meant”, and other implications that you think this post has, which are about Christiano-style proposals that were developed years after this post. I have disagreements with those, many disagreements. But it is definitely not what this post is about, one way or another, because this post predates Christiano being on the scene.
What this post is trying to illustrate is that if you try putting crisp physical predicates on reality, that won’t work to say what you want. This point is true! If you then want to take in a bunch of anachronistic ideas developed later, and claim (wrongly imo) that this renders irrelevant the simple truth of what this post actually literally says, that would be a separate conversation. But if you’re doing that, please distinguish the truth of what this post actually says versus how you think these other later clever ideas evade or bypass that truth.

Eliezer Yudkowsky 18 Oct 2024 17:08 UTC
35 points
33
in reply to: Matthew Barnett’s comment on: The Hidden Complexity of Wishes
The post is about the complexity of what needs to be gotten inside the AI. If you had a perfect blackbox that exactly evaluated the thing-that-needs-to-be-inside-the-AI, this could possibly simplify some particular approaches to alignment, that would still in fact be too hard because nobody has a way of getting an AI to point at anything. But it would not change the complexity of what needs to be moved inside the AI, which is the narrow point that this post is about; and if you think that some larger thing is not correct, you should not confuse that with saying that the narrow point this post is about, is incorrect.
I claim that having such a function would simplify the AI alignment problem by reducing it from the hard problem of getting an AI to care about something complex (human value) to the easier problem of getting the AI to care about that particular function (which is simple, as the function can be hooked up to the AI directly).
One cannot hook up a function to an AI directly; it has to be physically instantiated somehow. For example, the function could be a human pressing a button; and then, any experimentation on the AI’s part to determine what “really” controls the button, will find that administering drugs to the human, or building a robot to seize control of the reward button, is “really” (from the AI’s perspective) the true meaning of the reward button after all! Perhaps you do not have this exact scenario in mind. So would you care to spell out what clever methodology you think invalidates what you take to be the larger point of this post—though of course it has no bearing on the actual point that this post makes?

Eliezer Yudkowsky 18 Oct 2024 17:02 UTC
42 points
52
in reply to: Raemon’s comment on: The Hidden Complexity of Wishes
Wish there was a system where people could pay money to bid up what they believed were the “top arguments” that they wanted me to respond to. Possibly a system where I collect the money for writing a diligent response (albeit note that in this case I’d weigh the time-cost of responding as well as the bid for a response); but even aside from that, some way of canonizing what “people who care enough to spend money on that” think are the Super Best Arguments That I Should Definitely Respond To. As it stands, whatever I respond to, there’s somebody else to say that it wasn’t the real argument, and this mainly incentivizes me to sigh and go on responding to whatever I happen to care about more.
(I also wish this system had been in place 24 years ago so you could scroll back and check out the wacky shit that used to be on that system earlier, but too late now.)

Eliezer Yudkowsky 26 Aug 2024 15:56 UTC
48 points
0
on: Rationalism before the Sequences
I note that I haven’t said out loud, and should say out loud, that I endorse this history. Not every single line of it (see my other comment on why I reject verificationism) but on the whole, this is well-informed and well-applied.

Eliezer Yudkowsky 9 Aug 2024 18:14 UTC
LW: 21 AF: 12
8
AF
in reply to: Richard_Ngo’s comment on: A simple case for extreme inner misalignment
If you had to put a rough number on how likely it is that a misaligned superintelligence would primarily value “small molecular squiggles” versus other types of misaligned goals, would it be more like 1000:1 or 1:1 or 1000:1 or something else?
Value them primarily? Uhhh… maybe 1:3 against? I admit I have never actually pondered this question before today; but 1 in 4 uncontrolled superintelligences spending most of their resources on tiny squiggles doesn’t sound off by, like, more than 1-2 orders of magnitude in either direction.
Clocks are not actually very complicated; how plausible is it on your model that these goals are as complicated as, say, a typical human’s preferences about how human civilization is structured?
It wouldn’t shock me if their goals end up far more complicated than human ones; the most obvious pathway for it is (a) gradient descent turning out to produce internal preferences much faster than natural selection + biological reinforcement learning and (b) some significant fraction of those preferences being retained under reflection. (Where (b) strikes me as way less probable than (a), but not wholly forbidden.) The second most obvious pathway is if a bunch of weird detailed noise appears in the first version of the reflective process and then freezes.

Eliezer Yudkowsky 9 Aug 2024 2:19 UTC
LW: 37 AF: 16
13
AF
on: Self-Other Overlap: A Neglected Approach to AI Alignment
Not obviously stupid on a very quick skim. I will have to actually read it to figure out where it’s stupid.
(I rarely give any review this positive on a first skim. Congrats.)

Eliezer Yudkowsky 29 Jul 2024 19:13 UTC
LW: 7 AF: 5
2
AF
in reply to: Wei Dai’s comment on: Decision theory does not imply that we get to have nice things
By “dumb player” I did not mean as dumb as a human player. I meant “too dumb to compute the pseudorandom numbers, but not too dumb to simulate other players faithfully apart from that”. I did not realize we were talking about humans at all. This jumps out more to me as a potential source of misunderstanding than it did 15 years ago, and for that I apologize.

Eliezer Yudkowsky 28 Jul 2024 21:30 UTC
LW: 12 AF: 6
8
AF
in reply to: Wei Dai’s comment on: Decision theory does not imply that we get to have nice things
I don’t always remember my previous positions all that well, but I doubt I would have said at any point that sufficiently advanced LDT agents are friendly to each other, rather than that they coordinate well with each other (and not so with us)?

Eliezer Yudkowsky 16 Jul 2024 19:59 UTC
LW: 26 AF: 11
13
AF
in reply to: Eliezer Yudkowsky’s comment on: A simple case for extreme inner misalignment
Actually, to slightly amend that: The part where squiggles are small is a more than randomly likely part of the prediction, but not a load-bearing part of downstream predictions or the policy argument. Most of the time we don’t needlessly build our own paperclips to be the size of skyscrapers; even when having fun, we try to do the fun without vastly more resources, than are necessary to that amount of fun, because then we’ll have needlessly used up all our resources and not get to have more fun. We buy cookies that cost a dollar instead of a hundred thousand dollars. A very wide variety of utility functions you could run over the outside universe will have optima around making lots of small things because each thing scores one point, and so to score as many points as possible, each thing is as small as it can be as still count as a thing. Nothing downstream depends on this part coming true and there are many ways for it to come false; but the part where the squiggles are small and molecular is an obvious kind of guess. “Great giant squiggles of nickel the size of a solar system would be no more valuable, even from a very embracing and cosmopolitan perspective on value” is the loadbearing part.

Eliezer Yudkowsky 16 Jul 2024 14:11 UTC
LW: 26 AF: 10
19
AF
on: A simple case for extreme inner misalignment
The part where squiggles are small and simple is unimportant. They could be bigger and more complicated, like building giant mechanical clocks. The part that matters is that squiggles/paperclips are of no value even from a very cosmopolitan and embracing perspective on value.

Eliezer Yudkowsky 11 Jun 2024 18:07 UTC
145 points
34
on: My AI Model Delta Compared To Yudkowsky
I think that the AI’s internal ontology is liable to have some noticeable alignments to human ontology w/r/t the purely predictive aspects of the natural world; it wouldn’t surprise me to find distinct thoughts in there about electrons. As the internal ontology goes to be more about affordances and actions, I expect to find increasing disalignment. As the internal ontology takes on any reflective aspects, parts of the representation that mix with facts about the AI’s internals, I expect to find much larger differences—not just that the AI has a different concept boundary around “easy to understand”, say, but that it maybe doesn’t have any such internal notion as “easy to understand” at all, because easiness isn’t in the environment and the AI doesn’t have any such thing as “effort”. Maybe it’s got categories around yieldingness to seven different categories of methods, and/or some general notion of “can predict at all / can’t predict at all”, but no general notion that maps onto human “easy to understand”—though “easy to understand” is plausibly general-enough that I wouldn’t be unsurprised to find a mapping after all.
Corrigibility and actual human values are both heavily reflective concepts. If you master a requisite level of the prerequisite skill of noticing when a concept definition has a step where its boundary depends on your own internals rather than pure facts about the environment—which of course most people can’t do because they project the category boundary onto the environment, but I have some credit that John Wentworth might be able to do it some—and then you start mapping out concept definitions about corrigibility or values or god help you CEV, that might help highlight where some of my concern about unnatural abstractions comes in.
Entirely separately, I have concerns about the ability of ML-based technology to robustly point the AI in any builder-intended direction whatsoever, even if there exists some not-too-large adequate mapping from that intended direction onto the AI’s internal ontology at training time. My guess is that more of the disagreement lies here.
What links here?

Eliezer Yudkowsky 21 Apr 2024 1:37 UTC
LW: 16 AF: 9
8
AF
in reply to: Jan_Kulveit’s comment on: GPTs are Predictors, not Imitators
What the main post is responding to is the argument: “We’re just training AIs to imitate human text, right, so that process can’t make them get any smarter than the text they’re imitating, right? So AIs shouldn’t learn abilities that humans don’t have; because why would you need those abilities to learn to imitate humans?” And to this the main post says, “Nope.”
The main post is not arguing: “If you abstract away the tasks humans evolved to solve, from human levels of performance at those tasks, the tasks AIs are being trained to solve are harder than those tasks in principle even if they were being solved perfectly.” I agree this is just false, and did not think my post said otherwise.

Eliezer Yudkowsky 20 Mar 2024 0:13 UTC
15 points
−3
in reply to: Quintin Pope’s comment on: ‘Empiricism!’ as Anti-Epistemology
Unless I’m greatly misremembering, you did pick out what you said was your strongest item from Lethalities, separately from this, and I responded to it. You’d just straightforwardly misunderstood my argument in that case, so it wasn’t a long response, but I responded. Asking for a second try is one thing, but I don’t think it’s cool to act like you never picked out any one item or I never responded to it.
EDIT: I’m misremembering, it was Quintin’s strongest point about the Bankless podcast. https://www.lesswrong.com/posts/wAczufCpMdaamF9fy/my-objections-to-we-re-all-gonna-die-with-eliezer-yudkowsky?commentId=cr54ivfjndn6dxraD

Eliezer Yudkowsky 20 Mar 2024 0:08 UTC
12 points
4
in reply to: TurnTrout’s comment on: ‘Empiricism!’ as Anti-Epistemology
If Quintin hasn’t yelled “Empiricism!” then it’s not about him. This is more about (some) e/accs.

Eliezer Yudkowsky 11 Mar 2024 0:58 UTC
12 points
10
in reply to: River’s comment on: My Clients, The Liars
Wow, that’s fucked up.

Eliezer Yudkowsky 15 Feb 2024 18:29 UTC
12 points
4
in reply to: Martín Soto’s comment on: The “Commitment Races” problem
I am denying that superintelligences play this game in a way that looks like “Pick an ordinal to be your level of sophistication, and whoever picks the higher ordinal gets $9.” I expect sufficiently smart agents to play this game in a way that doesn’t incentivize attempts by the opponent to be more sophisticated than you, nor will you find yourself incentivized to try to exploit an opponent by being more sophisticated than them, provided that both parties have the minimum level of sophistication to be that smart.
If faced with an opponent stupid enough to play the ordinal game, of course, you just refuse all offers less than $9, and they find that there’s no ordinal level of sophistication they can pick which makes you behave otherwise. Sucks to be them!

Eliezer Yudkowsky 15 Feb 2024 18:23 UTC
35 points
0
in reply to: cubefox’s comment on: The Hidden Complexity of Wishes
You have misunderstood (1) the point this post was trying to communicate and (2) the structure of the larger argument where that point appears, as follows:
First, let’s talk about (2), the larger argument that this post’s point was supposed to be relevant to.
Is the larger argument that superintelligences will misunderstand what we really meant, due to a lack of knowledge about humans?
It is incredibly unlikely that Eliezer Yudkowsky in particular would have constructed an argument like this, whether in 2007, 2017, or even 1997. At all of these points in my life, I visibly held quite a lot of respect for the epistemic prowess of superintelligences. They were always going to know everything relevant about the complexities of human preference and desire. The larger argument is about whether it’s easy to make superintelligences end up caring.
This post isn’t about the distinction between knowing and caring, to be clear; that’s something I tried to cover elsewhere. The relevant central divide falls in roughly the same conceptual place as Hume’s Guillotine between ‘is’ and ‘ought’, or the difference between the belief function and the utility function.
(I don’t see myself as having managed to reliably communicate this concept (though the central idea is old indeed within philosophy) to the field that now sometimes calls itself “AI alignment”; so if you understand this distinction yourself, you should not assume that any particulary commentary within “AI alignment” is written from a place of understanding it too.)
What this post is about is the amount of information-theoretic complexity that you need to get into the system’s preferences, in order to have that system, given unlimited or rather extremely large amounts of power, deliver to you what you want.
It doesn’t argue that superintelligences will not know this information. You’ll note that the central technology in the parable isn’t an AI; it’s an Outcome Pump.
What it says, rather, is that there might be, say, a few tens of thousands of bits—the exact number is not easy to estimate, we just need to know that it’s more than a hundred bits and less than a billion bits and anything in that range is approximately the same problem from our standpoint—that you need to get into the steering function. If you understand the Central Divide that Hume’s Razor points to, the distinction between probability and preference, etcetera, the post is trying to establish the idea that we need to get 13,333 bits or whatever into the second side of this divide.
In terms of where this point falls within the larger argument, this post is not saying that it’s particularly difficult to get those 13,333 bits into the preference function; for all this post tries to say, locally, maybe that’s as easy as having humans manually enter 13,333 yes-or-no answers into the system. It’s not talking about the difficulty of doing the work but rather the amount and nature of a kind of work that needs to be done somehow.
Definitely, the post does not say that it’s hard to get those 13,333 bits into the belief function or knowledge of a superintelligence.
Separately from understanding correctly what this post is trying to communicate, at all, in 2007, there’s the question of whether modern LLMs have anything to say about—obviously not the post’s original point—but rather, other steps of the larger question in which this post’s point appears.
Modern LLMs, if you present them with a text-based story like the one in this parable, are able to answer at least some text-based questions about whether you’d prefer your grandmother to be outside the building or be safely outside the building. Let’s admit this premised observation at face value. Have we learned thereby the conclusion that it’s easy to get all of that information into a superintelligence’s preference function?
And if we say “No”, is this Eliezer making up post-hoc excuses?
What exactly we learn from the evidence of how AI has played out in 2024 so far, is the sort of thing that deserves its own post. But I observe that if you’d asked Eliezer-2007 whether an (Earth-originating) superintelligence could correctly predict the human response pattern about what to do with the grandmother—solve the same task LLMs are solving, to at least the LLM’s performance level—Eliezer-2007 would have unhesitatingly answered “yes” and indeed “OBVIOUSLY yes”.
How is this coherent? Because the post’s point is about how much information needs to get into the preference function. To predict a human response pattern you need (only) epistemic knowledge. This is part of why the post is about needing to give specifications to an Outcome Pump, rather than it depicting an AI being surprised by its continually incorrect predictions about a human response pattern.
If you don’t see any important distinction between the two, then of course you’ll think that it’s incoherent to talk about that distinction. But even if you think that Hume was mistaken about there existing any sort of interesting gap between ‘is’ and ‘ought’, you might by some act of empathy be able to imagine that other people think there’s an interesting subject matter there, and they are trying to talk about it with you; otherwise you will just flatly misunderstand what they were trying to say, and mispredict their future utterances. There’s a difference between disagreeing with a point, and just flatly failing to get it, and hopefully you aspire to the first state of mind rather than the second.
Have we learned anything stunningly hopeful from modern pre-AGIs getting down part of the epistemic part of the problem at their current ability levels, to the kind of resolution that this post talked about in 2007? Or from it being possible to cajole pre-AGIs with loss functions into willingly using that knowledge to predict human text outputs? Some people think that this teaches us that alignment is hugely easy. I think they are mistaken, but that would take its own post to talk about.
But people who point to “The Hidden Complexity of Wishes” and say of it that it shows that I had a view which the current evidence already falsifies—that I predicted that no AGI would ever be able to predict human response patterns about getting grandmothers out of burning buildings—have simply: misunderstood what the post is about, not understood in particular why the post is about an Outcome Pump rather than an AI stupidly mispredicting human responses, and failed to pick up on the central point that Eliezer expects superintelligences to be smart in the sense of making excellent purely epistemic predictions.

Eliezer Yudkowsky 14 Feb 2024 22:26 UTC
LW: 36 AF: 10
18
AF
on: Updatelessness doesn’t solve most problems
This deserves a longer answer than I have time to allocate it, but I quickly remark that I don’t recognize the philosophy or paradigm of updatelessness as refusing to learn things or being terrified of information; a rational agent should never end up in that circumstance, unless some perverse other agent is specifically punishing them for having learned the information (and will lose of their own value thereby; it shouldn’t be possible for them to gain value by behaving “perversely” in that way, for then of course it’s not “perverse”). Updatelessness is, indeed, exactly that sort of thinking which prevents you from being harmed by information, because your updateless exposure to information doesn’t cause you to lose coordination with your counterfactual other selves or exhibit dynamic inconsistency with your past self.
From an updateless standpoint, “learning” is just the process of reacting to new information the way your past self would want you to do in that branch of possibility-space; you should never need to remain ignorant of anything. Maybe that involves not doing the thing that would then be optimal when considering only the branch of reality you turned out to be inside, but the updateless mind denies that this was ever the principle of rational choice, and so feels no need to stay ignorant in order to maintain dynamic consistency.

Eliezer Yudkowsky 28 Dec 2023 17:53 UTC
LW: 10 AF: 6
15
AF
in reply to: Vanessa Kosoy’s comment on: Critical review of Christiano’s disagreements with Yudkowsky
They can solve it however they like, once they’re past the point of expecting things to work that sometimes don’t work. I have guesses but any group that still needs my hints should wait and augment harder.