I’m bumping into walls but hey now I know what the maze looks like.
Neil
Karma: 625
Poetry and practicality
I was staring up at the moon a few days ago and thought about how deeply I loved my family, and wished to one day start my own (I’m just over 18 now). It was a nice moment.
Then, I whipped out my laptop and felt constrained to get back to work; i.e. read papers for my AI governance course, write up LW posts, and trade emails with EA France. (These I believe to be my best shots at increasing everyone’s odds of survival).
It felt almost like sacrilege to wrench myself away from the moon and my wonder. Like I was ruining a moment of poetry and stillwatered peace by slamming against reality and its mundane things again.
But… The reason I wrenched myself away is directly downstream from the spirit that animated me in the first place. Whether I feel the poetry now that I felt then is irrelevant: it’s still there, and its value and truth persist. Pulling away from the moon was evidence I cared about my musings enough to act on them.
The poetic is not a separate magisterium from the practical; rather the practical is a particular facet of the poetic. Feeling “something to protect” in my bones naturally extends to acting it out. In other words, poetry doesn’t just stop. Feel no guilt in pulling away. Because, you’re not.
Too obvious imo, though I didn’t downnvote. This also might not be an actual rationalist failure mode; in my experience at least, rationalists have about the same intuition all the other humans have about when something should be taken literally or not.
As for why the comment section has gone berserk, no idea, but it’s hilarious and we can all use some fun.
Can we have a black banner for the FHI? Not a person, still seems appropriate imo.
See also Alicorn’s Expressive Vocabulary.
FHI at Oxford
by Nick Bostrom (recently turned into song):the big creaky wheel
a thousand years to turnthousand meetings, thousand emails, thousand rules
to keep things from changing
and heaven forbid
the setting of a precedentyet in this magisterial inefficiency
there are spaces and hiding places
for fragile weeds to bloom
and maybe bear some singular fruitlike the FHI, a misfit prodigy
daytime a tweedy don
at dark a superhero
flying off into the night
cape a-fluttering
to intercept villains and stop catastrophesand why not base it here?
our spandex costumes
blend in with the scholarly gowns
our unusual proclivities
are shielded from ridicule
where mortar boards are still in vogue
I’ve come to think that isn’t actually the case. E.g. while I disagree with Being nicer than clippy, it quite precisely nails how consequentialism isn’t essentially flawless:
I haven’t read that post, but I broadly agree with the excerpt. On green did a good job imo in showing how weirdly imprecise optimal human values are.
It’s true that when you stare at something with enough focus, it often loses that bit of “sacredness” which I attribute to green. As in, you might zoom in enough on the human emotion of love and discover that it’s just an endless tiling of Shrodinger’s equation.
If we discover one day that “human values” are eg 23.6% love, 15.21% adventure and 3% embezzling funds for yachts, and decide to tile the universe in exactly those proportions...[1] I don’t know, my gut doesn’t like it. Somehow, breaking it all into numbers turned humans into sock puppets reflecting the 23.6% like mindless drones.
The target “human values” seems to be incredibly small, which I guess encapsulates the entire alignment problem. So I can see how you could easily build an intuition from this along the lines of “optimizing maximally for any particular thing always goes horribly wrong”. But I’m not sure that’s correct or useful. Human values are clearly complicated, but so long as we haven’t hit a wall in deciphering them, I wouldn’t put my hands up in the air and act as if they’re indecipherable.
Unbounded utility maximization aspires to optimize the entire world. This is pretty funky for just about any optimization criterion people can come up with, even if people are perfectly flawless in how well they follow it. There’s a bunch of attempts to patch this, but none have really worked so far, and it doesn’t seem like any will ever work.
I’m going to read your post and see the alternative you suggest.
- ^
Sounds like a Douglas Adams plot
- ^
Interesting! Seems like you put a lot of effort into that 9,000-word post. May I suggest you publish it in little chunks instead of one giant post? You only got 3 karma for it, so I assume that those who started reading it didn’t find it worth the effort to read the whole thing. The problem is, that’s not useful feedback for you, because you don’t know which of those 9,000 words are presumably wrong. If I were building a version of utilitarianism, I would publish it in little bursts of 2-minute posts. You could do that right now with a single section of your original post. Clearly you have tons of ideas. Good luck!
You know, I considered “Bob embezzled the funds to buy malaria nets” because I KNEW someone in the comments would complain about the orphanage. Please don’t change.
Actually, the orphanage being a cached thought is precisely why I used it. The writer-pov lesson that comes with “don’t fight the hypothetical” is “don’t make your hypothetical needlessly distracting”. But maybe I miscalculated and malaria nets would be less distracting to LWers.
Anyway, I’m of course not endorsing fund-embezzling, and I think Bob is stupid. You’re right in that failure modes associated with Bob’s ambitions (eg human extinction) might be a lot worse than those of your typical fund-embezzler (eg the opportunity cost of buying yachts). I imagined Bob as being kind-hearted and stupid, but in your mind he might be some cold-blooded brooding “the price must be paid” type consequentialist. I didn’t give details either way, so that’s fair.
If you go around saying “the ends justify the means” you’re likely to make major mistakes, just like if you walk around saying “lying is okay sometimes”. The true lesson here is “don’t trust your own calculations, so don’t try being clever and blowing up TSMC”, not “consequentialism has inherent failure modes”. The ideal of consequentialism is essentially flawless; it’s when you hand it to sex-obsessed murder monkeys as an excuse to do things that shit hits the fan.
In my mind then, Bob was a good guy running on flawed hardware. Eliezer calls patching your consequentialism by making it bounded “consequentialism, one meta-level up”. For him, refusing to embezzle funds for a good cause because the plan could obviously turn sour is just another form of consequentialism. It’s like belief in intelligence, but flipped; you don’t know exactly how it’ll go wrong, but there’s a good chance you’re unfathomably stupid and you’ll make everything worse by acting on “the ends justify the means”.
From a practical standpoint though, we both agree and nothing changes: both the cold-hearted Bob and the kind Bob must be stopped. (And both are indeed more likely to make ethically dubious decisions because “the ends justify the means”.)
Post-scriptum:
Honestly the one who embezzles funds for unbounded consequentialist purposes sounds much more intellectually interesting
Yeah, this kind of story makes for good movies. When I wrote Bob I was thinking of The Wonderful Story of Mr.Sugar, by Roald Dahl and adapted by Wes Anderson on Netflix. It’s at least vaguely EA-spirited, and is kind of in that line (although the story is wholesome, as the name indicates, and isn’t meant to warn against dangers associated with boundless consequentialism at all).[1]
- ^
Let’s wait for the SBF movie on that one
- ^
Consequentialism is a compass, not a judge
Link is broken
Re: sociology. I found a meme you might enjoy, which would certainly drive your teacher through the roof: https://twitter.com/captgouda24/status/1777013044976980114
Privacy and writing
Yeah, that’s an excellent idea. I often spot typos in posts, but refrain from writing a comment unless I collect like three. Thanks for sharing!
A functionality I’d like to see on LessWrong: the ability to give quick feedback for a post in the same way you can react to comments (click for image). When you strong-upvote or strong-downvote a post, a little popup menu appears offering you some basic feedback options. The feedback is private and can only be seen by the author.
I’ve often found myself drowning in downvotes or upvotes without knowing why. Karma is a one-dimensional measure, and writing public comments is a trivial inconvience: this is an attempt at middle ground, and I expect it to make post reception clearer.
See below my crude diagrams.
I’m not clear on what you’re calling the “problem of superhuman AI”?
I was given clear instructions from a math phd about how to dump random lean files into the repository I created to confuse lesswrongers for at least a few minutes. But then I got confused while attempting to follow the instructions. There’s only so much my circuits can handle. I’m running most of my code on a Chromebook! Fear me.
Bonus song in I have been a good Bing: “Claude’s Anguish”, a 3-minute death-metal song whose lyrics were written by Claude when prompted with “how does the AI feel?”: https://app.suno.ai/song/40fb1218-18fa-434a-a708-1ce1e2051bc2/ (not for the faint of heart)
I’m glad “thought that faster” is the slowest song of the album. Also where’s the “Eliezer Yudkowsky” in the “ft. Eliezer Yudkowsky”? I didn’t click on it just to see Eliezer’s writing turned into song, I came to see Eliezer sing. Missed opportunity.
I’m not convinced. I felt the training video was incomplete, and the deadline too short.
I’m taking this post down, it was to set up an archive.org link as requested by Bostrom, and no longer serves that purpose. Sorry, this was meant to be discreet.