Zack_M_Davis

Karma: 16,536

Zack_M_Davis Jun 4, 2025, 11:48 PM
6 points
2
on: Briefly analyzing the 10-year moratorium amendment
Sen. Markey of Massachusetts has issued a press release condemning the proposed moratorium and expressing intent to raise a point of order:

I am committed to fighting this 10-year ban with every tool at my disposal. And that starts by making it clear that this 10-year ban on state AI regulation is a policy change that has no impact on the federal budget. That means it cannot be included in a reconciliation bill. If Senate Republicans keep the House language in their reconciliation bill, I will raise a point of order against it.

Zack_M_Davis Jun 2, 2025, 10:02 PM
14 points
9
in reply to: Duncan Sabien (Inactive)’s comment on: Truth or Dare
The issue is that I have no idea where you’re getting that hypothesis from. What have I written, anywhere, that makes you think I would disapprove of Alexei’s comment?

The seventh guideline doesn’t say that you shouldn’t hypothesize about what other people believe

In accordance with the Eighth Guideline, I would like to revise the wording of my invocation of the Seventh Guideline in the grandparent: given our history of communication failures, I think your comments would be better if you try to avoid posing hypotheses (not “making claims”) about what I believe in the absence of direct textual evidence, in accordance with the Seventh Guideline.

(But again, that’s just my opinion about how I think you could write better comments; I don’t consider it a “request.”)

I’d be interested in a statement of what Zack-guideline the above “here’s what I think he believes?” falls afoul of.

I still think your Seventh Guideline applies as written. All three of your examples of “ways a Seventh Guideline request might look” seem appropriate to me with some small adaptations for context (notwithstanding that I don’t believe in “requests”).

You wrote:

“wow, I support this way less than I otherwise would have, because your (hypothesized) straightforward diagnosis of what was going on in a large conflict over norms seems to me to be kind of petty” is contra both my norms and my understanding of Zack’s preferred norms; unless I miss him entirely neither one of us wants LessWrong to be the kind of place where that sort of factor weighs very heavily in people’s analysis.

The first example of a way a Seventh Guideline request might look says,

That’s not what I wrote, though. Can you please engage with what I wrote?

I can’t quite ask you to engage with what I wrote, because your hypothesis that I don’t “want LessWrong to be the kind of place where that sort of factor weighs very heavily in people’s analysis” bears no obvious resemblance to anything I’ve written, so it’s not clear what part of my writing I should be directing you to read more carefully.

In fact, I don’t even read the pettiness judgement as having weighed very heavily in Alexei’s analysis! Alexei wrote, “Overall strong upvote from me, but I’m not doing it because [...]”. I interpret this as saying that the pettiness of that section was enough of a detractor from the value of the post that he didn’t feel like awarding a strong-upvote, which I regard as distinct from weighing heavily in his analysis of the contents of the rest of the post themselves (as contrasted to his analysis of whether to strong-upvote). If it looks like Dante was motivated to write The Inferno in order to have a short section at the end depicting his enemies suffering divine punishment, that’s definitely something Dante scholars should be allowed to notice and criticize, without that weighing heavily into their analysis of the preceding 4000 lines: there’s a lot of stuff in those 4000 lines to be analyzed, separately from the fact that it’s all building up to the enemy torture scene. (I’m doing a decent amount of interpretation here; if Alexei happens to make the poor time-allocation decision of reading this subthread, he is encouraged to invoke the Seventh Guideline against this paragraph.)

The second example of a way a Seventh Guideline request might look says,

Er, you seem to be putting a lot of words in my mouth.

I think this applies? (A previous revision of this comment said “This applies straightforwardly”, but maybe you think the “my understanding”/”unless I miss him” disclaimers exclude the possibility of “putting words in someone’s mouth”?)

The third and final example of a way a Seventh Guideline request might look says,

I feel like I’m being asked to defend a position I haven’t taken. Can you point at what I said that made you think I think X?

“Asked to defend” doesn’t apply, but the question does. Can you point at what I said that made you think that I think that Alexei’s comment weighs the pettiness judgement very heavily in his analysis and that I don’t want Less Wrong to be the kind of place?

After being prompted by this thread and thinking for a minute, I was able to come up with a reason I should arguably disapprove of Alexei’s comment: that pettiness is not intellectually substantive (the section is correct or not separately from whether it’s a petty thing to point out) and letting a pettiness assessment flip the decision of whether to upvote makes karma scores less useful. I don’t feel that strongly about this and wouldn’t have come up with it without prompting because I’m not a karma-grubber: I think it’s, well, petty to complain about someone’s reasons for downvoting or witholding an upvote.

Zack_M_Davis Jun 2, 2025, 4:10 PM
8 points
5
in reply to: Steven Byrnes’s comment on: The Best Way to Align an LLM: Is Inner Alignment Now a Solved Problem?

Can you name any way to solve [chess but with rooks and bishops not being able to move more than four squares at a time] without RL (or something functionally equivalent to RL)?

This isn’t even hard. Just take a pre-2017 chess engine, and edit the rules code so that rooks and bishops can only move four spaces. You’re probably already done: the core minimax search still works, α–β pruning still works, quiescence still works, &c. To be fair, the heuristic evaluation function won’t be correct, but you could just … make bishops and rooks be respectively worth 2.5 and 3.5 points instead of the traditional 3 and 5? Even if my guess at those point values is wrong, that should still be easily superhuman with 2017 algorithms on 2017 hardware. (Stockfish didn’t incorporate neural networks until 2020.)

Zack_M_Davis Jun 1, 2025, 1:47 PM
5 points
−1
in reply to: Duncan Sabien (Inactive)’s comment on: Truth or Dare

my understanding of Zack’s preferred norms; unless I miss him entirely neither one of us wants LessWrong to be the kind of place where that sort of factor weighs very heavily in people’s analysis.

Um, I strong-upvoted and strong-agreement-voted Alexei’s comment.

Given our history of communication failures, I think your comments would be better if you try to avoid making claims about what I believe in the absence of direct textual evidence, in accordance with the Seventh Guideline.

But, crucially, that’s me saying I think your comments would be better comments if you did that. I’m not saying you shouldn’t try to extrapolate my views if you want to. You don’t owe me anything!

Zack_M_Davis Jun 1, 2025, 1:46 PM
12 points
0
in reply to: Duncan Sabien (Inactive)’s comment on: Truth or Dare

Zack’s [...] request

Clarification: I didn’t think of that as a “request.” I was saying that according to my standards, I would be embarrassed to publish criticism of someone that didn’t quote or link to their writings, and that it seemed to me to be in tension with your condemnations of strawmanning.

I don’t think of that as a request that you change it, because in general, I don’t think I have “jurisdiction” over other people’s writing. If someone says something I think is wrong, my response is to write my own comment or post explaining why I think it’s wrong (or perhaps mention it in person at Less Online), which they can respond or not-respond to as they see fit. You don’t owe me anything!

Zack_M_Davis May 29, 2025, 3:56 AM
1 point
4
on: Truth or Dare
The Rick and Morty analysis in Act III, Scene I is great. I guess the “To be fair, you have to have a very high IQ [...]” meme is for real!

Zack_M_Davis May 26, 2025, 2:07 PM
5 points
3
in reply to: Viliam’s comment on: AI #117: OpenAI Buys Device Maker IO
It’s good news for learning, not necessarily good news for jobs. If you care about creating “teaching” make-work jobs, but don’t care whether people know things, then it’s bad news.

Zack_M_Davis May 21, 2025, 1:53 AM
4 points
2
in reply to: Matthew Barnett’s comment on: AI Doomerism in 1879
Specifically, the idea is that AI going well for humans would require a detailed theory of how to encode human values in form suitable for machine optimization, and the relevance of deep learning is that Yudkowsky and Soares think that deep learning is on track to provide the superhuman optimization without the theory of values. You’re correct to note that this is a stance according to which “artificial life is by default bad, dangerous, or disvaluable,” but I think the way you contrast it with the claim that “biological life is by default good or preferable” is getting the nuances slightly wrong: independently-evolved biological aliens with superior intelligence would also be dangerous for broadly similar reasons.

Zack_M_Davis May 14, 2025, 7:50 PM
32 points
7
on: Eliezer and I wrote a book: If Anyone Builds It, Everyone Dies
I preordered my copy.

Something about the tone of this announcement feels very wrong, though. You cite Rob Bensinger and other MIRI staff being impressed. But obviously, those people are highly selected for already agreeing with you! How much did you engage with skeptical and informed prereaders? (I’m imagining people in the x-risk-reduction social network who are knowledgeable about AI, acknowledge the obvious bare-bones case for extinction risk, but aren’t sold on the literal stated-with-certainty headline claim, “If anyone builds it, everyone dies.”)

If you haven’t already done so, is there still time to solicit feedback from such people and revise the text? (Sorry if the question sounds condescending, but the tone of the announcement really worries me. It would be insane not to commission red team prereaders, but if you did, then the announcement should be talking about the red team’s reaction, not Rob’s!)

Zack_M_Davis May 9, 2025, 7:01 AM
9 points
2
in reply to: yams’s comment on: Eukryt Wrts Blg
Low-IQ voters can’t identify good policies or wise politicians; democracy favors political actors who can successfully propagandize and mobilize the largest number of people, which might not correspond to good governance. A political system with non-democratic elements that offers more formalized control to actors with greater competence or better incentives might be able to choose better policies.

I say “non-democratic elements” because it doesn’t have to be a strict binary between perfect democracy and perfect dictatorship. Consider, e.g., how the indirect election of U.S. Senators before the 17th Amendment was originally intended to make the Senate a more deliberative body by insulating it from the public.

(Maybe that’s all wrong, but you asked “what’s the model”, and this is an example model of why someone might be skeptical of democracy for pro-social structural reasons rather than just personally wanting their guy to be dictator.)

Zack_M_Davis May 4, 2025, 5:13 AM
22 points
6
in reply to: 1a3orn’s comment on: 1a3orn’s Shortform
From “But Hurting People Is Wrong”:

Look across the superclusters, and most entities either don’t do natural-number arithmetic at all, like stars and rocks; or they do it perfectly up to the limits of bounded cognition, like galaxy-spanning superintelligences. If there’s anything odd about humans, it’s the way that humans are only halfway finished being sucked into attractors like that.

Best wishes, Less Wrong Reference Desk

Zack_M_Davis Apr 19, 2025, 3:47 PM
6 points
0
in reply to: Sergii’s comment on: Sergii’s Shortform
(Previous commentary and discussion.)

Zack_M_Davis Apr 9, 2025, 10:15 PM
23 points
6
in reply to: johnswentworth’s comment on: Navigation by Moonlight
I’m again not sure how far this generalizes, but among the kind of men who read Less Wrong (which is a product of both neurotype and birth year), I think there’s a phenomenon where it’s not a matter of a man being cognitively unable to pick up on women’s cues, but of not being prepared to react in a functional way due to having internalized non-adaptive beliefs about the nature of romance and sexuality. (In a severe case, this manifests as the kind of neurosis described in Comment 171, but there are less severe cases.)

I remember one time from my youth where a woman was flirting with me in an egregiously over-the-top way that was impossible to not notice, but I just—pretended to ignore it? Not knowing what was allowed, it was easier to just do nothing. And that case was clearly not a good match, but that’s not the point—I somehow didn’t think through the obvious logic that if “yang doesn’t step up”, then relationships just don’t happen.

Zack_M_Davis Mar 24, 2025, 7:13 AM
9 points
12
in reply to: johnswentworth’s comment on: johnswentworth’s Shortform
Not sure how much this generalizes to everyone, but part of the story (for either the behavior or the pattern of responses to the question) might that some people are ideologically attached to believing in love: that women and men need each other as a terminal value, rather than just instrumentally using each other for resources or sex. For myself, without having any particular empirical evidence or logical counterargument to offer, the entire premise of the question just feels sad and gross. It’s like you’re telling me you don’t understand why people try to make ghosts happy. But I want ghosts to be happy.

Zack_M_Davis Feb 4, 2025, 7:03 AM
9 points
4
in reply to: GeneSmith’s comment on: Yudkowsky on The Trajectory podcast

No one with the money has offered to fund it yet. I’m not even sure they’re aware this is happening.

Um, this seems bad. I feel like I should do something, but I don’t personally have that kind of money to throw around. @habryka, is this the LTFF’s job??

Zack_M_Davis Jan 13, 2025, 11:40 PM
11 points
0
on: (The) Lightcone is nothing without its people: LW + Lighthaven’s big fundraiser
(I gave $2K.)

Zack_M_Davis Jan 13, 2025, 4:35 PM
6 points
1
in reply to: khafra’s comment on: Alignment Implications of LLM Successes: a Debate in One Act
Simplicia: But how do you know that? Obviously, an arbitrarily powerful expected utility maximizer would kill all humans unless it had a very special utility function. Obviously, there exist programs which behave like a webtext-next-token-predictor given webtext-like input but superintelligently kill all humans on out-of-distribution inputs. Obviously, an arbitrarily powerful expected utility maximizer would be good at predicting webtext. But it’s not at all clear that using gradient descent to approximate the webtext next-token-function gives you an arbitrarily powerful expected utility maximizer. Why would that happen? I’m not denying any of the vNM axioms; I’m saying I don’t think the vNM axioms imply that.

Zack_M_Davis Jan 13, 2025, 7:49 AM
8 points
−2
on: A Hill of Validity in Defense of Meaning
(Self-review.) I think this pt. 2 is the second most interesting entry in my Whole Dumb Story memoir sequence. (Pt. 1 deals with more niche psychology stuff than the philosophical malpractice covered here; pt. 3 is a more of a grab-bag of stuff that happened between April 2019 and January 2021; pt. 4 is the climax. Expect the denouement pt. 5 in mid-2025.)

I feel a lot more at peace having this out there. (If we can’t have justice, sanity, or language, at least I got to tell my story about trying to protect them.)

The 8 karma in 97 votes is kind of funny in how nakedly political it is. (I think it was higher before the post got some negative attention on Twitter.)

Given how much prereading and editing effort had already gone into this, it’s disappointing that I didn’t get the ending right the first time. (I ended up rewriting some of the paragraphs at the end after initial publication after it didn’t land in the comments section the way I wanted it to land.)

Subsection titles would have also been a better choice for such a long piece (which was rectified for the publication of pt.s 3 and 4); I may still yet add them.

Zack_M_Davis Jan 13, 2025, 7:05 AM
45 points
0
on: Alignment Implications of LLM Successes: a Debate in One Act
(Self-review.) I’m as proud of this post as I am disappointed that it was necessary. As I explained to my prereaders on 19 October 2023:

My intent is to raise the level of the discourse by presenting an engagement between the standard MIRI view and a view that’s relatively optimistic about prosaic alignment. The bet is that my simulated dialogue (with me writing both parts) can do a better job than the arguments being had by separate people in the wild; I think Simplicia understands things that e.g. Matthew Barnett doesn’t. (The karma system loved my dialogue comment on Barnett’s post; this draft is trying to scale that up.)

I’m annoyed at the discourse situation where MIRI thinks we’re dead for the same fundamental reasons as in 2016, but meanwhile, there are a lot of people who are looking at GPT-4, and thinking, “Hey, this thing seems pretty smart and general and good at Doing What I Mean, in contrast to how 2016-era MIRI said that we didn’t know how to get an agent to fill a cauldron; maybe alignment is easy??”—to which MIRI’s response has been (my uncharitable paraphrase), “You people are idiots who didn’t understand the core arguments; the cauldron thing was a toy illustration of a deep math thing; we never said Midjourney can’t exist”.

And just, I agree that Midjourney doesn’t refute the deep math thing and the people who don’t realize that are idiots, but I think the idiots deserve a better response!—particularly insofar as we’re worried about transformative AI looking a lot like the systems we see now, rather than taking a “LLMs are nothing like AGI” stance.

Simplicia isn’t supposed to pass the ITT of anyone in particular, but if the other character [...] doesn’t match the MIRI party line, that’s definitely a serious flaw that needs to be fixed!

I think the dialogue format works particularly well in cases like this where the author or the audience is supposed to find both viewpoints broadly credible, rather than an author avatar beating up on a strawman. (I did have some fun with Doomimir’s characterization, but that shouldn’t affect the arguments.)

This is a complicated topic. To the extent that I was having my own doubts about the “orthodox” pessimist story in the GPT-4 era, it was liberating to be able to explore those doubts in public by putting them in the mouth of a character with the designated idiot character name without staking my reputation on Simplicia’s counterarguments necessarily being correct.

Giving both characters perjorative names makes it fair. In an earlier draft, Doomimir was “Doomer”, but I was already using the “Optimistovna” and “Doomovitch” patronymics (I had been consuming fiction about the Soviet Union recently) and decided it should sound more Slavic. (Plus, “-mir” (мир) can mean “world”.)

Zack_M_Davis Jan 13, 2025, 12:51 AM
30 points
11
on: Shutting Down the Lightcone Offices
Retrospectives are great, but I’m very confused at the juxtaposition of the Lightcone Offices being maybe net-harmful in early 2023 and Lighthaven being a priority in early 2025. Isn’t the latter basically just a higher-production-value version of the former? What changed? (Or after taking the needed “space to reconsider our relationship to this whole ecosystem”, did you decide that the ecosystem is OK after all?)