MikkW

• MikkWMay 15, 2025, 8:53 PM

  6 points

  −1

  in reply to: MikkW’s comment on: Eliezer and I wrote a book: If Any­one Builds It, Every­one Dies

  I see that the numbers indicate people disagree with this post. Since there are several clauses, it’s hard to know which specifically (or all of them) are being disagreed with.

  The second paragraph (beginning “Contrary to what you wrote...”) is a list of factual statements, which as far as I can tell are all correct.

  The third paragraph (“Most importantly, the title is plenty big...”) is more subjective, but I’m currently not imagining that anyone is disagreeing with that paragraph (that is, that anyone thinks “actually, the title is too small”).

  The fourth paragraph (“In this case, part of the point...”) is more speculative, and I could easily imagine someone reading it and thinking “that’s not the point of publishing /​ writing a book”. There’s certainly a reason I put a “presumably” in there. I do still feel that there’s something to what I’m saying in that paragraph. My surprise would be of a limited extent if Soares and Yudkowsky said “that was not a consideration in our decision to do this”—but I would be somewhat surprised.

  I can see someone disagreeing with the first paragraph (“Given that the book...”), but my current state of mind is that such people would be simply wrong. The book is not being self-published, but is being published by Little, Brown and Company. Some excerpts from Wikipedia’s article on Little, Brown and Company:

  “The company was the most extensive law publisher in the United States, and also the largest importer of standard English law and miscellaneous works, introducing American buyers to the Encyclopædia Britannica, the dictionaries of William Smith, and many other standard works. In the early years [starting in 1837] Little and Brown published the Works of Daniel Webster, [...], [and] Letters of John Adams. [...] Little, Brown and Company was the American publisher for Edward Gibbon’s The Decline and Fall of the Roman Empire.”

  The firm was the original publisher of United States Statutes at Large beginning in 1845, under authority granted by a joint resolution of Congress.

  and

  During this time [about 1908 to 1985] the joint Atlantic Monthly Press/​Little Brown imprint published All Quiet on the Western Front, Herge’s The Adventures of Tintin, James Truslow Adams’s The Adams Family, [...] [and] J. D. Salinger’s The Catcher in the Rye.

  The point being, the company that is publishing Soares and Yudkowsky’s book, is an established company that has sold important and/​or bestselling works for two centuries. The people there know what they are doing, and that includes the people who design covers, as well as the bosses of the people who design the covers.

• MikkWMay 15, 2025, 2:45 PM

  5 points

  −28

  in reply to: MondSemmel’s comment on: Eliezer and I wrote a book: If Any­one Builds It, Every­one Dies

  Given that the book is being published by a major publisher, it can safely be assumed that the cover design was made by a professional cover designer, who knew what they were doing.

  Contrary to what you wrote, the title has a bigger font size than both the subtitle and the authors’ names (this is true of both the American and UK covers; I am primarily talking about the American cover, which I presume is the one you are referencing). Even if the author names were the same size as the title, it is immediately obvious which one is the title, and which one isn’t. Putting the subtitle in a dark grey, which is much closer to the background color (black) than the color of the title (white) is, also does a lot to move emphasis towards the title of the book (away from the subtitle)
  
  Most importantly, the title is plenty big. If it was small, then I would feel there is something to what you are saying; but the title is quite large and readable from a distance, and clearly delineated from the rest of the text on the cover.
  
  In this case, part of the point of publishing a book (including writing it in the first place), is presumably to promote the identity of the authors, to make them a known name /​ schelling point for discussion about AI safety. That would indicate making the names quite prominent on the cover.

• MikkWMay 14, 2025, 9:32 PM

  6 points

  2

  in reply to: tanagrabeast’s comment on: Eliezer and I wrote a book: If Any­one Builds It, Every­one Dies

  I would assume e-book orders will also play a role in encouraging the publisher to print more physical copies, because it indicates that more people are interested in reading the book.

• MikkWMay 14, 2025, 9:30 PM

  9 points

  4

  in reply to: peterbarnett’s comment on: Eliezer and I wrote a book: If Any­one Builds It, Every­one Dies

  Tim Urban has written about AI X-risk before, in a way that indicates that he’s spent a good bit of time thinking about the problem. But, the point of the book seems to be to speak to people who don’t have a deep knowledge of AI risk.

• MikkWJan 11, 2025, 12:43 PM

  2 points

  0

  in reply to: gwern’s comment on: Aris­toc­racy and Hostage Capital

  I agree that this description fits the paper.

• MikkWJan 10, 2025, 11:20 PM

  4 points

  0

  on: Aris­toc­racy and Hostage Capital

  If you read this and, like I did, felt unfulfilled after reading it, it’s worth noting that this paper (which was linked in the OP, but which I [and perhaps the unfulfilled reader] overlooked) goes into more detail:

  https://​​projects.panickssery.com/​​docs/​​allen-2009-a_theory_of_the_pre-modern_british_aristocracy.pdf

• MikkWOct 30, 2024, 2:58 PM

  2 points

  0

  on: The hos­tile telepaths problem

  This post does a good job of laying out compelling arguments for thoughts adjacent to areas I’ve previously already enjoyed thinking about.

  For the record, this sentence popped into my head while reading this: “Wait, but what if I’m Omega-V, and [Valentine] is a two boxer?”

  (Edit: the context for this thought is my previous thoughts having read other posts by Valentine, which I find both quite elucidating, but also somehow have left me feeling a bit creeped out; that being said, my opinion about this post itself is strongly positive)

• MikkWJun 19, 2024, 4:46 PM

  1 point

  −9

  on: Sur­viv­ing Seveneves

  If you dig deep enough, temperatures should be much cooler than on /​ near the surface of the earth. (Unless the heat gets very intense. I don’t know enough to rule that out). How much digging that deep (as opposed to the depths we usually did to) would cost, though

• MikkWJun 6, 2024, 10:08 AM

  2 points

  0

  in reply to: MikkW’s comment on: MikkW’s Shortform

  (The mentioned ACX post is https://​​www.astralcodexten.com/​​p/​​a-theoretical-case-against-education )

• MikkWJun 6, 2024, 10:08 AM

  2 points

  −4

  on: MikkW’s Shortform

  A recent Astral Codex Ten post contained this bit:

  Fewer than 50% (ie worse than chance) can correctly answer a true-false question about whether electrons are bigger than atoms.

  The linked source seems to indicate that the survey’s expected answer to the question “electrons are smaller than atoms” is “true”. However, I think this is likely based on a faulty understanding of reality, and in any case the question has a trickier nature than the survey or Scott Alexander give it credit for.

  There’s a common misconception that electrons (as well as e.g. protons and neutrons) are point particles, that is to say, that they can be said to exist at some precise location, just like a dot on a piece of graph paper.

  Even when people talk about the uncertainty principle, they often lean into this misconception by suggesting that the wave function indicates “the probability that the (point-like) particle is found at a given location”.

  However, an electron is not a point, but rather a wavefunction which has a wide extent in space. If you were to examine the electron at the tip of my pinky finger, there is in fact a non-zero (but very, very small) part of that electron that can be found at the base of the flag which Neil Armstrong planted on the moon (n.b. I’m not looking up whether it was actually Armstrong who did that), or even all the way out in the core of Alpha Centauri.

  We could still try to talk about the size of an electron (and the size of an atom, which is a similar question) by considering the volume that contains 99% of the electron (and likewise a volume that contains 99% of a proton or neutron).

  Considering this volume, the largest part of any given atom would be an electron, with the nuclear particles occupying a much larger volume (something something strong force). In this sense, the size of the atom is in fact coextensive with the size of its “largest” electron, and that electron is by no means smaller than the atom. There are of course in many atoms multiple electrons, and some of these may be “smaller” than the largest electron. However, I do not think the survey had this in mind as the justification for the answer it considered “correct” for the question.

  I think the appropriate next step for Scott Alexander is to retract the relevant sentence from his post.

• MikkWFeb 12, 2024, 11:33 PM

  2 points

  0

  in reply to: Rafael Harth’s comment on: An In­vi­ta­tion to Refrain from Down­vot­ing Posts into Net-Nega­tive Karma

  “authors will get hurt by people not appreciating their work” is something we just have to accept, even if it’s very harsh

  I don’t really agree with this. Sure, some people are going to write stuff that’s not very good, but that doesn’t mean that we have to go overboard on negative feedback, or be stingy with positive feedback.

  Humans are animals which learn by reinforcement learning, and the lesson they learn when punished is often “stay away from the thing /​ person /​ group that gave the punishment”, much more strongly than “don’t do the thing that made that person /​ thing /​ group punish me”.

  Wheras when they are rewarded, the lesson is “seek out the circumstances /​ context that let me be rewarded (and also do the thing that will make it reward me)”. Nobody is born writing amazingly, they have to learn it over time, and it comes more naturally to some, less to others.

  I don’t want bad writers (who are otherwise intelligent and intellectually engaged, which describes almost everybody who posts on LW) to learn the lesson “stay away from LW”. I want them to receive encouragement (mostly in forms other than karma, e.g. encouraging comments, or inclusion in the community, etc.), leading them to be more motivated to figure out the norms of LW and the art of writing, and try again, with new learning and experience behind them.

  I think the threshold of 0 is largely arbitrary

  It’s not all that arbitrary. Besides the fact that it’s one of the simplest numbers, which makes for an easy to remember /​ communicate heuristic (a great reason that isn’t arbitrary), I actually think it’s quite defensible as a threshold. If I write a post that has a +6 starting karma, and I see it drop down to 1 or 2 (or, yeah, −1), my thought is “that kinda sucked, but whatever, I’ll learn from my mistake and do better next time”.

  But if I see it drop down to, say, −5 or −6, my thought starts to become “why am I even posting on this stupid website that’s so full of anti-social jerks?”. And then I have to talk myself down from deleting my account and removing LW and the associated community from my life.

  (Not that I think LW is actually so full of jerks. There’s a lot of lovable people here who talk about interesting things, and I believe in LW’s raison d’etre, which is why I keep forcing myself to come back)

An In­vi­ta­tion to Refrain from Down­vot­ing Posts into Net-Nega­tive Karma

• MikkWJan 26, 2024, 7:59 PM

  2 points

  −2

  on: Values Darwinism

  I would like to make a meta-comment, not directly related to this post.

  When I came upon this post, it had a negative karma score. I don’t think it’s good form to have posts receiving negative net karma (except in extreme cases), so I upvoted to provide this with a positive net karma.

  It is unpleasant for an author when they receive a negative karma score on a post which they spent time and effort to make (even when that effort was relatively small), much more so than receiving no karma beyond the starting score. This makes the author less likely to post again in the future, which prevents communication of ideas, and keeps the author from getting better at writing. In particular this creates a risk of LessWrong becoming more like a bubble chamber (which I don’t think is desirable), and makes the community less likely to hear valuable ideas that go against the grain of the local culture.

  A writer who is encouraged to write more will become more clear in their communication, as well as in their thoughts. And they will also get more used to the particular expectations of the culture of LessWrong- norms that have good reason to exist, but which also go against some people’s intuitions or what has worked well for them in other, more “normie” contexts.

  Karma serves as a valuable signal to authors about the extent to which they are doing a good job of writing clearly about interesting topics in a way that provides value to members of the community, but the range of positive integers provides enough signal. There isn’t much lost in excluding the negative range (except in extreme cases).

  Let’s be nice to people who are still figuring writing out, I encourage you to refrain from downvoting them into negative karma.

  What links here?

  • An In­vi­ta­tion to Refrain from Down­vot­ing Posts into Net-Nega­tive Karma by MikkW (Jan 26, 2024, 8:13 PM; 2 points)

• MikkWJan 3, 2024, 10:12 PM

  2 points

  0

  in reply to: jefftk’s comment on: Trad­ing off Lives

  That statement of fact is indeed true. Would you mind saying more about your thoughts regarding it? There seems to be an unstated implication that this is bad. There is a part of me that agrees with that implication, but there are also parts of me that want to say “so what? that’s irrelevant”. (I feel ⌞explaining what the second set of shards is pointing to, would take more time and energy to write up than I am prepared to take right now⌝)

• MikkW 3 Jan 2024 15:31 UTC

  2 points

  −1

  on: Trad­ing off Lives

  On the other side, there’s the cost of ~10min of boredom, for every passenger, on every flight. Instead of playing games, watching movies, or reading, people would mostly be talking, looking out the window, or staring off into space.

  Tangent: I’m not completely sure that this is actually a cost and not an unintended benefit

• MikkW 21 Dec 2023 13:13 UTC

  5 points

  1

  in reply to: Daniel Kokotajlo’s comment on: Daniel Koko­ta­jlo’s Shortform

  Sharing my impression of the comic:

  Insofar as it supports sides, I’d say the first part of the meme is criticism of Eliezer

  The comic does not parse (to my eyes and probably most people’s) as the author intending to criticize Eliezer at any point

  Insofar as it supports sides, I’d say [...] the last part is criticism of those who reject His message

  Only in the most strawman way. It basically feels equivalent to me to “They disagree with the guy I like, therefore they’re dumb /​ unsympathetic”. There’s basically no meat on the bones of the criticism

• MikkW 21 Dec 2023 13:07 UTC

  4 points

  2

  in reply to: leogao’s comment on: leogao’s Shortform

  This subjectively seems to me to be the case.

• MikkW 25 Nov 2023 9:51 UTC

  2 points

  2

  in reply to: frontier64’s comment on: OpenAI: The Bat­tle of the Board

  The board’s statement doesn’t mention them having made such a request to Altman which was denied, that’s a strong signal against things having played out that way.

• MikkW 20 Nov 2023 13:46 UTC

  5 points

  1

  on: For Civ­i­liza­tion and Against Niceness

  In the case of the lawyers, this is actually not an example of non-niceness being good for society. The defense attorney who defends a guilty party, their job is not to be a jerk to the prosecutor or to the judge. It is to, as you say, provide the judge with information (including counter-arguments to the other side’s arguments). While his job involves working in an opposite direction from his counterpart, it does not involve being non-nice to his counterpart (and it is indeed most pro-society if the two sides treat eachother well /​ nicely outside of their equal-and-opposite professional duties), and it does not involve being non-nice to the judge, whose job the attorney (as you point is) is actually assisting with. Again, society expects maximum niceness from both attorneys towards the judge outside of ⌞their professional duty to imperfectly represent the truth⌝.

  Society expects niceness to be provided from each of these parties to each of the others: {the judge, the defense attorney, the prosecution attorney}

• MikkW 20 Nov 2023 11:19 UTC

  18 points

  20

  in reply to: lc’s comment on: Sam Alt­man, Greg Brock­man and oth­ers from OpenAI join Microsoft

  This is important news. I personally desire to be kept updated on this, and LW is a convenient (and appropriate) place to get this information. And I expect other users feel similarly.

  What’s different between this and e.g. the developments with Nonlinear, is that the developments here will have a big impact on how the AI field (and by one layer of indirection, the fate of the world) develops.