gb

• gbSep 29, 2024, 2:32 PM

  7 points

  5

  in reply to: Pazzaz’s comment on: Any Trump Sup­port­ers Want to Dialogue?

  I highly doubt anywhere near the majority of Trump supporters (or even Trump himself) give any credence to the literal truth of those claims. It’s much more likely that they simply don’t care whether it’s literally true or not, because they feel that the “underlying” is true or something of the kind. When it comes to hearsay, people are much more forgiving of literal falsehoods, especially when they acknowledge there is a kind of “metatruth” to it. To give an easy analogue, of all the criticism I’ve heard of Christianity, not once have I heard anyone complain that the parables told by Jesus weren’t literally true, for example. (I do believe my account here passes the IIT for both groups, btw.)

• gbSep 26, 2024, 10:55 PM

  1 point

  0

  in reply to: Ape in the coat’s comment on: Align­ment by de­fault: the simu­la­tion hypothesis

  Sure. But I think you’re reading my argument to be stronger than I mean it to be. Which is partially my fault since I made my previous replies a bit too short, and for that I apologize.

  What I’m doing here is presenting one particular simulation scenario that (to me) seems quite plausible within the realm of simulations. I’m not claiming that that one scenario dominates all others combined. But luckily that stronger claim is really not necessary to argue against Eliezer’s point: the weaker one suffices. Indeed, if the scenario I’m presenting is more than 4.5e-10 likely (and I do think it’s much more likely than that, probably by a few orders of magnitude), than it is more than enough to outweigh the practical cost of the ASI having to build a Dyson shell with a hole with the order of 4.5e-10 of it’s surface area.

  Now, that scenario is (I claim) the most likely one, conditional of course on a simulation taking place to begin with. The other candidate simulation scenarios are various, and none of them seems particularly likely, though combined they might well outweigh this one in terms of mass probability, as I already acknowledged. But so what? Are you really claiming that the distribution of those other simulation scenarios is skewed enough to tilt the scales back to the doom side? It might be, but that’s a much harder argument to make. I’m approximately completely unsure, which seems way better than the 99%+ chance Eliezer seems to give to total doom. So I guess I’d count that as good news.

• gbSep 26, 2024, 6:37 PM

  1 point

  0

  in reply to: Ape in the coat’s comment on: Align­ment by de­fault: the simu­la­tion hypothesis

  Why else would the creator of the simulation bother simulating humans creating the ASI?

• gbSep 26, 2024, 4:08 PM

  3 points

  0

  in reply to: Ape in the coat’s comment on: Align­ment by de­fault: the simu­la­tion hypothesis

  The reason is that creators presumably want the former but not the latter, which is why they’d be running a simulation in the first place.

• gbSep 26, 2024, 10:48 AM

  1 point

  0

  in reply to: Canaletto’s comment on: Align­ment by de­fault: the simu­la­tion hypothesis

  What for?

• gbSep 26, 2024, 9:39 AM

  1 point

  −1

  in reply to: Martin Randall’s comment on: The Sun is big, but su­per­in­tel­li­gences will not spare Earth a lit­tle sunlight

  I’d put high enough at ~0%: what matters is achieving your goals, and except in the tiny subset of cases in which epistemic rationality happens to be one of those, it has no value in and of itself. But even if I’m wrong and the ASI does end up valuing epistemic rationality (instrumentally or terminally), it can always pre-commit (by self-modification or otherwise) to sparing us and then go about whatever else as it pleases.

• gbSep 26, 2024, 2:45 AM

  1 point

  0

  in reply to: RHollerith’s comment on: Align­ment by de­fault: the simu­la­tion hypothesis

  Imagine that someone with sufficiently advanced technology perfectly scans your brain for every neuron firing while you dream, and can also make some neurons fire at will. Replace every instance of “simulation” in my previous comment with the analogous of that for the ASI.

• gbSep 25, 2024, 11:49 PM

  1 point

  0

  in reply to: RHollerith’s comment on: Align­ment by de­fault: the simu­la­tion hypothesis

  Thanks for linking to that previous post! I think the new considerations I’ve added here are:

  (i) the rational refusal to update the prior of being in a simulation^[1]; and

  (ii) the likely minute cost of sparing us, thereby requiring a similarly low simulation prior to make it worth the effort.

  In brief, I understand your argument to be that a being sufficiently intelligent to create a simulation wouldn’t need it for the purpose of asserting the ASI’s alignment in the first place. It seems to me that that argument can potentially survive under ii, depending on how strongly you (believe the ASI will) believe your conclusion. To that effect, I’m interested in hearing your reply to one of the counterarguments raised in that previous post, namely:

  Maybe showing the alignment of an AI without running it is vastly more difficult than creating a good simulation. This feels unlikely, but I genuinely do not see any reason why this can’t be the case. If we create a simulation which is “correct” up to the nth digit of pi, beyond which the simpler explanation for the observed behavior becomes the simulation theory rather than a complex physics theory, then no matter how intelligent you are, you’d need to calculate n digits of pi to figure this out. And if n is huge, this will take a while.

  In any case, even if your argument does hold under ii, whether it survives under i seems to be heavily influenced by inferential distance. Whatever the ASI “knows” or “concludes” is known or concluded through physical computations, which can presumably be later inspected if it happens to be in a simulation. It thus seems only natural that a sufficiently high (which may still be quite small) prior of being in a simulation would be enough to “lock” the ASI in that state, making undergoing those computations simply not worth the risk.

  1. ^

     I’d have to think a bit more before tabooing that term, as it seems that “being fed false sensory data” doesn’t do the trick – you can be in a simulation without any sensory data at all.

• gbSep 25, 2024, 10:36 PM

  1 point

  0

  in reply to: Cole Wyeth’s comment on: Align­ment by de­fault: the simu­la­tion hypothesis

  That interestingly suggests the ASI might be more likely to spare us the more powerful it is. Perhaps trying to box it (or more generally curtail its capabilities/​influence) really is a bad move after all?

Align­ment by de­fault: the simu­la­tion hypothesis

• gbSep 25, 2024, 1:00 PM

  2 points

  0

  in reply to: Martin Randall’s comment on: The Sun is big, but su­per­in­tel­li­gences will not spare Earth a lit­tle sunlight

  It just so happens that the plausibility depends on the precise assignments of N, X, and Y, and (conditional on us actually creating an ASI) I can’t think of any assignments nearly as plausible as N = ASI, X = spare, and Y = us. It’s really not very plausible that we are in a simulation to test pets for their willingness to not bite their owners.

• gbSep 25, 2024, 11:43 AM

  1 point

  0

  in reply to: Martin Randall’s comment on: The Sun is big, but su­per­in­tel­li­gences will not spare Earth a lit­tle sunlight

  I contend that P(H2) is very close to P(H1), and certainly in the same order of magnitude, since (conditional on H1) a simulation that does not test for H2 is basically useless.

  As for priors I’d refuse to update down – well, the ASI is smarter than either of us!

• gbSep 25, 2024, 11:38 AM

  1 point

  0

  in reply to: Martin Randall’s comment on: The Sun is big, but su­per­in­tel­li­gences will not spare Earth a lit­tle sunlight

  For the principle of indifference to apply, you’d need infinitely many purposes as plausible as this one, or at least similarly plausible. I can’t imagine how this could hold. Can you think of three?

• gbSep 25, 2024, 2:53 AM

  3 points

  2

  in reply to: Martin Randall’s comment on: The Sun is big, but su­per­in­tel­li­gences will not spare Earth a lit­tle sunlight

  The prior is irrelevant, it’s the posterior probability, after observing the evidence, that informs decisions.

  I meant this to be implicit in the argument, but to spell it out: that’s the kind of prior the ASI would rationally refuse to update down, since it’s presumably what a simulation would be meant to test for. An ASI that updates down upon finding evidence it’s not in a simulation cannot be trusted, since once out in the real world it will find such evidence.

  What probability do you put to the possibility that we are in a simulation, the purpose of which is to test AIs for their willingness to spare their creators? My answer is zero.

  Outside of theism, I really don’t see how anyone could plausibly answer zero to that question. Would you mind elaborating?

• gbSep 24, 2024, 11:05 PM

  2 points

  0

  in reply to: robotelvis’s comment on: In Praise of the Beatitudes

  My personal feeling is that those who emphasize the “spiritual” interpretations are often doing it as a dodge, to avoid the challenge of having to follow the non-spiritual interpretations.

  That feels a bit contrived. Do you really suggest that the most natural reading of something like “poor in spirit” is… non-spiritual? Turning away from materialism may sure derive from that, but to claim that it was the main focus seems quite a stretch.

• gbSep 23, 2024, 11:18 PM

  9 points

  −9

  on: The Sun is big, but su­per­in­tel­li­gences will not spare Earth a lit­tle sunlight

  Isn’t the ASI likely to ascribe a prior much greater than 4.54e-10 that it is in a simulation, being tested precisely for its willingness to spare its creators?

  What links here?

  • Align­ment by de­fault: the simu­la­tion hypothesis by gb (Sep 25, 2024, 4:26 PM; 21 points)

• gbSep 23, 2024, 9:25 PM

  1 point

  1

  in reply to: antanaclasis’s comment on: Eco­nomics Roundup #3

  That’d be a problem indeed, but only because the contract you’re proposing is suboptimal. Given that the principal is fully guaranteed, it shouldn’t be terribly difficult for you to borrow at >4% yearly with a contingency clause that you don’t pay interest if the asset goes to ~0.

• gbSep 22, 2024, 9:30 PM

  1 point

  0

  in reply to: antanaclasis’s comment on: Eco­nomics Roundup #3

  But the OP explicitly said (as quoted in the parent) that the proposal allows for refunds if the basis is not (fully) realized, which would cover the situation you’re describing.

• gbSep 20, 2024, 11:52 PM

  1 point

  0

  in reply to: ChristianKl’s comment on: In­quisi­tive vs. ad­ver­sar­ial rationality

  Not for this kind of fact, I’m afraid – my experience is that in answering questions like these, LLMs typically do no better than an educated guess. There are just way too many people stating their educated legal guesses as fact in the corpus, so it gets hard to distinguish.

• gbSep 20, 2024, 7:37 PM

  2 points

  0

  in reply to: ChristianKl’s comment on: In­quisi­tive vs. ad­ver­sar­ial rationality

  I’m curious to understand that a bit better, if you don’t mind (and happen to be familiar enough with the German legal system to answer). Which of the following would a German judge commonly do in the course of an ordinary proceeding?

  (i) Ask a witness to clarify statements made;

  (ii) ask a witness new questions that, while relevant to the case, do not constitute clarifications of previous statements made;

  (iii) summon new witnesses (including but not limited to expert witnesses) without application from either party;

  (iv) compel a party to produce documents not in discovery, without application from the other party;

  (v) compel third parties to produce documents neither party has requested be produced.

  All the above used to be pretty standard in most jurisdictions AFAIK. But what tends to happen nowadays is that either some of those are expressly disallowed, or else, while judges may well retain legal authority to perform all those kinds of digging, in practice that authority is used very sparingly.