ImmortalityOrDeathByAGI

• ImmortalityOrDeathByAGINov 3, 2023, 3:30 PM

  7 points

  1

  in reply to: ImmortalityOrDeathByAGI’s comment on: Say­ing the quiet part out loud: trad­ing off x-risk for per­sonal immortality

  For most people in their 20s or 30s, it is quite unlikely (around 10%) that they die before AGI. And if you basically place any value on the lives of people other than yourself, then the positive externalities of working on AI safety probably strongly outweigh anything else you could be doing.

  Acceleration probably only makes sense for people who are (1) extremely selfish (value their life more than everything else combined) and (2) likely to die before AGI unless it’s accelerated.

• ImmortalityOrDeathByAGINov 3, 2023, 3:23 PM

  14 points

  1

  on: Say­ing the quiet part out loud: trad­ing off x-risk for per­sonal immortality

  I wrote about this in Appendix A of this post.
  ------
  One might look at the rough ⁵⁰⁄₅₀ chance at immortality given surviving AGI and think “Wow, I should really speed up AGI so I can make it in time!”. But the action space is more something like:

  1. Work on AI safety (transfers probability mass from “die from AGI” to “survive AGI”)

     1. The amount of probability transferred is probably at least a few microdooms per person.

  2. Live healthy and don’t do dangerous things (transfers probability mass from “die before AGI” to “survive until AGI”)

     1. Intuitively, I’m guessing one can transfer around 1 percentage point of probability by doing this.

  3. Do nothing (leaves probability distribution the same)

  4. Preserve your brain if you die before AGI (kind of transfers probability mass from “die before AGI” to “survive until AGI”)

     1. This is a weird edge case in the model and it conditions on various beliefs about preservation technology and whether being “revived” is possible

  5. Delay AGI (transfers probability from “die from AGI” to “survive AGI” and from “survive until AGI” to “die before AGI”)

  6. Accelerate AGI (transfer probability mass from “survive AGI” to “die from AGI” and from “die before AGI” to “survive until AGI”)

  I think working on AI safety and healthy living seem like a much better choice than accelerating AI. I’d guess this is true even for a vast majority of purely selfish people.

  For altruistic people, working on AI safety clearly trumps any other action in this space as it has huge positive externalities. This is true for people who only care about current human lives (as one microdoom $\approx$ 8,000 current human lives saved), and it’s especially true for people who place value on future lives as well (as one microdoom = one millionth of the value of the entire long term future).

  This is a very simplified view of what it means to accelerate or delay AGI, which ignores that there are different ways to shift AGI timelines that transfer probability mass differently. In this model I assume that as timelines get longer, our probability of surviving AGI increases monotonically, but there are various considerations that make this assumption shaky and not generally true for every possible way to shift timelines (such as overhangs, different actors being able to catch up to top labs, etc.)

• ImmortalityOrDeathByAGISep 22, 2023, 9:04 PM

  3 points

  0

  in reply to: avturchin’s comment on: Im­mor­tal­ity or death by AGI

  As I said in another comment:

  I totally buy that we’ll see some life expectancy gains before AGI, especially if AGI is more than 10 years away. I mostly just didn’t want to make my model more complex, and if we did see life expectancy gains, the main effect this would have is to take probability away from “die before AGI”.

• ImmortalityOrDeathByAGISep 22, 2023, 9:00 PM

  1 point

  0

  in reply to: Going Durden’s comment on: Im­mor­tal­ity or death by AGI

  I totally buy that we’ll see some life expectancy gains before AGI, especially if AGI is more than 10 years away. I mostly just didn’t want to make my model more complex, and if we did see life expectancy gains, the main effect this would have is to take probability away from “die before AGI”.

• ImmortalityOrDeathByAGISep 22, 2023, 8:53 PM

  4 points

  1

  in reply to: pathos_bot’s comment on: Im­mor­tal­ity or death by AGI

  For a perfectly selfish actor, I think avoiding death pre-AGI makes sense (as long as the expected value of a post-AGI life is positive, which it might not be if one has a lot of probability mass on s-risks). Like, every micromort of risk you induce (for example, by skiing for one day), would decrease the probability you live in a post-AGI world by roughly ¹⁄_1,000,000. So, one can ask oneself, “would I trade this (micromort-inducing) experience for one millionth of my post-AGI life?”, and I the answer a reasonable person would give in most cases would be no. The biggest crux is just how much one values one millionth of their post-AGI life, which comes down to cruxes like its length (could be billions of years!), and its value per unit time (which could be very positive or very negative).

  Like, if I expect to live for a million years in a post-AGI world where I expect life to be much better than the life I’m leading right now, then skiing for a day would take away roughly one year away from my post-AGI life in expectation. I definitely don’t value skiing that much.

  This gets a bit complicated for people who are not perfectly selfish, as there are cases where one can trade micromorts for happiness, happiness for productivity, and productivity for impact on other people. So for instance, someone who works on AI safety and really likes skiing might find it net-positive to incur the micromorts because the happiness gained from skiing makes them better at AI safety, and them being better at AI safety has huge positive externalities that they’re willing to trade their lifespan for. In effect, they would be decreasing the probability that they themselves live to AGI, while increasing the probability that they and other people (of which there are many) survive AGI when it happens.

Im­mor­tal­ity or death by AGI