Thanks for this interesting and well-developed perspective. However, I disagree specifically with the claim “the existential risk caused by unaligned AI would cause high real interest rates”.
The idea seems to be that, anticipating doomsday, people will borrow money to spend on lavish consumption under the assumption that they won’t need to pay it back. But:
This is a bad strategy (in a game-theoretic and evolutionary sense).
I am skeptical that people actually act this way.
Studying a simple example may help to clarify.
Strategy: playing to your outs
There is a concept in card games of “playing to your outs”. The idea is, if you’re in a losing position, then the way to maximize your winning probability is to assume that future random events still provide a possibility of winning. A common example of this is to observe that you lose unless your next draw is a particular card, and as a result you play under the assumption that you will draw the card you need.
How the concept of playing to your outs applies to existential risk: if in fact extinction occurs, then it didn’t matter what you did. So you ought to plan for the scenario where extinction does not occur.
This same idea ought to apply to evolution for similar reasons that it applies to games. That said, obviously we’re not recalculating evolutionarily optimal strategies as we live our lives, so our actions may not be consistent with a good evolutionary strategy. Still, it makes me skeptical.
Do people actually act this way?
Section VI of the post discusses the empirical question of how people act if they expect doomsday, but I didn’t find it very persuasive. That said, I also haven’t done the work of finding contradictory evidence. (I’d propose looking at cases of religious movements with dated doomsday predictions.)
A lot of the cited evidence in section VI is about education, but I believe that’s a red herring. It makes sense that somebody would value education less if they expect to die before they can benefit from it. But that doesn’t necessarily mean they’re putting the resources towards lavish consumption instead: they might spend it on alternative investments that can outlive them (and therefore benefit their family). So it can’t tell us much about what doomsday does to consumption in particular.
Example
I had originally hoped to settle the matter through an example, but that doesn’t quite work. Still, it’s instructive.
Consider two people, Alice and Bob. They agree that there’s a 50% probability that tomorrow the Earth will explode, destroying humanity. They contemplate the following deal:
Today (day 0), Bob gives Alice $1 million.
The day after tomorrow (day 2), Alice gives Bob her $1.5 million vacation home.
One way to look at this:
Alice gains $1 million for sure.
With 50% probability, Alice doesn’t need to give Bob the home (because humanity is extinct), so her expected loss is only 50% * $1.5 million = $750k.
Therefore, Alice’s net expected profit is $250k.
From this perspective, it’s a good deal for Alice. (This seems to be the perspective taken by the OP.)
But here’s a different way to look at it:
In the scenario where the world ends, Alice, her family, and her friends are all dead, regardless of whether she made the deal, so she nets nothing.
In the scenario where the world doesn’t end, Alice has lost net $500k.
Therefore, Alice has a net expected loss of $250k.
From this perspective, it’s a bad deal for Alice. (This is more in line with my intuition.)
The discrepancy presumably arises from the possibility of Alice consuming the $1 million today, for example by throwing a mega-party.
But that just feels wrong, doesn’t it? Is partying hard actually worth so much to Alice that she’s willing to make herself worse off in the substantial (50%!) chance that the world doesn’t end?
Here’s an attempt to formalize the “is partying hard worth so much” aspect of your example:
It’s common (with some empirical support) to approximate utility as proportional to log(consumption). Suppose Alice has $5M of savings and expected-future-income that she intends to consume at a rate of $100k/year over the next 50 years, and that her zero utility point is at $100/year of consumption (since it’s hard to survive at all on less than that). Then she’s getting log(100000/100) = 3 units of utility per year, or 150 over the 50 years.
Now she finds out that there’s a 50% chance that the world will be destroyed in 5 years. If she maintains her old spending patterns her expected utility is .5*log(1000)*50 + .5*log(1000)*5 = 82.5. Alternately, if interest rates were 0%, she might instead change her plan to spend $550k/year over the next 5 years and then $50k/year subsequently (if she survives). Then her expected utility is log(5500)*5+.5*log(500)*45 = 79.4, which is worse. In fact her expected utility is maximized by spending $182k over the next five years and $91k after that, yielding an expected utility of about 82.9, only a tiny increase in EV. If she has to pay extra interest to time-shift consumption (either via borrowing or forgoing investment returns) she probably just won’t bother. So it seems like you need very high confidence of very short timelines before it’s worth giving up the benefits of consumption-smoothing.
I like this comment. Where the intuition pump breaks down is that you haven’t realistically described what Alice would do with a million dollars, knowing that the Earth has a 50% chance of being destroyed tomorrow. She’d probably spend it trying to diminish that probability—a more realistic possibility that total confidence in 50% destruction and total helplessness to do anything about it.
If we presume Alice can diminish the probability of doom somewhat by spending her million dollars, then it could easily have sufficient positive expected value to make the trade with Bob clearly beneficial.
Why would you expect her to be able to diminish the probability of doom by spending her million dollars? Situations where someone can have a detectable impact on global-scale problems by spending only a million dollars are extraordinarily rare. It seems doubtful that there are even ways to spend a million dollars on decreasing AI xrisk now when timelines are measured in years (as the projects working on it do not seem to be meaningfully funding-constrained), much less if you expected the xrisk to materialize with 50% probability tomorrow (less time than it takes to e.g. get a team of researchers together).
I agree it’s rare to have a global impact with a million dollars. But if you’re 50% confident the world will be destroyed tomorrow, that implies you have some sort of specific knowledge about the mechanism of destruction. The reason it’s hard to spend a million dollars to have a big impact is often because of a lack of such specific information.
But if you are adding the stipulation that there’s nothing Alice can do to affect the probability of doom, then I agree that your math checks out.
Thanks for this interesting and well-developed perspective. However, I disagree specifically with the claim “the existential risk caused by unaligned AI would cause high real interest rates”.
The idea seems to be that, anticipating doomsday, people will borrow money to spend on lavish consumption under the assumption that they won’t need to pay it back. But:
This is a bad strategy (in a game-theoretic and evolutionary sense).
I am skeptical that people actually act this way.
Studying a simple example may help to clarify.
Strategy: playing to your outs
There is a concept in card games of “playing to your outs”. The idea is, if you’re in a losing position, then the way to maximize your winning probability is to assume that future random events still provide a possibility of winning. A common example of this is to observe that you lose unless your next draw is a particular card, and as a result you play under the assumption that you will draw the card you need.
How the concept of playing to your outs applies to existential risk: if in fact extinction occurs, then it didn’t matter what you did. So you ought to plan for the scenario where extinction does not occur.
This same idea ought to apply to evolution for similar reasons that it applies to games. That said, obviously we’re not recalculating evolutionarily optimal strategies as we live our lives, so our actions may not be consistent with a good evolutionary strategy. Still, it makes me skeptical.
Do people actually act this way?
Section VI of the post discusses the empirical question of how people act if they expect doomsday, but I didn’t find it very persuasive. That said, I also haven’t done the work of finding contradictory evidence. (I’d propose looking at cases of religious movements with dated doomsday predictions.)
A lot of the cited evidence in section VI is about education, but I believe that’s a red herring. It makes sense that somebody would value education less if they expect to die before they can benefit from it. But that doesn’t necessarily mean they’re putting the resources towards lavish consumption instead: they might spend it on alternative investments that can outlive them (and therefore benefit their family). So it can’t tell us much about what doomsday does to consumption in particular.
Example
I had originally hoped to settle the matter through an example, but that doesn’t quite work. Still, it’s instructive.
Consider two people, Alice and Bob. They agree that there’s a 50% probability that tomorrow the Earth will explode, destroying humanity. They contemplate the following deal:
Today (day 0), Bob gives Alice $1 million.
The day after tomorrow (day 2), Alice gives Bob her $1.5 million vacation home.
One way to look at this:
Alice gains $1 million for sure.
With 50% probability, Alice doesn’t need to give Bob the home (because humanity is extinct), so her expected loss is only 50% * $1.5 million = $750k.
Therefore, Alice’s net expected profit is $250k.
From this perspective, it’s a good deal for Alice. (This seems to be the perspective taken by the OP.)
But here’s a different way to look at it:
In the scenario where the world ends, Alice, her family, and her friends are all dead, regardless of whether she made the deal, so she nets nothing.
In the scenario where the world doesn’t end, Alice has lost net $500k.
Therefore, Alice has a net expected loss of $250k.
From this perspective, it’s a bad deal for Alice. (This is more in line with my intuition.)
The discrepancy presumably arises from the possibility of Alice consuming the $1 million today, for example by throwing a mega-party.
But that just feels wrong, doesn’t it? Is partying hard actually worth so much to Alice that she’s willing to make herself worse off in the substantial (50%!) chance that the world doesn’t end?
Here’s an attempt to formalize the “is partying hard worth so much” aspect of your example:
It’s common (with some empirical support) to approximate utility as proportional to log(consumption). Suppose Alice has $5M of savings and expected-future-income that she intends to consume at a rate of $100k/year over the next 50 years, and that her zero utility point is at $100/year of consumption (since it’s hard to survive at all on less than that). Then she’s getting log(100000/100) = 3 units of utility per year, or 150 over the 50 years.
Now she finds out that there’s a 50% chance that the world will be destroyed in 5 years. If she maintains her old spending patterns her expected utility is .5*log(1000)*50 + .5*log(1000)*5 = 82.5. Alternately, if interest rates were 0%, she might instead change her plan to spend $550k/year over the next 5 years and then $50k/year subsequently (if she survives). Then her expected utility is log(5500)*5+.5*log(500)*45 = 79.4, which is worse. In fact her expected utility is maximized by spending $182k over the next five years and $91k after that, yielding an expected utility of about 82.9, only a tiny increase in EV. If she has to pay extra interest to time-shift consumption (either via borrowing or forgoing investment returns) she probably just won’t bother. So it seems like you need very high confidence of very short timelines before it’s worth giving up the benefits of consumption-smoothing.
I like this comment. Where the intuition pump breaks down is that you haven’t realistically described what Alice would do with a million dollars, knowing that the Earth has a 50% chance of being destroyed tomorrow. She’d probably spend it trying to diminish that probability—a more realistic possibility that total confidence in 50% destruction and total helplessness to do anything about it.
If we presume Alice can diminish the probability of doom somewhat by spending her million dollars, then it could easily have sufficient positive expected value to make the trade with Bob clearly beneficial.
Why would you expect her to be able to diminish the probability of doom by spending her million dollars? Situations where someone can have a detectable impact on global-scale problems by spending only a million dollars are extraordinarily rare. It seems doubtful that there are even ways to spend a million dollars on decreasing AI xrisk now when timelines are measured in years (as the projects working on it do not seem to be meaningfully funding-constrained), much less if you expected the xrisk to materialize with 50% probability tomorrow (less time than it takes to e.g. get a team of researchers together).
I agree it’s rare to have a global impact with a million dollars. But if you’re 50% confident the world will be destroyed tomorrow, that implies you have some sort of specific knowledge about the mechanism of destruction. The reason it’s hard to spend a million dollars to have a big impact is often because of a lack of such specific information.
But if you are adding the stipulation that there’s nothing Alice can do to affect the probability of doom, then I agree that your math checks out.