Lots of the comments here are pointing at details of the markets and whether it’s possible to profit off of knowing that transformative AI is coming. Which is all fine and good, but I think there’s a simple way to look at it that’s very illuminating.
The stock market is good at predicting company success because there are a lot of people trading in it who think hard about which companies will succeed, doing things like writing documents about those companies’ target markets, products, and leadership. Traders who do a good job at this sort of analysis get more funds to trade with, which makes their trading activity have a larger impact on the prices.
Now, when you say that:
the market is decisively rejecting – i.e., putting very low probability on – the development of transformative AI in the very near term, say within the next ten years.
I think what you’re claiming is that market prices are substantially controlled by traders who have a probability like that in their heads. Or traders who are following an algorithm which had a probability like that in the spreadsheet. Or something thing like that. Some sort of serious cognition, serious in the way that traders treat company revenue forecasts.
And I think that this is false. I think their heads don’t contain any probability for transformative AI at all. I think that if you could peer into the internal communications of trading firms, and you went looking for their thoughts about AI timelines affecting interest rates, you wouldn’t find thoughts like that. And if you did find an occasional trader who had such thoughts, and quantified how much impact they would have on the prices if they went all-in on trading based on that theory, you would find their impact was infinitesimal.
Market prices aren’t mystical, they’re aggregations of traders’ cognition. If the cognition isn’t there, then the market price can’t tell you anything. If the cognition is there but it doesn’t control enough of the capital to move the price, then the price can’t tell you anything.
I think this post is a trap for people who think of market prices as a slightly mystical source of information, who don’t have much of a model of what cognition is behind those prices.
i agree that there’s the 3rd alternative future that the post does not consider (unless i missed it!):
3. markets remain in an inadequate equilibrium until the end of times, because those participants (like myself!) who consider short timelines remain in too small minority to “call the bluff”.
see the big short for a dramatic depiction of such situation.
I agree with this comment. Don’t treat markets as an oracle regarding some issue, if there’s no evidence that traders in the market have even thought about the issue.
And in this case, the question is not just whether AI is coming soon, the question is whether AI will soon cause extinction or explosive economic growth. Have there been any Bloomberg op-eds debating this?
I agree with this comment. Don’t treat markets as an oracle regarding some issue, if there’s no evidence that traders in the market have even thought about the issue.
Agreed. A recent example of this was the long delay in early 2020 between “everyone who pays attention can see that corona will become a pandemic” until stocks actually started falling.
Reposting my agreement from the EA forum! (Personally I feel like it would be nice to have EA/Lesswrong crossposts have totally synced comments, such that it is all one big community discussion. Anyways --)
Definitely agree with this. Consider for instance how markets seemed to have reacted strangely / too slowly to the emergence of the Covid-19 pandemic, and then consider how much more familiar and predictable is the idea of a viral pandemic compared to the idea of unaligned AI:
The coronavirus was x-risk on easy mode: a risk (global influenza pandemic) warned of for many decades in advance, in highly specific detail, by respected & high-status people like Bill Gates, which was easy to understand with well-known historical precedents, fitting into standard human conceptions of risk, which could be planned & prepared for effectively at small expense, and whose absolute progress human by human could be recorded in real-time… If the worst-case AI x-risk happened, it would be hard for every reason that corona was easy. When we speak of “fast takeoffs”, I increasingly think we should clarify that apparently, a “fast takeoff” in terms of human coordination means any takeoff faster than ‘several decades’ will get inside our decision loops. -- Gwern
Those investors who limit themselves to what seems normal and reasonable in light of human history are unprepared for the age of miracle and wonder in which they now find themselves. The twentieth century was great and terrible, and the twenty-first century promises to be far greater and more terrible. …The limits of a George Soros or a Julian Robertson, much less of an LTCM, can be attributed to a failure of the imagination about the possible trajectories for our world, especially regarding the radically divergent alternatives of total collapse and good globalization.
In this comment, you are decisively rejecting the semi-strong form of the EMH, or at least carving out an exception where AI is concerned. Specifically, the semi-strong EMH states that “ because public information is part of a stock’s current price, investors cannot utilize either technical or fundamental analysis, though information not available to the public can help investors.”
The OP is explicitly and carefully written as an argument aimed at people who do subscribe to the (semi-strong or strong) EMH. For those people, it is correct to ascribe “semi-mystical” powers of prediction to the market. You can make a separate argument that the EMH is false, or that the reasoning of this article is flawed. But your comment makes it sound like it’s not the same thing to reject the EMH as it is to deny that “cognition is there” on this issue. I say that rejection of the EMH and denial of market cognition are the same belief.
It’s only an argument against the EMH if you take the exploitability of AI timeline prediction as axiomatic. If you unpack the EMH a little, traders not analyzing transformative AI can also be interpreted as evidence of inexploitability.
The OP described how a trader might exploit AI timeline prediction for financial gain if they believed in short timelines, such as by borrowing. Are you saying they are wrong, and that it’s not possible to exploit low real rates in this manner? I have to point out that this is not the argument you were making in the comment I responded to. Markets certainly are thinking very hard about AI, including the immediately transformative possibilities of LLMs like ChatGPT for search. Even if markets ignored the possibility of doom, it seems like they’d at least be focusing heavily on the possibility for mega-profits from controlling a beneficial AI. And it’s not like these ideas aren’t out there in the world—advances in AI are a major topic of world conversation.
At the very least, if markets were considering the possibility of AI takeoff and concluding it was inexploitable, that would not look like “if you could peer into the internal communications of trading firms, and you went looking for their thoughts about AI timelines affecting interest rates, you wouldn’t find thoughts like that.” It would look like firms thinking very hard about how to financially exploit AI takeoff and concluding explicitly that it is not possible.
Lots of the comments here are pointing at details of the markets and whether it’s possible to profit off of knowing that transformative AI is coming. Which is all fine and good, but I think there’s a simple way to look at it that’s very illuminating.
The stock market is good at predicting company success because there are a lot of people trading in it who think hard about which companies will succeed, doing things like writing documents about those companies’ target markets, products, and leadership. Traders who do a good job at this sort of analysis get more funds to trade with, which makes their trading activity have a larger impact on the prices.
Now, when you say that:
I think what you’re claiming is that market prices are substantially controlled by traders who have a probability like that in their heads. Or traders who are following an algorithm which had a probability like that in the spreadsheet. Or something thing like that. Some sort of serious cognition, serious in the way that traders treat company revenue forecasts.
And I think that this is false. I think their heads don’t contain any probability for transformative AI at all. I think that if you could peer into the internal communications of trading firms, and you went looking for their thoughts about AI timelines affecting interest rates, you wouldn’t find thoughts like that. And if you did find an occasional trader who had such thoughts, and quantified how much impact they would have on the prices if they went all-in on trading based on that theory, you would find their impact was infinitesimal.
Market prices aren’t mystical, they’re aggregations of traders’ cognition. If the cognition isn’t there, then the market price can’t tell you anything. If the cognition is there but it doesn’t control enough of the capital to move the price, then the price can’t tell you anything.
I think this post is a trap for people who think of market prices as a slightly mystical source of information, who don’t have much of a model of what cognition is behind those prices.
(Comment cross-posted with the EA forum version of this post)
i agree that there’s the 3rd alternative future that the post does not consider (unless i missed it!):
3. markets remain in an inadequate equilibrium until the end of times, because those participants (like myself!) who consider short timelines remain in too small minority to “call the bluff”.
see the big short for a dramatic depiction of such situation.
great post otherwise. upvoted.
Coincidentally, that scene in The Big Short takes place on January 11 (2007) :D
I agree with this comment. Don’t treat markets as an oracle regarding some issue, if there’s no evidence that traders in the market have even thought about the issue.
And in this case, the question is not just whether AI is coming soon, the question is whether AI will soon cause extinction or explosive economic growth. Have there been any Bloomberg op-eds debating this?
Agreed. A recent example of this was the long delay in early 2020 between “everyone who pays attention can see that corona will become a pandemic” until stocks actually started falling.
Is there any evidence at all that markets are good at predicting paradigm shifts? Not my field but I would not be surprised by the “no” answer.
Markets as often-efficient in-sample predictors, and poor out-of-sample predictors, would be my base intuition.
Reposting my agreement from the EA forum! (Personally I feel like it would be nice to have EA/Lesswrong crossposts have totally synced comments, such that it is all one big community discussion. Anyways --)
Definitely agree with this. Consider for instance how markets seemed to have reacted strangely / too slowly to the emergence of the Covid-19 pandemic, and then consider how much more familiar and predictable is the idea of a viral pandemic compared to the idea of unaligned AI:
Peter Thiel (in his “Optimistic Thought Experiment” essay about investing under anthropic shadow, which I analyzed in a Forum post) also thinks that there is a “failure of imagination” going on here, similar to what Gwern describes:
In this comment, you are decisively rejecting the semi-strong form of the EMH, or at least carving out an exception where AI is concerned. Specifically, the semi-strong EMH states that “ because public information is part of a stock’s current price, investors cannot utilize either technical or fundamental analysis, though information not available to the public can help investors.”
The OP is explicitly and carefully written as an argument aimed at people who do subscribe to the (semi-strong or strong) EMH. For those people, it is correct to ascribe “semi-mystical” powers of prediction to the market. You can make a separate argument that the EMH is false, or that the reasoning of this article is flawed. But your comment makes it sound like it’s not the same thing to reject the EMH as it is to deny that “cognition is there” on this issue. I say that rejection of the EMH and denial of market cognition are the same belief.
Source: https://www.investopedia.com/ask/answers/032615/what-are-differences-between-weak-strong-and-semistrong-versions-efficient-market-hypothesis.asp
It’s only an argument against the EMH if you take the exploitability of AI timeline prediction as axiomatic. If you unpack the EMH a little, traders not analyzing transformative AI can also be interpreted as evidence of inexploitability.
The OP described how a trader might exploit AI timeline prediction for financial gain if they believed in short timelines, such as by borrowing. Are you saying they are wrong, and that it’s not possible to exploit low real rates in this manner? I have to point out that this is not the argument you were making in the comment I responded to. Markets certainly are thinking very hard about AI, including the immediately transformative possibilities of LLMs like ChatGPT for search. Even if markets ignored the possibility of doom, it seems like they’d at least be focusing heavily on the possibility for mega-profits from controlling a beneficial AI. And it’s not like these ideas aren’t out there in the world—advances in AI are a major topic of world conversation.
At the very least, if markets were considering the possibility of AI takeoff and concluding it was inexploitable, that would not look like “if you could peer into the internal communications of trading firms, and you went looking for their thoughts about AI timelines affecting interest rates, you wouldn’t find thoughts like that.” It would look like firms thinking very hard about how to financially exploit AI takeoff and concluding explicitly that it is not possible.