The comments about Metaculus (“jumped ahead of 6 years earlier”) make more sense if you interpret them as being about Yudkowsky already having “priced in” a deep-learning-Actually-Works update in response toAlphaGo in 2016, in contrast to Metaculus forecasters needing to see DALLE 2/PaLM/Gato in 2022 in order to make “the same” update.
(That said, I agree that Yudkowsky’s sneering in the absence of a specific track record is infuriating; I strong-upvoted this post.)
(That said, I agree that Yudkowsky’s sneering in the absence of a specific track record is infuriating; I strong-upvoted this post.)
In particular I am irritated that Yudkowsky is criticizing metaculus forecasters when he literally doesn’t even have a metaculus account, at least one he posts. He’s pro-bets in theory, but then will asymmetrically criticize the people who make their prediction track record quantifiable and public. The reputational risk of making regular metaculus predictions would be a lot more psychologically and socially relevant to him than losing a thousand dollars, so the fact that he’s not doing so says a lot to me.
so the fact that he’s not doing so says a lot to me
How about Metaculus points being worth nothing, or it being a huge time commitment with no payoff? Last I heard (e.g. from Zvi, who wasn’t impressed with it), Metaculus still punished people for not continually updating their predictions, and occasionally rewarded them for making predictions, period (as in, both betting “no” or “yes” on some predictions granted points).
Metaculus generates lots of valuable metrics besides the “meaningless internet points” about which Zvi and others complained. If Yudkowsky had predicted regularly, he would have been able to know e.g. how well-calibrated he is, how his Brier score evolved over time, how it compares to the community’s, etc.
That doesn’t seem very relevant—they can have criticisms of a platform without necessarily doing forecasting. Also, a brier score on its own doesn’t tell us much meaningful without comparing with other people, due to how much different questions can vary in difficulty.
You’re right that that context makes it make more sense. One might even say it sounds impressive! But it’s a real shame he’s avoiding the format that could make these things a non-corrupt, meaningful forecasting track record.
The comments about Metaculus (“jumped ahead of 6 years earlier”) make more sense if you interpret them as being about Yudkowsky already having “priced in” a deep-learning-Actually-Works update in response to AlphaGo in 2016, in contrast to Metaculus forecasters needing to see DALLE 2/PaLM/Gato in 2022 in order to make “the same” update.
(That said, I agree that Yudkowsky’s sneering in the absence of a specific track record is infuriating; I strong-upvoted this post.)
In particular I am irritated that Yudkowsky is criticizing metaculus forecasters when he literally doesn’t even have a metaculus account, at least one he posts. He’s pro-bets in theory, but then will asymmetrically criticize the people who make their prediction track record quantifiable and public. The reputational risk of making regular metaculus predictions would be a lot more psychologically and socially relevant to him than losing a thousand dollars, so the fact that he’s not doing so says a lot to me.
How about Metaculus points being worth nothing, or it being a huge time commitment with no payoff? Last I heard (e.g. from Zvi, who wasn’t impressed with it), Metaculus still punished people for not continually updating their predictions, and occasionally rewarded them for making predictions, period (as in, both betting “no” or “yes” on some predictions granted points).
Have any of those things changed?
Metaculus generates lots of valuable metrics besides the “meaningless internet points” about which Zvi and others complained. If Yudkowsky had predicted regularly, he would have been able to know e.g. how well-calibrated he is, how his Brier score evolved over time, how it compares to the community’s, etc.
post brier score
That doesn’t seem very relevant—they can have criticisms of a platform without necessarily doing forecasting. Also, a brier score on its own doesn’t tell us much meaningful without comparing with other people, due to how much different questions can vary in difficulty.
I’m trolling
You’re right that that context makes it make more sense. One might even say it sounds impressive! But it’s a real shame he’s avoiding the format that could make these things a non-corrupt, meaningful forecasting track record.