I think this is an unreasonable characterization of the situation and my position, especially the claim:
Eliezer hasn’t seen a big list of prediction successes from Paul about this thing Paul claims to be unusually good at (whereas, again, EY makes no claim of being unusually good at timing arbitrary narrow-AI advances)
I responded to a long thread of Eliezer trash-talking me in particular (here), including making apparent claims about how this is not the kind of methodology that makes good forecasts. He writes:
It just seems very clear to me that the sort of person who is taken in by this essay is the same sort of person who gets taken in by Hanson’s arguments in 2008 and gets caught flatfooted by AlphaGo and GPT-3 and AlphaFold 2 [… the kind of person who is] going “Huh?” when AlphaGo or GPT-3 debuts[1]
He also writes posts like this one. Saying “the trick that never works” sure seems like it’s making a claim that something has a worse track record than whatever Eliezer is doing.
Overall it looks to me that Eliezer is saying, not once but many times, that he is better at predicting things than other people and that this should be taken as a reason to dismiss various kinds of argument.
I’m not claiming to be exceptional at making predictions. I’m claiming that Eliezer is mediocre at it and overconfident about it.
I’m glad we were able to make one bet, which will give one single bit of evidence for Eliezer’s position if he wins and a a measly 1/8th of a bit against if he loses. But I felt that Eliezer was unwilling to state almost any concrete predictions about anything (even one that was just “Paul is overconfident about X”). In light of that, I think Eliezer probably shouldn’t be ragging so hard on how other people are “caught flatfooted” (in contrast with his superior intuition).
If you want to say “X doesn’t work for forecasting” (and have it actually mean something rather than being mood affiliation) I think you basically need to finish the sentence by saying ”...as well as Y”. And if you want anyone to take that seriously you should be willing to stick your neck out on some X vs Y comparisons rather than just saying “how about the proponents of X go make some forecasts that I will cherry-pick and deride later.”
This is particularly frustrating to me because in 2013 I already expected beating the best human go players in the next few years based on the trend extrapolation in this document. As far as I know Eliezer appears to have had a smaller probability on human-level go performance soon (believing it would require some new insight instead of just extrapolating the curve out to expert performance in 2017-2018), and to have been more confident that the match would be either 0-5 or 5-0.
Likewise, I’m pretty confident that I had a higher probability on GPT-3 than Eliezer did. All of his public statements about language modeling suggests skepticism for getting GPT-3 like competencies at this point in the tech tree or producing it by stacking more layers, whereas in internal OpenAI discussions I was giving 25%+ probabilities for LM scaling working this well (admittedly not as high as some other even truer believers in stack more layers).
I’m not presenting these as explicit predictions that should be particularly convincing to others. But I do hope it explains why I disagree with Eliezer’s implications, and at least makes it plausible that his track record claim is backwards.
You’re not wrong, and I’m not saying you shouldn’t have replied in your current position, but the youtube drama isn’t increasing my respect for either you or Eliezer.
Yeah, I think I should probably stay out of this kind of interaction if I’m going to feel compelled to respond like this. Not that maximizing respect is the only goal, but I don’t think I’m accomplishing much else.
I’m also going to edit the the phrases “shouldn’t talk quite as much shit” and “full of himself,” I just shouldn’t have expressed that idea in that way. (Sorry Eliezer.)
(I think “role of a religious leader” is an apt description of what’s going on sociologically, even if no supernatural claims are being made; that’s why the “rightful caliph” language sticks.)
I used to find the hyper-arrogant act charming and harmless back in 2008, because, back in 2008, he actually was right about almost everything I could check myself. (The Sequences were very good.)
For reasons that are beyond the scope of this comment, I no longer think the hyper-arrogant act is harmless; it intimidates many of his faithful students (who genuinely learned a lot from him) into deferring to their tribal leader even when he’s obviously full of shit.
If he can’t actually live up to his marketing bluster, it’s important for our collective sanity that people with reputation and standing call bullshit on the act, so that citizens of the Caliphate remember that they have the right and the responsibility to think things through for themselves. I think that’s a more dignified way to confront the hazards that face us in the future—and I suspect that’s what the Yudkowsky of 2008 would want us to do. (He wrote then of being “not sure that human beings realistically can trust and think at the same time.”) If present-day Yudkowsky (who complains that “too many people think it’s unvirtuous to shut up and listen to [him]”) disagrees, all the more reason not to trust him anymore.
I think this is an unreasonable characterization of the situation and my position, especially the claim:
I responded to a long thread of Eliezer trash-talking me in particular (here), including making apparent claims about how this is not the kind of methodology that makes good forecasts. He writes:
He also writes posts like this one. Saying “the trick that never works” sure seems like it’s making a claim that something has a worse track record than whatever Eliezer is doing.
Overall it looks to me that Eliezer is saying, not once but many times, that he is better at predicting things than other people and that this should be taken as a reason to dismiss various kinds of argument.
I’m not claiming to be exceptional at making predictions. I’m claiming that Eliezer is mediocre at it and overconfident about it.
I’m glad we were able to make one bet, which will give one single bit of evidence for Eliezer’s position if he wins and a a measly 1/8th of a bit against if he loses. But I felt that Eliezer was unwilling to state almost any concrete predictions about anything (even one that was just “Paul is overconfident about X”). In light of that, I think Eliezer probably shouldn’t be ragging so hard on how other people are “caught flatfooted” (in contrast with his superior intuition).
If you want to say “X doesn’t work for forecasting” (and have it actually mean something rather than being mood affiliation) I think you basically need to finish the sentence by saying ”...as well as Y”. And if you want anyone to take that seriously you should be willing to stick your neck out on some X vs Y comparisons rather than just saying “how about the proponents of X go make some forecasts that I will cherry-pick and deride later.”
This is particularly frustrating to me because in 2013 I already expected beating the best human go players in the next few years based on the trend extrapolation in this document. As far as I know Eliezer appears to have had a smaller probability on human-level go performance soon (believing it would require some new insight instead of just extrapolating the curve out to expert performance in 2017-2018), and to have been more confident that the match would be either 0-5 or 5-0.
Likewise, I’m pretty confident that I had a higher probability on GPT-3 than Eliezer did. All of his public statements about language modeling suggests skepticism for getting GPT-3 like competencies at this point in the tech tree or producing it by stacking more layers, whereas in internal OpenAI discussions I was giving 25%+ probabilities for LM scaling working this well (admittedly not as high as some other even truer believers in stack more layers).
I’m not presenting these as explicit predictions that should be particularly convincing to others. But I do hope it explains why I disagree with Eliezer’s implications, and at least makes it plausible that his track record claim is backwards.
You’re not wrong, and I’m not saying you shouldn’t have replied in your current position, but the youtube drama isn’t increasing my respect for either you or Eliezer.
Yeah, I think I should probably stay out of this kind of interaction if I’m going to feel compelled to respond like this. Not that maximizing respect is the only goal, but I don’t think I’m accomplishing much else.
I’m also going to edit the the phrases “shouldn’t talk quite as much shit” and “full of himself,” I just shouldn’t have expressed that idea in that way. (Sorry Eliezer.)
I think the YouTube drama is serving an important function. Yudkowsky routinely positions himself in the role of a religious leader who is (in his own words) “always right”.
(I think “role of a religious leader” is an apt description of what’s going on sociologically, even if no supernatural claims are being made; that’s why the “rightful caliph” language sticks.)
I used to find the hyper-arrogant act charming and harmless back in 2008, because, back in 2008, he actually was right about almost everything I could check myself. (The Sequences were very good.)
For reasons that are beyond the scope of this comment, I no longer think the hyper-arrogant act is harmless; it intimidates many of his faithful students (who genuinely learned a lot from him) into deferring to their tribal leader even when he’s obviously full of shit.
If he can’t actually live up to his marketing bluster, it’s important for our collective sanity that people with reputation and standing call bullshit on the act, so that citizens of the Caliphate remember that they have the right and the responsibility to think things through for themselves. I think that’s a more dignified way to confront the hazards that face us in the future—and I suspect that’s what the Yudkowsky of 2008 would want us to do. (He wrote then of being “not sure that human beings realistically can trust and think at the same time.”) If present-day Yudkowsky (who complains that “too many people think it’s unvirtuous to shut up and listen to [him]”) disagrees, all the more reason not to trust him anymore.
This is correct.