Nice interview, liked it overall! One small question -
Heuristic: Imagine you were in a horror movie. At what point would the audience be like “why aren’t you screaming yet?” And how can you see GPT-3 and Dall-E (especially Dall-E) and not imagine the audience screaming at you?
I feel like I’m missing something; to me, this heuristic obviously seems like it’d track “what might freak people out” rather than “how close are we actually to AI”. E.g. it feels like I could also imagine an audience at a horror movie starting to scream in the 1970s if they were shown the sample dialogue with SHRDLU starting from page 155 here. Is there something I’m not getting?
Jonathan Blow had a thread on Twitter about this, like Eroisko SHRDLU has no published code, no similar system showing the same behaviour after 40-50 years. Just the author’s word. I think the performance of both was wildly exaggerated.
But if we are the movie audience seeing just the publication of the paper in the 70s, we don’t yet know that it will turn out to be a dead end with no meaningful follow-up after 40-50 years. We just see what looks to us like an impressive result at the time.
And we also don’t yet know if GPT-3 and Dall-E will turn out to be dead ends with no significant progress for the next 40-50 years. (I will grant that it seems unlikely, but when the SHRDLU paper was published, it being a dead end must have seemed unlikely too.)
If we start going to the exact specifics of what makes them different then yes, there are reasonable grounds for why GPT-3 would be expected to genuinely be more of an advance than SHRDLU was. But at least as described in the post, the heuristic under discussion wasn’t “if we look at the details of GPT-3, we have good reasons to expect it to be a major milestone”; the heuristic was “the audience of a horror movie would start screaming when GPT-3 is introduced”.
If the audience of a 1970s horror movie would have started screaming when SHRDLU was introduced, what we now know about why it was a dead end doesn’t seem to matter, nor does it seem to matter that GPT-3 is different. Especially since why would a horror movie introduce something like that only for it to turn out to be a red herring?
I realize that I may be taking the “horror movie” heuristic too literally but I don’t know how else to interpret it than “evaluate AI timelines based on what would make people watching a horror movie assume that something bad is about to happen”.
Nice interview, liked it overall! One small question -
I feel like I’m missing something; to me, this heuristic obviously seems like it’d track “what might freak people out” rather than “how close are we actually to AI”. E.g. it feels like I could also imagine an audience at a horror movie starting to scream in the 1970s if they were shown the sample dialogue with SHRDLU starting from page 155 here. Is there something I’m not getting?
Jonathan Blow had a thread on Twitter about this, like Eroisko SHRDLU has no published code, no similar system showing the same behaviour after 40-50 years. Just the author’s word. I think the performance of both was wildly exaggerated.
But if we are the movie audience seeing just the publication of the paper in the 70s, we don’t yet know that it will turn out to be a dead end with no meaningful follow-up after 40-50 years. We just see what looks to us like an impressive result at the time.
And we also don’t yet know if GPT-3 and Dall-E will turn out to be dead ends with no significant progress for the next 40-50 years. (I will grant that it seems unlikely, but when the SHRDLU paper was published, it being a dead end must have seemed unlikely too.)
Millions have personally used GPT-3 in this movie.
If we start going to the exact specifics of what makes them different then yes, there are reasonable grounds for why GPT-3 would be expected to genuinely be more of an advance than SHRDLU was. But at least as described in the post, the heuristic under discussion wasn’t “if we look at the details of GPT-3, we have good reasons to expect it to be a major milestone”; the heuristic was “the audience of a horror movie would start screaming when GPT-3 is introduced”.
If the audience of a 1970s horror movie would have started screaming when SHRDLU was introduced, what we now know about why it was a dead end doesn’t seem to matter, nor does it seem to matter that GPT-3 is different. Especially since why would a horror movie introduce something like that only for it to turn out to be a red herring?
I realize that I may be taking the “horror movie” heuristic too literally but I don’t know how else to interpret it than “evaluate AI timelines based on what would make people watching a horror movie assume that something bad is about to happen”.
Seems like he basically admits the thing was a fraud: