A possible example of the Ernest Rutherford effect (respected scientist says a thing isn’t going to happen and then the next day it does), Stuart Russell speaking in a recent podcast
Deep learning systems are needing, even for these relatively simple concepts, thousands, tens of thousands, millions of examples, and the idea within deep learning seems to be that well, the way we’re going to scale up to more complicated things like learning how to write an email to ask for a job, is that we’ll just have billions or trillions of examples, and then we’ll be able to learn really, really complicated concepts. But of course the universe just doesn’t contain enough data for the machine to learn direct mappings from perceptual inputs or really actually perceptual input history. So imagine your entire video record of your life, and that feeds into the decision about what to do next, and you have to learn that mapping as a supervised learning problem. It’s not even funny how unfeasible that is. The longer the deep learning community persists in this, the worse the pain is going to be when their heads bang into the wall.
I could be wrong but GPT3 probably could write a passable job application letter
For kicks, I wrote a job application prompt for GPT-3. Here’s the result: https://pastebin.com/BQDnqqjd (I suspect P&G will be hiring the first candidate—but you miss 100% of the shots you don’t take!).
Wow, this is really good and really funny! I don’t know if it counts as knowing how to write an email to ask for a job. On the one hand it knows like 99% of it… but on the other hand even the first letter comes across as immature.
True, but it’s not like I wrote a very serious, mature prompt either; real job ads are much more boring and jargony. (I could try harder but my poetry explorations are more interesting to me.) I suspect that with that prompt, it creates a humorous storytelling frame and thus I am tapping into the fiction-writing skills as well: the genre of fake job application letter does exist, and sounds a lot like the second letter (eg Joey Comeau’s Overqualified).
It’s not clear to me that Stuart was saying we won’t be able to use deep learning to write a job application letter—rather, perhaps he just meant that deep learning folks typically seem to think that we’ll be able to do this via supervised learning, but they are wrong because we’ll never have enough data. Idk. You might be right.
A possible example of the Ernest Rutherford effect (respected scientist says a thing isn’t going to happen and then the next day it does), Stuart Russell speaking in a recent podcast
I could be wrong but GPT3 probably could write a passable job application letter
For kicks, I wrote a job application prompt for GPT-3. Here’s the result: https://pastebin.com/BQDnqqjd (I suspect P&G will be hiring the first candidate—but you miss 100% of the shots you don’t take!).
Hilarious.
Wow, this is really good and really funny! I don’t know if it counts as knowing how to write an email to ask for a job. On the one hand it knows like 99% of it… but on the other hand even the first letter comes across as immature.
True, but it’s not like I wrote a very serious, mature prompt either; real job ads are much more boring and jargony. (I could try harder but my poetry explorations are more interesting to me.) I suspect that with that prompt, it creates a humorous storytelling frame and thus I am tapping into the fiction-writing skills as well: the genre of fake job application letter does exist, and sounds a lot like the second letter (eg Joey Comeau’s Overqualified).
It’s not clear to me that Stuart was saying we won’t be able to use deep learning to write a job application letter—rather, perhaps he just meant that deep learning folks typically seem to think that we’ll be able to do this via supervised learning, but they are wrong because we’ll never have enough data. Idk. You might be right.