At the moment we are seeing a host of simple generic ML algorithms. The type where GPT3 and DALLE2 are stereotypical examples. I wouldn’t go so far as saying this will last 10 years, let alone 20.
But before that, deployment of these systems will favor smaller, faster, and more auditable models leading companies to focus on distilled models specializing in specific tasks.
Suppose a task for which there is little training data on exactly that task. You can’t train a good chatbot on a page of text. But you can fine tune GPT3 on a page of text. Training a general model and then fine tuning is a useful strategy. (And someone with loads of compute will go as general as they can.)
So when a large ML model is trained on a wide variety of tasks, (lets suppose its trained using RL.) Can it be dangerous? I think this is a difficult question, and relates to how much large neural nets can learn deep patterns as opposed to shallow memorizing.
At the moment we are seeing a host of simple generic ML algorithms. The type where GPT3 and DALLE2 are stereotypical examples. I wouldn’t go so far as saying this will last 10 years, let alone 20.
Suppose a task for which there is little training data on exactly that task. You can’t train a good chatbot on a page of text. But you can fine tune GPT3 on a page of text. Training a general model and then fine tuning is a useful strategy. (And someone with loads of compute will go as general as they can.)
So when a large ML model is trained on a wide variety of tasks, (lets suppose its trained using RL.) Can it be dangerous? I think this is a difficult question, and relates to how much large neural nets can learn deep patterns as opposed to shallow memorizing.