That’s why I said “typically”, yes. What I meant was that if you choose 2 random tasks that neural networks are used for, most likely a neural net trained for one will not be useful for the other.
Also, even given transfer learning, the principle holds that you can have a neural net which works great for one task and not for another, just by retraining the last layer. That’s what I was getting at with the statement “a 2 hour machine learning tutorial beats weeks of philosophizing”—the fact that retraining the last layer dramatically changes performance across tasks demonstrates that “is intelligence unidimensional” is in some sense a wrong question. If you engage with territory then your ontology becomes finer-grained.
Aside from text, image, audio, point clouds, graphs etc., what have the Romans^Wconvolutions and Transformers done for us lately?
With exactly the same set of hyperparameters? Last I checked the optimal hyperparameters usually vary based on the task, but maybe that has changed.
Anyway, it sounds like you’ve changed the question from “do neural nets show intelligence is unidimensional” to “do convolutions / attention show intelligence is unidimensional [implicitly, within the scope of tasks for which neural nets work the best]”. There are some tasks where neural nets aren’t the best.
AI techniques seem to be something like a toolbox. There are cases where a tool works well in a wide variety of situations, and cases where one tool appears to almost strictly dominate another tool. And as you imply, even what might appear to be a single tool, such as “neural networks”, actually consists of a bunch of smaller tools which get recombined with each other in conventional ways. So far we haven’t found a single tool or way of recombining smaller tools which appears to be universally dominant over all the other approaches. Even if we did, the fact that there was no universally dominant approaches at some earlier phases of AI development suggests that a universally dominant tool may not be a permanent state of affairs. My personal guess is that we will discover something which looks a bit like a universally dominant approach around the time we develop transformative AI… but that doesn’t change the fact that AI is not a unidimensional thing from a philosophical perspective. (In particular, as I said, I think it will be possible to use the universally dominant approach to excel in particular narrow areas without creating something that looks like the AGI of science fiction.)
That’s why I said “typically”, yes. What I meant was that if you choose 2 random tasks that neural networks are used for, most likely a neural net trained for one will not be useful for the other.
Also, even given transfer learning, the principle holds that you can have a neural net which works great for one task and not for another, just by retraining the last layer. That’s what I was getting at with the statement “a 2 hour machine learning tutorial beats weeks of philosophizing”—the fact that retraining the last layer dramatically changes performance across tasks demonstrates that “is intelligence unidimensional” is in some sense a wrong question. If you engage with territory then your ontology becomes finer-grained.
With exactly the same set of hyperparameters? Last I checked the optimal hyperparameters usually vary based on the task, but maybe that has changed.
Anyway, it sounds like you’ve changed the question from “do neural nets show intelligence is unidimensional” to “do convolutions / attention show intelligence is unidimensional [implicitly, within the scope of tasks for which neural nets work the best]”. There are some tasks where neural nets aren’t the best.
AI techniques seem to be something like a toolbox. There are cases where a tool works well in a wide variety of situations, and cases where one tool appears to almost strictly dominate another tool. And as you imply, even what might appear to be a single tool, such as “neural networks”, actually consists of a bunch of smaller tools which get recombined with each other in conventional ways. So far we haven’t found a single tool or way of recombining smaller tools which appears to be universally dominant over all the other approaches. Even if we did, the fact that there was no universally dominant approaches at some earlier phases of AI development suggests that a universally dominant tool may not be a permanent state of affairs. My personal guess is that we will discover something which looks a bit like a universally dominant approach around the time we develop transformative AI… but that doesn’t change the fact that AI is not a unidimensional thing from a philosophical perspective. (In particular, as I said, I think it will be possible to use the universally dominant approach to excel in particular narrow areas without creating something that looks like the AGI of science fiction.)