Hmm. I’ve not seen any research about that possibility, which is obvious enough that I’d expect to see it if it were actually promising. And naively, it’s not clear that you’d get more powerful results from using 1M times the compute this way, compared to more direct scaling.
I’d put that in the exact same bucket as “not known if it’s even possible”.
Such possibility is explored at least here: https://arxiv.org/abs/2305.17066 but that’s not the point. The point is: even in hypothetical world where scaling laws and algorithmic progress hit the wall at smartest-human-level, you can do this and get an arbitrary level of intelligence. In real world, of course, there are better ways.
Hmm. I’ve not seen any research about that possibility, which is obvious enough that I’d expect to see it if it were actually promising. And naively, it’s not clear that you’d get more powerful results from using 1M times the compute this way, compared to more direct scaling.
I’d put that in the exact same bucket as “not known if it’s even possible”.
Such possibility is explored at least here: https://arxiv.org/abs/2305.17066 but that’s not the point. The point is: even in hypothetical world where scaling laws and algorithmic progress hit the wall at smartest-human-level, you can do this and get an arbitrary level of intelligence. In real world, of course, there are better ways.