Some evidence which I claim points in the direction of the plausibility of the feasibility of a highly parallelized search for algorithmic improvements: https://arxiv.org/abs/2403.17844
This paper indicates that even very small scale experiments on novel architectures can give a good approximation of how well those novel architectures will perform at scale. This means that a large search which does many small scale tests could be expected to find promising leads.
Some evidence which I claim points in the direction of the plausibility of the feasibility of a highly parallelized search for algorithmic improvements: https://arxiv.org/abs/2403.17844
This paper indicates that even very small scale experiments on novel architectures can give a good approximation of how well those novel architectures will perform at scale. This means that a large search which does many small scale tests could be expected to find promising leads.