I note that mixture-of-experts is referred to as the kind of thing that in principle could shorten timelines, but in practice isn’t likely to. Intuitively, and naively from neuroscience (different areas of the brain used for different things), it seems that mixture-of-experts should have a lot of potential, so I would like to see more detail on exactly why it isn’t a threat.
I note that mixture-of-experts is referred to as the kind of thing that in principle could shorten timelines, but in practice isn’t likely to. Intuitively, and naively from neuroscience (different areas of the brain used for different things), it seems that mixture-of-experts should have a lot of potential, so I would like to see more detail on exactly why it isn’t a threat.