AIs that are superhuman at just about any task we can (or simply bother to) define a benchmark, for
Something that I’m really confused about: what is the state of machine translation? It seems like there is massive incentive to create flawless translation models. Yet when I interact with Google translate or Twitter’s translation feature, results are not great. Are there flawless translation models that I’m not aware of? If not, why is translation lagging behind other text analysis and generation tasks?
this seems wrong. They’re probably not terribly old, they’re mostly just small. They might be out of date architectures, of course, because of implementation time.
The same reason SOTA models are only used in a few elite labs and nowhere else.
Cost, licensing issues, a shortage of people who know how to adapt them, problems with the technology being so new and still basically a research project.
Your question is equivalent to, a few years after transistors begin to ship in small packaged ICs, why some computers still used all vacuum tubes. It’s essentially the same question.
I am surprised that these issues would apply to, say, Google translate. Google appears unconstrained by cost or shortage of knowledgeable engineers. If Google developed a better translation model, I would expect to see it quickly integrated into the current translation interface. If some external group developed better translation models, I would expect to see them quickly acquired by Google.
google doesn’t use SOTA translation tools because they’re too costly per api call. they’re SOTA for the cost bucket they budgeted for google translate, of course, but there’s no way they’d use PaLM full size to translate.
also, it takes time for groups to implement the latest model. Google, microsoft, amazon, etc, are all internally is like a ton of mostly-separate companies networked together and sharing infrastructure; each team unit manages their own turf and is responsible for implementing the latest research output into their system.
Also do they have PaLM full size available to deploy like that? Are all the APIs in place where this is easy, or you can build a variant using PaLMs architecture but with different training data specifically for translation? Has Deepmind done all that API work or are they focused on the next big thing.
I can’t answer this, not being on the inside, but I can say on other projects, ‘research’ grade code is often years away from being deployable.
Something that I’m really confused about: what is the state of machine translation? It seems like there is massive incentive to create flawless translation models. Yet when I interact with Google translate or Twitter’s translation feature, results are not great. Are there flawless translation models that I’m not aware of? If not, why is translation lagging behind other text analysis and generation tasks?
Those translate engines are not using sota AI models, but something relatively old. (a few years)
this seems wrong. They’re probably not terribly old, they’re mostly just small. They might be out of date architectures, of course, because of implementation time.
Why haven’t they switched to newer models?
The same reason SOTA models are only used in a few elite labs and nowhere else.
Cost, licensing issues, a shortage of people who know how to adapt them, problems with the technology being so new and still basically a research project.
Your question is equivalent to, a few years after transistors begin to ship in small packaged ICs, why some computers still used all vacuum tubes. It’s essentially the same question.
I am surprised that these issues would apply to, say, Google translate. Google appears unconstrained by cost or shortage of knowledgeable engineers. If Google developed a better translation model, I would expect to see it quickly integrated into the current translation interface. If some external group developed better translation models, I would expect to see them quickly acquired by Google.
google doesn’t use SOTA translation tools because they’re too costly per api call. they’re SOTA for the cost bucket they budgeted for google translate, of course, but there’s no way they’d use PaLM full size to translate.
also, it takes time for groups to implement the latest model. Google, microsoft, amazon, etc, are all internally is like a ton of mostly-separate companies networked together and sharing infrastructure; each team unit manages their own turf and is responsible for implementing the latest research output into their system.
Also do they have PaLM full size available to deploy like that? Are all the APIs in place where this is easy, or you can build a variant using PaLMs architecture but with different training data specifically for translation? Has Deepmind done all that API work or are they focused on the next big thing.
I can’t answer this, not being on the inside, but I can say on other projects, ‘research’ grade code is often years away from being deployable.
Yeah strongly agreed, I think we’re basically trying to make the same point.