Google AI integrates PaLM with robotics: SayCan update [Linkpost]

Link post

Back in April, Google AI announced SayCan, a project which integrated a language model (FLAN) with robotics in order to produce a robot which could follow instructions. For example, cleaning up a mess in a kitchen. (LessWrong post from April which links to a tweet about SayCan.)

This week Google AI has released some new updates, dubbed PaLM-SayCan. This involved upgrading the integrated language model to Google AI’s top performing large language model (LLM) of 540-billion parameters, PaLM.

Website for PaLM-SayCan

Blog post announcing PaLM-SayCan

The following updates from this week are quoted from the original SayCan project website (linked in first sentence of this post, bold mine):

[8/16/2022] We integrated SayCan with Pathways Language Model (PaLM), and updated the results. We also added new capabilities including drawer manipulation, chain of thought prompting and multilingual instructions. You can see all the new results in the updated paper.
[8/16/2022] Our updated results show that SayCan combined with the improved language model (PaLM), which we refer to as PaLM-SayCan, improves the robotics performance of the entire system compared to a previous LLM (FLAN). PaLM-SayCan chooses the correct sequence of skills 84% of the time and executes them successfully 74% of the time, reducing errors by a half compared to FLAN. This is particularly exciting because it represents the first time we can see how an improvement in language models translates to a similar improvement in robotics.
[8/16/2022] We open-sourced a version of SayCan on a simulated tabletop environment.

Thanks to Jon Menaster for making me aware of this update from Google AI.