Since these systems don’t do things in the world yet, that’s the case. Robotics transformers are a thing though. This current system appears to have usable but slow vision.
A system that puts it all together: a sota LLM, with vision, with a subsystem that accepts tokens and uses them to set realtime robotic control policy, and the LLM has a fine tune from thousands of hours of robotics practice in simulation appears imminent.
Since these systems don’t do things in the world yet, that’s the case. Robotics transformers are a thing though. This current system appears to have usable but slow vision.
A system that puts it all together: a sota LLM, with vision, with a subsystem that accepts tokens and uses them to set realtime robotic control policy, and the LLM has a fine tune from thousands of hours of robotics practice in simulation appears imminent.