RoboCat learns across robots

Google DeepMind introduced RoboCat on 20 June, reporting that its robotic learning system can adapt to new tasks with 100 to 1,000 demonstrations. That follows substantial earlier training, according to the announcement.

Photo: Anthony Quintano, “Google at the 2012 Democratic National Convention North Carolina”. CC BY 2.0. Resized and cropped. Taken 6 September 2012. Historical Google identity photograph, not a RoboCat demonstration or June 2023 event image. No endorsement implied.

A specialised version practises the new task about 10,000 times on average. Researchers add that experience and human demonstrations to the dataset, then train another generalist.

The reported 86% success rate concerns gear-picking with a new arm after 1,000 demonstrations. It isn’t an overall reliability rating.

What self-improvement requires

The 20 June paper says goal images specify tasks, not exact target states.

Autonomous collection also needs success detection and resets. Human-labelled episodes train visual reward models; pools of complementary policies let one task reset another. Final evaluation results are counted manually.

Its 141 evaluated training-task variations aren’t 141 unseen tasks. Adaptation tests are separate. Demonstrations, success checks and resets still support the process.