Dobb-E

What is Dobb-E?

Dobb-E is an open-source framework for teaching robots household tasks through imitation learning — robots learn by watching demonstrations rather than being explicitly programmed. It tackles a core challenge in home robotics: collecting training demonstrations affordably. Data is gathered with “The Stick,” a simple capture tool built from a $25 Reacher-grabber stick, 3D-printed parts, and an iPhone.

With The Stick, the project collected the Homes of New York (HoNY) dataset: 13 hours of interactions recorded across 22 homes in New York City. That data trains Home Pretrained Representations (HPR), a model built on the ResNet-34 architecture using self-supervised learning objectives, designed to help robots adapt to new tasks in varied environments.

Who created Dobb-E?

Dobb-E was developed by researchers Nur Muhammad Mahi Shafiullah, Anant Rai, Haritheja Etukuru, Yiqian Liu, Ishan Misra, Soumith Chintala, and Lerrel Pinto, and released as an open-source project. The team’s goal is to advance how household robots learn and perform tasks, combining The Stick for data collection with the HPR model for initializing robot policies in new environments.

What is Dobb-E used for?

• Teaching robots household chores through demonstrations instead of explicit programming.
• Affordable demonstration collection via The Stick, lowering the cost barrier for home robotics research.
• Training the HPR model, a starting point for robot policies on new tasks in unfamiliar environments.
• Fast task learning — the framework has achieved an 81% average success rate on new tasks, with tasks learned in as little as 15–20 minutes.
• Open access to resources — pre-trained models, code, and documentation are on GitHub; the model is also available on Hugging Face or loadable via PyTorch Image Models (TIMM).
• Research reference — methods and results are detailed in the paper “On Bringing Robots Home.”

Why it matters

Home robotics has long been limited by the difficulty and expense of collecting real-world training data. Dobb-E shows that a low-cost capture tool, a modest dataset gathered across real homes, and self-supervised pretraining can produce policies that generalize to new tasks quickly. For researchers and hobbyists, the open-source release makes it practical to experiment with imitation learning for household robots without expensive custom hardware.