How many robot trajectories do you actually need

Nobody can read a task description and tell you how many trajectories it needs. But the published range is wide enough to contain a useful pattern, and the pattern is about how much of the problem you are solving from scratch.

What the published scale looks like

At the small end, single-task imitation learning works from tens of demonstrations when the setup is fixed. The ALOHA work trained bimanual tasks from roughly fifty demonstrations per task, and the Mobile ALOHA follow-up needed a similar order of magnitude per new task when fine-tuning from a pretrained base.

At the large end, generalist policies train on hundreds of thousands of episodes or more. RT-1 used on the order of a hundred and thirty thousand episodes across more than seven hundred tasks. Open X-Embodiment pooled over a million trajectories from dozens of datasets. DROID collected around seventy-six thousand trajectories across five hundred and sixty-four scenes. The gap between fifty and a hundred and thirty thousand is not a difference in efficiency, it is the difference between one task in one scene and one policy that has to work across many tasks, objects, and rooms.

The three multipliers

Whatever base number you start from, three things scale it.

• Task horizon. A pick-and-place is one behavior. A task with five sequential stages is roughly five behaviors, and each stage needs its own coverage plus episodes that cross the transitions between stages.
• Variant count. Every axis you vary multiplies the requirement: distinct objects, start poses, containers, lighting conditions, camera placements. Doubling the object set does not double the data if the policy already generalizes over objects, but it does if the new objects differ in shape, mass, or how they must be grasped.
• Starting point. Fine-tuning a pretrained vision-language-action model needs far fewer episodes per new task than training a policy from scratch. This is the largest single lever available, and it is the main reason published per-task numbers keep falling.

Recovery and failure data

A dataset of only successful demonstrations teaches a policy what success looks like but not how to get back to it. Projects that deploy on real hardware usually keep a fraction of episodes containing a mistake and a recovery — often somewhere between a tenth and a third of the set, depending on how often the task fails on its own.

Failures with a label are useful. Failures without one are worse than nothing, because a policy cannot distinguish an example to imitate from an example to avoid. If you keep failures, the annotation budget has to cover them, and that is a cost that shows up late if it is not planned for.

Measure your own curve instead of guessing

Take whatever data you have, train the same policy at a quarter, a half, and all of it, and evaluate each on a held-out set of scenes or objects. Plot success against dataset size.

If the curve is still climbing steeply at the full set, the binding constraint is data. If it has flattened, more episodes of the same kind will not help, and the constraint is diversity, model capacity, or the task itself. Those need different fixes, and the curve is what tells them apart. This costs a few training runs and replaces a guess with a measurement.

The uncomfortable part

No number transfers cleanly from someone else's project to yours. A figure from a paper was measured with a specific robot, control rate, camera set, task distribution, and model. Change the model and the number changes with it. What does transfer is the shape of the answer: tens of demonstrations for a fixed single task with a pretrained base, thousands for a task family across scenes, hundreds of thousands for a generalist policy. Start at the low end for your situation, measure the curve, and let the measurement tell you when to stop.