Human demonstration versus autonomous data: what each one can teach a policy

The two sources sample different distributions

A human demonstration is a trajectory through the states an expert chooses to occupy. An autonomous rollout is a trajectory through the states the policy actually reaches, which include the ones the expert never visits because they are the result of small errors compounding. That difference, not the cost, is the reason the two are not interchangeable.

A policy trained only on demonstrations is trained on the wrong distribution near the edges. It performs well where the expert went and degrades where its own small mistakes take it, and the degradation is not gradual: the further the policy drifts from the demonstrated states, the less the training data says about what to do. Autonomous data is the only source that covers exactly those states, which is why it is the corrective for the specific weakness demonstrations have.

What human demonstrations are good at

Demonstrations are the only source that works before a policy exists, and they carry things that a reward function cannot express.

• Cold start. No policy, no reward, no simulator needed, just a rig and an operator.
• Contact-rich skill. Insertion, in-hand adjustment, and anything where the right motion depends on feel rather than on geometry.
• Task structure. Multi-stage tasks come out of a demonstration already sequenced, with the transitions included.
• Intent and instruction pairing. Why the object was moved, and what a human would call the task, both come for free.
• A definition of success that a person can point at.

What autonomous data is good at

Once a policy succeeds some of the time, running it produces data at a cost per episode that human time cannot match, and the data is on the policy distribution by construction.

The catches are specific. The policy has to succeed often enough to be worth rolling out, which means a cold start has to come from somewhere else. A success signal is required, and producing one is often harder than producing the data: it is a scripted scene checker, a final-state measurement, or human review. And the data inherits the policy blind spots, because a policy that never tries a strategy never generates an episode about it, so autonomous collection explores around the current behavior rather than beyond it.

Interventions are the middle ground

The most valuable single category of robot data is an episode where the policy was in control, made a mistake, a human took over at the moment of failure, and the correction was recorded. The states are on the policy distribution, the actions are expert, and the human is idle until needed, which makes it far cheaper per episode than continuous teleoperation. This is the DAgger pattern, and the reason it works is that it converts scarce human time into exactly the states the policy gets wrong rather than into states it already handles.

It requires two pieces of engineering. Takeover has to be fast enough that the human can catch the failure before it completes, which means a low-latency interface and a clear signal for when the policy is about to go wrong. And the record has to carry an authority flag per step, marking who was in control, because an episode with two drivers and no marker is ambiguous data that most training pipelines will silently mangle. Keep the intervention point as a timestamped field rather than cutting the episode, since the segment before the takeover is the part that teaches failure prediction.

Scripted and planner-generated episodes

Motion-planned rollouts are the cheapest source of all and they are precise, but they only teach a task when the task is geometric. They are strong for covering start states, for pretraining perception, and for generating evaluation conditions on demand, and weak wherever contact is the hard part, because the planner has no model of what the object does when it is pushed.

Human video collected without a robot is a fourth category with a different limitation: it shows hands and objects in enormous variety and it contains no robot actions, no joint states, and no dynamics. It teaches visual priors and affordances, not control. Mixing it with robot data is a pretraining decision rather than a data collection one.

The order that usually works

Start with scripted or demonstrated episodes to get a policy that succeeds occasionally. Run that policy autonomously with a human able to intervene, and keep everything, including the failures. Use the corrected and failed episodes as the next training set, and keep a permanent trickle of human demonstration for the skills the policy still cannot perform, because those are the ones where an expert trajectory is still the only source.

The ratio shifts over the life of a project: mostly human at the beginning, mostly autonomous later, with human episodes retained for the hard tail. A project that stays all-demonstration pays the human cost forever, and a project that jumps to autonomous collection without a policy that works at all produces a large dataset of the same failure.