What is Imitation Learning?

Imitation learning covers the methods that learn from demonstrations instead of from a reward signal. The simplest is behavioral cloning: treat each demonstration as a set of observation-and-action pairs and train the policy as ordinary supervised learning. It works, and it fails in one specific way — the policy only ever sees the states the demonstrator visited, so the moment it drifts slightly off that path it meets a situation it was never trained on, decides badly, drifts further, and the error compounds. A long task then fails even though every individual decision was nearly right.

The standard answers to that are worth knowing because each one changes the data you need. DAgger runs the policy, asks the expert to label the states the policy actually reached, and retrains on the combined set — it targets the drift directly, and it needs an expert available for repeated rounds rather than one recording session. Diffusion policy sidesteps a different problem: when the same scene admits several valid actions, averaging them during training yields an action valid for none of them, so instead of regressing a single action it models the distribution and samples from it. Action chunking predicts a short sequence of actions at once, which reduces how often the policy gets a fresh chance to make a mistake.

The data consequence is that coverage and consistency beat volume. Ten thousand demonstrations of the same grasp teach one grasp very thoroughly; the demonstrations carrying real information are the ones where the object sat awkwardly, the first attempt failed, or the operator had to correct. Most collection pipelines discard exactly those. If a project is being specified for imitation learning, the brief should say how failure and recovery episodes are handled — kept, labeled, or dropped — because that decision gets made by default during collection and is expensive to reverse afterwards.