Embodied AI Data Collection

What the project actually involves

A collection engagement runs in four stages, and the first one is the longest. Specification means writing down the task list as verb plus object, the robot platform, the sensor configuration, the environments, and what counts as a completed attempt. This is done before any hardware is booked, because a change to the task list after production starts is a change to everything downstream.

The pilot is a short run — a day or two of collection in one environment, using the exact capture setup and annotation conventions intended for production. The buyer reviews the episodes and either signs off or changes the specification while the cost of changing it is still small.

Production is the scheduled work: sessions in each environment until the episode targets are met, with the same operators, the same calibration procedure, and the same reset discipline throughout. QA runs alongside production rather than at the end, because a stream-alignment fault found on day two is a fix, and the same fault found at delivery is a re-run.

What decides the cost

Embodied collection is priced by time-on-task, not by file size, and time-on-task is dominated by everything around the recording. A teleoperated arm needs an operator at the rig, a scene that is reset between attempts, objects put back in position, and batteries swapped. The fraction of a session day that becomes usable episode time is often well under half.

• Sensor count — each added camera or depth sensor multiplies data volume, adds synchronization work, and requires its own calibration. The third camera usually costs more in engineering than in hardware.
• Teleoperation versus autonomous capture — teleoperation is bound by operator hours; autonomous capture can run longer but yields fewer successful episodes per hour and needs a safety plan for every environment.
• Annotation depth — raw trajectories, episode segmentation, action labels, and language instructions are four different price points on the same recording. Instruction annotation is usually the largest single line.
• Scene diversity — ten environments cost more than ten repetitions in one environment, and they are the part that decides whether the model generalizes beyond the room it was recorded in.

How long an engagement takes

For a common research platform and a task list that is already written, a pilot can usually start within a few weeks. Production then depends almost entirely on the episode target and the environment count: a few hundred manipulation episodes across three environments is a matter of weeks, while a project spanning dozens of scenes and object categories runs for months.

Two factors stretch timelines more than buyers expect. The first is hardware availability — if the project needs a platform nobody has standing access to, the hardware has to be located or shipped, and that is calendar time nobody controls. The second is operator skill: teleoperation quality varies enormously between operators, and recruiting and screening them is part of the schedule, not an afterthought.

QA and packaging add one to two weeks on top of production, and they are not compressible without cutting the checks that catch delivery failures.

What you receive, and what you decide first

A delivery contains the episodes themselves with aligned sensor streams, a calibration record for every session, the annotation schema the data was labeled under, and a QA report describing what was checked and what was excluded. Failure episodes are included or excluded according to the success criteria agreed at specification, and either choice is documented.

Before work starts, the buyer has to settle the platform and its action-space convention, freeze the task list, decide the environment spread and object inventory, set the annotation depth, and state whether failed attempts are retained. These are the decisions that cannot be made late — each one changes the pilot, and the pilot is what the rest of the project is built on.