Setting up egocentric video collection: rigs, angles, sync, and privacy
Head, chest, or wrist mounting, what field of view costs you, how to keep cameras aligned, and how to handle the people who walk into frame.
Egocentric video is the cheapest way to get hand-object interaction at scale, because the collector is a person rather than a robot. The tradeoff is that every part of the setup becomes a choice you have to defend later: what the camera sees, how it is aligned with everything else, and whose faces end up in the recording.
Choose the mount by what the policy needs
Head-mounted rigs see roughly what the wearer is looking at, which is the closest available match to how a robot's head camera will be positioned later. A head strap with an action camera is the simple version, and a camera rigidly mounted on a helmet is more stable and easier to instrument. Chest-mounted rigs are steadier and suit tasks where the hands stay low and in front, but they do not follow gaze and miss whatever the wearer turns to look at.
Wrist-mounted cameras give close-up views of the hand and the object at the cost of a violently moving frame, which makes hand pose estimation harder and which some encoders handle badly, so they work best as a second view. A head camera plus one wrist camera covers both the scene and the contact, and two cameras mean two calibrations and a sync problem, so add the second view only if the task needs the close-up.
Field of view is the main optical tradeoff
A very wide lens, roughly a hundred and twenty degrees and up, keeps the hands in frame even at the edges, but it shrinks objects, bends straight lines at the corners, and produces small distorted hands, the opposite of what a manipulation policy needs. A narrower lens around ninety degrees keeps objects larger and less distorted but loses the hands whenever they leave the center, so mount it tilted down by roughly twenty to thirty degrees and let a natural working posture hold the hands in the lower half of the frame.
Thirty frames per second is workable for slow tasks and sixty is worth the storage for anything fast, since a quick grasp can begin and end within a few frames at thirty. Rolling-shutter cameras skew fast lateral motion, so choose a global-shutter sensor if that matters or accept the artifact deliberately. Auto-exposure produces visible brightness jumps when the wearer passes a window, so lock exposure and white balance in a controlled space and record the settings.
Sync and calibration
With one camera, sync only matters for aligning video against other sensors: a wrist inertial unit, a glove, or robot state. Use a visible event — a flash, a clap, or a marker crossing the frame — captured in both streams, and compute the offset from the difference in timestamps. With two or more cameras, capture the same event at the start and end of every session so you can measure drift and not only offset, since consumer cameras drift apart over long sessions and not linearly.
Calibrate intrinsics per unit, because two cameras of the same model are not interchangeable and a shared calibration file is a silent source of spatial error. Store the calibration identifier with the session, because footage whose extrinsics are unknown is nearly unusable for anything spatial no matter how good the video looks.
Privacy is a setup decision, not a post-processing one
The people who walk into frame did not agree to be in a training set. Decide before recording starts: collect in a space where bystanders are excluded, obtain consent from everyone who can plausibly appear, or plan to remove them. Faces are the obvious problem. The less obvious ones are name badges, whiteboards, screens showing customer data, and voices, since egocentric rigs usually record audio and a room with other people in it makes that audio a consent issue even when the video is clean.
Post-hoc blurring is possible, expensive, and imperfect, and it happens after the footage already sits on a disk you have to protect. On-device blurring reduces what you store, but the recording still happened, and in most jurisdictions the recording is the part that carries legal weight. Keep a log of where and when recording happened and who was present, because that log is what makes the dataset defensible later.
The operational details that end a session early
Storage and battery: a high-frame-rate camera fills a memory card fast enough that cards and batteries should be sized for a whole session rather than for a clip, and hot-swaps should not land in the middle of a sync event. Heat is the other silent failure, since head-mounted cameras in a warm room throttle or shut down without warning. Wearer comfort decides data volume more than anything else, because a rig that becomes uncomfortable after twenty minutes produces twenty-minute sessions regardless of the schedule.