Capture catalogue
Robot-ready capture, specified before it is collected
Every entry below is a capture specification we can scope and deliver: domain, sensor stack, annotation tier and delivery format fixed up front, so you know exactly what arrives. Read the FH-Ego v1 schema on any entry before committing to a pilot.
A target volume is what a full engagement commits to — agreed per brief before capture, rather than drawn from stock. Every entry states where it stands:
Pipeline validated the annotation pipeline has been run end-to-end on a real recording of this task family
Scoped on demand quoted per engagement, no hub committed yet
Research phase under evaluation, not yet commissionable
182,700 clips and 1,505 hours of egocentric capture are commissionable across 6 domains Target; these are volumes a brief commits to, not stock we hold. Each specification below states its own availability, and no two are the same. Simulated traversals are catalogued separately and are not counted here.
8 of 8 specifications
| Specification | Domain | Target volume | Availability | Capture device | Camera | Spatial tracks | Annotation |
|---|---|---|---|---|---|---|---|
| Precision Factory Assembly | Manufacturing | 35,000 clips / 290 hrs Target | Pipeline validated | Stereo Global-Shutter Head Rig | 2560x960 @ 90 FPS (Stereo) | 6-DoF Camera Trajectory (SLAM) & Sync IMU (400Hz) | Tier 1 · Tier 2 · Tier 3 |
| Bimanual Household Tasks | Kitchen | 42,500 clips / 350 hrs Target | Scoped on demand | Ray-Ban Meta | 1920x1080 @ 30 FPS | 3D Camera-Relative Cartesian Coordinates (X, Y, Z) | Tier 1 · Tier 2 · Tier 3 |
| Specialized Caregiving & Nursing | Healthcare | 18,200 clips / 150 hrs Target | Scoped on demand | GoPro Hero 12 | 3840x2160 @ 60 FPS | IMU Synchronized Telemetry (200Hz) | Tier 1 · Tier 2 |
| Warehouse & Logistics | Logistics | 50,000 clips / 410 hrs Target | Scoped on demand | GoPro Hero 12 | 1920x1080 @ 30 FPS | Wrist-mounted IMU sensor synchronization | Tier 1 · Tier 2 |
| Cleaning & Maintenance | Household | 22,000 clips / 180 hrs Target | Scoped on demand | Ray-Ban Meta | 1920x1080 @ 30 FPS | Wide-angle camera lens intrinsics | Tier 1 · Tier 2 |
| Agriculture & Outdoor Tasks | Outdoor | 15,000 clips / 125 hrs Target | Scoped on demand | iPhone LiDAR | 1920x1080 @ 30 FPS | LiDAR Depth Maps & Stereo Depth Extrinsics | Tier 1 · Tier 2 · Tier 3 |
| Video RLHF Policy Annotations | RLHF | 2,000,000 comparisons Target | Scoped on demand | Human Feedback Portal | Varies (Commissioning Lab Ingestion) | Reward score, preference rankings, and failure rationales | Tier 1 · Tier 2 |
| Game Environment & Simulator Data | Simulation | 12,000 scenes / 10,000 hrs Target | Research phase | Simulator Engine | Synthetic RGB-D @ 60 FPS | Ground-truth joint angles, SLAM trajectory, and depth matrices | Tier 1 · Tier 2 |
Common to all eight
What every brief includes, whatever you commission
The specifications above differ in domain, device, volume and tier. Everything in this section is identical across all eight, so it is written here once and linked from each brief rather than repeated on every one.
How a brief is scoped, and what "on demand" means
Six of the eight briefs below read scoped on demand. That is the honest description of where they stand: no hub, no recordings, no schedule. We would scope the brief, recruit for it and quote it as one engagement.
A pilot dataset is produced when a commissioning lab asks for one. We hold every brief here and expect to add more, so a brief is not a shelf we are stocking ahead of you — it is a specification we can execute against. One brief reads pipeline validated: a recording from that task family has been through the pipeline end to end and is published in full with its checksum, and that badge means the pipeline was validated, not the rig and not a hub. One reads research phase and is not commissionable today.
Three annotation tiers, one schema
Every brief is annotated against FH-Ego v1 and against nothing else. Tier 1 is the task and session record, Tier 2 is the action segment, and Tier 3 is the timeline tracks that carry the kinematics. The Annotation column above says which tiers a brief includes.
The exporter offers no custom field set. It emits one schema, and every field of it is published with its shape, its unit, its sampling rate and the tier it belongs to: FH-Ego v1, field by field.
Five delivery containers, and two we decline
The same annotated record is written into any of five containers, each with a writer in this repository and each read back off disk by its own verifier before it ships. Two more are declined rather than unbuilt, which is a position and not a gap.
Which five, which two, and the reasoning for each: what arrives, and in what container.
A consent receipt, written before the first frame is uploaded
Every recording is consented on the Capture Partner's own phone before capture, and the receipt is written before the upload begins. A receipt is eleven fields hashed into a chain, each entry carrying the hash of the one before it, so a receipt cannot be edited after the fact without breaking every entry that follows.
Consent is granted over the recording, not over a clip, and it can be withdrawn. A withdrawal makes byte-identical footage undeliverable — packaging refuses rather than skips.
You can test a receipt yourself, in your own browser, against the published chain: check a consent receipt. How the chain is built and what its digest does and does not prove: the consent ledger.
Faces blurred at ingestion, and what that does not cover
Faces are detected and blurred when footage arrives, before any person here opens it. If no detector loads, the clip does not proceed — nothing passes through unredacted.
It covers faces. It does not cover licence plates, screens or documents. Audio is handled by removal rather than by blurring: what a Commissioning lab receives carries no audio track, and that is checked rather than assumed — the encoder writes video only, and the stage reads its own written file back and raises if it finds sound. The original we retain keeps its audio. We have not measured a miss rate, because measuring one needs cohort footage and a labelled sample and we have neither.
The full position, including the failure this service shipped until 13 August 2026 and the claim we corrected the day after: what we blur, what we do not.
Nine reasons a clip can be rejected, and the partner is told which
Every clip is accepted, rejected or held. A rejection carries one or more reasons from a fixed list of nine, and the Capture Partner reads the same reason we recorded, in the same words, with guidance on what to do differently.
We publish no acceptance rate. We have not measured one — a rate needs a body of reviewed clips, and one recording has been through this pipeline end to end. How a clip is judged, and the nine reasons in full: how a clip is judged, and what a partner is told.
What has to happen before a sample appears
Every brief page carries a sample slot and every one of them is empty. That is not a build we have not finished — it is the honest state. One recording has been through the pipeline end to end: it belongs to precision factory assembly, and it is published in full, annotation and checksum, on the reference implementation.
What has to happen before a sample appears on any other brief is a commissioned brief and a Capture Partner cohort recording against it. We publish no blurred teaser, no greyed button and no date.
The stages a recording passes
Five stages are implemented. 3 of them do not run in the deployment we operate today — the pipeline parks before TRACK, and a page presenting them as live would be describing a pipeline nobody is running.
| Stage | What it does | Today |
|---|---|---|
BLUR | Faces detected and blurred, before any person here opens the footage. | Runs today |
SEGMENT | The continuous upload cut into task-bounded clips, and each clip judged. | Runs today |
TRACK | Hand landmarks read per sampled frame. | Parked |
CAPTION | A caption written for each action segment. | Parked |
EXPORT | The annotated record written into the delivery container. | Parked |
Every volume above is a target, not stock
Nothing in this catalogue is footage we are holding. Each row is a volume a brief would commit to, agreed before capture and carrying a Target badge wherever it is printed. We do not publish a count of what we hold, and a count of what we hold would not be the same number.
What has actually been through the pipeline end to end is 1 recording, published in full with its annotation and its checksum: the reference implementation.
The refusals
What a delivery does not include
Assembly refuses rather than skips, and every refusal names what caused it. These are conditions the code checks when a package is built, not undertakings we give.
Footage nobody consented to
Every clip's consent receipt is re-checked at the moment a package is assembled, not at the moment the release was cut. A partner who withdraws after a release is frozen still stops that delivery — byte-identical footage becomes undeliverable, and the package refuses rather than quietly shipping without it.
You can test any receipt against the published chain in your own browser: check a consent receipt.
Unredacted frames
Faces are detected and blurred at ingestion, before any person here opens the footage and before anything downstream runs. If no detector loads, the pipeline stops — there is no path that emits a clip the redaction pass could not process. What we blur, and what we do not.
Sound
The delivered clip carries no audio track, and the absence is checked rather than inherited: the encoder writes video only, and the stage reads its own written file back and raises if it finds one.
A package that does not verify as the package we named
A release that does not verify intact refuses. So does a clip with no exported annotation — a package with silently missing annotations is refused by name rather than shipped light. The delivery's own identifier is a hash over the sorted listing of every file inside it, each path paired with its own checksum, so rebuilding that listing is a test you can run on what arrives.
A container we have not written a reader for
A format outside the published set refuses, and so does a format whose independent verifier fails on any clip's export. Every container we offer is read back off disk before it ships. What arrives, and in what container.
A verdict you have to take on trust
Every clip in a delivery carries its QA verdict and any rejection reasons in the compliance log, alongside its content hash, its consent receipt identifier and the status that receipt held at packaging time. The judgement travels with the data rather than staying with us. How a clip is judged.
Anything we have not measured
Every figure we publish carries its provenance and the command that recomputes it, and the record is annotated field by field: what each one is, its shape, its unit, how often it is sampled and which layer produced it.
FH-Ego v1, field by field · every number we publish, and where it comes from