The pipelines

Three named workflows, each resumable and each recording what it did. They are wiring, not algorithms — every stage is a component with its own repository.

1 · geo_kinetic_discovery

No model, no checkpoint, no class list — and no class names out either. What it produces is instances: a discrete something is here, this is its box, this is its trajectory.

proprioception self-mask   forward kinematics from /ego/joint_states
  → terrain classification   Stone.mojo — connectivity, not height
  → voxel clustering         26-neighbourhood connected components
  → oriented box fit         PCA heading in the ground plane
  → gated association        Hungarian.mojo — solve_gated
  → RTS smoothing            Kalman.mojo — smooth_track
  → motion filter            the stage that decides whether the loop compounds

What geometry and motion can actually settle:

• Terrain against not-terrain — free and reliable.
• Instances — connected clusters standing on that terrain.
• Motion attributes — the strongest signal here by some distance, and the one the whole loop turns on.
• A size bucket — machine-sized and person-sized boxes differ by two and a half orders of magnitude, and a 5 m³ split separates them 97.5% of the time.
• The ego's own parts, exactly — kinematics knows which rigid body is the boom. The one place geometry hands over true named classes, free.

What it can never settle: excavator against dozer; worker against any person-shaped object; stockpile against spoil pile, where the discriminator is intent and simply is not present in a point cloud.

Why terrain is a connectivity question

Ground removal by height fails on a stockpile: a pile at its 34° angle of repose climbs 0.67 m across a one-metre cell, so the bottom is called ground and the top third survives as slivers that cluster into vehicle-shaped objects. Tightening the tolerance only trades those false positives for missed objects near slopes.

So the ground surface is grown, not thresholded — seeded from the cells the machine drove through, flooding outward one repose-limited step at a time. A pile is continuous with the grade beneath it; a truck is a three-metre jump. Height stops mattering, continuity starts mattering.

2 · bootstrap_new_classes

Gives those instances names, from two sources that fail in opposite directions.

select views          where each instance projects largest and is in frame
  → detect            Grounding DINO, prompt "excavator . haul truck . worker ."
  → associate         detection ↔ projected 3D box, by IoU
  → vote              aggregate across views, confidence-weighted
  → size prior        for what the detector could not name

The detector names machines and cannot see people. Over ten unobstructed views, nine returned haul truck with the correct label; a worker visible in three of them was never detected once.Geometry finds people and cannot name machines. So the detector is asked only where it has a chance, and the size prior covers the rest.

Every name carries cls_conf and cls_source, because the two paths are not equally trustworthy — seethe numbers.

3 · improve_offboard_model

Distil a detector from existing labels, then use it to label better. Run it repeatedly and it is a self-training loop.

filter labels        → the load-bearing parameter, not a knob
  → training set     NPZ in the layout CenterPillars already reads
  → train student    CenterPillars.py
  → infer            CenterPillars.mojo
  → track            the same association and smoothing as round 0
  → score            against the held-out oracle

Two further stages are implemented and evaluated but not in the default path — OfflinePoly.mojo(Offline-Poly) andLabelFormer.mojo(LabelFormer). What holds each back is on the papers page.

Nothing human-labelled enters at any point. The output is the same schema as the input, so feeding one round's labels into the next is the loop.

Running them

python -m workflows geo_kinetic_discovery \
    --scene site.mcap --work runs/a --truth runs/a/truth.csv

python -m workflows bootstrap_new_classes \
    --scene site.mcap --work runs/a --labels runs/a/labels.csv

python -m workflows improve_offboard_model \
    --scene site.mcap --work runs/a --labels runs/a/labels.csv --round r1

Every stage declares its inputs, outputs and parameters; the run context hashes them and skips any stage a previous run already produced. Re-running a finished workflow costs a second and touches nothing. That is not only convenience — durable-execution engines replay handlers, so idempotent steps are what would make adopting one later a wrapping job rather than a rewrite. The manifest doubles as the provenance record.