Numbers & learnings

Everything here was measured on one 60-second scene, scored against a held-out oracle at a 2 m centre-distance threshold. The interesting parts are the places the numbers disagreed with what had been written down first.

The loop is not monotonic

Round one made things worse. Trained on the raw pseudo-labels it inherited the systematic error and amplified it: 29.5% of teacher rows sat on a stockpile, and 37.4% of round-one output did. The label filter is not a refinement on the loop — it is what makes the loop work at all.

The student beats its own supervision

training labels   precision 0.865   recall 0.547
student output    precision 0.921   recall 0.656

Better on both axes. It recovered objects the filter had discarded and rejected false positives the filter had let through. Given clean enough examples the model's inductive bias works for you; below some label-precision floor it works against you. Somewhere between 0.653 and 0.865 is where this one flips — a more useful thing to know than any single score.

More precision is not always better, either. A stricter filter produced cleaner labels (0.942) and a worse student (F1 0.737), having thrown away a third of the training rows to get there.

The filter that worked was not the obvious one

Agreement — the obvious choice — is the weakest. In hindsight it could not have worked: the student was trained on those labels, so it corroborates the teacher's systematic errors rather than exposing them.

Motion works, and physically. A phantom is an artefact of which returns survived ground removal in one sweep, so it neither persists nor travels — median path length 0.75 m against8.26 m for a real object.

Terrain: connectivity beats thresholds

62% fewer false positives than the baseline, three true positives lost out of 1555, box quality unchanged. Pure region growing scores marginally better on precision and materially worse on everything about the boxes — a truck's lowest visible return sits inside the repose allowance, so the fill climbs onto vehicles and leaves floating tops. Connectivity decides where terrain extends; a per-cell thickness guard decides what is eligible to be terrain at all. Neither works alone.

Reading AOE properly

Orientation error runs 0 to π/2, and a uniformly random heading averages 0.785. Geometry's PCA heading scored0.750 — statistically indistinguishable from a coin flip. It carried no information at all. One round of distillation took it to0.553, which is genuinely informative.

That is the clearest case for the learned stages. A cluster is an object'svisible surface, not the object: a truck lit from one side has no far side in the cloud, so the box is short in depth and the principal axis follows the illuminated face. The missing information is not in the sweep to be extracted. It is in the trajectory, and in a size prior only a trained model carries.

Naming: one source is trustworthy, the other is not

Every name the detector gave was right. The size prior's failure is structural: it cannot abstain — shown a cluster of stockpile it will confidently return person. So an instance the detector was shown and did not corroborate is now recorded as unknownrather than guessed at. Detector silence is evidence.

Recall depends on what you count

The first honest score came out at recall 0.27, which felt wrong. Breaking the oracle down by class explained it: grade stakes are 60% of all ground-truth rows, and a stake is 50 mm square and collects a couple of returns per sweep. Excluding them, the same run scores0.69.

Nothing about the pipeline changed. The metric had been measuring the LiDAR rather than the labeler.

Synthetic pixels are not photographs

The control proves the model and prompt are fine. Untextured geometry does not carry what an open-vocabulary detector keys on. Enough to build a naming pipeline on; not enough to trust its labels.

Camera placement mattered more than expected. A single centre-forward camera spent 22.8% of every frame looking at its own boom, and that boom captured the only detection it made. Four cameras at the house corners, panned outward: 0.0% ego occlusion at every swing angle — and the label changed from “excavator haul” on the machine's own boom to haul truck on the actual truck.

What the numbers changed

Five times a measurement contradicted what had already been written down, and each time the write-up changed rather than the number:

• The obvious LiDAR mount fired two thirds of its beams into its own cab — caught by checking the return histogram, not by looking at a render.
• Recall was measuring the sensor, not the method.
• Thresholding cells was the wrong shape of answer; the question was connectivity.
• “More refinement will help” — it did not. Adding an offline tracking stage improved what it derives from trajectory and degraded what it takes on faith from the boxes.
• The label filter expected to work was the weakest of three.

The full build log, including the bugs that plausible-looking numbers nearly hid, is inRefinery's journal.