Papers

Each of these is its own repository — a from-scratch implementation, most in pure Mojo, several with a PyTorch reference alongside and numerical parity verified stage by stage.

In the pipelines today

STONE: A Scalable Multi-Modal Surround-View 3D Traversability Dataset for Off-Road Robot Navigation

Park et al., ICRA 2026 · Stone.mojo · used in geo_kinetic_discovery

Per-cell plane fits over a 2.5-D grid, calibrated from the machine's own driven path. Supplies the terrain stage — adapted to reason about connectivity rather than traversability, because slope separates drivable ground from a pile and must not separate terrain from an object.

A shortest augmenting path algorithm for dense and sparse linear assignment problems

Jonker & Volgenant, Computing 38, 1987 · Hungarian.mojo · used in all three pipelines

Data association, frame to frame. Extended with a distance gate so a track whose object left the scene goes unmatched rather than being bound to whatever is cheapest.

Maximum likelihood estimates of linear dynamic systems

Rauch, Tung & Striebel, AIAA Journal 3(8), 1965 · Kalman.mojo · used in geo_kinetic_discovery, improve_offboard_model

The RTS backward pass — the offboard advantage in one algorithm. Every frame is conditioned on the whole trajectory, so early distant detections inherit the accuracy of later close ones.

Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

Liu et al., ECCV 2024 · GroundingDino.mojo · used in bootstrap_new_classes

Names instances from a text prompt. Measured on this data: perfect precision on the machines it does detect, and blind to people — which is why the size prior sits beside it rather than under it.

PointPillars: Fast Encoders for Object Detection from Point Clouds

Lang et al., CVPR 2019 · CenterPillars · used in improve_offboard_model

The pillar encoder — points scattered into a BEV grid.

SECOND: Sparsely Embedded Convolutional Detection

Yan et al., Sensors 18(10), 2018 · CenterPillars · used in improve_offboard_model

The BEV backbone with a multi-scale concat neck.

Center-based 3D Object Detection and Tracking

Yin et al., CVPR 2021 · CenterPillars · used in improve_offboard_model

The centre head — per-class Gaussian heatmaps plus box regression. No anchors, no rotated-IoU NMS. This is the distilled student that turns geometry's pseudo-labels into a model with a shape prior.

Implemented, not yet in the default path

These are complete implementations. What holds each one back is stated plainly rather than left as “future work”.

Offline-Poly: A Polyhedral Framework For Offline 3D Multi-Object Tracking

Li et al., 2026 · OfflinePoly.mojo · Evaluated, not currently in the default path

Learning-free offline MOT that fuses tracklets from several upstream trackers. Measured here: it improves what it derives from trajectory (heading) and degrades what it takes on faith from the boxes (size), because it was designed downstream of a detector that emits well-formed boxes. A good stage waiting for a good input.

LabelFormer: Object Trajectory Refinement for Offboard Perception from LiDAR Point Clouds

Yang et al., CoRL 2023 · LabelFormer.mojo · Needs a domain checkpoint

Refines a whole trajectory rather than a frame — which is exactly what orientation error needs, and the clearest remaining target on this data.

Offline Tracking with Object Permanence

Liu & Caesar, 2023 · TrackPermanence.mojo · Needs a domain checkpoint

Re-identifies a terminated tracklet with a later one and regresses the occluded poses between — the stage aimed at the dust and stockpile occlusions this scene generates deliberately.

Datasets referenced

Not implemented — cited because they shaped decisions.

3D Point Cloud Dataset of Heavy Construction Equipment (3D-ConHE)

CAD models of excavators, dump trucks, dozers, graders and rollers. Raycasting these instead of cuboids is the clearest path to narrowing the synthetic-imagery gap.

Rohbau3D: A Shell Construction Site 3D Point Cloud Dataset

504 real LiDAR scans across 14 construction sites. Static, so nothing to track — but real backgrounds.

Argoverse 2

What the PyTorch reference implementations train on, and the reason their detector class lists have never heard of an excavator boom.

Every repository