labelrefinery

Seeing the data and the labels

The scene and every round of labels are published. Both open inFoxglove with no custom panel and nothing to install.

Open it in the browser

Foxglove's web app can stream both files straight from where they are published — nothing to download, nothing to install.

Open the scene and every round of labels →Scene only →

The 86 MB recording is read over HTTP range requests, so playback starts without pulling the whole file. Then followwhat to turn on below.

If the viewer cannot reach the files, download them instead — the local path below is identical once they are open, and Foxglove merges the two files into one timeline either way.

Download

curl -O https://samples.magmalake.org/sitegen/v0.2.0/site_seed1_60s.mcap    #  86 MB
curl -O https://samples.magmalake.org/sitegen/v0.2.0/labels_rounds.mcap     # 471 KB

foxglove site_seed1_60s.mcap labels_rounds.mcap    # or drag both in at once

Foxglove merges multiple local files into a single playback timeline, so the labels line up with the recording frame for frame. Predictions live in their own file on purpose: the recording stays immutable, and one 86 MB scene serves any number of pipeline runs.

What to turn on

Add a 3D panel and set display frame tomap. Then:

TopicSchema
/lidar/pointsfoxglove.PointCloudthe sweep — set colour by to intensity
/ego/joint_statesfoxglove.JointStatesswing, boom, stick, bucket
/camera/<name>/imagefoxglove.CompressedImagefour cameras at the house corners
/terrain/heightmapfoxglove.PointCloudground and stockpiles, published once
/ground_truth/actorsfoxglove.SceneUpdateheld out — the oracle cuboids
/pred/*foxglove.SceneUpdateone topic per pipeline round, from the labels file

Scrub to t ≈ 25 s: round 0 has objects standing on the stockpile and round 2 does not. That is the 824 → 136 false-positive drop, visible rather than tabulated. Add a Plot panel on/ego/joint_states for the dig cycle — it repeats every 15 s, and the flat stretch at t ≈ 26–34 s is the machine walking to its second station.

/ground_truth/* is the oracle. It is in the file so you cansee what a labeler was up against — a labeler must never read it. Keeping the truth and the data in one recording is deliberate: the same pass wrote both, so the truth cannot drift from what it describes.

What is in the scene

A 20-tonne excavator running a dig–swing–dump cycle, loading an articulated hauler that backs in, is loaded and hauls off before a second truck takes its place. Part-way through, the machine walks 6.5 m along the trench to a second dig station — house squared up, boom tucked — and the hauler repositions with it. A spotter holds station; a second worker crosses the swing radius. Two stockpiles sit at the angle of repose, a row of grade stakes runs along the north edge, and dust kicks up when the loaded truck pulls away.

Four things are deliberate:

The excavator is articulated. Base pose plus swing, boom, stick and bucket — four degrees of freedom on top of the body. Every link is its own ground-truth box, so a scorer can ask whether a labeler got thebucket right, which is the part that will actually hit something.

Proprioception is a published topic. The machine's own joint angles are free, exact labels. The LiDAR rides the house, so15% of every sweep is the ego looking at itself, and forward kinematics is what removes it before anything else runs.

The difficulty is in the sensor, not the scene. Points fall off as 1/r², range noise grows with distance, dropout rises quadratically and dust eats returns over a region. A worker at 14 m gets about 20 points.

Nothing is balanced. Terrain is ~83% of returns; the two workers together are ~0.3%. Real class imbalance, not a curated benchmark.

Reproducing it

The scene is deterministic in its seed — two runs produce byte-identical files, verified by checksum, so two pipelines under comparison consume exactly the same input.

git clone https://github.com/labelrefinery/sitegen
cd sitegen && make install

uv run sitegen generate --out site.mcap --seed 1 --duration 60 --camera-hz 1
# sha256 b3140981...  the published file, byte for byte

sitegen overlay writes a label file from any tracker-schema CSV, so your own pipeline's output can be viewed the same way.