dataset v3 500 2026 05 03 500 case curated RCA evaluation set, rebuilt 2026 05 03 with the manifest driven causal graph reasoner (rcabench platform PR 370 / forge rework). Same 1464 case raw pool as v1/v2 ( pool v1 2026 05 02 , sourced from FSE/openrca2 + AegisLab detector success last 48h), but the ground truth causal graph.json for every case is regenerated by the new fault kind manifest expansion instead of the legacy rule only propagator. What changed vs v2 v2's causal graph.json came from the legacy rule only propagator. That propagator produced an average of 463 candidate paths per case (35× more than the new builder) and relied on per edge gates to drop the spurious ones at the end. The new manifest driven path builder enumerates layer by layer from the registered fault manifest, so the graph it ships with each case is what the fault contract itself says is propagation — not whatever post filtered out of an exhaustive walk. The shift in graph structure is uniform: smaller mean degree, fewer hallucinated transit nodes, and 41/500 v2 cases now have cyclic service graphs under the new reasoner (graph F1 metric becomes applicable=False for those, but the rest of the eval still s…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy