Data on Trial — Benchmark Artifacts
Benchmark artifacts for the data_on_trial jury-system pipeline. The pipeline code lives on GitHub: QuiZet/data_on_trial.
These files are .gitignored in the code repo because of their size, and mirror the
repository's directory layout so they can be dropped back in place.
Contents
| Path | Description |
|---|---|
datasets/ | Downloaded / generated HF datasets used as pipeline inputs (Arrow/JSON). |
experiments/ | Experiment snapshots for MMLU-Pro & GPQA: judge results, calibration configs, recovery runs. |
analysis/results/ | Aggregated analysis outputs. |
analysis/exp/results/ | Logs and outputs from the e1–e13 experiment suite. |
analysis/exp/figures/ | Generated figures (PNG). |
datagen_related/ | Archive — full snapshot of the original production run (the pre-refactor datagen_related/data_on_trial working dir): ~1.4 GB of judge results, raw generations, calibration outputs, logs, and SLURM scripts across GPQA & MMLU-Pro. Preserved before the compute cluster was decommissioned. |
Usage
Clone the code and rehydrate the artifacts alongside it:
git clone https://github.com/QuiZet/data_on_trial
cd data_on_trial
hf download yungisimon/data_on_trial --repo-type dataset --local-dir .
Notes
No API keys or secrets are included in this dataset. See the GitHub repository for pipeline code, configuration, and reproduction instructions.