Counterfactual VLM Benchmark Data

This dataset repository contains the data payload for the Counterfactual VLM Benchmark.

Uploaded at: 20260520T092554Z

• Code repository: https://github.com/JingyuSunUOE/counterfactual-vlm-benchmark
• Dataset repository: https://huggingface.co/datasets/JingyuSun/counterfactual-vlm-benchmark-data

Use this dataset together with the GitHub repository. From the repository root, run:

python scripts/download_hf_dataset.py \
  --repo-id JingyuSun/counterfactual-vlm-benchmark-data \
  --local-dir .

Contents

dataset/                         Original images for non-medical benchmarks
cf_dataset/                      Counterfactual images and question JSON files
vision_dataset/                  Generated visual evidence
medical/modality_swapping/        Medical question JSON and generated medical visualizations, if included
eval_results/*/metadata/          Runtime metadata and evidence manifests needed by evaluators
MANIFEST.json                     Release manifest and payload summary

Not Included

This release intentionally does not include:

• eval_results/*/raw_runs, reports, tables, figures, or raw API outputs
• BraTS source NIfTI files under medical/BraTS2023_GLI/
• API keys, provider caches, or local environment files

The included eval_results/*/metadata/ directories are lightweight runtime artifacts used by the evaluation scripts. They are included so a fresh GitHub checkout plus this dataset payload has the same directory layout expected by the benchmark runners.

Size Summary

Total files: 45383

Total size: 6.3GB

Medical Data Note

BraTS source data is not redistributed. Any medical generated images in this dataset should be used only under the relevant source-data license constraints.