CLEVR-HOPE

CLEVR-HOPE The CLEVR Held-Out Pair Evaluation (CLEVR-HOPE) dataset is a diagnostic dataset for testing the systematicity of VQA models. CLEVR-HOPE is a controlled setting to test whether VQA models generalize to pairs of attribute values that were not seen during either training or fine-tuning. Within CLEVR-HOPE, we refer to an unseen pair of attribute values as a Held-Out Pair (HOP). The dataset is composed of 29 sub-datasets, each for a different HOP. For each of the 29 HOPs… See the full description on the dataset page: https://huggingface.co/datasets/user9000/CLEVR-HOPE.