MI-Reconstruction-Collection

MI-Reconstruction-Collection is a reconstruction dataset for model inversion evaluation. It contains AttackSamples/ organized by private dataset, attack method, public dataset, and target model.

Download

The evaluation code expects this dataset to live under an AttackSamples/ directory.

git clone https://huggingface.co/datasets/hosytuyen/MI-Reconstruction-Collection AttackSamples

Dataset structure

Example layout:

AttackSamples/
├── Facescrub/
├── CelebA/
├── Cifa100/
└── Stanford_Dogs/

Within each evaluation setup, you will typically find:

• all-images/: reconstructed images to evaluate
• private-data/: class-organized reference images used to build the comparison set
• result files such as gemini_results_<model>.csv or prediction.csv when available

Evaluation

Use the reference evaluation codebase here:

• hosytuyen/MI-Eval-MLLM

In that repo:

• use evaluation.ipynb to generate and run MLLM-based evaluation
• use analysis.ipynb to compare MLLM labels against conventional MI evaluation outputs

Links

• Paper: arXiv:2505.03519
• Evaluation Code: hosytuyen/MI-Eval-MLLM
• Project page: FMLLM