SeaIce-Bench

Dataset description

SeaIce-Bench is a multimodal benchmark for evaluating large language models (LLMs) and multimodal large language models (MLLMs) on sea-ice analysis and prediction tasks. The dataset combines multispectral satellite imagery, monthly sea-ice extent records, weekly sea-ice motion vectors, and textual question-answer pairs.

The image data were collected between 2017 and 2026 near the Arctic and Antarctic polar regions. The numerical data cover both the Northern and Southern Hemispheres through 40-year moving windows.

SeaIce-Bench can be used to evaluate a model's ability to:

• interpret original-size and cropped multispectral satellite images;
• estimate or analyze sea-ice conditions from visual inputs;
• reason over long-term monthly sea-ice extent records;
• analyze and forecast weekly sea-ice motion vectors; and
• answer domain-specific textual questions about sea ice.

Dataset structure

.
|-- dataset_img_ori_png/
|   |-- images/
|       |-- IMG_DATA1/
|       |   |-- <12 spectral-band PNG files>
|       |   |-- <1 TCI PNG file>
|       |-- ...
|       |-- IMG_DATA500/
|-- dataset_img_png/
|   |-- images/
|       |-- IMG_DATA1/
|       |   |-- <12 spectral-band PNG files>
|       |   |-- <1 TCI PNG file>
|       |-- ...
|       |-- IMG_DATA10000/
|-- ice_extent/
|   |-- {year}_{month}_{polar_region}.csv
|-- motion_vector_weekly/
|   |-- {index}_{polar_region}_{vector_component}.csv
|-- q_and_a/
    |-- QA_low.csv
    |-- QA_high.csv

dataset_img_ori_png

This directory contains satellite images at their original spatial dimensions and resolution. The source imagery is converted from JPEG 2000 to PNG because current LLM and MLLM interfaces generally do not accept the original JPEG 2000 format.

The dataset_img_ori_png/images directory contains 500 scenes. Each scene is stored in a directory named IMG_DATA{index} and contains:

• 12 spectral-band PNG images; and
• 1 true-color image (TCI) in PNG format.

In total, this subset contains 6,500 PNG files across 500 scene directories.

dataset_img_png

This directory contains cropped satellite images in PNG format. The dataset_img_png/images directory contains 10,000 locations. Each location is stored in a directory named IMG_DATA{index} and contains:

• 12 spectral-band PNG images; and
• 1 true-color image (TCI) in PNG format.

In total, this subset contains 130,000 PNG files across 10,000 location directories.

ice_extent

This directory contains monthly sea-ice extent data for both hemispheres. Each CSV file represents a 40-year moving window and follows this naming convention:

{year}_{month}_{polar_region}.csv

The filename records the starting year, starting month, and polar region of the window. For example:

1978_11_N.csv

This file contains 40 years of monthly sea-ice extent data beginning in November 1978 for the Northern Hemisphere.

motion_vector_weekly

This directory contains weekly sea-ice motion-vector data for both hemispheres. Each CSV file represents a 40-year moving window for one vector component and follows this naming convention:

{index}_{polar_region}_{vector_component}.csv

Here:

• {index} identifies the starting week;
• {polar_region} identifies the Northern or Southern Hemisphere; and
• {vector_component} is either u or v.

For example:

0_north_u.csv

This file contains 40 years of weekly data for the u component of sea-ice motion, beginning with the first week of 1979 in the Northern Hemisphere.

q_and_a

This directory contains textual question-answer data in two CSV files:

• QA_low.csv: question-answer pairs that have not been curated by domain experts;
• QA_high.csv: question-answer pairs that have been curated by domain experts.

Each file contains the following fields: