OmniReasoner-SFT

OmniReasoner-SFT is a mixed-source, research-only supervised fine-tuning dataset for audio-visual and long-video reasoning. It contains two-stage cold-start SFT trajectories with interval selection, zoom-in evidence, and final answers.

Contents

• data/train.jsonl: HF-ready training JSONL with repo-relative media paths.
• media/: raw and derived media referenced by train.jsonl.
• manifests/media_manifest.jsonl: media inventory with repo paths, source family, media type, file size, and reference counts.
• manifests/dataset_stats.json: dataset and media statistics.
• configs/: lmms-engine SFT configs and launch scripts used for training.
• scripts/materialize_lmms_jsonl.py: rewrites repo-relative paths into local absolute file:// paths after download.

Source Families

This dataset includes both original-source and derived media:

• CG-Bench long videos.
• FineVideo videos.
• AVQA-R1 image/audio samples derived from OmniInstruct/AVQA/VGGSound sources.
• Composition videos generated by concatenating multiple FineVideo videos.
• Anomaly videos generated by inserting LLaVA-Video-178K / scaled-source anomaly clips into CG-Bench long videos, with anomaly type and ground-truth intervals.

The final SFT trajectories were produced with Gemini hindsight generation and then filtered with leakage checks, hard rules, judge scoring, and fusion.

License And Usage

The annotations, reasoning traces, metadata, and construction recipes are released by the dataset authors for non-commercial academic research only.

Raw and derived media retain the copyright and usage restrictions of their source datasets, original creators, and original platforms. Access to this repository does not grant additional rights beyond those source licenses and terms. Users must not redistribute the raw or derived media.

If you are a rights holder and want a file removed, please contact the dataset authors through the Hugging Face repository discussion page.

Citations And Source Notices

Users should cite this dataset and the applicable upstream datasets, including:

• CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding.
• FineVideo and YouTube-Commons where applicable.
• LLaVA-Video-178K for anomaly inserted-source clips where applicable.
• AVQA-R1 / EchoInk-R1, OmniInstruct/OmniBench, AVQA, and VGGSound where applicable.

Local Training

data/train.jsonl stores relative paths such as media/videos/.... To recreate the absolute file:// format expected by the original lmms-engine training environment after downloading this repository, run:

python scripts/materialize_lmms_jsonl.py \
  --input data/train.jsonl \
  --output data/train.local.jsonl \
  --root "$(pwd)"

Then point the lmms-engine config to data/train.local.jsonl.