TAAM — Typology Aware Adaptive Mixing — TAAM v2 seed 42 This is a BabyLM 2026 Multilingual Track submission checkpoint. It was trained on English + Dutch + Mandarin Chinese under a ≤100M unique token budget with the TAAM method. Method : TAAM v2 Seed : 42 Repo : amosluna/babylm 2026 taam v2 seed42 Final π (per language sampling probability): eng=0.157, nld=0.280, zho=0.563 Total token exposures : 655360000 Training wall clock : 10126.835841417313 s Source run dir : runs/2026 05 14 TAAM v2 seed42 Intermediate checkpoints This repo exposes 24 intermediate checkpoints as branches following the BabyLM 2026 naming convention: chck 1M, chck 2M, ..., chck 10M, chck 20M, ..., chck 100M, chck 200M, ..., chck 600M . The eval pipeline at babylm org/babylm eval pulls these revisions automatically with: Usage Method summary TAAM combines (a) a URIEL/lang2vec derived typological prior over initial sampling probabilities, (b) EXP3 online updates over per language sampling probabilities, and (c) byte premium aware token budgeting. The two reward variants are normalized excess loss (v1, delta based) and cross lingual deficit (v2, level based). See the paper and the public repo for full details, inc…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy