ALMA ( A dvanced L anguage M odel based tr A nslator) is an LLM based translation model, which adopts a new translation model paradigm: it begins with fine tuning on monolingual data and is further optimized using high quality parallel data. This two step fine tuning process ensures strong translation performance. Please find more details in our paper. ALMA R (NEW!) is released now! ALMA R builds upon ALMA models, with further LoRA fine tuning with our proposed Contrastive Preference Optimization (CPO) as opposed to the Supervised Fine tuning used in ALMA. CPO fine tuning requires our triplet preference data for preference learning. ALMA R now can matches or even exceeds GPT 4 or WMT winners! We release six translation models presented in the paper: ALMA 7B : Full weight Fine tune LLaMA 2 7B on 20B monolingual tokens and then Full weight fine tune on human written parallel data ALMA 7B LoRA : Full weight Fine tune LLaMA 2 7B on 20B monolingual tokens and then LoRA fine tune on human written parallel data ALMA 7B R (NEW!) : Further LoRA fine tuning upon ALMA 7B LoRA with contrastive preference optimization. ALMA 13B : Full weight Fine tune LLaMA 2 7B on 12B monolingual tokens and th…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy