Pleias 3b Preview is an early preview of a 3 billion parameters base model trained by Pleias on Common Corpus. Pleias 3b Preview was pretrained at Jean Zay (compute grant n°GC011015451) with support from Etalab Like all the base and specialized models from Pleias, Pleias 3b Preview has only been trained on open data out of copyright (public domain) or under a permissible license. Description Pleias 3b Preview is a transformer base model, entirely pretrained from scratch, using an architecture similar to Llama/GPT Neox for easier deployment/inference. It includes the following features, that would apply to any responsibly trained variant: Only trained on open data under a permissible license and in compliance with the European AI Act. By design, all Pleias model are unable to output copyrighted content. Extensive multilingual support for main European languages. A new tokenizer designed for enhanced document processing tasks and better multilingual support. Extremely low level of toxicity and problematic content. Fully supported languages include English, French, Spanish, German, Italian, Dutch, Latin and Portuguese. Recommended use As a base model, Pleias 3b Preview is only able to…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy