JuriBERT: A Masked Language Model Adaptation for French Legal Text Introduction JuriBERT is a set of BERT models (tiny, mini, small and base) pre trained from scratch on French legal domain specific corpora. JuriBERT models are pretrained on 6.3GB of legal french raw text from two different sources: the first dataset is crawled from Légifrance and the other one consists of anonymized court’s decisions and the pleadings from the Court of Cassation (mémoires ampliatifs). The latter contains more than 100k long documents from different court cases. It is now on Hugging Face in four different versions with varying number of parameters. JuriBERT Pre trained models Model params Architecture dascim/juribert tiny 6M Tiny (L=2, H=128, A=2) dascim/juribert mini 15M Mini (L=4, H=256, A=4) dascim/juribert small 42M Small (L=6, H=512, A=8) dascim/juribert base 110M Base (L=12, H=768, A=12) JuriBERT Usage Load JuriBERT and its sub word tokenizer : Filling masks using pipeline Extract contextual embedding features from JuriBERT output Authors JuriBERT was trained and evaluated at École Polytechnique in collaboration with HEC Paris by Stella Douka, Hadi Abdine, Mihcalis Vazirgiannis, Rajaa El Hamd…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy