BanglaBERT This repository contains the pretrained discriminator checkpoint of the model BanglaBERT . This is an ELECTRA discriminator model pretrained with the Replaced Token Detection (RTD) objective. Finetuned models using this checkpoint achieve state of the art results on many of the NLP tasks in bengali. For finetuning on different downstream tasks such as Sentiment classification , Named Entity Recognition , Natural Language Inference etc., refer to the scripts in the official GitHub repository. Note : This model was pretrained using a specific normalization pipeline available here. All finetuning scripts in the official GitHub repository uses this normalization by default. If you need to adapt the pretrained model for a different task make sure the text units are normalized using this pipeline before tokenizing to get best results. A basic example is given below: Using this model as a discriminator in transformers (tested on 4.11.0.dev0) Benchmarks Zero shot cross lingual transfer learning Model Params SC (macro F1) NLI (accuracy) NER (micro F1) QA (EM/F1) BangLUE score mBERT 180M 27.05 62.22 39.27 59.01/64.18 50.35 XLM R (base) 270M 42.03 72.18 45.37 55.03/61.83 55.29 XLM…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy