distilbert base multilingual cased sentiments student This model is distilled from the zero shot classification pipeline on the Multilingual Sentiment dataset using this script. In reality the multilingual sentiment dataset is annotated of course, but we'll pretend and ignore the annotations for the sake of example. Teacher model: MoritzLaurer/mDeBERTa v3 base mnli xnli Teacher hypothesis template: "The sentiment of this text is {}." Student model: distilbert base multilingual cased Inference example Training procedure Notebook link: here Training hyperparameters Result can be reproduce using the following commands: If you are training this model on Colab, make the following code changes to avoid Out of memory error message: Training log Framework versions Transformers 4.28.1 Pytorch 2.0.0+cu118 Datasets 2.11.0 Tokenizers 0.13.3
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy