japanese roberta base This repository provides a base sized Japanese RoBERTa model. The model was trained using code from Github repository rinnakk/japanese pretrained models by rinna Co., Ltd. How to load the model ~~~~ from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from pretrained("rinna/japanese roberta base", use fast=False) tokenizer.do lower case = True due to some bug of tokenizer config loading model = AutoModelForMaskedLM.from pretrained("rinna/japanese roberta base") ~~~~ How to use the model for masked token prediction Note 1: Use [CLS] To predict a masked token, be sure to add a [CLS] token before the sentence for the model to correctly encode it, as it is used during the model training. Note 2: Use [MASK] after tokenization A) Directly typing [MASK] in an input string and B) replacing a token with [MASK] after tokenization will yield different token sequences, and thus different prediction results. It is more appropriate to use [MASK] after tokenization (as it is consistent with how the model was pretrained). However, the Huggingface Inference API only supports typing [MASK] in the input string and produces less robust prediction…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy