BERT large model (uncased) whole word masking finetuned on SQuAD Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. This model is uncased: it does not make a difference between english and English. Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same. The training is identical each masked WordPiece token is predicted independently. After pre training, this model was fine tuned on the SQuAD dataset with one of our fine tuning scripts. See below for more information regarding this fine tuning. Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by the Hugging Face team. Model description BERT is a transformers model pretrained on a large corpus of English data in a self supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic p…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy