BETO: Spanish BERT BETO is a BERT model trained on a big Spanish corpus. BETO is of size similar to a BERT Base and was trained with the Whole Word Masking technique. Below you find Tensorflow and Pytorch checkpoints for the uncased and cased versions, as well as some results for Spanish benchmarks comparing BETO with Multilingual BERT as well as other (not BERT based) models. Download : : : : : : BETO uncased tensorflow weights pytorch weights vocab, config BETO cased tensorflow weights pytorch weights vocab, config All models use a vocabulary of about 31k BPE subwords constructed using SentencePiece and were trained for 2M steps. Benchmarks The following table shows some BETO results in the Spanish version of every task. We compare BETO (cased and uncased) with the Best Multilingual BERT results that we found in the literature (as of October 2019). The table also shows some alternative methods for the same tasks (not necessarily BERT based methods). References for all methods can be found here. Task BETO cased BETO uncased Best Multilingual BERT Other results : : : : POS 98.97 98.44 97.10 [2] 98.91 [6], 96.71 [3] NER C 88.43 82.67 87.38 [2] 87.18 [3] MLDoc 95.60 96.12 95.70 [2] 8…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy