La descripción en Español se encuentra después de la descripción en Inglés. (English) GPT2 small spanish: a Language Model for Spanish text generation (and more NLP tasks...) GPT2 small spanish is a state of the art language model for Spanish based on the GPT 2 small model. It was trained on Spanish Wikipedia using Transfer Learning and Fine tuning techniques . The training took around 70 hours with four GPU NVIDIA GTX 1080 Ti with 11GB of DDR5 and with around 3GB of (processed) training data. It was fine tuned from the English pre trained GPT 2 small using the Hugging Face libraries (Transformers and Tokenizers) wrapped into the fastai v2 Deep Learning framework. All the fine tuning fastai v2 techniques were used. The training is purely based on the GPorTuguese 2 model developed by Pierre Guillou. The training details are in this article: "Faster than training from scratch — Fine tuning the English GPT 2 in any language with Hugging Face and fastai v2 (practical case with Portuguese)". This preliminary version is now available on Hugging Face. Limitations and bias (Copied from original GPorTuguese 2 model)The training data used for this model come from Spanish Wikipedia. We know i…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy