Swedish BERT Models The National Library of Sweden / KBLab releases three pretrained language models based on BERT and ALBERT. The models are trained on aproximately 15 20GB of text (200M sentences, 3000M tokens) from various sources (books, news, government publications, swedish wikipedia and internet forums) aiming to provide a representative BERT model for Swedish text. A more complete description will be published later on. The following three models are currently available: bert base swedish cased ( v1 ) A BERT trained with the same hyperparameters as first published by Google. bert base swedish cased ner ( experimental ) a BERT fine tuned for NER using SUC 3.0. albert base swedish cased alpha ( alpha ) A first attempt at an ALBERT for Swedish. All models are cased and trained with whole word masking. Files name files bert base swedish cased config, vocab, pytorch model.bin bert base swedish cased ner config, vocab pytorch model.bin albert base swedish cased alpha config, sentencepiece model, pytorch model.bin TensorFlow model weights will be released soon. Usage requirements / installation instructions The examples below require Huggingface Transformers 2.4.1 and Pytorch 1.3.…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy