Table of contents 1. Introduction 2. Using PhoBERT with transformers Installation Pre trained models Example usage 3. Using PhoBERT with fairseq 4. Notes PhoBERT: Pre trained language models for Vietnamese Pre trained PhoBERT models are the state of the art language models for Vietnamese (Pho, i.e. "Phở", is a popular food in Vietnam): Two PhoBERT versions of "base" and "large" are the first public large scale monolingual language models pre trained for Vietnamese. PhoBERT pre training approach is based on RoBERTa which optimizes the BERT pre training procedure for more robust performance. PhoBERT outperforms previous monolingual and multilingual approaches, obtaining new state of the art performances on four downstream Vietnamese NLP tasks of Part of speech tagging, Dependency parsing, Named entity recognition and Natural language inference. The general architecture and experimental results of PhoBERT can be found in our paper: @inproceedings{phobert, title = {{PhoBERT: Pre trained language models for Vietnamese}}, author = {Dat Quoc Nguyen and Anh Tuan Nguyen}, booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020}, year = {2020}, pages = {1037 1042}…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy