Please use 'Bert' related functions to load this model! This repository contains the resources in our paper "Revisiting Pre trained Models for Chinese Natural Language Processing" , which will be published in "Findings of EMNLP". You can read our camera ready paper through ACL Anthology or arXiv pre print. Revisiting Pre trained Models for Chinese Natural Language Processing Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, Guoping Hu You may also interested in, Chinese BERT series: https://github.com/ymcui/Chinese BERT wwm Chinese ELECTRA: https://github.com/ymcui/Chinese ELECTRA Chinese XLNet: https://github.com/ymcui/Chinese XLNet Knowledge Distillation Toolkit TextBrewer: https://github.com/airaria/TextBrewer More resources by HFL: https://github.com/ymcui/HFL Anthology Introduction MacBERT is an improved BERT with novel M LM a s c orrection pre training task, which mitigates the discrepancy of pre training and fine tuning. Instead of masking with [MASK] token, which never appears in the fine tuning stage, we propose to use similar words for the masking purpose . A similar word is obtained by using Synonyms toolkit (Wang and Hu, 2017), which is based on word2vec (Mikolo…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy