MobileBERT fine tuned on SQuAD v2 MobileBERT is a thin version of BERT LARGE, while equipped with bottleneck structures and a carefully designed balance between self attentions and feed forward networks. This model was fine tuned from the HuggingFace checkpoint google/mobilebert uncased on SQuAD2.0. Details Dataset Split samples SQuAD2.0 train 130k SQuAD2.0 eval 12.3k Fine tuning Python: 3.7.5 Machine specs: CPU: Intel(R) Core(TM) i7 6800K CPU @ 3.40GHz Memory: 32 GiB GPUs: 2 GeForce GTX 1070, each with 8GiB memory GPU driver: 418.87.01, CUDA: 10.1 script: It took about 3.5 hours to finish. Results Model size : 95M Metric Value Original (Table 5) EM 75.2 76.2 F1 78.8 79.2 Note that the above results didn't involve any hyperparameter search. Example Usage Created by Qingqing Cao GitHub Twitter Made with ❤️ in New York.
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy