BERT L6 H768 (uncased) Mini BERT models from https://arxiv.org/abs/1908.08962 that the HF team didn't convert. The original conversion script is used. See the original Google repo: google research/bert Note: it's not clear if these checkpoints have undergone knowledge distillation. Model variants H=128 H=256 H=512 H=768 : : : : : : : : L=2 [2/128 (BERT Tiny)][2 128] [2/256][2 256] [2/512][2 512] [2/768][2 768] L=4 [4/128][4 128] [4/256 (BERT Mini)][4 256] [4/512 (BERT Small)][4 512] [4/768][4 768] L=6 [6/128][6 128] [6/256][6 256] [6/512][6 512] [ 6/768 ][6 768] L=8 [8/128][8 128] [8/256][8 256] [8/512 (BERT Medium)][8 512] [8/768][8 768] L=10 [10/128][10 128] [10/256][10 256] [10/512][10 512] [10/768][10 768] L=12 [12/128][12 128] [12/256][12 256] [12/512][12 512] [12/768 (BERT Base, original)][12 768] [2 128]: https://huggingface.co/gaunernst/bert tiny uncased [2 256]: https://huggingface.co/gaunernst/bert L2 H256 uncased [2 512]: https://huggingface.co/gaunernst/bert L2 H512 uncased [2 768]: https://huggingface.co/gaunernst/bert L2 H768 uncased [4 128]: https://huggingface.co/gaunernst/bert L4 H128 uncased [4 256]: https://huggingface.co/gaunernst/bert mini uncased [4 512]: http…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy