CodeBERT base mlm Pretrained weights for CodeBERT: A Pre Trained Model for Programming and Natural Languages. Training Data The model is trained on the code corpus of CodeSearchNet Training Objective This model is initialized with Roberta base and trained with a simple MLM (Masked Language Model) objective. Usage Expected results: Reference 1. Bimodal CodeBERT trained with MLM+RTD objective (suitable for code search and document generation) 2. 🤗 Hugging Face's CodeBERTa (small size, 6 layers) Citation
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy