ESM 1b ESM 1b (paper, repository) is a transformer protein language model, trained on protein sequence data without label supervision. The model is pretrained on Uniref50 with an unsupervised masked language modeling (MLM) objective, meaning the model is trained to predict amino acids from the surrounding sequence context. This pretraining objective allows ESM 1b to learn generally useful features which can be transferred to downstream prediction tasks. ESM 1b has been evaluated on a variety of tasks related to protein structure and function, including remote homology detection, secondary structure prediction, contact prediction, and prediction of the effects of mutations on function, producing state of the art results. Important note : ESM 2 is now available in a range of checkpoint sizes. For most tasks, ESM 2 performance will be superior to ESM 1 and ESM 1b, and so we recommend using it instead unless your goal is explicitly to compare against ESM 1b. The ESM 2 checkpoint closest in size to ESM 1b is esm2 t33 650M UR50D. Model description The ESM 1b model is based on the RoBERTa architecture and training procedure, using the Uniref50 2018 03 database of protein sequences. Note t…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy