IgBert unpaired model Model pretrained on protein and antibody sequences using a masked language modeling (MLM) objective. It was introduced in the paper Large scale paired antibody language models. The model is finetuned from ProtBert BFD using unpaired antibody sequences from the Observed Antibody Space. Use The model and tokeniser can be loaded using the transformers library The tokeniser is used to prepare batch inputs Note that the tokeniser adds a [CLS] token at the beginning of each sequence, a [SEP] token at the end of each sequence and pads using the [PAD] token. For example a batch containing sequences E V V M , A L will be tokenised to [CLS] E V V M [SEP] and [CLS] A L [SEP] [PAD] [PAD] . Sequence embeddings are generated by feeding tokens through the model To obtain a sequence representation, the residue tokens can be averaged over like so For sequence level fine tuning the model can be loaded with a pooling head by setting add pooling layer=True and using output.pooler output in the down stream task.
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy