bacformer large masked MAG 2025 05 15 : We identified a bug in the Bacformer Large code on HuggingFace which resulted in a significant drop in the quality of the output embeddings. This is now fixed, but if you downloaded or cached the model before this date, re download and use the latest model revision before running evaluations. KEY CHANGES from Bacformer base (26M) : Number of parameters: Bacformer Large is much larger ( 300M parameters) Updated Transformer architecture: we use SwigLU activation function Larger protein language base model: we use ESM C 300M No special tokens Changed the argument name in forward() from token type ids contig ids Bacformer is a foundational model for bacterial genomics, modeling the whole bacterial genome as a sequence of proteins. Bacformer takes as input the set of proteins present in a genome, ordered by their positions on the chromosome and plasmid(s), and computes contextual protein representations with a transformer that conditions each protein on every other protein in the genome. Each protein is treated as a token; thus, Bacformer may be viewed as a contextualized protein language model that captures the protein–protein interactions underl…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy