HyenaDNA Welcome! HyenaDNA is a long range genomic foundation model pretrained on context lengths of up to 1 million tokens at single nucleotide resolution . See below for an overview of the model and training. Better yet, check out these resources. Resources: arxiv blog colab github Links to all HuggingFace models: We've uploaded a collection of all the pretrained HyenaDNA checkpoints. You'll see models of different sizes and sequence lengths. There are also original weights only versions of each model in the LongSafari organization, which are designed to be loaded with the original github repo. These models have identical outputs to the models in the collection above, just different interfaces. See GPU requirements for each model. Using HyenaDNA In this brief code sample we demonstrate fine tuning HyenaDNA on a sequence classification task. This sample uses the medium checkpoint, with a maximum sequence length of 160k nucleotides. Note that training will fail if you use a sequence length longer than the maximum supported length for your chosen checkpoint. In testing, we have been able to train at a sequence length up to about 250k nucleotides on a Colab T4 GPU (16GB VRAM). For lo…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy