Huginn 0125 This is Huginn, version 01/25, a latent recurrent depth model with 3.5B parameters, trained for 800B tokens on AMD MI250X machines. This is a proof of concept model, but surprisingly capable in reasoning and code given its training budget and size. All details on this model can be found in the tech report: "Scaling up Test Time Compute with Latent Reasoning: A Recurrent Depth Approach." (https://www.arxiv.org/abs/2502.05171) For more information, see the paper page: https://huggingface.co/papers/2502.05171. 8 intermediate checkpoints of the model can be found in its collection. Additional intermediate checkpoints are available upon request while we find a place to host all ~350 of them. The data used to train this model is publicly available (entirely on Hugging Face), and scripts provided with the pretraining code at https://github.com/seal rg/recurrent pretraining can be used to repeat our preprocessing and our entire training run. Table of Contents 1. How to Use 2. Advanced Usage 3. Model Summary 4. Limitations 5. Technical Details 6. License 7. Citation Downloading and Using the Model Load the model like this: Modifying the Model's Depth at Test Time: By providing t…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy