🦅 Supra 50M Instruct Supra 50M Instruct is a compact 50M parameter chat/instruct causal language model built by SupraLabs, trained from scratch using a Llama style architecture on 20 billion tokens of high quality educational web text. Despite being significantly smaller than comparable open models, it achieves competitive or superior results on several key benchmarks. It's our first SupraLabs Scaling Up Plan model. 📚 Training Data For the SFT (supervised finetuning) we used the full Alpaca Cleaned dataset for 4 epochs. See the full SFT code in sft.py Benchmarks Task Metric Value : : : : arc easy acc,none 0.4659 arc easy acc stderr,none 0.0102 arc easy acc norm,none 0.4423 arc easy acc norm stderr,none 0.0102 arc challenge acc,none 0.2287 arc challenge acc stderr,none 0.0123 arc challenge acc norm,none 0.2756 arc challenge acc norm stderr,none 0.0131 hellaswag acc,none 0.2794 hellaswag acc stderr,none 0.0045 hellaswag acc norm,none 0.2922 hellaswag acc norm stderr,none 0.0045 winogrande acc,none 0.5154 winogrande acc stderr,none 0.0140 piqa acc,none 0.5558 piqa acc stderr,none 0.0114 piqa acc norm,none 0.5952 piqa acc norm stderr,none 0.0115 openbookqa acc,none 0.1580 openbookqa…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy