🦅 Supra 50M Reasoning Supra 50M Reasoning is the reasoning version of Supra 50M Instruct. It does "reasoning" before every answer for proving better answers. 📚 Training Data For the SFT (supervised finetuning) for making the model reason, we used a custom synthetically dataset with 500 samples by Qwen3 1.7B for 6 epochs. See the full SFT code in sft.py and the dataset in data.jsonl 🏆 Benchmarks Category Benchmark Metric Score / Value Status Linguistics & Grammar BLiMP Accuracy 64.14% Success Commonsense & Reasoning PIQA Normalized Accuracy 59.47% Success COPA Accuracy 59.00% Success WinoGrande Accuracy 51.07% Success BoolQ Accuracy 46.06% Success TruthfulQA MC2 Accuracy 42.55% Success SWAG Normalized Accuracy 42.33% Success HellaSwag Normalized Accuracy 29.16% Success RACE Accuracy 27.85% Success CommonsenseQA Accuracy 21.46% Success Academic & Knowledge SciQ Normalized Accuracy 64.10% Success ARC Easy Normalized Accuracy 45.16% Success OpenBookQA Normalized Accuracy 28.80% Success ARC Challenge Normalized Accuracy 26.54% Success MMLU Accuracy 23.58% Success Language Modeling LAMBADA Accuracy 16.53% Success WikiText 2 Word Perplexity 166.27 Success 🧩 Answer Structure All answer…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy