Quasar Foundation Models (RoPE Base) Quasar Foundation Models are SILX AI’s core models designed for long context reasoning , agentic systems , and persistent memory based intelligence . This release is NOT a state of the art final model . It is a base pretraining model designed specifically for distributed knowledge distillation on Bittensor (SN24 Quasar subnet) . The goal is to create a shared architecture where miners continuously distill knowledge from frontier models (e.g., Qwen, GLM) into Quasar. ⚠️ Important Note This model is: A base model Pretrained for only a few billion tokens Designed for distillation and scaling , not benchmarking Performance will improve through iterative subnet training + distillation cycles . Model Overview Model Name: Quasar 3B (RoPE Base) Organization: SILX AI Architecture: Quasar RoPE Hybrid Transformer Total Parameters: 3B Active Parameters: ~1B (Mixture of Experts) Training Stage: Stage 1 (Base Pretraining) Sequence Length: 16K tokens (RoPE phase) Training Strategy Quasar follows a multi stage training pipeline : Stage 1 — RoPE Pretraining Train using Rotary Positional Embeddings (RoPE) Context length: 16K tokens Objective: stabilize training a…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy