Trinity Mini Trinity Mini is an Arcee AI 26B MoE model with 3B active parameters. It is the medium sized model in our new Trinity family, a series of open weight models for enterprise and tinkerers alike. This model is tuned for reasoning, but in testing, it uses a similar total token count to competitive instruction tuned models. Trinity Mini is trained on 10T tokens gathered and curated through a key partnership with Datology, building upon the excellent dataset we used on AFM 4.5B with additional math and code. Training was performed on a cluster of 512 H200 GPUs powered by Prime Intellect using HSDP parallelism. More details, including key architecture decisions, can be found on our blog here Try it out now at chat.arcee.ai Model Details Model Architecture: AfmoeForCausalLM Parameters: 26B, 3B active Experts: 128 total, 8 active, 1 shared Context length: 128k Training Tokens: 10T License: OpenMDW 1.1 Recommended settings: temperature: 0.15 top k: 50 top p: 0.75 min p: 0.06 Benchmarks Running our model Transformers VLLM llama.cpp LM Studio API Transformers Use the main transformers branch If using a released transformers, simply pass "trust remote code=True": VLLM Supported in V…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy