APEX MTP Vision Apache-2.0

🤖 About Agents-A1

Agents-A1 is a 35B-parameter Mixture-of-Experts agentic model from InternScience, post-trained on top of Qwen3.5-35B-A3B via a three-stage paradigm: full-domain SFT → domain-level teacher training → multi-teacher multi-domain on-policy distillation.

Despite operating in the ~35B model class, Agents-A1 delivers highly competitive performance against frontier-scale systems such as GPT-5.5, DeepSeek-V4-pro, and Kimi-K2.6 — achieving SOTA on Seal-0 (56.4), HiPhO (46.4), FrontierScience-Olympiad (79.0), IFBench (80.6), IFEval (94.8), and best-among-comparable on BrowseComp (75.5), XBench-DS-2510 (86.0), GAIA (96.0), SciCode (44.3), HLE (47.6), and MolBench-bind (56.8).

This GGUF package includes the mmproj-Agents-A1-Uncensored-MTP-BF16.gguf vision projector for multimodal (image + text) capabilities with llama.cpp. MTP layers are extracted from Qwen3.5-35B-A3B and injected into Agents-A1's safetensors (see MTP Extraction & Injection section). License: Apache-2.0.

⚠️ Uncensored Adaptation

This release merges the selected behavior-modification LoRA into the BF16 base before MTP injection and APEX quantization. The uncensored adaptation was produced with abliterix. It is intended to reduce refusal behavior and can produce responses that differ materially from the original model.

Evaluation result	2 refusals / 100 prompts
KL divergence	0.016058

Use responsibly: evaluate the model for your deployment context and apply appropriate safeguards.

🧠 Model Details

Architecture	Qwen3.5 MoE (Mixture of Experts)
Parameters	35B total, 3B active per token
Experts	256 routed experts, 8 active per token
Layers	40 transformer layers + 1 MTP layer
Context	262,144 tokens
MTP Source	Qwen3.5-35B-A3B (1 layer, 785 tensors, injected)
Block Count	41 (blk.0–39 + blk.40 MTP)
License	Apache-2.0

🔧 MTP Extraction & Injection

The released InternScience/Agents-A1 checkpoint is a 40-layer Qwen3.5-35B-A3B MoE without MTP (Multi-Token Prediction) layers. To enable MTP acceleration in llama.cpp (which speeds up long-context generation by 10–30%), we extract the 1 MTP layer from Qwen3.5-35B-A3B and inject it into Agents-A1's safetensors before GGUF conversion.

The full pipeline has 4 steps: (1) Extract 785 MTP tensors from Qwen3.5-35B-A3B (filter keys containing "mtp"); (2) append as a new shard model-15-of-15.safetensors and update model.safetensors.index.json's metadata.total_size and weight_map (do not modify the original 14 shards); (3) convert to BF16 GGUF via the master-branch llama.cpp convert_hf_to_gguf.py — the master build auto-detects regular layers blk.0–39 + MTP layer blk.40.nextn.* (785 tensors); (4) quantize with APEX llama-quantize using qwen36_35b_mtp_*.txt configs which already include the blk.40 override (MTP kept at Q8_0 across all tiers — no manual patching needed). The imatrix Qwen3.5-35B-A3B.imatrix.gguf is reused directly (same architecture, compatible weights).

Reproduction command (example: I-Compact tier)

F:\llama.cpp\...\llama-quantize.exe ^
  --imatrix J:\Models\Qwen3.5-35B-A3B.imatrix.gguf ^
  --tensor-type-file E:\apex-quant\configs\qwen36_35b_mtp_compact.txt ^
  J:\Models\Agents-A1-Uncensored-MTP-GGUF\Agents-A1-Uncensored-MTP-BF16.gguf ^
  J:\Models\Agents-A1-Uncensored-MTP-APEX-GGUF\Agents-A1-Uncensored-MTP-APEX-I-Compact.gguf ^
  Q4_K_M

📊 BenchLocal Results (APEX-I-Compact, 16.14 GB)

Mode	ToolCall-15	BugFind-15	HermesAgent-20	Max	Eff.
Thinking	100	88	87	91.2	71.2
No Thinking	97	100	85	93.1	57.1

RTX 5070 Ti 16GB + 128GB RAM · No-thinking mode achieves higher ceiling (BugFind +12) but suffers more retries on complex agent scenarios.

🚀 Usage

llama.cpp (text only)

hf download SC117/Agents-A1-Uncensored-MTP-APEX-GGUF --include "*.gguf" --local-dir ./models
./llama-server -m ./models/Agents-A1-Uncensored-MTP-APEX-I-Compact.gguf -ngl 99 -c 131072

llama.cpp (vision + text)

./llama-server -m ./models/Agents-A1-Uncensored-MTP-APEX-I-Compact.gguf --mmproj ./models/mmproj-Agents-A1-Uncensored-MTP-BF16.gguf -ngl 99 -c 131072

vLLM

vllm serve SC117/Agents-A1-Uncensored-MTP-APEX-GGUF --port 8000 --tensor-parallel-size 1 --max-model-len 262144 --reasoning-parser qwen3
·
Tool-call variant
vllm serve SC117/Agents-A1-Uncensored-MTP-APEX-GGUF --port 8000 --tensor-parallel-size 1 --max-model-len 262144 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder

SGLang

python3 -m sglang.launch_server --model-path "SC117/Agents-A1-Uncensored-MTP-APEX-GGUF" --host 0.0.0.0 --port 30000

🎛️ Recommended Sampling Parameters

From the official Agents-A1 model card:

temperature	0.85
top_p	0.95
top_k	20
min_p	0.0
presence_penalty	1.1
repetition_penalty	1.0

💡 What is APEX?

These GGUF files are quantized using APEX, an MoE-aware mixed-precision quantization technique. APEX classifies every tensor by its role — routed expert, shared expert, SSM, or attention — and applies a layer-wise precision gradient, giving sensitive edge layers (including the MTP layer) higher precision and compressing redundant middle layers more aggressively.

APEX beats Q8_0 perplexity at half the size — and even beats F16 in some cases.

The qwen36_35b_mtp_*.txt configs include overrides for blk.40 (the MTP layer), preserving it at Q8_0 across all four I- tiers. The same Qwen3.5-35B-A3B.imatrix.gguf is reused (same architecture, compatible MoE expert layout).

📦 APEX Quantization Tiers

File	Size	Profile	Best For
*-APEX-I-Quality.gguf	21.75 GB	I-Quality	High quality (Q6_K + iq4_xs attention)
*-APEX-I-Balanced.gguf	24.21 GB	I-Balanced	Best all-rounder (Q6_K + Q5_K experts)
*-APEX-I-Compact.gguf	16.14 GB	I-Compact	Best quality/size ratio (Q4_K default)
*-APEX-I-Mini.gguf	13.36 GB	I-Mini	Most compact, fits in 16GB VRAM (Q3_K + iq2_s)

BF16 source: Agents-A1-Uncensored-MTP-BF16.gguf (66.19 GB). imatrix: Qwen3.5-35B-A3B.imatrix.gguf (reused from base model).