Qwen3.6-35B-Abliterated-Heretic — 4-bit AWQ Quantized Multimodal MoE

A 4-bit AWQ-quantized multimodal Mixture-of-Experts (MoE) model derived from Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-BF16. This is a quantized variant of Youssofal's Abliterated-Heretic-Uncensored model — an unrestricted, uncensored iteration of the Qwen3.6-35B family — compressed to 4-bit via Activation-aware Weight Quantization for deployment-friendly inference.

Model Type & Architecture

Vision Encoder

Quantization Details

The quantization strategy preserves critical components in full precision — including the language model head, all shared expert gating mechanisms across 40 layers, the vision-language merger, and the entire vision encoder — while quantizing only the language model layer weights. This hybrid approach maintains output quality while achieving significant memory reduction.

Multimodal Capabilities

This model supports multimodal inputs including:

• Text: Full text generation, understanding, and chat
• Images: Image-to-text via <|image|> tokens (ID: 248056)
• Video: Video understanding with up to 768 frames at 2 FPS via <|video|> tokens (ID: 248057)
• Audio: Audio token support (BOS/EOS defined)
• Tool Calling: Full function/tool calling with XML-style tool call formatting
• Reasoning: Chain-of-thought with <think>/</think> blocks

Chat Template

The model uses a sophisticated multimodal chat template with:

• Full tool/function calling support with XML-style formatting
• Reasoning block support (<thinking> / </think>)
• Image/video token insertion with optional numbering
• System message handling
• Multi-turn conversation with tool results

See chat_template.jinja for the complete template.

Tokenizer

See tokenizer_config.json for full configuration.

Generation Parameters

Quick Start — Inference Examples

Basic Text Generation

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "./Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit"

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

# Simple text generation
messages = [
    {"role": "user", "content": "Hello, can you tell me a story?"}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Multimodal (Image) Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
from PIL import Image

model_id = "./Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

# Load image
image = Image.open("path/to/image.jpg").convert("RGB")

# Multimodal conversation
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": "Describe this image in detail."}
        ]
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

# Note: For actual image processing, use Qwen3VLProcessor
# from transformers import Qwen3VLProcessor
# processor = Qwen3VLProcessor.from_pretrained(model_id)

inputs = processor(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
result = processor.decode(outputs[0], skip_special_tokens=True)
print(result)

Chat with Tool Calling

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "./Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

# Define tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name or coordinates"}
                },
                "required": ["location"]
            }
        }
    }
]

messages = [
    {"role": "system", "content": "You are a helpful assistant with weather lookup capabilities."},
    {"role": "user", "content": "What's the weather in Tokyo?"}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    tools=tools,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)
# The model may output a tool call in XML format:
# <function=get_weather>