Model Details This model is an int4 model with group size 128 of Qwen/Qwen3.5 122B A10B generated by intel/auto round. Please follow the license of the original model. The main branch provides the tuned quantized model, and revision 3f4ba63 corresponds to the RTN version. In general, the tuned version is preferred; however, it has not been validated for this particular model. vllm Infernece Example ~~~bash pip install git+https://github.com/vllm project/vllm.git@main pip install git+https://github.com/huggingface/transformers.git ~~~ ~~~bash vllm serve Intel/Qwen3.5 122B A10B int4 AutoRound port 8000 tensor parallel size 1 max model len 2048 reasoning parser qwen3 served model name qwen ~~~ ~~~bash curl http://localhost:8000/v1/chat/completions H "Content Type: application/json" d ' { "model": "qwen", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Summarize Qwen 3.5 in one sentence."} ], "temperature": 1, "max tokens": 512 } ' ~~~ Transformers Inference Transformers v5.2.0 auto round and This pr is required https://github.com/intel/auto round/pull/1476 ~~~python from transformers import AutoProcessor, Qwen3 5MoeForCondition…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy