Model Details This model is a int4 model with group size 128 of Qwen/Qwen3.6 27B generated by intel/auto round. Please follow the license of the original model. vllm Inference Example ~~~bash vllm serve Intel/Qwen3.6 27B int4 AutoRound port 8000 tensor parallel size 1 max model len 2048 reasoning parser qwen3 served model name qwen speculative config '{"method":"qwen3 next mtp","num speculative tokens":2}' curl http://localhost:8000/v1/chat/completions H "Content Type: application/json" d ' { "model": "qwen", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Summarize Qwen 3.6 in one sentence."} ], "temperature": 1, "max tokens": 512 } ' ~~~ Transformers Inference Example Generate the Model Ethical Considerations and Limitations The model can produce factually incorrect output, and should not be relied on to produce factually accurate information. Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs. Therefore, before deploying any applications of the model, developers should perform safety testing. Caveats and…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy