Qwen 2.5 72B Instruct Dynamic FP8 This is FP8 Dynamic Quantization (A8W8) for https://huggingface.co/Qwen/Qwen2.5 72B Instruct, we use it for vLLM==0.8.5.post1 and above.
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy