prudant/Qwen3 Reranker 0.6B seq cls W8A8 This is a compressed version of tomaarsen/Qwen3 Reranker 0.6B seq cls using llm compressor with the following scheme: W8A8 Serving python3 m vllm.entrypoints.openai.api server model 'dolfsai/Qwen3 Reranker 0.6B seq cls vllm W8A8' task classify Important : You MUST read the following guide for correct usage of this model here Guide Model Details Original Model : tomaarsen/Qwen3 Reranker 0.6B seq cls Quantization Method : GPTQ Compression Libraries : llm compressor Calibration Dataset : ultrachat 200k (2048 samples) Optimized For : Inference with vLLM License : same as original model
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy