Qwen3 Reranker 4B seq cls vllm fixed This is a fixed version of the Qwen3 Reranker 4B model converted to sequence classification format, optimized for use with vLLM. Model Description This model is a pre converted version of Qwen/Qwen3 Reranker 4B that: Has been converted from CausalLM to SequenceClassification architecture Includes proper configuration for vLLM compatibility Provides ~75,000x reduction in classification head size Offers ~150,000x fewer operations per token compared to using the full LM head Key Improvements The original converted model (tomaarsen/Qwen3 Reranker 4B seq cls) was missing critical vLLM configuration attributes. This version adds: These configurations are essential for vLLM to properly handle the pre converted weights. Usage with vLLM Python Example Performance This model performs identically to the original Qwen3 Reranker 4B when used with proper configuration, while providing significant efficiency improvements: Memory : ~600MB → ~8KB for classification head Compute : 151,936 logits → 1 logit per forward pass Speed : Faster inference due to reduced computation Technical Details Architecture : Qwen3ForSequenceClassification Base Model : Qwen/Qwen3 Rer…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy