QMD Query Expansion Fine Tuning Train small language models to expand search queries for QMD's hybrid retrieval pipeline. What This Does Given a raw search query like "auth config" , the trained model produces structured expansions: These feed into QMD's three search backends: lex: lines go to BM25 full text search (short, keyword focused) vec: lines go to vector similarity search (natural language phrases) hyde: is a hypothetical document passage for embedding based retrieval (HyDE technique) Quick Start Cloud training via HuggingFace Jobs (no GPU needed) Local training (if you have a GPU) Monitoring HF Jobs Prompt Format All tools use the same prompt — Qwen3 chat template with /no think : The /no think directive suppresses Qwen3's chain of thought mode, producing direct lex:/vec:/hyde: output without blocks. File Structure Training Pipeline Stage 1: SFT (Supervised Fine Tuning) Teaches the model the lex:/vec:/hyde: output format from labeled examples. Parameter Value Base model Qwen/Qwen3 1.7B Method LoRA (rank 16, alpha 32) Target modules All projection layers (q/k/v/o/gate/up/down) Dataset ~2,290 examples (train split) Effective batch size 16 (4 × 4 gradient accumulation) Epoch…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy