All MiniLM L6 v2 GGUF Original Model sentence transformers/all MiniLM L6 v2 Run with LlamaEdge LlamaEdge version: v0.8.2 and above Context size: 384 Vector size: 256 Run as LlamaEdge service Quantized GGUF Models Name Quant method Bits Size Use case all MiniLM L6 v2 Q2 K.gguf Q2 K 2 19.2 MB smallest, significant quality loss not recommended for most purposes all MiniLM L6 v2 Q3 K L.gguf Q3 K L 3 20.5 MB small, substantial quality loss all MiniLM L6 v2 Q3 K M.gguf Q3 K M 3 19.9 MB very small, high quality loss all MiniLM L6 v2 Q3 K S.gguf Q3 K S 3 19.2 MB very small, high quality loss all MiniLM L6 v2 Q4 0.gguf Q4 0 4 19.7 MB legacy; small, very high quality loss prefer using Q3 K M all MiniLM L6 v2 Q4 K M.gguf Q4 K M 4 21 MB medium, balanced quality recommended all MiniLM L6 v2 Q4 K S.gguf Q4 K S 4 20.7 MB small, greater quality loss all MiniLM L6 v2 Q5 0.gguf Q5 0 5 21 MB legacy; medium, balanced quality prefer using Q4 K M all MiniLM L6 v2 Q5 K M.gguf Q5 K M 5 21.7 MB large, very low quality loss recommended all MiniLM L6 v2 Q5 K S.gguf Q5 K S 5 21.5 MB large, low quality loss recommended all MiniLM L6 v2 Q6 K.gguf Q6 K 6 24.2 MB very large, extremely low quality loss all MiniLM…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy