vLLM Compatible TranslateGemma 12B This is a modified version of google/translategemma 12b it optimized for deployment with vLLM. No retraining was performed. Only configuration files and the chat template were modified. Model weights are identical to the original. Why This Exists As of 2025 01 29, vLLM does not natively support TranslateGemma's custom structured input format. See vllm project/vllm 32446 for the upstream tracking issue. Until that is merged, this repo provides a workaround by modifying configuration files to make TranslateGemma compatible with vLLM's standard chat API. Acknowledgements This conversion is based entirely on the work done by Infomaniak AI/vllm translategemma 4b it. The same conversion approach was applied to the 12B model. Full credit goes to the Infomaniak AI team for figuring out the necessary changes. Modified Files The following files were modified from the original google/translategemma 12b it for vLLM compatibility. All other files (model weights, tokenizer, etc.) are unchanged. config.json — RoPE parameters simplified for vLLM generation config.json — Added temperature, bos token id, max length; adjusted eos token id ordering chat template.jinj…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy