Advancing Open source Language Models with Mixed Quality Data Online Demo GitHub Paper Discord Sponsored by RunPod Llama 3 Version: OPENCHAT 3.6 20240522 🏆 The Overall Best Performing Open source 8B Model 🏆 🚀 Outperforms Llama 3 8B Instruct and open source finetunes/merges 🚀 Llama 3 Instruct often fails to follow the few shot templates. See example . Usage To use this model, we highly recommend installing the OpenChat package by following the installation guide in our repository and using the OpenChat OpenAI compatible API server by running the serving command from the table below. The server is optimized for high throughput deployment using vLLM and can run on a consumer GPU with 24GB RAM. To enable tensor parallelism, append tensor parallel size N to the serving command. Once started, the server listens at localhost:18888 for requests and is compatible with the OpenAI ChatCompletion API specifications. Please refer to the example request below for reference. Additionally, you can use the OpenChat Web UI for a user friendly experience. If you want to deploy the server as an online service, you can use api keys sk KEY1 sk KEY2 ... to specify allowed API keys and disable log req…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy