Refact 1.6B Finally, the model we started training with our blog post is ready 🎉 After fine tuning on generated data, it beats Replit 3b, Stability Code 3b and many other models. It almost beats StarCoder ten times the size! Model Size HumanEval pass@1 HumanEval pass@10 DeciCoder 1b 1b 19.1% Refact 1.6 fim 1.6b 32.0% 53.0% StableCode 3b 20.2% 33.8% ReplitCode v1 3b 21.9% CodeGen2.5 multi 7b 28.4% 47.5% CodeLlama 7b 33.5% 59.6% StarCoder 15b 33.6% Likely, it's the best model for practical use in your IDE for code completion because it's smart and fast! You can start using it right now by downloading the Refact plugin. You can host the model yourself, too, using the open source docker container. And it's multi language (see MultiPL HumanEval and other metrics below) and it works as a chat (see the section below). It Works As a Chat The primary application of this model is code completion (infill) in multiple programming languages. But it works as a chat quite well. HumanEval results using instruction following (chat) format, against models specialized for chat only: Model Size pass@1 pass@10 Refact 1.6 fim 1.6b 38.4% 55.6% StableCode instruct 3b 26.9% 36.2% OctoGeeX 6b 44.7% CodeLla…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy