Kanana 🤗 Models     📕 Blog     📜 Technical Report   💻 Github Introduction We introduce Kanana, a series of bilingual language models (developed by Kakao) that demonstrate exceeding performance in Korean and competitive performance in English. The computational cost of Kanana is significantly lower than that of state of the art models of similar size. The report details the techniques employed during pre training to achieve compute efficient yet competitive models, including high quality data filtering, staged pre training, depth up scaling, and pruning and distillation. Furthermore, the report outlines the methodologies utilized during the post training of the Kanana models, encompassing supervised fine tuning and preference optimization, aimed at enhancing their capability for seamless interaction with users. Lastly, the report elaborates on plausible approaches used for language model adaptation to specific scenarios, such as embedding, function calling, and Retrieval Augmented Generation (RAG). The Kanana model series spans from 2.1B to 32.5B parameters with 2.1B models (base, instruct, embedding, function call, and RAG) publicly released to promote resea…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy