Update @ 2024.05.20: Re Upload RoPE fixed model Update @ 2024.05.01: Pre Release Llama 3 KoEn 8B model & Llama 3 KoEn 8B Instruct preview Update @ 2024.04.24: Release Llama 3 Open Ko 8B model & Llama 3 Open Ko 8B Instruct preview Model Details Llama 3 Open Ko 8B Llama 3 Open Ko 8B model is continued pretrained language model based on Llama 3 8B. This model is trained fully with publicily available resource, with 60GB+ of deduplicated texts. With the new Llama 3 tokenizer, the pretraining conducted with 17.7B+ tokens, which slightly more than Korean tokenizer(Llama 2 Ko tokenizer). The train was done on TPUv5e 256, with the warm support from TRC program by Google. Note for Llama 3 Open Ko 8B Instruct preview With applying the idea from Chat Vector paper, I released Instruction model named Llama 3 Open Ko 8B Instruct preview. Since it is NOT finetuned with any Korean instruction set(indeed preview ), but it would be great starting point for creating new Chat/Instruct models. Meta Llama 3 Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instructio…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy