InCoder 1B A 1B parameter decoder only Transformer model trained on code using a causal masked objective, which allows inserting/infilling code as well as standard left to right generation. The model was trained on public open source repositories with a permissive, non copyleft, license (Apache 2.0, MIT, BSD 2 or BSD 3) from GitHub and GitLab, as well as StackOverflow. Repositories primarily contained Python and JavaScript, but also include code from 28 languages, as well as StackOverflow. For more information, see our: Demo Project site Examples Paper A larger, 6B, parameter model is also available at facebook/incoder 6B. Requirements pytorch , tokenizers , and transformers . Our model requires HF's tokenizers = 0.12.1, due to changes in the pretokenizer. Usage See https://github.com/dpfried/incoder for example code. Model model = AutoModelForCausalLM.from pretrained("facebook/incoder 1B") Tokenizer tokenizer = AutoTokenizer.from pretrained("facebook/incoder 1B") (Note: the incoder 1B and incoder 6B tokenizers are identical, so 'facebook/incoder 6B' could also be used.) When calling tokenizer.decode , it's important to pass clean up tokenization spaces=False to avoid removing spac…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy