DistilGPT2 DistilGPT2 (short for Distilled GPT2) is an English language model pre trained with the supervision of the smallest version of Generative Pre trained Transformer 2 (GPT 2). Like GPT 2, DistilGPT2 can be used to generate text. Users of this model card should also consider information about the design, training, and limitations of GPT 2. Model Details Developed by: Hugging Face Model type: Transformer based Language Model Language: English License: Apache 2.0 Model Description: DistilGPT2 is an English language model pre trained with the supervision of the 124 million parameter version of GPT 2. DistilGPT2, which has 82 million parameters, was developed using knowledge distillation and was designed to be a faster, lighter version of GPT 2. Resources for more information: See this repository for more about Distil\ (a class of compressed models including Distilled GPT2), Sanh et al. (2019) for more information about knowledge distillation and the training procedure, and this page for more about GPT 2. Uses, Limitations and Risks Limitations and Risks Click to expand CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy