Model Summary Phi 2 is a Transformer with 2.7 billion parameters. It was trained using the same data sources as Phi 1.5, augmented with a new data source that consists of various NLP synthetic texts and filtered websites (for safety and educational value). When assessed against benchmarks testing common sense, language understanding, and logical reasoning, Phi 2 showcased a nearly state of the art performance among models with less than 13 billion parameters. Our model hasn't been fine tuned through reinforcement learning from human feedback. The intention behind crafting this open source model is to provide the research community with a non restricted small model to explore vital safety challenges, such as reducing toxicity, understanding societal biases, enhancing controllability, and more. How to Use Phi 2 has been integrated in the transformers version 4.37.0, please ensure that you are using a version equal or higher than it. Phi 2 is known for having an attention overflow issue (with FP16). If you are facing this issue, please enable/disable autocast on the PhiAttention.forward() function. Intended Uses Given the nature of the training data, the Phi 2 model is best suited for…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy