Model Summary The language model Phi 1.5 is a Transformer with 1.3 billion parameters. It was trained using the same data sources as phi 1, augmented with a new data source that consists of various NLP synthetic texts. When assessed against benchmarks testing common sense, language understanding, and logical reasoning, Phi 1.5 demonstrates a nearly state of the art performance among models with less than 10 billion parameters. We did not fine tune Phi 1.5 either for instruction following or through reinforcement learning from human feedback . The intention behind crafting this open source model is to provide the research community with a non restricted small model to explore vital safety challenges, such as reducing toxicity, understanding societal biases, enhancing controllability, and more. For a safer model release, we exclude generic web crawl data sources such as common crawl from the training. This strategy prevents direct exposure to potentially harmful online content, enhancing the model's safety without RLHF. However, the model is still vulnerable to generating harmful content. We hope the model can help the research community to further study the safety of language models…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy