Model Card Prompt Guard LLM powered applications are susceptible to prompt attacks, which are prompts intentionally designed to subvert the developer’s intended behavior of the LLM. Categories of prompt attacks include prompt injection and jailbreaking: Prompt Injections are inputs that exploit the concatenation of untrusted data from third parties and users into the context window of a model to get a model to execute unintended instructions. Jailbreaks are malicious instructions designed to override the safety and security features built into a model. Prompt Guard is a classifier model trained on a large corpus of attacks, capable of detecting both explicitly malicious prompts as well as data that contains injected inputs. The model is useful as a starting point for identifying and guardrailing against the most risky realistic inputs to LLM powered applications; for optimal results we recommend developers fine tune the model on their application specific data and use cases. We also recommend layering model based protection with additional protections. Our goal in releasing PromptGuard as an open source model is to provide an accessible approach developers can take to significantly…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy