Perception Encoder Audio Frame (PE A Frame) PE A Frame is a state of the art audio text embedding model. For text, the model produces a single embedding. For audio, it produces a sequence of embeddings (onen for every 40ms of audio). These embeddings can then be used for audio event localization. For convienience, model outputs temporal spans (start and end timestamps) indicating when that event (freeform audio description) occurs in the audio. Model Description PE A Frame uses contrastive learning to align frame level audio representations with text descriptions. The model can identify precise time ranges when described audio events occur Model Variants We release multiple model checkpoints with varying sizes: Model Parameters pe a frame small 450M pe a frame base 560M pe a frame large 1.4B Quick Start Basic Usage: Audio Event Localization Example Output: Batch Processing Multiple Audio Files Adjusting Detection Threshold The threshold parameter controls sensitivity for event detection. Lower values detect more events (higher recall), while higher values are more selective (higher precision): Extracting Embeddings Without Spans If you only need embeddings without temporal localiza…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy