VoxEval Github repository for paper: VoxEval: Benchmarking the Knowledge Understanding Capabilities of End to End Spoken Language Models Also check out our survey paper at Recent Advances in Speech Language Models: A Survey ! VoxEval is a novel speech question answering benchmark specifically designed to assess SLMs' knowledge understanding through purely speech based interactions. Below are the three highlights of our VoxEval benchmark: End to end speech based evaluation: Both input and output are audio based. Diverse audio conditions: VoxEval includes audio files featuring a variety of speakers, speaking styles, and audio qualities. Complex spoken evaluations: It supports advanced spoken assessments, including spoken math tasks. Download Data You can access our VoxEval Dataset Repository on 🤗 Hugging Face to directly download the dataset. Below is the layout of the dataset folder. all fewshot examples folder contains the few shot examples for the evaluation. math CoT fewshot folder contains the few shot examples for evaluating the math subjects via Chain of Thought prompting. test folder contains the actual test data in VoxEval. Evaluation results of existing end to end Spoken L…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy