MMAU-Pro

dataset·general audio reasoning·active

Expert-created audio questions covering speech, sound, music, mixtures, spatial audio and long-form reasoning.

Verified facts

Official site	https://sonalkum.github.io/mmau-pro/ ↗
Geography	United States
arxiv	2508.13992
huggingface repository	gamma-lab-umd/MMAU-Pro
size	skills: 49 · instances: 5305 · evaluated models: 22
scope	Holistic audio-general-intelligence evaluation with deliberately difficult questions.
genres	music subset is multicultural but majority Western
license	unknown/mixed upstream
steward	University of Maryland-led Gamma Lab collaboration
creators	Sonal Kumar; Šimon Sedláček; Vaibhavi Lokegaonkar
languages	English questions; multilingual audio included
modalities	single and multiple audio; multiple-choice QA; open-ended QA; spatial audio; long-form audio
geographies	Global
access method	Official page links dataset and code.
intended uses	audio reasoning evaluation; multimodal model evaluation
consent claims	Source-person and annotator consent details not verified.
related papers	MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligence
annotation method	Human experts authored multi-hop questions and answers.
collection method	Audio was sourced from the wild rather than fixed benchmark corpora.
provenance claims	Official project describes wild-source collection and expert authorship.
current availability	Project page and Hugging Face dataset available.
commercial use limits	Wild-source media rights require separate assessment.
us market scope basis	Publicly available or materially used in US-facing music/audio-AI research.
limitations biases disputes	Approximately sixty percent of music content is Western.; Wild-source audio creates rights and stability risk.