MuChoMusic

dataset·music audio language qa·active

Human-validated multiple-choice questions for evaluating music understanding and reasoning in audio-language models.

Verified facts

Official site	https://mulab-mir.github.io/muchomusic/ ↗
Geography	Global
paper venue	ISMIR 2024
size	tracks: 644 · questions: 1187 · choices per question: 4
scope	Musical concepts, culture, function and reasoning over recorded music.
genres	varied
license	unknown at benchmark level
steward	MuLab / Queen Mary University of London / Universal Music Group
creators	Benno Weck; Ilaria Manco; Emmanouil Benetos; Elio Quinton; George Fazekas; Dmitry Bogdanov
languages	English questions
modalities	audio; multiple-choice text; human validation labels
geographies	Global
access method	Project page links paper, data, code and supplement.
intended uses	music understanding evaluation; audio-language model evaluation
consent claims	Inherited from source datasets; validator consent details not verified.
related papers	MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Models
annotation method	Human validators checked questions and carefully designed distractors.
collection method	Questions were written for tracks from MusicCaps and Song Describer.
provenance claims	Official project names the two public source datasets.
current availability	Project artifacts are publicly linked.
commercial use limits	MusicCaps and Song Describer source-audio terms must be checked.
us market scope basis	Publicly available or materially used in US-facing music/audio-AI research.
limitations biases disputes	Multiple-choice format can reward test-taking strategies.; Coverage inherits the two source datasets' biases.