Six more August papers show where music models still fail

Six more August papers examine whether music models follow instructions, judge songs, understand underrepresented traditions, preserve audio detail and survive more realistic evaluation. They cover different tasks, but share one useful lesson: a high score can hide the exact failure a real user cares about.

Prompt agreement is not the same as control

A counterfactual study tests key and beat grouping in ACE-Step 1.5, Stable Audio 3 Medium and LeVo2. ACE-Step and Stable Audio showed substantial key control; LeVo2 did not. The more interesting result concerns four-beat grouping: Stable Audio produced it in 97% of neutral cases but only 56% when explicitly asked for it in the matched treatment.

The model often produced the requested attribute because it was already common, not necessarily because the instruction caused it. Neutral prompts and target swaps make that distinction visible.

A song reward model that explains itself

MuseCritic generates a natural-language critique across five aesthetic dimensions before predicting a reward score. Its authors report 71.35% accuracy on 733 out-of-domain Music Arena preference pairs and improvements when the reward was used to tune a smaller song model.

Readable criticism is more useful than an unexplained number, but the reported performance remains an experiment by the model's authors. It does not turn aesthetic judgement into an objective measurement.

Deepfake detection expands beyond speech

The AT-ADD challenge covers speech, environmental sound, singing voice and music. Its best type-agnostic system reached 96.10% macro-F1 on the final evaluation set. The organisers still identify unseen generators, realistic distortions and balanced performance across audio types as open problems.

That is a stronger setting than testing clean speech alone, but it remains a controlled challenge rather than a guarantee for every platform upload.

The best representation layer depends on the task

A study of 12 music foundation models across 15 downstream tasks finds that several label-free metrics can help choose layers for genre, emotion, tagging and beat tracking. The same metrics fail on tonal tasks such as key and chord recognition. A new pitch-transposition measure tracks tonal quality more consistently.

The practical point is that one “best layer” or one general embedding score is not enough for every music-analysis job.

Cultural coverage is still shallow

UniVerseBench contains 5,042 question-and-answer pairs across more than 38 cultural and linguistic entities. Training on an automatically generated companion dataset improved large audio-language models, but the authors report continued difficulty with fine-grained acoustic features.

More language coverage can improve surface recognition without producing deep understanding of a musical tradition.

Better audio latents target audible failures

ear-VAE2 works with complex spectral representations to address high-frequency loss, phase incoherence and stereo collapse in compressed music autoencoders. On a 546-track dataset, the paper reports the best point estimates on five of seven reconstruction measures; its refiner reduced Mel Distance by 19.4%, and a downstream generator improved on all 12 reported automatic metrics.

Those are the authors' results on their dataset. The useful advance is the focus on failures engineers and listeners can actually hear, not the claim that reconstruction is solved.

The practical reading

Ask whether the benchmark isolates causal instruction following, whether preferences come with understandable reasons, whether cultural coverage goes beyond labels, and whether detector or fidelity scores survive real production conditions. These preprints and conference papers sharpen the questions; they do not replace listening, independent replication or deployment tests.