The C@merata task at MediaEval 2016: Natural language queries derived from exam papers, articles and other sources against classical music scores in MusicXML

Cl@ssical Music Extraction of Relevant Aspects by Text Analysis (C@merata) is a shared evaluation held at MediaEval and this is the third time the task has run. The input is a natural language query ('F# in the cello') and the output is a passage in a MusicXML score which contains this note played on the instrument in question. There are 200 such questions each year and evaluation is via modified versions of Precision, Recall and FMeasure. In 2014 and 2015 the best Beat F (BF) scores were 0.797 and 0.620, both attained by CLAS. This year, queries were more difficult and in addition the most experienced groups from previous years were unable to take part. In consequence, the best BF was 0.070. This year, there was progress concerning the development of the queries, many of these being derived from real sources such as exam papers, books and scholarly articles. We are thus converging on our goal of relating musical references in complex natural language texts to passages in music scores.