TY - JOUR
T1 - Overview of the CLEF 2004 multilingual question answering track
AU - Magnini, Bernardo
AU - Vallin, Alessandro
AU - Ayache, Christelle
AU - Erbach, Gregor
AU - Peñas, Anselmo
AU - De Rijke, Maarten
AU - Rocha, Paulo
AU - Simov, Kiril
AU - Sutcliffe, Richard
PY - 2005
Y1 - 2005
N2 - Following the pilot Question Answering Track at CLEF 2003, a new evaluation exercise for multilingual QA systems took place in 2004. This paper reports on the novelties introduced in the new campaign and on participants' results. Almost all the cross-language combinations between nine source languages and seven target languages were exploited to set up more than fifty different tasks, both monolingual and bilingual. New types of questions (How-questions and definition questions) were given as input to the participating systems, while just one exact answer per question was allowed as output. The evaluation exercise has highlighted some difficulties in assessing definition questions and can be improved in the future, but the overall analysis of submissions shows encouraging results.
AB - Following the pilot Question Answering Track at CLEF 2003, a new evaluation exercise for multilingual QA systems took place in 2004. This paper reports on the novelties introduced in the new campaign and on participants' results. Almost all the cross-language combinations between nine source languages and seven target languages were exploited to set up more than fifty different tasks, both monolingual and bilingual. New types of questions (How-questions and definition questions) were given as input to the participating systems, while just one exact answer per question was allowed as output. The evaluation exercise has highlighted some difficulties in assessing definition questions and can be improved in the future, but the overall analysis of submissions shows encouraging results.
UR - https://www.scopus.com/pages/publications/24944457277
U2 - 10.1007/11519645_38
DO - 10.1007/11519645_38
M3 - Conference article
AN - SCOPUS:24944457277
SN - 0302-9743
VL - 3491
SP - 371
EP - 391
JO - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
JF - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
T2 - 5th Workshop of the Cross-Language Evaluation Forum, CLEF 2004: Multilingual Information Access for Text, Speech and Images
Y2 - 15 September 2004 through 17 September 2004
ER -