TY - GEN
T1 - QA4MRE 2011-2013
T2 - 4th International Conference of the CLEF Initiative, CLEF 2013
AU - Peñas, Anselmo
AU - Hovy, Eduard
AU - Forner, Pamela
AU - Rodrigo, Álvaro
AU - Sutcliffe, Richard
AU - Morante, Roser
PY - 2013
Y1 - 2013
N2 - This paper describes the methodology for testing the performance of Machine Reading systems through Question Answering and Reading Comprehension Tests. This was the attempt of the QA4MRE challenge which was run as a Lab at CLEF 2011-2013. The traditional QA task was replaced by a new Machine Reading task, whose intention was to ask questions that required a deep knowledge of individual short texts and in which systems were required to choose one answer, by analysing the corresponding test document in conjunction with background text collections provided by the organization. Four different tasks have been organized during these years: Main Task, Processing Modality and Negation for Machine Reading, Machine Reading of Biomedical Texts about Alzheimer's disease, and Entrance Exams. This paper describes their motivation, their goals, their methodology for preparing the data sets, their background collections, their metrics used for the evaluation, and the lessons learned along these three years.
AB - This paper describes the methodology for testing the performance of Machine Reading systems through Question Answering and Reading Comprehension Tests. This was the attempt of the QA4MRE challenge which was run as a Lab at CLEF 2011-2013. The traditional QA task was replaced by a new Machine Reading task, whose intention was to ask questions that required a deep knowledge of individual short texts and in which systems were required to choose one answer, by analysing the corresponding test document in conjunction with background text collections provided by the organization. Four different tasks have been organized during these years: Main Task, Processing Modality and Negation for Machine Reading, Machine Reading of Biomedical Texts about Alzheimer's disease, and Entrance Exams. This paper describes their motivation, their goals, their methodology for preparing the data sets, their background collections, their metrics used for the evaluation, and the lessons learned along these three years.
UR - http://www.scopus.com/inward/record.url?scp=84886376548&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-40802-1_29
DO - 10.1007/978-3-642-40802-1_29
M3 - Conference contribution
AN - SCOPUS:84886376548
SN - 9783642408014
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 303
EP - 320
BT - Information Access Evaluation
Y2 - 23 September 2013 through 26 September 2013
ER -