Overview of QA4MRE at CLEF 2012: Question Answering for Machine Reading evaluation

Anselmo Peñas; Eduard Hovy; Pamela Forner; Álvaro Rodrigo; Richard Sutcliffe; Caroline Sporleder; Corina Forascu; Yassine Benajiba; Petya Osenova

Overview of QA4MRE at CLEF 2012: Question Answering for Machine Reading evaluation

Anselmo Peñas
, Eduard Hovy
, Pamela Forner
, Álvaro Rodrigo
, Richard Sutcliffe
, Caroline Sporleder
, Corina Forascu
, Yassine Benajiba
, Petya Osenova

Research output: Contribution to journal › Conference article › peer-review

Abstract

This paper describes the Question Answering for Machine Reading (QA4MRE) task at the 2012 Cross Language Evaluation Forum. In the main task, systems answered multiple-choice questions on documents concerned with four different topics. There were also two pilot tasks, Processing Modality and Negation for Machine Reading, and Machine Reading on Biomedical Texts about Alzheimer's disease. This paper describes the preparation of the data sets, the definition of the background collections, the metric used for the evaluation of the systems' submissions, and the results. Eleven groups participated in the task submitting a total of 43 runs in seven languages.

Original language	English
Journal	CEUR Workshop Proceedings
Volume	1178
Publication status	Published - 2012
Event	2012 Cross Language Evaluation Forum Conference, CLEF 2012 - Rome, Italy Duration: 17 Sep 2012 → 20 Sep 2012

Cite this

@article{6445fad5b7144ec1ba1eaa0e3a65afe3,

title = "Overview of QA4MRE at CLEF 2012: Question Answering for Machine Reading evaluation",

abstract = "This paper describes the Question Answering for Machine Reading (QA4MRE) task at the 2012 Cross Language Evaluation Forum. In the main task, systems answered multiple-choice questions on documents concerned with four different topics. There were also two pilot tasks, Processing Modality and Negation for Machine Reading, and Machine Reading on Biomedical Texts about Alzheimer's disease. This paper describes the preparation of the data sets, the definition of the background collections, the metric used for the evaluation of the systems' submissions, and the results. Eleven groups participated in the task submitting a total of 43 runs in seven languages.",

author = "Anselmo Pe{\~n}as and Eduard Hovy and Pamela Forner and {\'A}lvaro Rodrigo and Richard Sutcliffe and Caroline Sporleder and Corina Forascu and Yassine Benajiba and Petya Osenova",

year = "2012",

language = "English",

volume = "1178",

journal = "CEUR Workshop Proceedings",

issn = "1613-0073",

note = "2012 Cross Language Evaluation Forum Conference, CLEF 2012 ; Conference date: 17-09-2012 Through 20-09-2012",

}

TY - JOUR

T1 - Overview of QA4MRE at CLEF 2012

T2 - 2012 Cross Language Evaluation Forum Conference, CLEF 2012

AU - Peñas, Anselmo

AU - Hovy, Eduard

AU - Forner, Pamela

AU - Rodrigo, Álvaro

AU - Sutcliffe, Richard

AU - Sporleder, Caroline

AU - Forascu, Corina

AU - Benajiba, Yassine

AU - Osenova, Petya

PY - 2012

Y1 - 2012

N2 - This paper describes the Question Answering for Machine Reading (QA4MRE) task at the 2012 Cross Language Evaluation Forum. In the main task, systems answered multiple-choice questions on documents concerned with four different topics. There were also two pilot tasks, Processing Modality and Negation for Machine Reading, and Machine Reading on Biomedical Texts about Alzheimer's disease. This paper describes the preparation of the data sets, the definition of the background collections, the metric used for the evaluation of the systems' submissions, and the results. Eleven groups participated in the task submitting a total of 43 runs in seven languages.

AB - This paper describes the Question Answering for Machine Reading (QA4MRE) task at the 2012 Cross Language Evaluation Forum. In the main task, systems answered multiple-choice questions on documents concerned with four different topics. There were also two pilot tasks, Processing Modality and Negation for Machine Reading, and Machine Reading on Biomedical Texts about Alzheimer's disease. This paper describes the preparation of the data sets, the definition of the background collections, the metric used for the evaluation of the systems' submissions, and the results. Eleven groups participated in the task submitting a total of 43 runs in seven languages.

UR - https://www.scopus.com/pages/publications/84922021542

M3 - Conference article

AN - SCOPUS:84922021542

SN - 1613-0073

VL - 1178

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

Y2 - 17 September 2012 through 20 September 2012

ER -

Overview of QA4MRE at CLEF 2012: Question Answering for Machine Reading evaluation

Abstract

Other files and links

Fingerprint

Cite this