Overview of the CLEF 2007 multilingual question answering track

Danilo Giampiccolo; Pamela Forner; Jesús Herrera; Anselmo Peñas; Christelle Ayache; Corina Forascu; Valentin Jijkoun; Petya Osenova; Paulo Rocha; Bogdan Sacaleanu; Richard Sutcliffe

doi:10.1007/978-3-540-85760-0_27

Overview of the CLEF 2007 multilingual question answering track

Danilo Giampiccolo, Pamela Forner, Jesús Herrera, Anselmo Peñas, Christelle Ayache, Corina Forascu, Valentin Jijkoun, Petya Osenova, Paulo Rocha, Bogdan Sacaleanu, Richard Sutcliffe

Retired Staff

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

The fifth QA campaign at CLEF [1], having its first edition in 2003, offered not only a main task but an Answer Validation Exercise (AVE) [2], which continued last year's pilot, and a new pilot: the Question Answering on Speech Transcripts (QAST) [3, 15]. The main task was characterized by the focus on cross-linguality, while covering as many European languages as possible. As novelty, some QA pairs were grouped in clusters. Every cluster was characterized by a topic (not given to participants). The questions from a cluster possibly contain co-references between one of them and the others. Finally, the need for searching answers in web formats was satisfied by introducing Wikipedia as document corpus. The results and the analyses reported by the participants suggest that the introduction of Wikipedia and the topic related questions led to a drop in systems' performance.

Original language	English
Title of host publication	Advances in Multilingual and Multimodal Information Retrieval - 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Revised Selected Papers
Publisher	Springer Verlag
Pages	200-236
Number of pages	37
ISBN (Print)	3540857591, 9783540857594
DOIs	https://doi.org/10.1007/978-3-540-85760-0_27
Publication status	Published - 2008
Event	8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007 - Budapest, Hungary Duration: 19 Sep 2007 → 21 Sep 2007

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	5152 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007
Country/Territory	Hungary
City	Budapest
Period	19/09/07 → 21/09/07

Access to Document

10.1007/978-3-540-85760-0_27

Cite this

Giampiccolo, D., Forner, P., Herrera, J., Peñas, A., Ayache, C., Forascu, C., Jijkoun, V., Osenova, P., Rocha, P., Sacaleanu, B., & Sutcliffe, R. (2008). Overview of the CLEF 2007 multilingual question answering track. In Advances in Multilingual and Multimodal Information Retrieval - 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Revised Selected Papers (pp. 200-236). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5152 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-540-85760-0_27

Giampiccolo, Danilo ; Forner, Pamela ; Herrera, Jesús et al. / Overview of the CLEF 2007 multilingual question answering track. Advances in Multilingual and Multimodal Information Retrieval - 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Revised Selected Papers. Springer Verlag, 2008. pp. 200-236 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{15cac6b3a60d4c37959ba139a2edd5e9,

title = "Overview of the CLEF 2007 multilingual question answering track",

abstract = "The fifth QA campaign at CLEF [1], having its first edition in 2003, offered not only a main task but an Answer Validation Exercise (AVE) [2], which continued last year's pilot, and a new pilot: the Question Answering on Speech Transcripts (QAST) [3, 15]. The main task was characterized by the focus on cross-linguality, while covering as many European languages as possible. As novelty, some QA pairs were grouped in clusters. Every cluster was characterized by a topic (not given to participants). The questions from a cluster possibly contain co-references between one of them and the others. Finally, the need for searching answers in web formats was satisfied by introducing Wikipedia as document corpus. The results and the analyses reported by the participants suggest that the introduction of Wikipedia and the topic related questions led to a drop in systems' performance.",

author = "Danilo Giampiccolo and Pamela Forner and Jes{\'u}s Herrera and Anselmo Pe{\~n}as and Christelle Ayache and Corina Forascu and Valentin Jijkoun and Petya Osenova and Paulo Rocha and Bogdan Sacaleanu and Richard Sutcliffe",

year = "2008",

doi = "10.1007/978-3-540-85760-0_27",

language = "English",

isbn = "3540857591",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "200--236",

booktitle = "Advances in Multilingual and Multimodal Information Retrieval - 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Revised Selected Papers",

note = "8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007 ; Conference date: 19-09-2007 Through 21-09-2007",

}

Giampiccolo, D, Forner, P, Herrera, J, Peñas, A, Ayache, C, Forascu, C, Jijkoun, V, Osenova, P, Rocha, P, Sacaleanu, B & Sutcliffe, R 2008, Overview of the CLEF 2007 multilingual question answering track. in Advances in Multilingual and Multimodal Information Retrieval - 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5152 LNCS, Springer Verlag, pp. 200-236, 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Budapest, Hungary, 19/09/07. https://doi.org/10.1007/978-3-540-85760-0_27

Overview of the CLEF 2007 multilingual question answering track. / Giampiccolo, Danilo; Forner, Pamela; Herrera, Jesús et al.
Advances in Multilingual and Multimodal Information Retrieval - 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Revised Selected Papers. Springer Verlag, 2008. p. 200-236 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5152 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Overview of the CLEF 2007 multilingual question answering track

AU - Giampiccolo, Danilo

AU - Forner, Pamela

AU - Herrera, Jesús

AU - Peñas, Anselmo

AU - Ayache, Christelle

AU - Forascu, Corina

AU - Jijkoun, Valentin

AU - Osenova, Petya

AU - Rocha, Paulo

AU - Sacaleanu, Bogdan

AU - Sutcliffe, Richard

PY - 2008

Y1 - 2008

N2 - The fifth QA campaign at CLEF [1], having its first edition in 2003, offered not only a main task but an Answer Validation Exercise (AVE) [2], which continued last year's pilot, and a new pilot: the Question Answering on Speech Transcripts (QAST) [3, 15]. The main task was characterized by the focus on cross-linguality, while covering as many European languages as possible. As novelty, some QA pairs were grouped in clusters. Every cluster was characterized by a topic (not given to participants). The questions from a cluster possibly contain co-references between one of them and the others. Finally, the need for searching answers in web formats was satisfied by introducing Wikipedia as document corpus. The results and the analyses reported by the participants suggest that the introduction of Wikipedia and the topic related questions led to a drop in systems' performance.

AB - The fifth QA campaign at CLEF [1], having its first edition in 2003, offered not only a main task but an Answer Validation Exercise (AVE) [2], which continued last year's pilot, and a new pilot: the Question Answering on Speech Transcripts (QAST) [3, 15]. The main task was characterized by the focus on cross-linguality, while covering as many European languages as possible. As novelty, some QA pairs were grouped in clusters. Every cluster was characterized by a topic (not given to participants). The questions from a cluster possibly contain co-references between one of them and the others. Finally, the need for searching answers in web formats was satisfied by introducing Wikipedia as document corpus. The results and the analyses reported by the participants suggest that the introduction of Wikipedia and the topic related questions led to a drop in systems' performance.

UR - http://www.scopus.com/inward/record.url?scp=70349835056&partnerID=8YFLogxK

U2 - 10.1007/978-3-540-85760-0_27

DO - 10.1007/978-3-540-85760-0_27

M3 - Conference contribution

AN - SCOPUS:70349835056

SN - 3540857591

SN - 9783540857594

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 200

EP - 236

BT - Advances in Multilingual and Multimodal Information Retrieval - 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Revised Selected Papers

PB - Springer Verlag

T2 - 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007

Y2 - 19 September 2007 through 21 September 2007

ER -

Giampiccolo D, Forner P, Herrera J, Peñas A, Ayache C, Forascu C et al. Overview of the CLEF 2007 multilingual question answering track. In Advances in Multilingual and Multimodal Information Retrieval - 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Revised Selected Papers. Springer Verlag. 2008. p. 200-236. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-540-85760-0_27

Overview of the CLEF 2007 multilingual question answering track

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this