Automated Question Answering for Improved Understanding of Compliance Requirements: A Multi-Document Study

Sallam Abualhaija, Chetan Arora, Amin Sleimi, Lionel C. Briand

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Software systems are increasingly subject to regulatory compliance. Extracting compliance requirements from regulations is challenging. Ideally, locating compliance-related information in a regulation requires a joint effort from requirements engineers and legal experts, whose availability is limited. However, regulations are typically long documents spanning hundreds of pages, containing legal jargon, applying complicated natural language structures, and including cross-references, thus making their analysis effort-intensive. In this paper, we propose an automated question-answering (QA) approach that assists requirements engineers in finding the legal text passages relevant to compliance requirements. Our approach utilizes large-scale language models fine-tuned for QA, including BERT and three variants. We evaluate our approach on 107 question-answer pairs, manually curated by subject-matter experts, for four different European regulatory documents. Among these documents is the general data protection regulation (GDPR) - a major source for privacy-related requirements. Our empirical results show that, in $\approx 94$% of the cases, our approach finds the text passage containing the answer to a given question among the top five passages that our approach marks as most relevant. Further, our approach successfully demarcates, in the selected passage, the right answer with an average accuracy of $\approx$91%.

Original languageEnglish
Title of host publicationProceedings - 30th IEEE International Requirements Engineering Conference, RE 2022
EditorsEric Knauss, Gunter Mussbacher, Chetan Arora, Muneera Bano, Jean-Guy Schneider
PublisherIEEE Computer Society
Pages39-50
Number of pages12
ISBN (Electronic)9781665470001
DOIs
Publication statusPublished - 2022
Externally publishedYes
Event30th IEEE International Requirements Engineering Conference, RE 2022 - Virtual, Online, Australia
Duration: 15 Aug 202219 Aug 2022

Publication series

NameProceedings of the IEEE International Conference on Requirements Engineering
Volume2022-August
ISSN (Print)1090-705X
ISSN (Electronic)2332-6441

Conference

Conference30th IEEE International Requirements Engineering Conference, RE 2022
Country/TerritoryAustralia
CityVirtual, Online
Period15/08/2219/08/22

Keywords

  • BERT
  • Language Models (LMs)
  • Natural Language Processing (NLP)
  • Question Answering
  • Regulatory Compliance
  • Requirements Engineering

Fingerprint

Dive into the research topics of 'Automated Question Answering for Improved Understanding of Compliance Requirements: A Multi-Document Study'. Together they form a unique fingerprint.

Cite this