How Summary Length Affects ROUGE Scores: A Case Study on BERT Summarization of Clinical Case Reports

Extractive summarization is a technique that selects key sentences directly from a document to create a shorter version while preserving its core meaning. Since this technique reuses original text, it reduces the risk of introducing factual errors, an important advantage in clinical contexts where accuracy is critical. In this work, we apply a BERT extractive summarizer to clinical case reports, aiming to study how the number of retained sentences affects summary quality. We evaluated summaries of three different lengths, 3, 7 and 10 sentences, using three variants of ROUGE: ROUGE-1, ROUGE-2 and ROUGE-L metrics. Our results show a consistent trade-off: recall increases with summary length, while precision decreases. ROUGE-N F-scores peaked at seven sentences, suggesting this length offers the best balance between informativeness and conciseness. However, ROUGE-L scores declined steadily as more sentences were added, indicating that longer summaries may disrupt the structural coherence and narrative flow of the original text. These findings offer practical insights for optimizing extractive summarization in clinical applications, where summaries must be both accurate and easy to process.