Personalized PHQ-9 test length using probability density estimation based on conditional probability and K-Nearest Neighbours

The Patient Health Questionnaire-9 (PHQ-9) is a tool consisting of nine items designed to assess the severity of depression in individuals. Shorter versions have been developed such as the PHQ-DEP-4, which includes four items, and the PHQ-2, which consists of just two. These fixed-length formats have been developed to facilitate rapid screening, particularly for identifying individuals eligible for clinical trials. In this study, we propose and evaluate a dynamic version of the PHQ-9, in which the number of questions administered varies according to the respondent's answers. This adaptive approach estimates the likelihood of depression conditional on the responses given thus far and can terminate the assessment early when a confident classification (depressed or non-depressed) can be made before all nine questions are completed. The model relies on a historical datasets of completed PHQ-9 interviews to inform these decisions. When a matching response pattern is not available in the historical data, a K-Nearest Neighbours (KNN) model is applied to estimate the probability density for this pattern. Experimental results demonstrate that the dynamic PHQ-9 model outperforms the PHQ-DEP-4, achieving higher sensitivity, specificity, and Youden index, while also reducing respondent burden by requiring fewer questions on average.