Does Medium Matter? Exploring Validity in Paper and Digital Spatial Ability Tests in Secondary Education

Spatial ability has long been linked with success in science, technology, engineering, and mathematics (STEM) disciplines with evidence suggesting that higher spatial abilities at younger ages can be predictive of higher academic attainment within STEM degree areas. Over the past few years there has been an increased recognition of the need for the development of spatial abilities in technology education, and the importance of developing such skills within adolescents has since been identified as an explicit area of importance within Irish secondary school curricula. In response to this curricular need, work has been done to investigate the development of such skills through direct instruction in a secondary level setting to examine the feasibility and explore potential difficulties associated with such implementations. A key element to such research studies is the measurement of spatial skills through psychometric testing, yet some uncertainty remains around the medium of this testing and the validity associated with the various mediums. To date some work has been done to investigate this relationship within a third level context, but the transferability of these findings to a secondary level context is still unclear due to the difference in developmental stages between these student cohorts. This paper aims to provide clarity around the use of paper-based and digital spatial testing mediums within a secondary level setting in regard to validity, to inform the methodological decisions of a larger study and similar future research. The data presented in this paper forms part of a study investigating the development of spatial ability within Irish secondary level technology education. Secondary school students in Ireland completed spatial testing of the Purdue Spatial Visualisation Test of Rotations (PSVT:R) and the Paper Folding Test (PFT) in both paper-based and digital formats. Item response theory analysis was conducted on both the PSVT:R (consisting of 30 items) and the PFT (consisting of 20 items) to determine the validity associated with the testing medium. The results from this study provide initial insights around the use of such testing mediums within secondary level technology education research and offer suggestions for further investigation.