TY - JOUR
T1 - Online student evaluations of teaching
T2 - what are we sacrificing for the affordances of technology?
AU - Risquez, Anglica
AU - Vaughan, Elaine
AU - Murphy, Maura
N1 - Publisher Copyright:
© 2014 Taylor & Francis.
PY - 2015/1/2
Y1 - 2015/1/2
N2 - In the context of increased emphasis on quality assurance of teaching, it is crucial that student evaluations of teaching (SET) methods be both reliable and workable in practice. Online SETs particularly tend to raise criticisms with those most reactive to mechanisms of teaching accountability. However, most studies of SET processes have been conducted with convenience, small and cross-sectional samples. Longitudinal studies are rare, as comparison studies on SET methodological approaches are generally pilot studies followed shortly after by implementation. The investigation presented here significantly contributes to the debate by examining the impact of the online administration method of SET on a very large longitudinal sample at the course level rather than attending to the student unit, thus compensating for the inter-dependency of students’ responses according to the instructor variable. It explores the impact of the administration method of SET (paper based in-class vs. out-of-class online collection) on scores, with a longitudinal sample of over 63,000 student responses collected over a total period of 10 years. Having adjusted for the confounding effect of class size, faculty, year of evaluation, years of teaching experience and student performance, it is observed that the actual effect of the administration method exists, but is insignificant.
AB - In the context of increased emphasis on quality assurance of teaching, it is crucial that student evaluations of teaching (SET) methods be both reliable and workable in practice. Online SETs particularly tend to raise criticisms with those most reactive to mechanisms of teaching accountability. However, most studies of SET processes have been conducted with convenience, small and cross-sectional samples. Longitudinal studies are rare, as comparison studies on SET methodological approaches are generally pilot studies followed shortly after by implementation. The investigation presented here significantly contributes to the debate by examining the impact of the online administration method of SET on a very large longitudinal sample at the course level rather than attending to the student unit, thus compensating for the inter-dependency of students’ responses according to the instructor variable. It explores the impact of the administration method of SET (paper based in-class vs. out-of-class online collection) on scores, with a longitudinal sample of over 63,000 student responses collected over a total period of 10 years. Having adjusted for the confounding effect of class size, faculty, year of evaluation, years of teaching experience and student performance, it is observed that the actual effect of the administration method exists, but is insignificant.
KW - educational technology
KW - Ireland
KW - online survey
KW - response rates
KW - student evaluation of teaching
UR - http://www.scopus.com/inward/record.url?scp=84918510984&partnerID=8YFLogxK
U2 - 10.1080/02602938.2014.890695
DO - 10.1080/02602938.2014.890695
M3 - Article
AN - SCOPUS:84918510984
SN - 0260-2938
VL - 40
SP - 120
EP - 134
JO - Assessment and Evaluation in Higher Education
JF - Assessment and Evaluation in Higher Education
IS - 1
ER -