A watershed moment for opinions of teaching surveys (or student evaluations of teaching) (SOTS/SETs) in Canadian postsecondary education came with the 2018 Toronto Metropolitan University (formerly Ryerson University) arbitration ruling which recognized that the “body of research has concluded for years: student evaluations are not a barometer with which to judge a professor’s teaching effectiveness and therefore should not be used for employment-related decisions such as promotion and tenure” (Canadian Association of University Teachers, 2018).
One of the most comprehensive, recent, and damning summaries of the available evidence is Heffernan’s (2022) study on sexism, racism, prejudice, and bias in student evaluations. After reviewing 30 years (1990-2020) of published research on SETS, their conclusion was unequivocal:
…analysis of the existing literature makes it clear that SET results are strongly influenced by external factors unrelated to course content or teacher performance. In addition, these factors are frequently based on student demographics, and students’ biases and prejudices based on the teaching academic’s gender…
Women and marginalised groups are losing jobs, failing to achieve promotion, and are being negatively impacted at every step where SETs are concerned, and will continue to be discriminated against every time SET data is collected until the practice is stopped. These practices not only harm the sector’s women and most underrepresented and vulnerable, it cannot be denied that SETs also actively contribute to further marginalising the groups universities declare to protect and value in their workforces. (p. 151-152)
A continued mandating of SETS in tenure and promotion processes is the more difficult to defend when we consider that other, far less flawed mechanisms are available. Heffernan (2022) points to research documenting positive experiences at some universities with “students as partners” initiatives and the use of focus groups and student interviews to achieve genuine improvements in teaching and learning.
Closer to home, a Special Senate Committee for Evaluating Course and Teaching Effectiveness at Memorial University of Newfoundland (MUN) delivered a report recommending optional peer review of teaching for instructors, combined with a significantly reimagined approach to the design and usage of course experience questionnaires (CEQ aka SOTS). The work on CEQs is ongoing at MUN, and it is notable that MUNFA successfully negotiated removing reference to CEQs in tenure and promotion in its most recent round of collective bargaining in 2023.
The UPEI Board of Governors has publicly stated that that equity, diversity, inclusion, and Indigenization is one of their top priorities. Given the overwhelming evidence and the student support (see tweet below), the need for immediate policy change regarding SOTS at UPEI is clear. As Hefferman (2022) states, “no university … can declare to be a gender equal employer or have an interest in growing a safe, inclusive and diverse workforce if they continue using SETs to evaluate course and teacher quality.”
References
Heffernan, T. (2022). Sexism, racism, prejudice, and bias: A literature review and synthesis of research surrounding student evaluations of courses and teaching. Assessment & Evaluation in Higher Education, 47(1), 144-154. https://doi.org/10.1080/02602938.2021.1888075
Memorial University of Newfoundland. (2021). Developing a comprehensive system of evaluating courses and teaching effectiveness: Report and recommendations of the special senate committee for evaluating course and teaching effectiveness. https://www.mun.ca/senate/media/production/memorial/administrative/senate/media-library/about/ECTE_report.pdf
Canadian Association of University Teachers. (n.d.) The end of student questionnaires? https://www.caut.ca/bulletin/2018/11/end-student-questionnaires
