Memorial University, like most universities, has a course evaluation questionnaire (CEQ) that is administered for at least two functions of significant interest to ASMs. The first of these functions is the formative one of providing an opportunity for feedback to faculty on their teaching; and the second of these functions is the summative one of helping to make personnel decisions, i.e., the role in Promotion and Tenure (P&T) evaluation.
There are serious questions about whether Student Evaluations of Teaching (SETs), such as MUN’s CEQs, are an appropriate or reliable measure of teaching effectiveness and whether they contain inherent biases, with gender bias being an especially marked concern.
MUNFA wishes to draw your attention to some of the potential pitfalls in using CEQ results, particularly given the following combination of factors: difficulties in measuring teaching effectiveness; relative ease of obtaining CEQ results; poor return rates of CEQ questionnaires by on-line surveys; expanding interest by university administrations in quantifying and obtaining “metrics” on faculty; and the potential for serious consequences (denial of promotion or tenure).
In particular, we draw your attention to two recent publications:
- An evaluation of course evaluations by P.B. Stark and R. Freishtat.
- Student evaluations of teaching (mostly) do not measure teaching effectiveness by Boring et al.
The paper by Stark and Freishtat offers some constructive guidelines on how to evaluate teaching effectiveness in a more effective and holistic way based on experience at their own institution. It also, however, states in their RECAP and RECOMMENDATIONS Sections that:
- “SET does not measure teaching effectiveness”;
- Survey rates matter and, when low, extrapolating to the whole class is unreliable;
- “Do not average or compare averages of SET scores: such averages do not make sense statistically”;
- “Avoid comparing teaching in course of different types, levels, sizes, functions, or disciplines.”
The paper by Boring et al. concludes that, “gender biases prevent SET from measuring teaching effectiveness accurately and fairly”. They recommend using the results from SET/CEQ with caution for formative reasons, and not using them at all for summative purposes.
MUNFA and CAUT (see attached) understand the need to evaluate teaching as part of the P&T process. However, whether they are submitting or evaluating P&T files, ASMs need to be aware that surveys of student opinion about teaching do not measure teaching effectiveness, and because research shows that they involve prejudices to the disadvantage of equity-seeking groups, student opinion surveys should not be used in Promotion and Tenure evaluation.
For more information on the use of CEQs and the evaluation of teaching, please consult the CAUT policy statements on the Use of Student Opinion Surveys and the Evaluation of Teaching.
