How reliable are student evaluations of teaching?

Last week I got my results back on semester 2’s teaching, and much hilarity ensued!

As many universities, Edinburgh now uses an online system to collect numerical scores on each course, along with some free text comments. The comments are sometimes very helpful, though I found the old system of distributing paper forms in class much more successful in this regard, both in terms of response rates and usefulness of comments.

The numerical scores offer some satisfaction too, since (humblebrag alert) I do tend to do pretty well. Individual lecturers are rated on four areas: X “was organised and well prepared”, “was good at explaining the subject”, “was approachable and willing to help”, and “stimulated my interest in the subject”.

Now we can certainly debate whether or not these are the best factors to rate, and certainly we can query the decision to use a numerical grading system given widespread evidence of unconscious bias affecting results, with gender bias well documented and other forms of bias also likely.

However, this year’s results offered further evidence of the unreliability of such scores. As I looked through the different ratings for the different courses I contributed to, I was disappointed to see that, yet again, my contribution to a particular team-taught course in another school was pulling down my averages. This has happened before, due – I suspect – not to my weaker teaching, but to a general lower satisfaction with the course overall.

Then a penny dropped. This year I did not actually deliver my lecture on this course, as it was during the industrial action. Suddenly an 85% student satisfaction rate didn’t look so bad, given that I had achieved it without having any contact with any of the students! Indeed, looking back at last year’s responses, I only managed 86% satisfaction WITH the lecture!

So how had students found themselves so satisfied with my preparation, explanation, approachability and stimulation of interest, given my total absence from their learning experience? Presumably they had no idea who I was, and gave a general response based on their overall impression of the course.

This year the results will be removed from my record, but in future, when I do do the lecture, will the feedback be any more reliable? I think I will continue to take such scores with a very large pinch of salt.

Advertisements

About naomiappleton

I work in the Divinity School at the University of Edinburgh, where I research and teach subjects related to South and Southeast Asian religions.
This entry was posted in Academia, Teaching. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s