How reliable are student evaluations of teaching?

Last week I got my results back on semester 2’s teaching, and much hilarity ensued!

As many universities, Edinburgh now uses an online system to collect numerical scores on each course, along with some free text comments. The comments are sometimes very helpful, though I found the old system of distributing paper forms in class much more successful in this regard, both in terms of response rates and usefulness of comments.

The numerical scores offer some satisfaction too, since (humblebrag alert) I do tend to do pretty well. Individual lecturers are rated on four areas: X “was organised and well prepared”, “was good at explaining the subject”, “was approachable and willing to help”, and “stimulated my interest in the subject”.

Now we can certainly debate whether or not these are the best factors to rate, and certainly we can query the decision to use a numerical grading system given widespread evidence of unconscious bias affecting results, with gender bias well documented and other forms of bias also likely.

However, this year’s results offered further evidence of the unreliability of such scores. As I looked through the different ratings for the different courses I contributed to, I was disappointed to see that, yet again, my contribution to a particular team-taught course in another school was pulling down my averages. This has happened before, due – I suspect – not to my weaker teaching, but to a general lower satisfaction with the course overall.

Then a penny dropped. This year I did not actually deliver my lecture on this course, as it was during the industrial action. Suddenly an 85% student satisfaction rate didn’t look so bad, given that I had achieved it without having any contact with any of the students! Indeed, looking back at last year’s responses, I only managed 86% satisfaction WITH the lecture!

So how had students found themselves so satisfied with my preparation, explanation, approachability and stimulation of interest, given my total absence from their learning experience? Presumably they had no idea who I was, and gave a general response based on their overall impression of the course.

This year the results will be removed from my record, but in future, when I do do the lecture, will the feedback be any more reliable? I think I will continue to take such scores with a very large pinch of salt.

Advertisements
Posted in Academia, Teaching | Leave a comment

Digital scroll of the Vessantara Jataka

Following my post yesterday, about a digitised scroll of the Mahābhārata, Leedom Lefferts got in touch to share a link to a digitised scroll of the Vessantara Jātaka! It is a rather different – even more magical – experience, since you get to walk virtually around an exhibition and view the hanging illustrated scroll within, as well as the other artefacts included. You can also zoom in to appreciate the detail.

DevotionThe exhibition was called “DEVOTION – Image, Recitation, and Celebration of the Vessantara Epic in Northeast Thailand” and was curated by Thomas Kaiser at the University of Zurich’s ethnographic museum this past year. Sadly I could not make it to the exhibition itself, though I have enjoyed perusing the catalogue, which is really a substantial scholarly book, with the same title as the exhibition. Now I can also transport myself into the room and investigate all the treasures on display!

Do take a tour! Enjoy!

Posted in Academia, Buddhism, Jataka, Publications, Religious narrative | Leave a comment

Mahabharata scroll digitised

A couple of years ago, when teaching a course on the Hindu Epics, I took my class to the Centre for Research Collections at the University’s main library to look at a late 18th century Mahabharata scroll. It was stunning. Housed in a glass-topped wooden box and mounted on rollers, the scroll was moved using a key in the side of the box. At around 70 metres long, it took some time to move through the manuscript, admiring the tiny tiny Devanagari characters and the beautiful illustrations.

scroll

I am delighted to say that now everyone with a decent internet connection can also scroll through the manuscript. While you may not get the thrill of watching the scroll move as you turn a key, you do get to see the scroll magnified, and without the reflections from the glass top. You can zoom in to the small text and read, or admire details in the painted miniatures. It really is a treasure.

You can see it at the following link:

A library blog post on the project giving some more details about how the conservation and digitisation came about can be found here: http://libraryblogs.is.ed.ac.uk/diu/2018/06/22/a-stitch-in-time-mahabharata-delivered-online/

Please do take a look, for idle pleasure or for research purposes!

Posted in Academia, Religious narrative, Teaching | Tagged | 1 Comment

Daily Writing and Daily Reading

I was recently identified – wrongly – as one of those people who does “daily writing”. Although I know the comment was meant positively – an obvious cause for my apparent productivity – I found myself strangely affronted by the idea. I have never really liked the fashion for writing in little tiny pockets of time (such as “pomodoros” of twenty-five minutes), nor the implication that we therefore don’t need long expanses of time in order to do research.

Each of my three monographs was written during a sustained period of research – my PhD, a three-year postdoc, and a funded research project. Even within these periods of research I tended to have bursts of writing and long months without. I am not a magician who can write books around the edges of normal teaching and administrative duties.

The only piece of research I managed to fit around the last semester’s workload was a single conference paper. I read and planned and sketched out the overall structure in snatched half-days, then wrote almost the entire thing in a single day, at home on the sofa. That was the grand total of my writing for January to April. Unless you count the 100-page application for the School’s Athena SWAN award, and even that was not written in daily pockets, but in sustained days and half-days.

The only time I have done anything that resembled daily writing was during the final year of my PhD, when I set myself the task of writing 1,000 words per day. I would start at around 8am, curled up at my desk in my PJs, with a pot of tea by my side. Usually I would finish writing late morning, and then spend the afternoon reviewing notes or sources ready for the next day’s writing. (I would also walk into town to use the university facilities to check my emails, as I did not have internet at home at that point – happy undistracted days!) In other words my daily writing was part of a broader routine of doing research all day. It was not tucked in around the edges of other responsibilities.

Daily reading is, however, a routine I return to periodically. I find it particularly useful at this time of year, when various duties are still punctuating my week – meetings, marking and moderating, reviewing and planning – but when I am also trying to get my head back into research mode. As I reminded myself through various futile attempts last week, I cannot really write at this time of year, as there are too many interruptions and my brain is too tired. Remembering to read for a few hours every day helps me to gradually reorient and refresh my brain. Then, later in the summer, I hope to have a few sustained weeks of writing!

Posted in Academia | Leave a comment

On student finances

Last month I paid off the last of my student loan.

I was one of the first generations of UK students to pay my own way at university. I paid no fees (they were means-tested in those days, and only about a thousand pounds even if you did pay) so my loan was just the total of three years of living costs in a fairly affordable city. I worked part-time jobs alongside my degree, and also during the summer breaks. And I was fully funded throughout my postgraduate studies. In other words, I was very lucky and fairly frugal, and yet I still had to wait until ten years into my career before clearing the debt.

What have we done to the next generations? Nine thousand pounds per year of fees debt, plus the soaring costs of living (especially renting), adds up to a pretty hefty loan. Most graduates will never pay off their debt, and indeed some politicians like to say it isn’t really debt because large chunks won’t be repaid, but it sure feels like debt to the students!

Meanwhile there’s a real squeeze on postgraduate funding while more and more students are recruited to boost the income streams of the universities. When I was planning for my MPhil and doctorate I applied directly to the national funder (the AHRC) and was judged against the national pool of applicants. Nowadays the funding is allocated to consortia in a manner I don’t fully understand, but I do understand this: It is no longer a level playing field. Weak applicants get funding in one region while strong applicants are denied funding in another region. Meanwhile local funding schemes are often fees-only, meaning that students have to accrue even more debt in pursuit of an elusive career.

So paying off my student debt makes me feel very lucky. I am wholeheartedly grateful to the tax-payers and government funds that made all my degrees possible. However, it also makes me sad, knowing that if I was going through the system now I would be in a very different position, and perhaps would not have felt able to pursue my higher degrees and this career.

Sorry, next generation. We messed up.

Education is a public good as well as a private good. I want my taxes to support students, as I was supported by the taxes of others. Time for a new system, please, or a return to the old.

Posted in Academia | Leave a comment

New project: Jatakas in Indian texts and art

As I sit at home watching the snow build up outside the window (the university is closed for a second day running due to the weather) I am musing about my new research project, an exploration of the uses of jātaka stories in early Indian texts and art.

In many ways it is not a new project at all. My doctoral work, into the Pāli jātaka tradition, set me on a long and ongoing quest to explore how the jātaka genre is understood in a range of Buddhist contexts. Since my work on the Pāli tradition I have also worked on the jātakas of the Avadānaśataka, and spent some time looking at those in the Mahāvastu. I have also always had an interest in how (and why) jātakas are present as visual depictions at Buddhist stūpa and temple sites, though I never have a helpful answer when someone sends me a photo and asks “what’s this?”!

However, thanks to a recent award of a Philip Leverhulme Prize, I will soon be able to approach my jātaka research more systematically, and with the help of a research resource that I hope will also be of use to others. The prize is allowing me to employ a Research Assistant (for twelve months full-time for the 2018/19 academic year) as well as get the technical support necessary to create an online searchable database of jātakas in Indian Buddhist texts and art.

The creation of such a resource has been a longstanding ambition, and the idea developed through long conversations with my friend and fellow jātaka-enthusiast Arthid Sheravanichkul of Chulalongkorn University. I look forward to finally getting it off the ground, with the help of a Research Assistant, and with the guidance of a number of scholars – including Arthid – who have agreed to form an advisory team for the project.

Once complete, the database should allow a variety of search functions, making it easy, for example, to look up all stories about jackals, or all stories that feature the Buddha’s mother, or all stories addressing the theme of deceit. It should also provide a bank of images of stories in art, helping with future identifications and scholarship. These features will certainly help me in my research, and I hope they will also be of use to other scholars, of art or text, as well as to Buddhists and other story enthusiasts.

If you know of anyone who might be interested in the Research Assistant post please do encourage them to apply. They must have a PhD (or be due to have one by the start of September) in Buddhist Studies or another relevant field (eg early Indian religion). Further details are available in the job advert here: http://www.jobs.ac.uk/job/BHY516/research-assistant/.

Once semester is over and time opens up once more I hope to post further details of the plans for this resource and the wider research project that I envisage building around it.

Posted in Academia, Buddhism, Buddhist texts, Jataka, Religious narrative | Leave a comment

Why I spent my morning on a picket line

This week I am on strike, for the first time in my life. I spent four hours this morning in the freezing cold handing out leaflets in front of New College, the building where I usually work. I did the same yesterday morning, and I will do the same again tomorrow unless the strikes are called off.

The reasons for the strike are set out clearly on the website of the Universities and Colleges Union. In short, the higher-ups have decided – on the basis of doubtful calculations – that our pension scheme needs reform, and that this reform should involve utterly devaluing the scheme for members. Not only will there be a huge cut in benefits as a result, there will be no predictability, as the value of individual pension pots will depend entirely on the vagaries of the market. Risk will be individual, not collective, and each of us will have to take a gamble based on how long we think we might live. It is, to put it bluntly, a disaster for all of us – academics and professional support staff – in the scheme.

I don’t consider myself a greedy person. I didn’t oppose previous devaluations of the pension, such as the closing of the final salary scheme or the introduction of a cap to the defined benefits scheme. All I want is a decent and predictable income in my retirement. I don’t really see that this is too much to ask.

But this strike is also about so much more than pensions. It is about a system where university management can make decisions that impoverish all of its staff without proper consultation. It is about a system where student fees – as our own PM recently admitted (and it isn’t often that I agree with her) – bear no relation to how much a course is worth or costs to provide, and relate only to the amount universities think they can get away with charging. Fees are soaring (and likewise student debt, which is now scandalous), pay has stagnated, an obscene proportion of UK academics are employed on precarious contracts, and now they want our pensions too? All while the Vice-Chancellors take their pay packets of hundreds of thousands of pounds, and spend millions more on vanity building projects? No thank you.

I love my job. It is stressful at times, as I have shared with honesty in previous posts. But I love being an educator and a researcher, and evidence would suggest that I’m quite good at both. However, I am starting to feel embarrassed about working in universities, which are becoming more and more like businesses. I passionately believe in the social good of higher education, and I like to think that I work in the public sector, for the good of others. These strikes have demonstrated plainly to me that I am far from the only person who worries that our university system is going down the pan in pursuit of profit.

So striking really has been an act of solidarity. We’ve sung and danced and eaten cake, and handed out “solidarity biscuits” alongside leaflets. The support from students has been overwhelming and humbling. They are losing valuable education, yet they can see the bigger picture, and they understand that none of us have taken the decision to strike lightly. Fingers crossed that negotiations re-open and we can all get in out of the cold, back into our offices and lecture halls where we want to be.20180226_082226

Posted in Academia | Tagged , , | Leave a comment