Last year I was involved in the REF exercise (a UK-wide research audit process), serving on sub-panel 31, assessing the health of Theology and Religious Studies units through their outputs, impact case studies and environment statements. Now the results are out, it amuses me to categorise some reflections within a Buddhist framework.
First up, let’s be “protestant Buddhists” for a moment and advocate a return to the scriptures! The REF website has so much information on it, including the full criteria and policies, a list of submissions (including impact case studies) and lengthy reports at both panel and sub-panel level. There is a lot there, and, as a sub-panel member bound by confidentiality agreements, there is nothing I can say that is not also there. In other words, I will always be in tune with the scripture, but I can reframe its messages to address audiences of my own 🙂
And for this particular audience I would like to reflect on the fact that the REF, like everything else, is subject to the three characteristics of existence: it is impermanent, unsatisfactory, and not-self.
As anyone working in the UK HE sector knows, the REF process has changed and will change again. Even its name has changed, and the criteria and submission rules have been different each time the exercise has run. The main change since the last time is that institutions cannot be so selective in which staff they include, but the number of outputs submitted per staff-member is lower. As such, there has been a general increase in the proportion of outputs given the higher grades of 3* and 4*, making comparison between 2021 and 2014 scores somewhat meaningless (though that hasn’t stopped people making the comparisons, of course!).
Impermanence, in a Buddhist sense, results from the inherent conditionality of the world. Everything is part of a complex system of causes and conditions, and once again this featured in the REF work. Research outputs were scored – amongst other things – according to their signficance for the field. Is the work “cumulative” (2*) or “catalytic” (3*)? Is it a “primary or essential point of reference” (4*)? This framework helped to remind me of how very interconnected all our work is as scholars. Not only do we rely on prior work, but our own contribution is of most value when it provides new insight for others. It is not only the “impact” that obviously impacts others – all our work, at its best, makes its mark on other scholars, students, and the wider world.
But the evaluation of research environments was trickier given the inherent impermanence of the world. One of the criteria was “sustainability” yet this is such a challenging thing to assess. We had to evaluate environment statements as they were presented to us, setting aside any knowledge we might have directly of departments, the pressures on them, and the changes that had happened between submission and evaluation. And one of the hardest things about the REF is knowing that its results will inform all sorts of subsequent decisions about departments up and down the country, especially in a small and vulnerable field like TRS.
And this of course is one of the reasons the REF is unsatisfactory, or dukkha. However carefully the exercise is done, its results will always be used for unintended purposes, and groups of people will selectively quote or compare scores, or seek to use the information to suit different ends.
But while we might reasonably debate whether or not the REF should exist, and how the criteria and different aspects rank, it was reassuring to me to see how robust the process was. It is at its heart a massive academic peer-review exercise, and we should have faith in its results in that respect.
This time around there was a new aspect to the dukkha of the process: long (and many) Zoom meetings! Although I don’t believe the need to mostly conduct our meetings on Zoom in any way compromised the process, it was impressively exhausting. I was sad not to have the chance to spend real time with other panel members, and I was troubled by back and neck pain (though even that turned out to have a silver lining in ways too unexpected to relate). Yet we all pushed through, accepting the unsatisfactoriness in the process, while making it the best it could be under the circumstances.
Looking back, I can see that the lack of self or ego in the REF process is why it was so enjoyable to be a part of. My sub-panel colleagues were so amazing to work with, because we all took the work incredibly seriously, but none of us took ourselves seriously. This was academics at our best.
The lack of self is also why we shouldn’t fear the REF, for it really is an assessment of groups not individuals. Nobody ever finds out their individual score for outputs, and nobody ever will. It is about the health of the field, and of units at institutions. For the field, we saw plenty of positive signs, with a wide variety of valuable outputs, and some great early career scholarship coming through, as well as some really impressive examples of research impacting wider society.