Benchmarking – the good, the bad and the incomparable

By Laura Jackson, Principal Consultant / 19 March 2025

The obsession for benchmarking in higher education is all about what everyone else is doing, and whether we are doing the same. Comparisons and metrics are everywhere. Alongside the obvious measures in league tables and various frameworks (TEF, REF, KEF, etc), higher education institutions also have their comparator groups, which provide a logical starting point for having a good old nosey at what other people are doing and working out where they sit in the grand scheme of things. Notions of standard deviations, baselines and performance create a buzz of excitement as institutions try to work out just 'How are we doing?'. And on the face of it, benchmarking is an at-a-glance way for senior leaders and boards to understand what everyone else is doing, and how we are different (both better and worse). 

But the benchmarking exercise itself is hugely problematic. Don’t misunderstand, benchmarking can be a hugely useful exercise – but if not exercised with a little caution it can actually create more problems than it solves. And why is that? 

Looking backwards, not forwards  

To understand how we compare to others, we need to look at the measurable factors we are interested in. And invariably, these are things that have already happened and have been reported on. We can track our own performance against these measures and see how we fared. But this retrospective look implies that there is an ability to recreate the conditions that created those results elsewhere in the first place. Understanding the reasons why performance happened in the way that it did is just as important as the metrics themselves. The limitation to lagging measures makes it difficult to recreate the conditions for success. 

Data integrity 

One of the most common refrains in higher education is about the poor quality of data. Systems, processes and people all impact on the quality and integrity of the data we use. We are all aware of the data quality issues in our own institutions – but what about those issues in others? How do we know that the data presented by others is any good? The obvious safeguard that we have is the data provided to the Designated Data Body which has been curated and prepared by everyone to the same definitions and standards, and thoroughly assessed by the regulator. But if the last few years in the world of higher education data wrangling has taught us anything, it is that the journey our data has had to go on to get into the ‘acceptable standards’ for submission means the distance travelled in what the data now represents can be further from the truth than we would like to believe. How definitions get applied within an institution is also a mystery. What is a student? What is a course? And how simple and transparent are the ways we have of coding and standardising? Just talk to anyone involved in the great HECOS recoding of the late 2010s to know what a pain that is. If we don’t trust our own data, how can we trust others? 

'What gets measured, gets done'

Benchmarking and performance are a key part in resource allocation. And in these times of strained budgets, there is an increasing requirement to evidence that there is a need for resourcing in our areas of interest. And whilst comparison is the thief of joy, it can also be the thief of innovation. When we look to benchmark, we are reliant on there being data there to support our thinking – and more often than not, it isn’t. There is a reticence across the sector to start measuring things from a zero base – we find a strange comfort in five years’ worth of historic data to base our conclusions on. But branching out on our own is a much more uncomfortable space. We end in a self-perpetuating loop of activity based on what we can measure and compare, rather than thinking about what would make a real difference. 

Bigger Picture 

And here is the crux of the problem with benchmarking, and performance metrics in general. These are indicators which are there to support us in our strategic direction. To help us understand just how we are doing on our journey towards our goals, and to support us in developing the right strategies to get there. But without the bravery to zero base, and the luxury of being able to access the same data from others it will always be of limited value. And perhaps by embracing a collaborative approach grounded in simple, meaningful data, higher education institutions can transform benchmarking from a competitive exercise into a shared journey of continuous improvement, fostering innovation and excellence across the sector. 

If you’re interested in having a conversation with us about data confidence, strategy or governance, get in touch.