Limited Numbers: What University Rankings Can (and Can’t) Tell Us

Times Higher Education World University Rankings (Photo credit: Wikipedia)

by Gwilym Croucher, Higher Education Policy Adviser at University of Melbourne, The Conversation: http://theconversation.edu.au

The release of The Times Higher Education World University Rankings will be welcomed by many people in the Australian university sector.

---

See the full list of The Times Higher Education World University Ranking here
View rankings by region

---

Australia now has eight universities in the top 200, one more than last year, with the University of Adelaide joining this top grouping at 176.

Six of this group improved their positions, with Melbourne University rising to 28 (up from 37 last year) while ANU, moved from 38 to 37. The other Australian universities were: Sydney University (62), Queensland University (65), UNSW (85), Monash (99) and UWA (190).

Australian Institution	2011-12 rank	2012-13 rank
University of Melbourne	37	28
Australian National University	38	37
University of Sydney	58	62
Queensland University	74	65
University of New South Wales	173	85
Monash University	117	99
University of Adelaide	201-225	176
University of Western Australia	189	190

Sourced from The Times Higher Education World University Rankings.

But an important story here is not how well individual universities have done, but what the rankings say about the sector overall.

What goes into the rankings ‘sausage’

While this is generally good news for Australian universities, we should view the results carefully and be mindful of the limitations in the story they tell us. The challenge of rankings, is recognising their value without using them in perverse ways.

The Times Higher Education (THE) ranking was developed in 2004, based initially on surveys of reputation, staffing ratios, research and other indicators. In recent years, the majority of the ranking (around 70%) has been derived from indicators around research, predominantly research reputation, citations and funding. The rest of the ranking captures teaching, learning and a few other markers.

This creates limits in what the rankings can tell us. The reliance on surveys means that “perceptions” can distort the story that (more) objective indicators, such as citations, tell us about an institution. The rankings become a delicate mix of hard data and ingrained prejudice by the world’s academics.

A second issue is the (necessarily) retrospective nature of much of the hard data. Citations measures pick up research often started a decade or longer ago, failing to tell us much about important research just building momentum now.

The bigger picture

Despite the limitations in what any ranking can tell us, they still have an important story to tell. It is not what individual institutions have done in the past or how their peers view them, but rather that the Australian system is doing well, and in this way we “punches above our weight” (to use the obligatory phrase since Les Darcy when discussing anything Australia does well in the international arena).

Like any imperfect proxy, the rankings of individual institutions hint at the health of the system overall, even if there are inevitable instances where we can do better. As the Times press release reflects, Australia does well on the average movement of our top 200 institutions, with our universities from this top group raising an average of 15 places.

To read further, go to: http://theconversation.edu.au/limited-numbers-what-university-rankings-can-and-cant-tell-us-9949?utm_medium=email&utm_campaign=Latest+from+The+Conversation+for+4+October+2012&utm_content=Latest+from+The+Conversation+for+4+October+2012+CID_f854c801de6c785bfe260d692fd58edc&utm_source=campaign_monitor&utm_term=Limited%20numbers%20what%20university%20rankings%20can%20and%20cant%20tell%20us