New Parameters Can Upset QS World University Rankings
Quacquarelli Symonds (QS) World University Rankings 2019 places the two Australian universities, The University of Sydney and The University of Melbourne at 5th and 6th spot in Graduate Employability, ahead of the University of Oxford, University of Cambridge, UC Berkeley and ETH Zurich. Interestingly, both the Australian universities don’t even feature in the top 30 World universities.
In another instance, ‘QS World Ranking’ forgot to place KU Leuven in its Graduate Employability Ranks. Notably, KU Leuven is one of the best European universities ranking 81st in the QS Overall University Ranks and Graduate Employability counts towards 10% of the overall university score.
This is difficult for me to assimilate and this article would give an insight into some inherent flaw and bias in the ranking system.
Ranking Parameters
Quacquarelli Symonds (QS) ranks the universities using 6 parameters. The following infographic gives an insight into the parameters.
Clearly, academic reputation weighs 40% of the overall score. Employer Reputation is a purely qualitative factor which is measured using a questionnaire. Academic reputation involves a mix of qualitative questionnaire and quantifiable statistics.
Citations per faculty is collected from SCOPUS data. Faculty and Student data of the universities are collected from the national database or directly from the university.
1. Regional Bias In Academic Reputation
29 UK-based universities feature in the top 200 ranks.
Academic Reputation counts towards 40% of the overall score in QS rankings. It is affected by the regional bias of the people answering the questionnaire. In one of the critical reviews on the World Rankings, Mu-Hsuan Huang talks about the regional bias in the questionnaires in his article published in the journal Research Evaluation.
According to Mr. Huang, “ The way questionnaires were distributed and calculated provided clues that QS Rankings generally tended to be more advantageous for the Commonwealth Nations like UK and Australia.”
He stated that the surveyed population who answered the questionnaires in 2012 QS rankings pertaining to the Academic Reputation comprised of 8.86% UK professionals, next to the US. Mr. Huang found that there is a positive correlation between the number of return questionnaires from a country and the performance of its universities.
Interestingly, 3.59% of questionnaires were returned from Indonesia and 3.16% from Philippines, which are not even placed in the top 50 countries by research output in Nature Index. France, Sweden, and Russia which are significant contributors to scientific research contributed to merely 1.97%, 1.45% and 0.65% of the answered questionnaires.
Removing the regional bias can definitely turn the tables for German, French and Scandinavian universities.
2. Nobel Laureates As A Measure Of Academic Reputation
Nobel Prize, Field Medal and Pulitzer Prize winners bring laurels to a university. The number of such winners is a quantifiable measure of the Academic reputation.
Currently, the academic reputation measurement method is mostly a qualitative method that involves answering a few questionnaires by people whom QS calls ‘experts’.
QS should include ‘People with Academic Distinctions’ as part of the Academic Reputation. Further, it should be normalized for Engineering as Nobel Prize and many other famous prizes are not awarded for excellence in Engineering. Instead, QS can include highly-cited researchers from Clarivate Analytics as a measure of the ‘Quality of Academics’.
3. Reducing Weight Of Academic Reputation To 25%
Image Source: University of Bologna Website
‘Alma Mater Studiorum-University of Bologna’, founded in 1088, is considered to be the oldest university in the Western World. It ranks 180 in QS Rankings 2019 with an overall score of 45.9/100. It scores 72.1/100 in academic reputation placing it at 77th rank. How can a reputed university which ranks in the top 100 for its reputation in the World be ranked so low?
Now have a glance at other parameters. It scores a mere 7.3/100 in terms of faculty: student ratio because the overall strength of the university is over 75,000. It scores a little over 10/100 in terms of international outlook. The normalized research score for citations per faculty is 46/100.
The rank of the University of Bologna doesn’t correspond to the quality of research or academic work from the scores. It seems that the high weight on Academic reputation still keeps the university under 200 although it is not close to its similarly ranked German and Dutch counterparts in terms of research budget and infrastructure.
4. Graduate Outcome
QS should include Graduate Outcome and Ease of Getting The First Job rather than merely employer reputation in its overall university ranking parameters. These are significant factors which QS ignores. In fact, for most students getting a job is the ultimate purpose of university education and that’s what matters to the most. (I mentioned ‘University Education’ and not ‘education’) Further, a robust ranking system should also be able to capture the non-traditional career choices made by the alumni.
Relevant: Graduate Outcome Parameters by HESA
Under the current circumstances, getting a job after graduation for international students in the UK is a tough job, especially for international students. The UK allows the international students only 4–6 months to look for a job after graduation. With most UK postgraduate Master’s degree lasting 1 year and often without a thesis, it is tough for a student to build a professional network and find a job immediately without work experience. The Guardian reports the harsh reality, “Get a job or get out.”
On the other hand, most abroad destinations like the US, Canada, Germany, France, and the Netherlands offer at-least 1 year post-study Visa to find work. Further, the Master’s degree in these countries has a duration of 2 years.
Relevant: Public Support For International Students In UK
If the weight on graduate employability is increased to 25% and the evaluation doesn’t merely rely on Employer Reputation, I am quite sure less than 20 UK universities would find a place in the top 200 World ranks.
Note: Currently, QS measures only Employer Reputation and it counts towards 10% of the overall score.
5. Local Recruiters vs International Managers
Universidad Nacional de Colombia ranks higher than CalTech, Carnegie Mellon and the University of Waterloo in employer reputation of QS Ranking. US News Rankings places the Colombian university at 713th spot in the World rankings. Even QS ranking places it at 272. So, it’s surprising to find the Colombian university so high in the ranking table of Employer Reputation.
The Indian Institute of Technology (IITs) are quite popular among recruiters in India and abroad. Unlike European countries, companies come to recruit students to universities in India. Most of the IITs see on-campus recruitment of over 80% of its graduates. The remaining students either end up in higher education or find a job off-campus. Typically, 80% of the students in IITs are working in an industry 4 months after graduation.
The ‘IITs’ have a global brand image. Recruiters like Oracle, Optiver, Samsung, Cohesity, Micron Semiconductor, Works Applications and Microsoft hire at least 200 students from different IITs for overseas positions right as a fresher. International job offers have seen a surge year after year now. 2018 has seen a significant rise in Japanese recruiters.
However, IITs are placed way in the bottom, beyond 140 in QS Employer reputation. IIT Bombay scores 63.3 in Employer Reputation and IIT Madras 35.6. This is contrary to the National Index Data and employers’ perception.
Being an alumnus from an IIT, I am speculative that there is an inherent selection bias of the employers or performance bias of the evaluators of the questionnaire. I am quite sure that judging the employer reputation by giving different weights to the National Recruiters And International Managers would disrupt the ranks. This is likely to give a boost to the German and Indian universities.
6. Industry Outcome
Come on, industry-academia partnership can boost the research in a university. Collaborations with industries play a crucial role in knowledge transfer. Yes, let me tell you that many of these collaborations don’t yield a peer-reviewed journal publication. However, they often yield patents. QS ranking doesn’t take into account the patents.
Don’t you think including granted patents is also a measure of research output? Is it just merely the number of citations which matter?
Conclusion
I have cited several bizarre statistics in the QS rankings 2019.
I am a Masters student in Biomedical Engineering at RWTH Aachen, Germany. Currently, QS publishes ranks of universities in Anthropology with less than 100 universities, but not in Biomedical Engineering. In the summer of 2018, I had to choose between Imperial College London, Carnegie Mellon University, University of Glasgow, KU Leuven and RWTH Aachen. Except for RWTH Aachen, the rest 4 universities were ranked in the top 100 in QS rankings. QS places RWTH Aachen at 144 while Times Higher Education ranking, another popular ranking system places it at 78. That’s when I found the loopholes in the ranking system. I hope QS rankings become less biased in the coming years.
Disclaimer: The views expressed by the author are his own and doesn’t involve any endorsement. You are free to comment on this and bring to my notice any point that’s devoid of logic.
