The World University Rankings 2007

The World University Rankings 2007 Martin Ince Communications Limited The Challenges of University Ranking Presentation by Martin Ince Contributing editor, THES, Université Libre, Brussels December 12, 2007 Martin Ince Communications Limited The THES     Since 1971 Weekly newspaper formerly associated with The Times [of London] Group including TES Online at www.thes.co.uk since 1994 Martin Ince Communications Limited Why rank universities?  - - Interest in ranking things and people Hospitals Schools Local authorities Rich lists; Britain, world, Asians, footballers Universities: The Times Martin Ince Communications Limited Does it matter?   - - Some of these rankings are fun or pornography, eg Rich Lists Others are serious Bad marks for school or hospital Likewise for a university department Over 1000 “failing” public bodies in UK – schools, police forces etc Martin Ince Communications Limited National Rankings The Times - produced by John O’Leary, former editor of THES - Institutions as well as subjects Criteria for subjects include: - Teaching quality - Research quality - Entry standards - Employability  Martin Ince Communications Limited National rankings (2)  - - - Criteria for institutions include Teaching standards Staff/student ratio Library spending Facilities spending Good degrees Jobs Research Martin Ince Communications Limited The US Comparison  - - - US News and World Report “America’s Best Colleges” Mainly about how likely you are to graduate Also student experience eg class size However, many other tables eg liberal arts, business, engineering colleges Likewise McLean’s (Canada) et al Martin Ince Communications Limited Richness of data    Main table in US News has 18 main columns Likewise Times Good University Guide How can we do this globally? Martin Ince Communications Limited Why world rankings?      Long overdue: higher education has always been very international Unique position of the THES Universities becoming more global Knowledge the real factor in international competitiveness Increasing desire for comparative information Martin Ince Communications Limited Why world rankings (2)?         GATS EU and Bologna 3 million students outside home country Forecast to be 5 million by 2010 BTA UK as a major source and destination UK as major collaborator UK universities opening in China and elsewhere Martin Ince Communications Limited And Tony says so  “In 10 years we will think nothing of students going off to university anywhere in the world” Tony Blair at the Labour Party conference, September 2006 Martin Ince Communications Limited In addition     Interest from governments – UK Treasury, Lambert EU, Germany Shanghai Jiao Tong OECD from 2010 Martin Ince Communications Limited How to do it?     - - Audience includes – Internationally mobile students Internationally mobile staff Internationally mobile money Focus on: Teaching Research International orientation Martin Ince Communications Limited Peer review     Peer review is the way academic value is measured We decided to make it the centrepiece of this ranking It is the least understood aspect of our work So here is the explanation Martin Ince Communications Limited Peer review (2)     Total 5,101 over three years International spread Subject spread Active academics Martin Ince Communications Limited The peers     Gathered by QS listbuilding 41 per cent in Europe 30 per cent the Americas 29 per cent Asia Pacific • Aggregate no more than three years Martin Ince Communications Limited The question   - - Online survey The top 30 universities in the topics they know about Arts and humanities Social sciences Science Biomedicine Technology Martin Ince Communications Limited Plusses      Simple Understandable Robust Self-correcting if large enough sample Hard to cheat Martin Ince Communications Limited Minuses  - Biases include Age Size Name - Beijing - Loughborough - Brussels, Leuven Audience conservatism Martin Ince Communications Limited Employers        Another group who know about university quality Innovation in 2005 Mainly private sector At 10 per cent of total Therefore academics cut from 50 to 40 per cent Will be improved 2007 sample 1,471 Martin Ince Communications Limited Employers are:    Americas 43 per cent Europeans 32 per cent Asia Pacific 25 per cent • Again, maximum three years aggregation Martin Ince Communications Limited Quantitative measures  - Aim to measure universities in terms of Student commitment Research commitment International commitment and competitiveness Martin Ince Communications Limited How to do this       Extensive data gathering exercise By UK firm QS Mix of data sources National Institutional Direct contact Martin Ince Communications Limited First quantitative criterion…    Staff/student ratio Classic measure of commitment to teaching 20 per cent of final score Martin Ince Communications Limited How international?     - Two criteria rated at 5 per cent each Is this somewhere where people want to be? Staff Students Again raises issues Visiting scholars? EU cross-border students? Doing full courses? Geography advantage Martin Ince Communications Limited Citations    Like peer review Classic measure of research quality Use Scopus data Martin Ince Communications Limited Citations (2) Citations per staff member to see density of brain power  Not citations per paper  Well-understood bias - against non-English publication - against arts and humanities - against national-oriented topics This accounts for the final 20 per cent  Martin Ince Communications Limited Comparison with Shanghai Jiao Tong       Not a newspaper Nobel + Fields prizes Nobel 6/9 in 2007 Fields 4/6 in 21st century These used twice Science and Nature Science and Social Science citations 500 rather than our 200 Martin Ince Communications Limited Comparison with Shanghai Jiao Tong  But: Ends up looking rather similar near the top Share 133/200   Next – OECD Martin Ince Communications Limited Changes in 2007    Scopus: better at Asia Also more transparent Better at non-English sources Martin Ince Communications Limited Changes (2) Z score: Measure distance from centre Normalise top to 100  Effects outliers such as CalTech, LSE Martin Ince Communications Limited Changes (3) Harder to cheat Main effects in South Asia  FTEs more rigorously defined Martin Ince Communications Limited What did we find?    Harvard The US – 57 in top 200 Yale Oxbridge logjam at 2 Martin Ince Communications Limited But…      The top 200 includes universities in 28 states US, UK, Australia, Netherlands Korea, China, Japan Continental Europe Developing world: Unam, Brazil (2), Cape Town 200= Martin Ince Communications Limited International commitment     US shows up badly HKUST top for international staff, LSE in sixth place London School of Economics top for international students Yale among few US with international staff Martin Ince Communications Limited Peer review      Berkeley the winner Cambridge, Oxford, Stanford, Harvard popular Well-liked universities all over the world Little evidence of patriotism bias US, UK, Australia, Japan, China, Canada,Singapore dominate the top 20 Martin Ince Communications Limited Citations     Medical faculty is a big plus Or major biomedical income CalTech the winner by some distance, then Stanford, MIT, ENS Big country effect is at work here Martin Ince Communications Limited Staff/student ratio      Winner CalTech, US US, French, Swiss, etc institutions all well placed Harvard shows badly here at 15th Asian and European universities wellplaced Weak correlation with research – but not zero Martin Ince Communications Limited Take home message    Small variations don’t matter much A position can be gained by many combinations of weakness and strength eg, many Asian institutions do well despite scoring zero on citations Martin Ince Communications Limited What we didn’t find    Data on 500 institutions with significant citations Have to teach undergraduates Have to teach in at least two areas Martin Ince Communications Limited Things that don’t work       Library spending Course cost Completion Entry standards Wealth Alumni giving Martin Ince Communications Limited Response      More work than writing the thing Last year about 30 newspaper articles in Mexico alone Interest from media, universities etc across Europe and Asia Less from the US Political response – Ireland, Malaysia, Switzerland Martin Ince Communications Limited Types of response      Reject the whole idea Complain about their position Think it is about right Wonder how to do better Compare Shanghai… Martin Ince Communications Limited Belgian institutions        Leuven 61 (was 96 in 2006) Louvain 123 Ghent 124 ULB 154 Antwerp 187 (up from 252) VUB 229 (down from 133) Liege 262 (201 in 2006) Martin Ince Communications Limited This means…     Strong showing cf comparable countries Cf Netherlands 11 in top 200, 12 in 500 Perhaps less international Martin Ince Communications Limited Good at….     Peer review Employer review Not very international Somewhat less cited Martin Ince Communications Limited How to do better      Publish more in the right places Be more international Be better represented academically around the world Have better employer links Have enough staff to teach your students Martin Ince Communications Limited The future      - - Important for staff, students, parents, management, employers Important for governments Important for business Important globally, eg for the EU Management tool: How do we look? Who to talk to? What are our ambitions? Martin Ince Communications Limited Future developments  - New data Any suggestions? Subject specific? Management indicators Refine existing data, eg from employers More global reach, eg Africa New analyses New entrants And most importantly…. Martin Ince Communications Limited Job creation     More people looking at it than doing it Leiden, CEPES IREG All highly valuable Martin Ince Communications Limited The book     Published 2006 and again early 2008 Yours from www.topuniversities.com 500 institutions including articles on the top 100 and shorter details on the rest Data in groups after 200 Martin Ince Communications Limited …really the last slide  Thanks to Gerard Kelly, editor of The THES Nunzio Quacquarelli, QS Ben Sowter, QS Staff of Scopus Martin Ince Communications Limited