Academic Ranking of World Universities Methodologies and Problems

Document Sample
Academic Ranking of World Universities Methodologies and Problems Powered By Docstoc
					Academic Ranking of World Universities
     Methodologies and Problems


1   Purposes

2   Methodologies & Results

3   Problems & Discussion

4   Ranking by Subject Fields

5   Final Remarks


          Dream of Chinese for WCU

 World-class university (WCU) is a dream for
   generations of Chinese. It’s not only for pride, but also
   for the future of China.

 Recently, Chinese government has launched several
   initiatives for research universities. The best-known one
   is specially designed to build WCU (985 Project).

      Goals of Top Chinese Universities

 Many top Chinese universities have setup their
   strategic goals as WCU.

 Most of them have also set time tables for achieving the
   goal of WCU. For example:
   2016 for Peking University
   2020 for Tsinghua University

           Questions About WCU

 Is there a clear definition for WCU?

 How many WCU should there be in the world?

 What are the positions of top Chinese universities in
   the world?

 How can Chinese universities improve themselves to
   reach the goal of WCU?

 Academic Ranking of World Universities

 Our original purpose of doing the Academic Ranking of
   World Universities (ARWU) was to find out the position
   of Chinese universities in the world and the gap between
   them and WCU.

 ARWU was put on the internet upon the encouragement
   of colleagues from all over the world.

                 Features of ARWU

 ARWU uses a few carefully selected, objective criteria and
   internationally comparable data that everyone could
   verify in some way.

 It has been carried out by a ranking team of four
   researchers at Shanghai Jiao Tong University for their
   academic interests.

 It has been done independently without any financial
   support from any external sources.

 & Results

              Selection of Universities
 Any university that has any Nobel Laureates, Fields
  Medals, Highly Cited Researchers, or papers published
  in Nature or Science.

 Major universities of every country with significant
  amount of papers indexed by Thomson.

 Number of universities scanned: >2000

 Number of universities actually ranked: >1000

 Number of ranked universities on our web: 500

Ranking Criteria and Weights

         Definition of Indicator: Alumni
 The total number of the alumni of an institution
  winning Nobel Prizes and Fields Medals.

 Alumni are defined as those who obtain bachelor,
  Master’s or doctoral degrees from the institution.

 Different weights are set according to the periods of
  obtaining degrees. The weight is 100% for alumni of
  1991-2000, 90% for alumni of 1981-1990, 80% for
  alumni of 1971-1980, and so on.

 If a person obtains more than one degrees from an
  institution, the institution is considered once only.

            Definition of Indicator: Award
   The total number of the staff of an institution winning Nobel
    prizes in physics, chemistry, medicine and economics and Fields
    Medal in Mathematics.
   Staff is defined as those who work at an institution at the time of
    winning the prize.
   Different weights are set according to the periods of winning the
    prizes. The weight is 100% for winners since 2001, 90% for
    winners in 1991-2000, 80% for winners in 1981-1990, 70% for
    winners in 1971-1980, and so on.
   If a winner is affiliated with more than one institution, each
    institution is assigned the reciprocal of the number of institutions.
   For Nobel prizes, if a prize is shared by more than one person,
    weights are set for winners according to their proportion of prize.
          Definition of Indicator: HiCi

 The number of highly cited researchers in 21 broad
  subject categories in life sciences, medicine, physical
  sciences, engineering and social sciences.
 The definition of categories and detailed procedures
  can be found at the website of Institute of Scientific
 The total number of HiCi is about 5000, about 4000
  of which is university staff.

          Definition of Indicator: N&S

 The annual average number of articles published in
  Nature and Science in the past five years.
 To distinguish the order of author affiliation, a
  weight of 100% is assigned for corresponding author,
  50% for first author (second author if the first
  author is the same as corresponding author), 25%
  for the next author, and 10% for other authors.
 Only publications of article type are considered.

          Definition of Indicator: PUB

 Total number of articles indexed in Science Citation
  Index-expanded (SCIE) and Social Science Citation
  Index (SSCI) in the past year.

 A weight of 2 is assigned to articles indexed in SSCI
  to compensate the bias against humanities and social

 Only publications of article type are considered.

          Definition of Indicator: PCP
 The sub-total scores of the above five indicators
  divided by the number of full-time equivalent
  academic staff.

 If the number of academic staff for institutions of a
  country cannot be obtained, the total scores of the
  above five indicators is used.

 For ranking 2005, the number of full-time equivalent
  academic staff is obtained for institutions in USA,
  China, Australia, Italy, Netherlands, Sweden, and
  Belgium etc.
        Main Sources of Data
 Nobel laureates:
 Fields Medals:
 Highly-cited researchers:
 Articles published in Nature and Science:
 Articles indexed in SCIE and SSCI:

                  Results of ARWU
 Top 500 universities in the world
 Top 100 universities in North and Latin America
 Top 100 universities in Asia/Oceania
 Top 100 universities in Europe
 Statistics of top universities by region and country
 Percentage distribution of top universities by country
  as compared with the share of global population and
Problems &

   Methodological: Education and Service

 Education is the basic function of any university, however,
  it would be impossible to rank the quality of education
  due to the huge differences among the national systems.

 Contribution to the national economic development is
  becoming increasingly important for universities, however,
  it is impossible to obtain internationally comparable
  indicators and data.

 The academic or research performance of universities, a
  good indication of their reputation, can be ranked

  Methodological: Humanities & Social Sciences
 Many well-known institutions specialized in humanities
  and social sciences are ranked relatively low.

 Since 2004, the indicator of N&S is not considered for
  institutions specialized in humanities and social sciences,
  its weight is relocated to other indicators.

 Since 2005, a weight of 2 for articles indexed by SSCI is

 Nevertheless, if a university specialized in social sciences
  and humanities had Nobel Laureates in economics and
  Highly Cited Researchers in social sciences, it should have
  good standing.
         Methodological: Language Bias
 English is the language of international academic

 Any ranking based on academic performance will be
  biased towards institutions in English-speaking

 One possible solution: papers published in non-native
  languages are offered a special weight.

 Another possible solution: normalization of total
  articles by the proportion of journal editors of each
      Methodological: Award and Alumni
 Universities which started after 1911 do not have a fair

 Disciplines not related to the awarding fields do not have
  a fair chance. Other important awards include Abel,
  Pulitzer, Turing, Tyler, Pritzker, etc.

 Institutions for winning awards and those for doing the
  researches may not be the same.

 Institutions for obtaining degrees and those for pursuing
  the studies may not be the same.

 Postdoctoral training is not considered.
    Methodological: Per Capita Performance

 The weight of the PCP indicator for per capita
  performance is rather low. Large institutions have
  relatively high positions in the ranking.

 However, it’s very difficult to obtain internationally
  comparable data on the number of academic staff.

 The types of academic staff: such as purely teaching staff,
  teaching and research staff, purely research staff.

 The ranks of academic staff: such as professor, associate
  professor, reader, lecturer, research scientist etc.

              Technical: Attributions

 Many universities have more than one commonly used
  names: such as Virginia Tech and Virginia Polytechnic
  and State University.

 Variations due to translation: such as Univ Koln and
  Univ Cologne, Univ Vienna and Univ Wien.

 Abbreviated names: such as ETH Zurich for Swiss
  Federal Institute of Technology Zurich.

 Some authors only write their departmental or institute
  names without mentioning their university names.

      Technical: Definition of Institution

 University systems: such as Univ California system,
  Univ London system.

 Affiliated institutions and research organizations:
  such as Ecole Polytechnique Montreal (affiliated to
  University of Montreal), CNRS Labs (affiliated to
  French universities).

 Teaching and affiliated Hospitals: complex!

 Our answer: according to author’s expression.

            Other Technical Problems

   Merging, splitting, inheriting, discontinuing, name-
   changing of institutions such as:

 Univ Kwazulu-Natal in South Africa merged from Univ
  Natal and Univ Durban-Westville.

 University of Innsbruck in Austria splitted into Univ
  Innsbruck and Innsbruck Medical Univ.

 Humboldt Univ Berlin and Free Univ Berlin inheriting
  the Nobel Prizes of the Berlin University before world
  war II.

    Ranking by
Broad Subject Fields

  Requests for Ranking of World Universities by

 Broad subject fields or schools, colleges


 Subject fields or programs, departments

★ In addition, many top Chinese universities want to
  learn their positions in the world by broad subject
  fields or disciplines.

        Definition of Broad Subject Fields

 Natural Sciences and Mathematics (SCI)

 Engineering/Technology and Computer Sciences (ENG)

 Life and Agriculture Sciences (LIFE)

 Clinical Medicine and Pharmacy (MED)

 Social Sciences (SOC)

 Arts and humanities are not ranked

 ARWU-FIELD Indicators and Weights

Code     SCI   ENG        LIFE   MED   SOC

Alumni   10%              10%    10%   10%

Award    15%              15%    15%   15%

 HiCi    25%   25%        25%    25%   25%

 TOP     25%   25%        25%    25%   25%

 PUB     25%   25%        25%    25%   25%

Fund           25%

     Changes in Indicators and Definition
 N&S in ARWU is not used in ARWU-FIELD.

 TOP is the percentage of articles published in the top 20%
  journals of each broad subject field.

 Fund is the total engineering-related research
  expenditures. It’s used only for ENG ranking.

 Alumni and Award since 1951 are used for all rankings
  fields except ENG.

 PUB is the total number of articles indexed by Thomson
  in the past year.
             Results of ARWU-FIELD

 Top 100 universities in SCI

 Top 100 universities in ENG

 Top 100 universities in LIFE

 Top 100 universities in MED

 Top 100 universities in SOC

 Statistics of top universities by region & country

 List of top universities by number of top fields
      Special Problems in ARWU-FIELD

 It’s difficult to obtain data on engineering-related
  research expenditures and make them comparable.
  For 2007, Fund was obtained only for US and
  Canadian universities.

 It’s difficult to separate the Nobel Laureates in
  Physiology or Medicine. They are used in both LIFE
  and MED ranking.

Final Remarks

             Controversy of Ranking

 Any ranking is controversial and no ranking is
  absolutely objective.

 University rankings become popular in many
  countries. Whether we agree or not, ranking systems
  clearly are here to stay.

 The key issue then becomes how to improve ranking
  systems for the benefits of higher education (IREG).

                  Use of Ranking

 Rankings should be used with cautions. Their
  methodologies must be read carefully before
  reporting or using their results.

 Rankings should be used in combination with other
  types of evaluation whenever possible, such as
  bench-marking, peer review, etc.

                 Future Efforts

 Study all the above mentioned problems and
  continuously improve the ranking methodologies.

 Establish more comprehensive databases of WCU.

 Update ARWU (every August) and ARWU-
  FIELD annually (every February).

            Future Efforts (Cont’ed)

 Provide ranking of universities specialized or
  strong in engineering, medicine, etc. based on the
  classification of world universities.

 Provide ranking of universities with different size,
  history, budget and function etc. once
  internationally comparable data are obtained.