Document Sample
LIU Powered By Docstoc
					         Academic Ranking of World Universities
                 Methodologies and Problems

                           May 15, 2007

                  By Professor Nian Cai Liu
Institute of Higher Education and Center for World-Class Universities
             Shanghai Jiao Tong University, China


 Purposes

 Methodologies & Results

 Problems & Discussion

 Ranking by Subject Fields

 Final Remarks


        Dream of Chinese for WCU

 World-class university (WCU) is a dream for
  generations of Chinese. It’s not only for pride,
  but also for the future of China.

 Recently, Chinese government has launched
  several initiatives for research universities.
  The best-known one is specially designed to
  build WCU (985 Project).

    Goals of Top Chinese Universities

   Many top Chinese universities have setup
    their strategic goals as WCU.

   Most of them have also set time tables for
    reaching the goal of WCU. For example:
    2016 for Peking University
    2020 for Tsinghua University

           Questions About WCU

   Is there a clear definition for WCU?

   How many WCU should there be in the world?

   What are the positions of top Chinese
    universities in the world?

   How can Chinese universities improve
    themselves to reach the goal of WCU?

       Academic Ranking of World Universities

   Our original purpose of doing the Academic
    Ranking of World Universities (ARWU) was to find
    out the position of Chinese universities in the
    world and the gap between them and WCU.

   ARWU was put on the internet upon the
    encouragement of colleagues from all over the
    world. There have been an average of 2000
    visitors every day since 2003.

                Features of ARWU

   ARWU uses a few carefully selected, objective
    criteria and internationally comparable data that
    everyone could verify in some way.
   It has been carried out by a ranking team of four
    persons in the Institute of Higher Education of
    Shanghai Jiao Tong University for their academic
   It has been done independently without any
    financial support from any sources outside the
    Institute of Higher Education.

Methodologies & Results

             Selection of Universities

   Any university that has any Nobel Laureates, Fields
    Medals, Highly Cited Researchers, or papers published
    in Nature or Science.

   Major universities of every country with significant
    amount of papers indexed by Thomson.

   Number of universities scanned: >2000

   Number of universities actually ranked: >1000

   Number of ranked universities on our web: 500

               Ranking Criteria and Weights

           Criteria                             Indicator                        Code      Weight

         Quality of          Alumni of an institution winning Nobel
                                                                     Alumni                  10%
          Education           Prizes and Fields Medals
                              Staff of an institution winning Nobel
                                                                                 Award       20%
                              Prizes and Fields Medals
     Quality of Faculty
                              Highly cited researchers in 21 broad
                                                                                  HiCi       20%
                              subject categories

                              Articles published in Nature and Science*          N&S         20%
      Research Output
                              Articles in SCIE, SSCI and AHCI                     SCI        20%

                              Academic performance with respect to
      Size of Institution                                                         Size       10%
                              the size of an institution

             Total                                                                          100%
    For institutions specialized in humanities and social sciences such as London School of Economics,
    N&S is not considered, and the weight of N&S is relocated to other indicators.

           Definition of Indicator: Alumni

   The total number of the alumni of an institution winning
    Nobel Prizes and Fields Medals.
   Alumni are defined as those who obtain bachelor,
    Master’s or doctoral degrees from the institution.
   Different weights are set according to the periods of
    obtaining degrees. The weight is 100% for alumni of
    1991-2000, 90% for alumni of 1981-1990, 80% for
    alumni of 1971-1980, and so on.
   If a person obtains more than one degrees from an
    institution, the institution is considered once only.

               Definition of Indicator: Award

   The total number of the staff of an institution winning Nobel prizes
    in physics, chemistry, medicine and economics and Fields Medal in
   Staff is defined as those who work at an institution at the time of
    winning the prize.
   Different weights are set according to the periods of winning the
    prizes. The weight is 100% for winners since 2001, 90% for
    winners in 1991-2000, 80% for winners in 1981-1990, 70% for
    winners in 1971-1980, and so on.
   If a winner is affiliated with more than one institution, each
    institution is assigned the reciprocal of the number of institutions.
   For Nobel prizes, if a prize is shared by more than one person,
    weights are set for winners according to their proportion of prize.

              Definition of Indicator: HiCi

   The number of highly cited researchers in 21 broad
    subject categories in life sciences, medicine,
    physical sciences, engineering and social sciences.

   The definition of categories and detailed procedures
    can be found at the website of Institute of Scientific

   The total number of HiCi is about 5000, about 4000
    of which is university staff.

            Definition of Indicator: N&S

   The annual average number of articles published in
    Nature and Science in the past five years.

   To distinguish the order of author affiliation, a weight of
    100% is assigned for corresponding author, 50% for
    first author (second author if the first author is the same
    as corresponding author), 25% for the next author, and
    10% for other authors.

   Only publications of article type are considered.

           Definition of Indicator: SCI

   Total number of articles indexed in Science
    Citation Index-expanded (SCIE) and Social
    Science Citation Index (SSCI) in the past year.

   A weight of 2 is assigned to articles indexed in
    SSCI to compensate the bias against humanities
    and social sciences.

   Only publications of article type are considered.

         Definition of Indicator: Size

   The sub-total scores of the above five indicators
    divided by the number of full-time equivalent
    academic staff.

   If the number of academic staff for institutions of a
    country cannot be obtained, the total scores of the
    above five indicators is used.

   For ranking 2005, the number of full-time equivalent
    academic staff is obtained for institutions in USA,
    China, Australia, Italy, Netherlands, Sweden, and
    Belgium etc.

       Main Sources of Data

   Nobel laureates:
   Fields Medals:
   Highly-cited researchers:
   Articles published in Nature and Science:
   Articles indexed in SCIE and SSCI:

                   Results of ARWU


   Top 500 universities in the world
   Top 100 universities in North and Latin America
   Top 100 universities in Asia/Oceania
   Top 100 universities in Europe
   Statistics of top universities by region and country
   Percentage distribution of top universities by
    country as compared with the share of global
    population and GDP
Problems & Discussion

          Methodological: Education and Service

   Education is the basic function of any university, however,
    it would be impossible to rank the quality of education due
    to the huge differences among the national systems.

   Contribution to the national economic development is
    becoming increasingly important for universities, however,
    it is impossible to obtain internationally comparable
    indicators and data.

   The academic or research performance of universities, a
    good indication of their reputation, can be ranked

     Methodological: Humanities & Social Sciences

   Many well-known institutions specialized in humanities
    and social sciences are ranked relatively low.
   Since 2004, the indicator of N&S is not considered for
    institutions specialized in humanities and social sciences,
    its weight is relocated to other indicators.
   Since 2005, a weight of 2 for articles indexed by SSCI is
   Nevertheless, if a university specialized in social sciences
    and humanities had Nobel Laureates in economics and
    Highly Cited Researchers in social sciences, it should have
    good standing.

           Methodological: Language Bias

   English is the language of international academic
   Any ranking based on academic performance will be
    biased towards institutions in English-speaking
   One possible solution: papers published in non-native
    languages are offered a special weight.
   Another possible solution: normalization of total articles
    by the proportion of journal editors of each country.

         Methodological: Award and Alumni

   Universities which started after 1911 do not have a
    fair chance.
   Disciplines not related to the awarding fields do not
    have a fair chance. Other important awards include
    Abel, Pulitzer, Turing, Tyler, Pritzker, etc.
   Institutions for winning awards and those for doing
    the researches may not be the same.
   Institutions for obtaining degrees and those for
    pursuing the studies may not be the same.
   Postdoctoral training is not considered.

     Methodological: Per Capita Performance

   The weight of the Size indicator for per capita
    performance is rather low. Large institutions have
    relatively high positions in the ranking.

   However, it’s very difficult to obtain internationally
    comparable data on the number of academic staff.

   The types of academic staff: such as purely teaching
    staff, teaching and research staff, purely research staff.

   The ranks of academic staff: such as professor, associate
    professor, reader, lecturer, research scientist etc.

                Technical: Attributions

   Many universities have more than one commonly used
    names: such as Virginia Tech and Virginia Polytechnic and
    State University.
   Variations due to translation: such as Univ Koln and Univ
    Cologne, Univ Vienna and Univ Wien.
   Abbreviated names: such as ETH Zurich for Swiss Federal
    Institute of Technology Zurich.
   Some authors only write their departmental or institute
    names without mentioning their university names.

      Technical: Definition of Institution

   University systems: such as Univ California
    system, Univ London system.

   Affiliated institutions and research organizations:
    such as Ecole Polytechnique Montreal (affiliated
    to University of Montreal), CNRS Labs (affiliated
    to French universities).

   Teaching and affiliated Hospitals: complex!

   Our answer: according to author’s expression.

             Other Technical Problems

    Merging, splitting, inheriting, discontinuing,
    name-changing of institutions such as:
   Univ Kwazulu-Natal in South Africa merged from
    Univ Natal and Univ Durban-Westville.
   University of Innsbruck in Austria splitted into
    Univ Innsbruck and Innsbruck Medical Univ.
   Humboldt Univ Berlin and Free Univ Berlin
    inheriting the Nobel Prizes of the Berlin
    University before world war II.
Ranking by Broad Subject Fields


    Requests for Ranking of World Universities by

 Broad subject fields or schools, colleges

 Subject fields or programs, departments

   In addition, many top Chinese universities
    want to learn their positions in the world by
    broad subject fields or disciplines.

      Definition of Broad Subject Fields

   Natural Sciences and Mathematics (SCI)

   Engineering/Technology and Computer Sciences (ENG)

   Life and Agriculture Sciences (LIFE)

   Clinical Medicine and Pharmacy (MED)

   Social Sciences (SOC)

   Arts and humanities are not ranked

    ARWU-FIELD Indicators and Weights

Code     SCI   ENG   LIFE   MED   SOC

Alumni   10%         10%    10%   10%

Award    15%         15%    15%   15%

 HiCi    25%   25%   25%    25%   25%

 TOP     25%   25%   25%    25%   25%

 PUB     25%   25%   25%    25%   25%

Fund           25%
        Changes in Indicators and Definition

   N&S in ARWU is not used in ARWU-FIELD.
   TOP is the percentage of articles published in the top
    20% journals of each broad subject field.
   Fund is the total engineering-related research
    expenditures. It’s used only for ENG ranking.
   Alumni and Award since 1951 are used for all
    rankings fields except ENG.
   PUB is the total number of articles indexed by
    Thomson in the past year.

              Results of ARWU-FIELD


   Top 100 universities in SCI
   Top 100 universities in ENG
   Top 100 universities in LIFE
   Top 100 universities in MED
   Top 100 universities in SOC
   Statistics of top universities by region & country
   List of top universities by number of top fields
      Special Problems in ARWU-FIELD

   It’s difficult to obtain data on engineering-
    related research expenditures and make them
    comparable. For 2007, Fund was obtained only
    for US and Canadian universities.

   It’s difficult to separate the Nobel Laureates in
    Physiology or Medicine. They are used in both
    LIFE and MED ranking.

Final Remarks

          Controversy of Ranking

   Any ranking is controversial and no ranking
    is absolutely objective.

   University rankings become popular in many
    countries. Whether we agree or not, ranking
    systems clearly are here to stay.

   The key issue then becomes how to improve
    ranking systems for the benefits of higher

                   Future Efforts

   Study all the above mentioned problems and
    continuously improve the ranking methodologies.

   Establish more comprehensive databases of WCU.

   Update ARWU (every August) and ARWU-FIELD
    annually (every February).

            Future Efforts (Cont’ed)

   Provide ranking of universities specialized or
    strong in engineering, medicine, etc. based on
    the classification of world universities.

   Provide ranking of universities with different
    size, history, budget and function etc. once
    internationally comparable data are obtained.

Thank you very much
 for your attention!


Shared By: