Measuring Research and Experimental Development by lIeu1Dl8


									 Measuring R&D in Developing Countries:
      Annex to the Frascati Manual

West Africa Regional Science, Technology and Innovation Policy Reviews and Statistics Workshop
                                                                                  Bamako, Mali
                                                                                10-13 May 2010


 The problem
 The process
 Contents of the Technical Guide “Challenges
  Facing the Measurement of R&D in Developing
 Thinking ahead

          The problem

 Recognition, meeting targets, evidence-based
  S&T policy, but:
  • lack of interest at the level of policy makers (low policy-
  • S&T is still not properly represented in economic/social
    public policies. Lack of resources devoted to statistics in
  • lack of technical knowledge for the production of cross-
    nationally comparable R&D statistics
  • difficulties in applying FM concepts and methods
  • weak statistical institutions
           The process (1)

 Experience acquired through the UIS work, in particular
  through direct contact with S&T statisticians in numerous
  workshops and other meetings around the developing
 Advisory Meeting to the UIS S&T Statistics Programme
  held in Montreal, Canada, December 2007.
 Papers commissioned by UIS to Jacques Gaillard (IRD,
  Paris), Michael Kahn et al (HSRC, South Africa), and
  Gustavo Arber et al (RICYT, Argentina).
 Proposal for an annex to the Frascati Manual on
  measuring R&D in developing countries was presented at
  the OECD 2008 and 2009 NESTI meeting.

         The process (2)

 Expert Meeting on Measuring R&D in Developing
  Countries in Windhoek, Namibia, 14 to 16
  September 2009.
 Consultant drafted:
  • Working paper on Measuring R&D in Developing
  • Proposed Annex to the Frascati Manual
 Both to be released in 2010
 Some of the issues might also present
  measurement challenges for a future revision of
  the Frascati Manual.
         Main outcomes of the Namibia
 Developing countries a very heterogeneous
 Problems not unique to developing countries
 Stay within boundaries of FM: additional areas
  may be addressed within FM framework
 Most recommendations stood up
 Much additional work needed

           Contents of the Technical Guide

1. Introduction
2. The growing importance of R&D in developing countries
3. Information on R&D expenditure
4. The R&D workforce, internal and international mobility
5. Specific fields of R&D activity
6. Foreign and internationally controlled entities
7. Strengthening R&D statistics systems
8. Thinking ahead

             2. R&D systems in developing
 Particular characteristics of R&D activities to be taken into
   • R&D performers function within the specific context of a national,
     cultural, political, financial and economic system, frequently
     carrying with them the legacies of colonial, post colonial and other
     forms of governance.
   • different structures in terms of state/government,
     research/innovation system, higher education system, statistical
   • particular ‘culture of information’
   • Users of R&D stat: Gov, analysts. + international donor agencies

 However, international comparability is foremost.

         Nature of R&D activities

 There is often more ‘R’ than ‘D’ in developing
 Strong presence of the government and higher
  education sectors in the performance of R&D.
 Degree of informality is common. Informal R&D is
  difficult to capture and therefore it is usually
  considered beyond the scope of R&D surveys.
 S&T indicators need to be adapted to particular
  policy needs, and need to provide answers to
  actual policy questions of the developing

            Heterogeneity and concentration

 R&D activities and their institutional framework present
  distinctive characteristics  structures needs to be
  properly understood.
 Developing countries are a heterogeneous group:
   • Group A: countries with consolidated R&D systems and developed
     S&T statistics systems  no major difficulties in applying Frascati
     Manual concepts.
   • Group B: countries with consolidated R&D systems and less
     developed S&T statistics systems  need specific guidance on
     how to establish and consolidate sound R&D statistics systems.
   • Group C: countries with incipient R&D systems  need specific
     guidelines on how to start creating a regular R&D statistical
 High degree of concentration (in group of countries, in
  particular institutions, in major projects, etc)  lead to
  volatility and inconsistencies in statistics.
          Other issues

 Informal R&D:
  • Occasional R&D
  • R&D in the informal sector / Informally organized R&D
   difficult to measure / not peculiar to developing

              3.Patterns in research funding and
 R&D used to be largely funded by the government, but now new
  sources of funds are emerging (Foundations, NGOs, foreign
  organizations, private business)  use of public sector budgetary
  information (widely used in developing countries as GERD proxy) is no
  longer valid.
 Funding in developing countries provide support to individuals and
  groups rather than institutions  remain unaccounted for, seldom
 Discrepancy between voted and allocated budgets; confusions
  between S&T and R&D budget; difficulties in identifying R&D
  components in the national budget; lacking separate research budget;
  budget commitments are not frequently followed up  raises problems
  with their use for GERD estimates; lead to an over- or underestimation;
  use of different classification limit the availability of key data; source of
  funds accounted for budget data create incompatibilities with FM
  classifications; use of combination of budget data and information from
  annual reports from performing units lead to double counting; capital
  expenditure frequently unaccounted.

         4. Researchers and research
Underestimation of researchers
 Unpaid research
 Informal research
 Research outside of the normal work setting with
  external funding
 Multiple part time positions not taken into account
  or undercounted (“Taxi –professors”)
 Master’s research

          Researchers and research profession

Overestimation of researchers
 Counting the contract instead of the real effort
  (research-professors or enseignant-chercheur)
 Multiple full-time research positions
Special cases
 FTE calculation >1 and FTE>HC
 R&D in times of crisis
 Visiting researchers
 Brain circulation
          Counting researchers

 Recommendations
 Peer interviews of researchers
 Include a module on barriers
 Use secondary sources
  • Publication databases, both national and international
  • STMIS and other databases of researchers
  • Databases and registers of clinical trials
  • Databases and registers of the main foreign donors
    involved in funding R&D in the countries
  • University accreditation databases
         5. Dealing with special types of R&D -
         Traditional knowledge
Traditional knowledge
A cumulative body of knowledge, know-how, practices and
representations maintained and developed by peoples with
extended histories of interaction with the natural environment.
These sophisticated sets of understandings, interpretations and
meanings are part and parcel of a cultural complex that
encompasses language, naming and classification systems,
resource use practices, ritual, spirituality and worldview.
Dichotomy between traditional and scientific knowledge
• substantive grounds – because of differences in the subject
  matter and characteristics of traditional and scientific knowledge.
• methodological and epistemological grounds – because the two
  forms of knowledge employ different methods to investigate
• contextual grounds – because traditional knowledge is more
  deeply rooted in its environment.
           Special types of R&D - Traditional
Links between traditional and scientific knowledge
 Traditional knowledge (in general) as an object of scientific
  study (ethno-botany, ethno-pedology, ethno-forestry,
  ethno-veterinary medicine, ethno-ecology, etc).
 Application of scientific methods to traditional knowledge,
  converting it into a source of scientific information (in
  biodiversity science or nature conservation).
 Application of science to unlocking the potential of
  traditional knowledge (research on traditional medicinal
  practices, traditional pharmacopeia, etc).
 Interaction between scientists and communities in
  participatory technology development using the traditional

            Special types of R&D - Traditional
Measurement issues and recommendations
 Establish the boundaries of R&D for the purposes of the
  Frascati Manual related to traditional knowledge.
 The activities establishing an interface between traditional
  knowledge and R&D (also the cases where R&D
  component can be appropriately measured), are to be
  counted as R&D activities, The production, storage and
  communication of traditional knowledge, in traditional
  ways, should not be counted as R&D.
 Need to consider particular scientific disciplines currently
  not explicitly incorporated into the classification of Fields of
  Sciences. Some of these fields are trans-disciplinary (e.g.
  ethno-botany), making them extremely difficult to map into
  the current classification’s structure.
          Special types of R&D - Clinical trials

Clinical trials
 (Can) involve a significant amount of R&D
 Growth area for developing countries (outsourcing
  of R&D, decentralization of the laboratories,
  activities of pharmaceutical companies, need to
  conduct clinical trials among a wide population of
  potential users).

            Special types of R&D - Clinical trials

Measurement of clinical trials
 Registers of clinical trials available, e.g. WHO but also
  national level
 Funding often from abroad (headquarters of the
  pharmaceutical companies involved)
 Different types of performing units:
   • a local branch of the foreign main sponsor
   • universities and university hospitals
   • individual researchers
   • local medical clinics
   • locally registered PNPs
   • international PNPs

          Special types of R&D - Clinical trials

Measurement issues and recommendations
 Occupation category of local staff
  • Medical doctors and other professionals with at least
    ISCED 5A degrees should be considered as
  • Nurses and other staff with qualifications below ISCED
    5A should be accounted for as technicians

 FTE calculation is important (often part-time)
 Attribution of sector of performance must be done
  with care to avoid double counting.
             Special types of R&D - Industrial
 Reverse engineering: understanding the structure and functioning
  of an object (in order to make a new device or program creates a
  similar object in a different way), copying it, or improving it.
  Recommendation: If reverse engineering is carried out in the
  framework of an R&D project to develop a new (and different)
  product, it should be considered as R&D.
   This is dealt within FM measurements.
 Community development and other social projects should be
  considered R&D only as long as they are in a development and
  testing phase, in which case they should be counted as
  experimental development, most probably in the field of social
   This falls within Social science R&D activities.
 In some developing countries, religious research has a particular
  importance. In principle, religious research is a part of humanities,
  and institutions performing it should be included in R&D surveys.
   This falls within Humanities R&D activities.
           6. The foreign institutions sector

 Create a “foreign institutions” (FI) sector as a separate
  sector of performance to make the resulting data more
  policy relevant.
 Funding flowing from this sector to other sectors should be
  considered from “Abroad” as stated in the main body of the
  Frascati Manual
What is included?
 Foreign antennas
 International organizations
 Foreign company’s R&D labs  (remains in the BE sector)
 Foreign universities  (remains in the HE sector)
             Foreign research centres

 “Foreign antennas”, or foreign research centres, are based in the
  country, but have foreign researchers and foreign funding  direct
  impact on R&D measurement and on the use of R&D stats in policy
  making; strongly distort the countries’ R&D indicators.
 Foreign research labs/MNCs set up by foreign companies may cater
  to the R&D needs of their headquarters, with decision making taking
  place outside the host country  little involvement of the local
  innovation system; need to distinguish such foreign-owned institutions.
 International organizations with R&D activities, involving local staff
  and addressing local issues  significant weight in the total GERD and
  R&D personnel.
 Foreign universities based and conducting R&D in campuses set up
  in the country.  pose particular problems, not accounted for in the

                The foreign institutions sector

The principal sector sub-classification
   Business enterprises
   Government
   Higher Education
   Private non-profit
   International organizations
Practical consequences of introducing the Foreign Institution sector
   FI sector should be treated at the same level as other sectors of performance.
   Specific questionnaire for the FI sector should be designed, addressing the
    particular characteristics of these institutions (demographic characteristics of
    researchers, nationality and country of birth, parameters related to the
    internationalization of R&D, linkages between these institutions and the
    national innovation system).
   Resulting data for the FI sector should be published separately from other
          7. Strengthening S&T statistics
          systems in developing countries
 Institutionalizing S&T statistics
 Establishing registers
 Structural issues in the private sector and the
  private not-for-profit sector
 User-producer networks
 Science & Technology Management Information
  Systems and other secondary sources
 Survey procedures and estimation

          Institutionalization of S&T statistics

 Political support
 Infrastructure and sustained staff training/capacity
 Involvement of NSOs: “Official statistics” status for
  R&D surveys.
 Adequate legal framework

           Establishing registers

 R&D in developing countries tends to be very much the
  purview of public bodies
 Establishing a database of public sector R&D projects
      • include human and financial resources; align with national policies.
      • design could reflect the R&D statistical reporting/definitions.
      • source for evaluation of such projects.

 Establishing STMIS
      • provide overview of research system.
      • framework for establishing complete registers as sample frames
        for R&D surveys.

          Establishing registers

 Other sources
  • associations (trade, academic).
  • learned societies.
  • registers or databases of scientists and engineers.
  • database of research grants.
  • databases of scientific publications.
  • patents and other IP documents.
  • business registers.

          Structural issues in the private sector
          and the PNP sector
 Publicly-owned businesses play a major role in
  R&D in some developing countries
  • should consider issuing data for ‘publicly-owned
    businesses’ separately from the ‘fully private enterprise
  • private enterprises could also be disaggregated by
    ownership, in particular the various degrees of foreign

            Structural issues in the private sector
            and the PNP sector
 Business enterprise R&D is presumed to be generally weak in
  developing countries when compared to industrial countries.
   • this fact needs to be taken into account when conducting sample
     surveys, perhaps by over-sampling, especially amongst larger
   • big companies should not be missed out as it might imply
     significant error.
   • countries should invest some time in interviewing key firms to
     understand their R&D function and obtain a clear picture of their
 High-tech companies: R&D has more added value.
   • need for careful identification of potential R&D actors.
 Private-non-profit sector: make a significant contribution to R&D
  in developing countries, but the sector tends to be very volatile.
         User-producer networks

• user-producer networks and other forms of stakeholder consultation
  should be instituted.
• establishing national S&T statistics groups.
• involve multiple actors.
• coordinating/networking among institutions/databases.
• partnering with business associations.
• conducting face-to-face visits by statisticians and project leaders.
• exploit pre-existing personnel ties.
• get NSO involved; to deal with privacy of information.
• training of interviewers/primary data producers.

            Science and Technology Management
            Information Systems and other secondary
 STMIS (such as CV-LAC, database of scientists, research
  grants, etc): frequent source for the production of R&D
   • need close integration between the statistical system and the
   • need adjustments to produce comparable statistics, taking into
     account issues of definitions and coverage.
   • need a balanced approach using both STMIS and surveys.
   • need different approach to Private sector organizations as they are
     frequently not covered by these systems.
 Combined R&D and innovation surveys
   • the relative rarity of occurrence of R&D in businesses needs to be
     taken into account.
         Survey procedure and estimation

• attention needs to be paid to questionnaire design.
• frequency of survey.
• prioritize area of work; accompanied by step-by-step approach.
• use of survey questionnaires of other countries for inspiration: need
  adaptations to local situation.
• get expertise from the NSO, in conducting survey, in sampling ….
• different questionnaires might be designed for different sectors
  based on stakeholder consultations; “one size does not fit all”.
• procedures need to be developed for estimating missing data.

         8. Thinking ahead: Other products –
         beyond R&D
 Redefine the concepts of scientific and
  technological education and training at broadly the
  third level (STET), Scientific and technological
  services (STS) and S&T activities (STA)
 Better integrate education statistics with R&D
 Hands on guidance
 Metadata
 Model questionnaire

  Thank you!


To top