Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Inspired by Technology, Driven by Pedagogy by OECD

VIEWS: 73 PAGES: 163

" "

More Info
									 inspired by Technology,
 Driven by Pedagogy
 A SySTEmiC APPrOACh TO
 TEChnOlOgy-bASED SChOOl
 innOvATiOnS




Centre for Educational Research and Innovation
Inspired by Technology,
  Driven by Pedagogy
A SYSTEMIC APPROACH TO TECHNOLOGY-BASED
           SCHOOL INNOVATIONS
              ORGANISATION FOR ECONOMIC CO-OPERATION
                         AND DEVELOPMENT
      The OECD is a unique forum where governments work together to address the economic,
social and environmental challenges of globalisation. The OECD is also at the forefront of efforts
to understand and to help governments respond to new developments and concerns, such as
corporate governance, the information economy and the challenges of an ageing population.
The Organisation provides a setting where governments can compare policy experiences, seek
answers to common problems, identify good practice and work to co-ordinate domestic and
international policies.
      The OECD member countries are: Australia, Austria, Belgium, Canada, Chile, the
Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Israel,
Italy, Japan, Korea, Luxembourg, Mexico, the Netherlands, New Zealand, Norway, Poland, Portugal,
the Slovak Republic, Slovenia, Spain, Sweden, Switzerland, Turkey, the United Kingdom and the
United States. The European Commission takes part in the work of the OECD.
      OECD Publishing disseminates widely the results of the Organisation’s statistics gathering
and research on economic, social and environmental issues, as well as the conventions,
guidelines and standards agreed by its members.



          This work is published on the responsibility of the Secretary-General of the OECD. The opinions
        expressed and arguments employed herein do not necessarily reflect the official views of the
        Organisation or of the governments of its member countries.




ISBN 978-92-64-09478-9 (print)
ISBN 978-92-64-09443-7 (PDF)




Series: Educational Research and Innovation
ISSN 2076-9660 (print)
ISSN 2076-9679 (online)




Photo credits: Cover ©. Ariel Skelley/Blend Images/Corbis.

Corrigenda to OECD publications may be found on line at: www.oecd.org/publishing/corrigenda.
© OECD 2010

You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications,
databases and multimedia products in your own documents, presentations, blogs, websites and teaching materials, provided
that suitable acknowledgment of OECD as source and copyright owner is given. All requests for public or commercial use and
translation rights should be submitted to rights@oecd.org. Requests for permission to photocopy portions of this material for
public or commercial use shall be addressed directly to the Copyright Clearance Center (CCC) at info@copyright.com or the Centre
français d’exploitation du droit de copie (CFC) at contact@cfcopies.com.
                                                                                 PREFACE – 3




                                              Preface

           Human capital is at the heart of the innovation process, and our educa-
       tional systems bear the primary responsibility for nurturing and developing
       the capacities and innovative capabilities of our fellow citizens. Yet, educa-
       tion is costly; for many countries, educational expenditures constitute a large
       proportion of public spending. In the light of the current recession and conse-
       quent budget constraints that every country faces, governments are looking
       at ways to maximise the returns on their investments in education. This is not
       a purely economic perspective: human capital and talent are critical for the
       development of our societies; thus, investing in education and getting returns
       on it are important for the well being of all.
            The OECD Centre for Educational Research and Innovation (CERI) has
       been addressing the issue of how education systems manage innovation from
       two different perspectives. On one hand, in the broader context of the OECD
       Innovation Strategy,* CERI has analysed what human capital is contributing
       to innovation as well as how innovation in education can be promoted. On the
       other hand, CERI has also approached the innovation capacity of education
       systems from a systemic perspective.** The aim is to better understand the
       process of innovation and to facilitate the policy process involved in promot-
       ing, sustaining, assessing and scaling up innovations. In this respect, it is
       worth acknowledging that innovation is not only an elusive concept but also
       extremely context-dependent.
           Technology has come to play an integral and important role in education.
       Despite the current recession, and even seeing an opportunity in it, many
       countries are now investing heavily again in promoting technology-based
       school innovations by way of universalising access (one computer per stu-
       dent) and of producing digital learning resources and platforms. If innova-
       tion at large can be seen as a means to capitalise on educational investments,
       the same can be said of investments in technology in education. Countries
       have to develop systemic approaches to make the most of their confidence in


       * More at www.oecd.org/innovation/strategy.
       ** More at www.oecd.org/edu/systemicinnovation.


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
4 – PREFACE

      technology-based school innovations because, in the long run, that is prob-
      ably the most effective way of securing such investments.
           The perspective presented in this publication addresses precisely the
      issue of how technology-based school innovations can not only be fostered
      and supported but also monitored, assessed and eventually scaled-up – which
      is even more important from a systemic perspective. In this respect, this pub-
      lication is an attempt to capture the key issues that matter for a better under-
      standing of how a systemic approach to technology-based school innovations
      can contribute to quality education for all, promoting a more equal and effec-
      tive education system. Sufficient return on public investments in education
      and our ability to innovate are today more important than ever.
          In particular, this publication focuses on the novel concept of systemic
      innovation; on the emerging opportunities to generate innovations that stem
      from Web 2.0 and the important investments and efforts that have gone
      into the development and promotion of digital resources; and on alternative
      ways to monitor, assess and scale up technology-based innovations. In par-
      ticular, some country cases, as well as alternative research frameworks, are
      highlighted.
          This publication results from a successful co-operation with the State of
      Santa Catarina (Brazil), which started with an international conference that
      was convened in Florianopolis, the capital of the State, in November 2009.
      This conference, organised by CERI with the support of the Secretariat of
      Education for Santa Catarina, had the overall goal of addressing the issue of
      how education systems go about technology-based innovations. The meeting
      had the inspiring title of “The School of Tomorrow, Today”. Experts from dif-
      ferent parts of the world came together to discuss issues of innovation, policy
      development, research and technology development with representatives from
      Santa Catarina.
          Francesc Pedró, from CERI, designed and managed that event and is
      the main editor of this resulting publication. Øystein Johannessen, from the
      Norwegian Ministry of Education, contributed to both the event and this
      publication during his secondment period at CERI and well after it. Therese
      Walsh and Lynda Hawe provided invaluable editorial support.
          The Santa Catarina meeting was an extremely successful event, and I
      would like to express our most sincere gratitude not only for this support
      but also for the sustained interest that the State of Santa Catarina has had on
      OECD’s work in education, reflected in a number of joint projects. None of
      these would have come true without the impetus of former State Governor,
      Luis Henrique Silveira, and both his former Secretary of State for Education
      Paulo Bauer, as well as Silvestre Heerdt, currently in office, and Director
      General Antonio Pazeto. Wilson Schuelter, on the Brazilian side, has been



                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                                              PREFACE – 5



       instrumental to the success of these projects as Ian Whitman has been on the
       OECD side.
           I am convinced that this publication will honor the efforts and expecta-
       tions of all of them and serve the purpose of helping governments and con-
       cerned stakeholders to revisit their assumptions about how education systems
       can maximise the benefits of technology-based school innovations.




                                           Barbara Ischinger
                                Director, Directorate for Education




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                                                                  TABLE OF CONTENTS – 7




                                             Table of contents


Executive summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

Introduction. The need for a systemic approach to technology-based school
              innovations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
   The growing relevance of technology-based school innovations . . . . . . . . . . . . .14
   Are education systems failing to scale up technology-based innovations? . . . . . 15
   Why a systemic approach to technology-based school innovations can be useful . . .16
   The agenda on systemic innovation and how this report contributes to it. . . . . . .17
   References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

                                        Part I. A changing landscape

Chapter 1. Web 2.0 and the school of the future, today . . . . . . . . . . . . . . . . . . . 23
   What is Web 2.0 and why is it of educational importance? . . . . . . . . . . . . . . . . .                                24
   Acknowledging the realities of Web 2.0 use in the school of today . . . . . . . . . .                                     26
   Popular solutions for overcoming the “problem” of schools in a Web 2.0 world                                              28
   Towards a more reasoned response to Web 2.0 and the school of the future . . .                                            33
   Conclusion: Towards a more critical understanding of Web 2.0, schools and
   schooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   35
   References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .    39

Chapter 2. Can digital learning resources spur innovation? . . . . . . . . . . . . . . . 45
   Background, objectives and methodological approach . . . . . . . . . . . . . . . . . . . .                                46
   Main findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .     48
   Government-initiated innovations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .                  51
   Innovations initiated by commercial actors . . . . . . . . . . . . . . . . . . . . . . . . . . . . .                      52
   Bottom-up innovations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .            54
   Looking at the future of DLR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .                57
   Conclusions and policy implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .                   59
   References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .    62
   Appendix 2.A. Cases studied in the DLR project . . . . . . . . . . . . . . . . . . . . . . . . .                          63


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
8 – TABLE OF CONTENTS

                Part II. How technology-based innovations are monitored,
                                 assessed and scaled up

Chapter 3. Monitoring and assessing the use of ICT in education:
           The case of Australia. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
  Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   68
  Building interest in the educational use of ICT: 1990 to 2000 . . . . . . . . . . . . . .                                   69
  Educational goals and plans for ICT in education . . . . . . . . . . . . . . . . . . . . . . . .                            70
  Supporting the use of ICT in education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .                      71
  Monitoring and evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .               74
  References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .      84

Chapter 4. Extending and scaling technology-based innovations through
           research: The case of Singapore. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
  Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
  A way forward . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
  From research projects to extension and scaling . . . . . . . . . . . . . . . . . . . . . . . . . 95
  Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
  References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101


                              Part III. Promising avenues for research

Chapter 5. The third lever: Innovative teaching and learning research . . . . .105
  Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
  ITL research background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
  ITL research design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
  Methods, sample and outputs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115
  Policy implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .118
  References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Chapter 6. Design research on technology-based innovations. . . . . . . . . . . . . 125
  Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
  Curriculum: What’s in a name? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
  The vulnerable curriculum spider web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
  Perspectives on substantive choices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .131
  Development strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .132
  The potential of curriculum design research . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
  Features of curriculum design research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .135
  Emphasis on formative evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .137



                                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                                                              TABLE OF CONTENTS – 9



  Generalisation of curricular design research findings. . . . . . . . . . . . . . . . . . . . .138
  Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
  References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .141

Conclusion. Lessons learnt and policy implications . . . . . . . . . . . . . . . . . . . . . .143
  Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
  Lessons learnt. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
  Policy principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146
  The analysis of technology-based innovation in education . . . . . . . . . . . . . . . . .148
  Policy implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
  The road ahead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .157
  References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159


Figures
Figure 4.1      Framework for translation and extension/scaling innovations . . . . . . 93
Figure 5.1      Educational transformation model. . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Figure 5.2      Innovative Teaching Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Figure 5.3      ITL research logic model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .110
Figure 5.4      ITL research timeline – annual milestones for 2010-2012 . . . . . . . . .118
Figure 6.1      Curricular spider web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Figure 7.1      Simplified model of coherence between key policy elements. . . . . . .149


Tables
Table 2.1       The process of innovation related to context, output and stakeholders 49
Table 2.2       Teachers’ access, competence, and motivation to use ICT . . . . . . . . . 56
Table 3.1       ICT literacy proficiency level descriptions and percentage
                distributions, 2005 and 2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Table 4.1       Three stages from research projects to extension and scaling . . . . . . 97
Table 5.1       Global ITL research team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115
Table 5.2       ITL research methods summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116
Table 6.1       Typology of curriculum representations . . . . . . . . . . . . . . . . . . . . . . 128
Table 6.2       Curriculum components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                                       EXECUTIVE SUMMARY – 11




                                  Executive summary

           This book is an attempt to contextualise the issues described above by
       providing an analytical framework made up of three different sections: the
       opportunities offered by technology, how technology-based innovations are
       monitored and assessed, and the role of research in documenting innovations.
            The first part offers a look at the changing landscape of technology in edu-
       cation and, in particular, to the emergence of the Web 2.0 and digital learning
       resources to see where the new opportunities and challenges are located. Neil
       Selwyn’s contribution highlights, from a well-documented perspective, the
       changes brought about by Web 2.0 in society, the opportunities that schools
       might benefit from and, sadly, how little use teachers are making of these
       opportunities. In doing so, he raises once again the issue of the difficulties that
       schools and teachers face when trying to digest new technological develop-
       ments. Finally he rightly asks whether society should really expect them to do
       it without an ongoing dialogue about educational expectations. The second con-
       tribution in this section is made by Jan Hylén, formerly at CERI, who summa-
       rises the main findings of an OECD report on the production and use of digital
       learning resources in the five Nordic countries. Hylén’s perspective on this
       long-awaited development, digital learning resources, is extremely nuanced and
       raises important questions about the role that governments, public and private
       providers and the teaching community should play if digital learning resources
       were to become a standard feature in mainstream education, thus transcending
       the stage of being yet another technology-based innovation.
            In the second part, the report looks at ways in which the use of tech-
       nology is being monitored and assessed at national level in order to better
       support the spreading and scaling up of what works and at what types of evi-
       dence are used in this process. This is a crucial question that most education
       systems have failed to address properly so far. The section presents two dif-
       ferent cases. First, John Ainley documents how Australia has had an increas-
       ingly complex perspective on this issue, and in particular on how to monitor
       technology use in schools, as a true recognition not only if the variance in
       scope and depth that technology-based innovations have across schools and
       territories. In addition he also suggests the need for substantial progress in



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
12 – EXECUTIVE SUMMARY

     the collection of evidence about how young people become equipped with
     digital literacy skills and, in a broader sense, with 21st century skills. David
     Hung, Kenneth Lim, and David Huang present the case of Singapore in their
     contribution. Certainly, this case is quite particular in many respects, and
     among others in its emphasis on the design, implementation and evaluation of
     national master plans. In those plans, the support to and the monitoring and
     assessing of technology-based innovations have been playing an important
     role and this contribution discusses the different ways in which practitioners,
     researchers and policy makers have been involved in the process of docu-
     menting successful innovations and planning for scaling up.
          The report also provides in the third part a fresh look into the role of
     research by presenting some insights into ongoing efforts to compare the
     effects of technology use internationally and by exploring the promising
     avenue of design research. First, Maria Langworthy, Linda Shear and Barbara
     Means introduce what might be one of the most important international
     and comparative research efforts to develop and contribute a set of tools to
     measure educators’ adoption of innovative teaching practices and the degree
     to which those practices provide students with learning experiences that pro-
     mote the skills they will need to live and work in the 21st century. Still in its
     initial phase, this major research effort represents an important challenge to
     existing assumptions about the lack of connection between teachers’ innova-
     tion practices involving technology and students’ achievements. Second, in
     a completely different vein, Jan van den Akker discusses the benefits and
     limitations of curriculum design research in this domain and how its results
     could represent an important contribution to curriculum policies and develop-
     ment. Its main point is not to elaborate and implement complete interventions
     but to come to (successive) prototypes that increasingly meet the innovative
     aspirations and requirements. The process is often iterative, cyclic or spiral:
     analysis, design, evaluation and revision activities are iterated until a satisfy-
     ing balance between ideals and realisation has been achieved.
          Finally, the main conclusions that can be drawn from all these contri-
     butions and the relevant discussions that took part in the Santa Catarina
     conference are presented. While there seems to be clear support for a sys-
     temic approach to technology-based school innovations, particularly when it
     comes to the complex issue of assessing them and using validated evidence
     to decide upon their scaling up, there is also the recognition that this new
     approach challenges predominant assumptions about innovation in educa-
     tion. Governments and teachers alike need to rethink how innovations are
     supported, monitored and assessed, whether the right strategies and tools are
     in place and used to their full potential, and finally whether the whole discus-
     sion about technology in education has to be redefined in the light of what we
     all should be caring about: the quality of students’ learning.



                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
     INTRODUCTION. THE NEED FOR A SYSTEMIC APPROACH TO TECHNOLOGY-BASED SCHOOL INNOVATIONS – 13




                                           Introduction

                      The need for a systemic approach to
                      technology-based school innovations


                                Francesc Pedró
               OECD Centre for Educational Research and Innovation




    While access to new digital technology in schools has increased measurably in
    the past ten years, it has not been adopted as quickly and intensively as expected
    despite policy efforts to promote and support technology-based school innova-
    tions. This chapter explores possible reasons for this response on the part of
    schools and teachers from the perspective of systemic innovation. Specifically, it
    addresses the question of how more effective knowledge management at the system
    level of technology-based school innovations could contribute to educational
    change.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
14 – INTRODUCTION. THE NEED FOR A SYSTEMIC APPROACH TO TECHNOLOGY-BASED SCHOOL INNOVATIONS

           The purpose of this chapter is to apply the perspective of systemic
      innovation to the analysis of technology-based school innovations. For the
      purposes of this chapter, and consistently with previous CERI works,3 edu-
      cational innovation is defined as any dynamic change intended to add value
      to the educational processes and resulting in measurable outcomes, be that
      in terms of stakeholder satisfaction or educational performance. In particular,
      this systemic approach focuses on how systems monitor and assess innova-
      tions and use the results to cumulate knowledge for action in this domain.
      More specifically, it looks at how innovations are generated and diffused in
      the system; to what extent knowledge is the basis of these innovations; how
      knowledge circulates throughout the process; and how stakeholders interact
      to generate and benefit from this knowledge.
          This perspective has been successfully applied to two different areas
      so far: vocational education and training (VET) (OECD, 2009a), and digital
      learning resources (DLR) (OECD, 2009b). This chapter attempts to test its
      limits when applied to the analysis of technology-based school innovations.

The growing relevance of technology-based school innovations

           Our increasingly technology-rich world raises new concerns for educa-
      tion while also engendering expectations that schools become the vanguard
      of knowledge societies. Firstly, technology can provide the necessary tools
      for improving the teaching and learning process, and for opening new oppor-
      tunities and avenues. In particular, it can enhance the customisation of the
      learning process, adapting it to the particular needs of the student. Secondly,
      education has the role of preparing students for adult life, and, therefore, it
      must provide students with the skills necessary to join a society where tech-
      nology-related competencies are becoming increasingly indispensable. The
      development of these competencies, which are part of the set of the so-called
      21st century competencies, is increasingly becoming an integral part of the
      goals of compulsory education. Finally, in a knowledge economy driven by
      technology, people who do not acquire and master these competencies may
      suffer from a new form of the digital divide that may affect their capacity to
      fully integrate into the knowledge economy and society (OECD, 2010).
          Countries have been supporting technology-based innovations in a vari-
      ety of ways during the past three decades. Typically at policy level, important
      efforts are being made in three different directions:
          1. Determining the conditions that enable the adoption of technology.
             These conditions cover a wide range of issues, ranging from the
             availability of equipment and connectivity, to teacher training, tech-
             nical and pedagogical support, as well as the production and distribu-
             tion of digital learning materials.


                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
     INTRODUCTION. THE NEED FOR A SYSTEMIC APPROACH TO TECHNOLOGY-BASED SCHOOL INNOVATIONS – 15



            2. Empowering schools and teachers to generate discrete innovations at
               the school or classroom level by providing different forms of incen-
               tives, mostly seed money, through open calls for innovation.
            3. Providing support for the research community interested in docu-
               menting and analysing emerging educational innovations.
            Although little is known about the size and intensity of the investments
       made in this domain, there are clear indications that the basic conditions for
       creating a propitious environment for using technology in schools have been
       there for a long time. By 1999, the limited available data on trends in tech-
       nology investment and use (technology spending, schools connected to the
       Internet) were headed up sharply (OECD, 1999). In 2003, more robust data
       from PISA confirmed exponential growth in the presence of technology in
       education (OECD, 2006). Between 2000 and 2003, students-per-computer
       ratios dropped by more than half in most countries and even more in those
       that were lagging. While less than a third of secondary schools had Internet
       access in 1995, it was already virtually universal by 2001, as is currently the
       case with broadband connectivity in a growing number of OECD countries.

Are education systems failing to scale up technology-based innovations?

            With the turn of the century and the burst of the Internet bubble, policy
       makers had to re-adjust their expectations. As they could not see schools and
       teachers adopting technology at the desired pace and with the expected inten-
       sity or clear-cut evidence of the expected benefits, a certain discomfort, if not
       skepticism, began to silently propagate. As a result, in many OECD countries
       the incorporation of technology in education has lost its status as policy prior-
       ity number one although for a number of political reasons investments have
       not been stopped. In many respects, the principle of “build it, and they will
       come” seems to be firmly in place, and education systems keep investing in
       technology, based on the belief that, sooner or later, schools and teachers will
       adopt it and benefit from it.
           Why, despite the impressive policy efforts made to promote technology-
       based school innovations, is change at the system level not happening? There
       is a growing body of literature pointing out the reasons why a majority of
       teachers is still unable to find feasible ways to use technology to support a
       much desired pedagogical change. In most OECD countries, when it comes to
       teachers’ access to technology this is no longer a problem, nor is the required
       baseline of technical skills. As a matter of fact, most teachers are convinced
       of the benefits that the use of technology can bring to the classroom. The
       reasons that explain the resulting paradox are likely to be related to any of
       the following areas:



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
16 – INTRODUCTION. THE NEED FOR A SYSTEMIC APPROACH TO TECHNOLOGY-BASED SCHOOL INNOVATIONS

          1. Knowledge base: The question of what works, that is, the connection
             between pedagogical practices involving technology and their effects
             on quality, equity and performance, remains ill-addressed. This is
             partly due to the complexity of the issue that “technology is a blind
             artefact that can be used for a variety of purposes”, but it is also due
             in part to an ineffective use of the existing evidence, a problem that
             can be addressed from a knowledge management perspective.
          2. Teacher training: An ongoing OECD comparative study on how tech-
             nology is used in teacher training colleges reveals that in most cases
             these crucial institutions are unable to provide prospective teachers with
             real hands-on experience in technology-enhanced pedagogies and fail,
             consequently, to provide clear directions on effective uses of technol-
             ogy in the classroom. Instead of being real showcasing opportunities,
             teacher-training institutions seem to instill a reluctant attitude towards
             technology – presented as an add-on which would require more time.
          3. Incentives: Pedagogical change requires a huge investment of effort
             by teachers, individually and collectively. Education systems provide
             no clear incentives to support and reward the required effort. In addi-
             tion, the empirical evidence that could eventually drive this change,
             for instance, connecting particular uses of technology with improve-
             ments in student performance, is scarce and not communicated in
             proper ways to teachers.
          In the following pages, this chapter will address the first area only: how
      more effective knowledge management at the system level of technology-
      based school innovations could contribute to educational change.

Why a systemic approach to technology-based school innovations can
be useful
          The underlying assumption of this chapter is that there is a need to know
      more about how governments promote, monitor, evaluate and scale up suc-
      cessful technology-based or supported innovations, paying particular attention
      to the role played by research, monitoring and evaluation, and the resulting
      knowledge base, both at national and international level. To put it briefly, is
      there any kind of knowledge management cycle in place in the domain of tech-
      nology-based school questions? Or, in other words, do governments (be they at
      national, regional or local level) have a systemic approach to technology-based
      school innovations? The knowledge management perspective, previously used
      by CERI in the area of educational research and development,4 emphasises
      how knowledge is produced, shared and disseminated, and effectively used
      in any decision-making process whether it be in policy making or in profes-
      sional practice. Again, it should be stressed that this may be the first time that


                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
     INTRODUCTION. THE NEED FOR A SYSTEMIC APPROACH TO TECHNOLOGY-BASED SCHOOL INNOVATIONS – 17



       such an approach has been applied to the analysis of systemic innovation and
       represents a first step in a promising analytical field.
            From a knowledge management perspective a number of questions arise.
       First, it is unclear how the different strands of policy effort are aligned and
       interrelated in optimal ways to limit the amount of policy investment that, aimed
       at seeding innovation, may end up having a very limited systemic impact. A
       second question is how innovations are monitored and assessed, who does this
       and where the resulting knowledge goes or becomes disseminated. Third, the
       role that educational research could eventually play in leveraging knowledge
       may also be questioned: who decides what kind of research should be funded in
       this domain? How relevant are the methodologies and results obtained? How are
       they communicated and eventually used? Particular attention should be given to
       how countries initiate innovation, the processes involved, the role of drivers and
       barriers, the relationships between main actors, the knowledge base being drawn
       upon, and the procedures and criteria for assessing progress and outcomes.

The agenda on systemic innovation and how this report contributes to it

           Although there is an increasing interest in the role played by research
       evidence in policy formation in education, not enough is known about the
       connections among research findings, public policies, and educational inno-
       vations. Previous CERI work on knowledge management, on educational
       R&D and, particularly, on evidence-based policy research (OECD, 2003;
       OECD, 2004; OECD, 2007) points to the current difficulties experienced
       when trying to align these three elements. The systemic analysis of inno-
       vation in education provides another opportunity to continue and refine
       the work carried out so far, paying particular attention to the connections
       between evidence and the innovation processes in education.
           In the particular domain of technology-based school innovations, here are
       three questions that can be developed as research questions:
                Competing concepts of technology-based school innovation: How
                is technology-based innovation defined and understood in different
                education systems? Why should technology-based school innovation
                systems be fostered?
                The dynamics of technology-based school innovation from a knowl-
                edge management perspective: What are the main models of innova-
                tion in OECD countries? What are the systemic factors involved?
                Innovation policies regarding technology in education: From the
                perspective of evidence-based policy research, how are innovation
                policies designed? What is the role of research evidence in nurturing
                innovation policies? How are these policies monitored and evaluated?


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
18 – INTRODUCTION. THE NEED FOR A SYSTEMIC APPROACH TO TECHNOLOGY-BASED SCHOOL INNOVATIONS




                                           Notes

1.    More at www.oecd.org/innovation/strategy.
2.    More at www.oecd.org/edu/systemicinnovation.
3.    More at www.oecd.org/edu/systemicinnovation.
4.    The definitions of research and development used then are also applied throughout
      this project. Research is defined as the process of knowledge creation that con-
      forms to the agreed scholarly standards intended to warrant its validity and trust-
      worthiness. In this project, basic research is differentiated from applied research.
      The former is driven by curiosity and an inherent interest in a phenomenon or
      problem, while the latter is consciously designed to solve a problem in policy or
      practice. In both cases, the process of knowledge creation is carried out within
      the framework of a theory, which might be either validated or challenged by new
      research. Development is defined as any form of knowledge creation designed to
      improve practice. Thus, the main purpose of development is to facilitate change
      in a particular context. A number of educational developments are teacher-lead
      activities and consist of enquiry-based activities that take place within schemes
      for the professional development of teachers (more at www.oecd.org/edu/rd).




                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
     INTRODUCTION. THE NEED FOR A SYSTEMIC APPROACH TO TECHNOLOGY-BASED SCHOOL INNOVATIONS – 19




                                           References

       OECD (1999), Education Policy Analysis 1999, OECD Publishing.
       OECD (2003), New Challenges for Educational Research, Knowledge
         management, OECD Publishing.
       OECD (2004), Innovation in the Knowledge Economy: Implications for
         Education and Learning, Knowledge management, OECD Publishing.
       OECD (2006), Are Students Ready for a Technology-Rich World? What PISA
         Studies Tell Us, PISA, OECD Publishing.
       OECD (2009a), Working Out Change: Systemic Innovation in Vocational
         Education and Training, Educational Research and Innovation, OECD
         Publishing.
       OECD (2009b), Beyond Textbooks: Digital Learning Resources as Systemic
         Innovation in the Nordic Countries, Educational Research and Innovation,
         OECD Publishing.
       OECD (2010), Are the New Millennium Learners Making the Grade?
         Technology Use and Educational Performance in PISA 2006, Educational
         Research and Innovation, OECD Publishing.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                           PART I. A CHANGING LANDSCAPE – 21




                                                Part I

                                   A changing landscape




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                  1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY – 23




                                             Chapter 1

                 Web 2.0 and the school of the future, today


                                      Neil Selwyn
                     London Knowledge Lab, Institute of Education
                        University of London, United Kingdom




    The future of schools and schooling constitutes one of the major areas of current
    education debate, especially in light of the increasing importance of digital tech-
    nologies in contemporary society. While having undoubted educational potential,
    these digital technologies mark a significant area of uncertainty, which are encap-
    sulated in current debates over the place of so-called “Web 2.0” technologies in
    education. This chapter offers a critical perspective on the emergence of Web 2.0
    applications and the hype surrounding their uptake in education. It chapter looks
    at the changes brought about by Web 2.0 in society, the opportunities that schools
    might benefit from and, sadly, how little use teachers are making of these oppor-
    tunities. It concludes by arguing for the need to retain a realistic, if not critical,
    perspective on schools and Web 2.0 – seeking to find ways of using Web 2.0 tech-
    nologies to work with the schools of today, rather than against them.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
24 – 1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY

What is Web 2.0 and why is it of educational importance?

           Alongside other tags such as the “social web”, “modern web” and “social
      software”, the notion of “Web 2.0” provides a convenient umbrella term for a
      host of recent Internet tools and practices ranging from social networking and
      blogging to “folksonomies” and “mash-ups”. In a technical sense, Web 2.0 can
      be argued to refer to an increased socialisation of Internet tools, applications
      and services. As Matthew Allen (2008) describes, the notion of Web 2.0 reflects
      “approaches to the design and functionality of Web sites and the services they
      offer, emerging in recent years, and essentially describing technological imple-
      mentations that prioritise the manipulation and presentation of data through the
      interaction of both human and computer agents”. Of course, many computer
      scientists dispute the technical necessity of such rebranding of the Internet. As
      Scholz (2008) argues, many claims for the technical novelty of Web 2.0 appli-
      cations are misleading, with much use of the term driven by a commercial and
      political “branding mania and obsession with newness”. Yet issues of original-
      ity notwithstanding, the notion of Web 2.0 is an important framing device for
      understanding contemporary Internet use – defining “what enters the public
      discourse about the impact of the Internet on society” (Scholz, 2008).
            In particular the “Web 2.0” label reflects the changing nature of con-
      temporary online activity – not least what is described as a “mass” Internet
      connectivity based around the collective actions of online user communities
      rather than individual users (see O’Reilly, 2005; Shirky, 2008; Brusilovsky,
      2008). Thus in contrast to the “broadcast” mode of information exchange that
      characterised Internet use in the 1990s, the Web applications of the 2000s are
      seen to rely on openly shared digital content that is authored, critiqued and
      re-configured by a mass of users – what has been described as “many-to-
      many” connectivity as opposed to “one-to-many” transmission. Put simply,
      the current prominence of “Web 2.0” within popular and academic discus-
      sion of the Internet reflects the growing importance that is being placed
      on interaction between and within groups of Internet users. In this sense,
      Web 2.0 re-invigorates many of the debates that began in the 1990s about the
      transformatory nature of the Internet (Roberts, 2009). Yet unlike the Internet
      of the 1990s “Web 1.0” as some commentators have now retrospectively
      branded it), current debates are propelled by notions of online immediacy
      where users can get what they want, when they want it – “propelled … by a
      fantasy of intuition in which the Web already knows what you want because
      it is you” (Evens, 2009).
          The notion of Web 2.0 therefore merits serious consideration – not least
      in describing what is seen to be a renewed “spirit” and “ethos” of contem-
      porary Internet use. In this sense the individual technological artefacts that
      are positioned under the Web 2.0 label are of less importance than the wider
      ideals that are seen to drive their use (Allen, 2008). For example, Web 2.0 can


                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                  1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY – 25



       be seen as conveying a new sense of economy of Internet use (based around
       notions such as the “attention economy” and the provision of ostensibly
       “free” services in return for greater audience share. Web 2.0 can also be seen
       as conveying a new sense of Internet user – one “who is more engaged, active
       and a participant in the key business of the Internet: creating, maintaining
       and expanding the ‘content’ which is the basis for using the Internet in the
       first place” (Allen, 2008). Finally, and perhaps most significantly, Web 2.0
       can also be seen as conveying a new sense of the politics of the Internet
       – based around libertarian notions of empowered individuals and diminished
       institutions – “expressed in traditional democratic terms, emphasising free-
       dom of choice and the empowerment of individuals through the ‘architecture
       of participation’” (Allen, 2008). In all these instances, Web 2.0 therefore pre-
       sents a significant and specific challenge to orthodox notions of technologi-
       cal and institutional arrangements, economic structures and social relations.
       In particular, Web 2.0 is felt by many commentators to address a number of
       “participatory deficits” in terms of public/state services – positioning state
       services around the needs of the individual, with services such as education
       driven by the individual rather than the institution (Evens, 2009).
            Yet despite the significant political, economic and institutional ramifica-
       tions of Web 2.0 for the formal provision of education, most of the reaction to
       Web 2.0 within educational circles has been shaped by concerns over learn-
       ing and individual learners. In particular the privileging of participatory and
       collaborative group activity within Web 2.0 debate has been welcomed as
       having clear parallels with contemporary understandings of learning and edu-
       cation, and it is perhaps unsurprising that it has been the learning and ped-
       agogy-related aspects of Web 2.0 that have prompted greatest enthusiasm of
       late amongst educators and educationalists (see Davies and Merchant, 2009).
       In particular, it has been argued that Web 2.0 practices have a strong affinity
       with socio-cultural accounts of “authentic” learning where knowledge is co-
       constructed actively by learners with the support of communal social settings
       – taking the form of constantly reassessed “collective agreement” (see Dede,
       2008). A great deal of attention has been paid to the personalised and socially
       situated forms of learning (intended or otherwise) that can be found within
       Web 2.0 practices, with learners said to gain from participatory experiences
       in the co-construction of online knowledge (e.g. Lameras et al., 2009). Thus,
       Web 2.0 has now come to embody the long-held belief amongst education
       technologists that learning best takes place within technology-supported net-
       works of learners involved in the creation as well as consumption of content.
       For these reasons alone, Web 2.0 is now being touted in some quarters as “the
       future of education” (Hargadon, 2008).
           As these latter sentiments illustrate, growing numbers of educational
       commentators are promoting the educational potential of Web 2.0 tech-
       nologies in defiantly transformatory terms. Aside from the cognitive and


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
26 – 1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY

      pedagogical benefits of Web 2.0 use, it is now being argued widely that
      Web 2.0 tools, applications and services offer schools an opportunity to (re)
      connect with otherwise disaffected and disengaged learners. For example,
      as Mason and Rennie (2007, p. 199) reason, “shared community spaces
      and inter-group communications are a massive part of what excites young
      people and therefore should contribute to [their] persistence and motivation
      to learn”. These expectations of enhanced motivation and interest are often
      accompanied by presumptions of an enhanced equality of opportunity, with
      much popular and academic commentary celebrating (at least implicitly) the
      capacity of Web 2.0 practices to recast the social arrangements and relations
      of school-based learning along open and democratic lines. As Solomon and
      Schrum (2007, p. 8) conclude, “everyone can participate thanks to social
      networking and collaborative tools and the abundance of Web 2.0 sites …
      The Web is no longer a one-way street where someone controls the content.
      Anyone can control content in a Web 2.0 world”.

Acknowledging the realities of Web 2.0 use in the school of today

           While much hope and excitement surrounds the educational potentials
      of Web 2.0 tools and applications, many education technologists remain pro-
      foundly frustrated by the apparent lack of effective Web 2.0 use in schools.
      In particular, a consistent picture is emerging from the empirical literature of
      a noticeable disjuncture between the rhetoric of mass socialisation and active
      community-led learning and the rather more individualised and passive reali-
      ties of Web 2.0 use in schools. Concerns are therefore beginning to be raised
      that Web 2.0 technologies do not appear to be used to their full potential even
      in relatively well-resourced, “high-technology” classrooms.
           This “digital disconnect” between the rhetoric and reality of Web 2.0 use
      in schools was demonstrated in a recent UK study that deliberately targeted
      schools that were known to make extensive use of Web 2.0 technologies in
      their teaching and learning (Luckin et al., 2009). These researchers found
      most students to be making some use of Web 2.0 technologies, with the most
      prominent activities in the classroom being social networking sites, Weblogs,
      wikis, discussion forums and online chat and uploading and downloading
      of online material. While the study was able to identify some examples of
      engaging and educationally worthwhile Web 2.0 approaches, a range of
      impediments to effective use were identified. For example, the study found
      that teachers were generally cautious in adopting collaborative and communal
      Web 2.0 practices that many felt could challenge traditional school structures.
      More importantly, a number of practical barriers relating to technological
      access, infrastructure and bandwidth continued to impede Web 2.0 use even
      in the more well-resourced schools. The study also underlined that the edu-
      cational use of Web 2.0 tools largely depended on the rigidity or flexibility of


                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                  1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY – 27



       the school curriculum. Additionally, teacher fears related to Internet safety
       and school policy constraints, such as school Internet restrictions and fire-
       walls, were reported to often impose barriers for the adoption of Web 2.0
       practices. The study also drew attention to the fact that “learners spend, on
       average, more time working on school work on a computer outside school
       than at school itself” (Luckin et al., 2009).
           While institutional factors undoubtedly influence the varying levels of
       Web 2.0 in schools, Luckin’s study was also significant in highlighting the
       rather narrow nature of Web 2.0 use in school. The study reported that for most
       students Web 2.0 applications appeared to be used to engage with learning
       content and other learners in a number of bounded and passive ways, rather
       than supporting unconstrained active interaction with information and knowl-
       edge. As Luckin et al. (2009) concluded, even in schools with high levels of
       Web 2.0 use in the classroom, there was “little evidence of critical enquiry or
       analytical awareness, few examples of collaborative knowledge construction,
       and little publication or publishing outside of social networking sites”. At best,
       many students’ engagement can be said to lead to what Crook (2008) terms a
       “low bandwidth exchange” of information and knowledge, with any potential
       for socially-situated authentic learning realised more accurately in terms of
       co-operation rather than collaboration between individuals. This, of course,
       contradicts the rhetoric of “the Web 2.0 ethos of establishing and sustaining
       collaborative learning communities” (Crook and Harrison, 2008, p. 19).
            The unsatisfactory use of Web 2.0 tools in school settings is also reflected
       in the emerging qualitative research literature on the nature of students’ use
       of Web 2.0 tools in school settings. A growing number of in-depth observa-
       tional studies also suggest that the compulsion for convivial creation that is
       seen to lie at the heart of many Web 2.0 practices does not translate easily
       into many classroom contexts. For instance, recent qualitative studies have
       illustrated how fostering a spirit of “commons-based peer production” within
       a community of Web 2.0 users is especially difficult in formal education set-
       tings. Grant’s (2009) case study approach to the use of wiki technologies by
       13 and 14-year-old science and technology students in the UK offers some
       revealing insights into the clash between the communitarian ideals of many
       education technology designers and the rather more “closed” approaches
       towards technology-based learning which are fostered in learners from what
       Grant (2009) terms their “experience of the broader economy of education
       and school practices”. Similarly, Lund and Smørdal’s (2005) earlier study of
       collaborative wiki construction in Norwegian secondary schools showed how
       learners preferred to create new entries indefinitely at the expense of editing
       and improving their own or their classmates’ contributions. Students were
       observed to “not immediately embrace any notion of collective ownership
       or epistemology but continued a practice where the institutionally cultivated
       individual ownership persisted” (Lund and Smørdal, 2005, p. 41).


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
28 – 1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY

           These findings are replicated in other studies of different Web 2.0 tools.
      For example, Knobel and Lankshear’s (2006) study of blogs produced by
      United States school children revealed a lack of creativity and development
      of ideas in terms of students’ writing process, coupled with a subsequent lack
      of supportive feedback and commentary from other members of the group.
      The overall conclusion drawn from the study was what the authors termed as
      a “why bother?” attitude amongst students and teachers alike. These reactions
      are, perhaps, unsurprising as learners’ participation in school-based learning
      activities (by their very nature) are coerced rather than chosen. Thus, as Kate
      Orton-Johnson reasons, the communicative and communal activities most
      readily associated with Web 2.0 technologies are often, in effect, only “sec-
      ondary activities” which contribute little to the “real” practices of academic
      study which remain “grounded in traditional offline activities; reading, note
      taking and the production of assessed work” (Orton-Johnson, 2007).

Popular solutions for overcoming the “problem” of schools in a Web 2.0
world

          It is clear that as they currently stand, some aspects of Web 2.0 use “fit”
      better than others with the realities of contemporary schools and schooling.
      From this perspective, increasing numbers of educationalists have therefore
      started to search for reasons that may underpin the apparent “failure” of
      Web 2.0 technologies in schools. As is often the case with debates over the
      “short-comings” of public education, “blame” has tended to be most read-
      ily attributed to the perceived deficiencies of educational institutions and
      practitioners. In particular, the last five years have seen a consensus forming
      amongst educational technologists that the structure of contemporary schools
      and schooling is responsible primarily for “emasculating” the potential of
      Web 2.0 technology (Somekh, 2007). In particular, schools’ continued reliance
      on broadcast pedagogies of various kinds, structured hierarchical relationships
      and formal systems of regulation is seen to leave them “poorly placed to deal
      well” with the challenges posed by Web 2.0 technologies (Bigum and Rowan,
      2008, p. 250). These criticisms reflect long-running dissatisfactions with
      schools in the “digital age”. As Luke (2003, p. 398) concluded just before the
      mainstream emergence of Web 2.0 technologies, twenty-first century educa-
      tors can be accused of failing increasingly to “come to terms with the contra-
      dictions” between the complexities and fluidities of digitally-based learning
      and the persistence of a model of schooling “based on static print/book culture
      and competitive individualism where learning is geographically tied to a desk
      … and old-style transmission and surveillance pedagogy”.
           As such, many practice-based reasons are beginning to be put forward
      for the poor showing of Web 2.0 in schools. For instance, school buildings are
      criticised as being architecturally unsuitable for widespread networked and


                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                  1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY – 29



       wireless technology use. Teachers are criticised as being too old, incompetent
       or disinterested to integrate Web 2.0 applications into their teaching. Students
       are said to lack the skills or application to make the most of educational
       (rather than leisure) applications of Web 2.0 applications and tools. School
       leaders and administrators are felt to lack the required direction or foresight
       to adopt collective and communal approaches into their school organisation
       and management. School curricula are criticised as remaining too rigid and
       entrenched in top-down paradigms of information transfer. All told, the
       emerging received wisdom amongst many educationalists and technologists
       is that schools and those within them lack what it takes “to go with the tech-
       nological flow” (Dale et al., 2004).
           All of these factors therefore underpin a growing sense in the minds of
       many proponents of Web 2.0 use in education that schools are simply unable
       to deal with the challenges posed by Web 2.0 technologies for a number of
       intractable structural reasons. As the sociologist Manuel Castells was led to
       conclude recently, “education is the most conservative system as to changing
       anything since the Middle Ages […] the rules, the format, the organisation of
       the schools are completely different in terms of interactivity and hypertex-
       tuality” (Castells, 2008). With these thoughts in mind, much of the current
       debate concerning Web 2.0 and schools is now beginning to focus on how
       best to re-structure the school to fit with the demands and needs of the tech-
       nology use. These solutions for change tend to take one of two forms – either
       the complete replacement of the school through Web 2.0 technologies and
       practices, or else the reinvention of the school through the use of Web 2.0
       tools and practices.

       Replacing the school with Web 2.0 technologies
            In the minds of some commentators, the seriousness of the “school prob-
       lem” little choice but to renounce the school as a viable site for learning. In
       this spirit, growing number of educationalists are concluding that the school
       is a “dead” site for technology use that will never be able to adapt sufficiently
       to the challenge and disruption of the emerging forms of Web 2.0 technolo-
       gies. In this sense the school is conceived as an outmoded technology from a
       past industrial age that should be dismantled. The education technology aca-
       demic literature, at least, is increasingly featuring the promotion of reasoned
       arguments that all of the structural impediments and challenges to technol-
       ogy (i.e. the school) must be removed in order to facilitate the realisation of
       the digital transformation of education.
           Indeed, powerful arguments have long been advanced that children are
       better off learning amongst themselves through Web 2.0 and other Internet
       technologies – gaining an education through the “hard fun” of creating and
       playing in online environments rather than being subjected to the “teaching


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
30 – 1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY

      disabled” pedagogies of the conventional classroom (Negroponte, 1995;
      Shaffer, 2008). Now Web 2.0 technologies are seen to provide a ready basis
      for young people’s circumvention of the traditional structures of their schools
      and generally “finding something online that schools are not providing them”
      as Henry Jenkins (2004) has put it. For example, Web 2.0 tools such as wikis,
      social networking and folksonomy software are seen to be able to change
      education away from being “a special activity that takes place in special
      places at special times, in which children are instructed in subjects for rea-
      sons they little understand” (Leadbeater, 2008a, p. 149). As Nicole Johnson
      concluded from her study of “expert” Web 2.0 users in Australian secondary
      schools, home-based Web 2.0 technologies are allowing students to learn
      despite (rather than because) their schools:
          The [students] were able to choose what they learned and when they
          learned. They viewed the medium in which they did it as a form of
          leisure. They were also able to choose who and what they learned
          from – not just what has been set up as exclusive and privileged.
          They were able to both learn and receive pleasure from their engage-
          ment and not have to be concerned about the hierarchisation and fail-
          ure in relation to how traditional schooling determines competence.
          They were in fact designing and engaging in their own learning. The
          teenage experts did not gain a significant amount of learning in the
          area of computing from formal education and traditional school-
          ing [….] what is significant is that these participants accomplished
          (in their own eyes) a level of expertise that schooling had not been
          chiefly responsible for. Indeed, all of the participants alleged that
          schooling had had little influence in their trajectory toward expertise
          (Johnson, 2009, p. 70).
          As Johnson infers, Web 2.0 tools are seen as having the capacity to make
      learning a “looser” arrangement for the individual student – involving a vari-
      ety of people and places throughout a community for a variety of reasons.
      In this respect, much faith continues to be vested in twenty-first century
      Web 2.0 technologies as a catalyst for the total substitution of twentieth cen-
      tury modes of teaching, learning and schooling.
          There is a distinct ground swell of support within the education technol-
      ogy community for non-school based technology enhanced learning. From
      James Gee’s celebration of the learning potential of computer games through
      to Futurelab’s “Out-Space” agenda, some influential elements of the educa-
      tion technology community appear keen to hasten the decline of the school
      as the primary site of learning. Indeed a spirit of using digital technologies
      to bypass traditional education institutions is evident in online services such
      as the School of Everything – a popular Web space in the UK designed to
      put teachers in contact with learners and therefore aiming to be “an eBay



                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                  1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY – 31



       for stuff that does not get taught in school” (Leadbeater, 2008b). Similarly,
       NotSchool.Net is a well established and officially endorsed online platform
       which aims to re-engage UK teenagers otherwise excluded from the formal
       education system with learning and the pursuit of qualifications. In a higher
       education context, the International University of the People represents a
       not-for-profit volunteer university offering courses provided entirely online
       and free of charge based around principles of open source courseware and
       social networking (Swain, 2009a). Yet rather than being cursory additions to
       traditional schooling, these examples and others like them are seen to mark
       the first steps in a radical rethinking and reorganisation of existing structures
       and organisation of education provision. As Leadbeater (2008b, p. 26) rea-
       sons, the imperative of Web 2.0 based education provision…
            … require[s] us to see learning as something more like a computer
            game, something that is done peer-to-peer, without a traditional
            teacher … We are just at the start of exploring how we can be organ-
            ised without the hierarchy of top-down organisations. There will
            be many false turns and failures. But there is also huge potential to
            create new stores of knowledge to the benefit of all, innovate more
            effectively, strengthen democracy and give more people the opportu-
            nity to make the most of their creativity.

       Reinventing the school through Web 2.0 technologies
            While these “replacement discourses” are growing in popularity, sup-
       port remains amongst many educationalists and some technologies for the
       use of Web 2.0 tools as a means to re-configure and re-invent the school
       – retaining the overall notion of the school as an institution, but along more
       fluid and flexible lines of “school 2.0” (e.g. Wang and Chern, 2008). Such
       “reschooling” arguments are advanced most commonly via proposals for the
       development of digitally aligned modes of schooling that are built around
       the active communal creation of knowledge (rather than passive individual
       consumption), and imbued with a sense of play, expression, reflection and
       exploration. As such, any re-conceptualisation of the school and classroom is
       deliberately learner-centred – focused on “learner participation and creativity
       and online identity formation, and how these intersect with, support, or sug-
       gest desired competencies, teaching practices, and policies” (Greenhow et al.,
       2009, p. 225). These imperatives to change and reinvent have been expressed
       most fully in terms of curriculum and pedagogy, as evidenced in the variety
       of recent proposals from education commentators and stakeholders for “peda-
       gogical mash-ups”, “remix curricula” and pedagogies of social interaction
       (e.g. Fisher and Baird, 2009; Code and Zaparyniuk, 2009).
           All of these curricular reconfigurations are predicated upon the notion
       that Web 2.0 technologies are leading to different types of information and


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
32 – 1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY

      knowledge production that is based around fast-changing, non-textual forms
      that require new forms of more critical and reflexive information skills and
      literacies (Buschman, 2009). In this sense the argument is increasingly being
      made that it no longer makes sense to retain “pre-digital” models of curricular
      organisation focused on rigidly hierarchic organisation of static content under
      the control of the teacher. Instead, questions are now being asked in relation to
      how best to develop Web 2.0-inspired curricula that can be negotiated rather
      than prescribed, that are driven by learner needs, and based on providing
      learners with skills in managing and accessing knowledge and being in control
      of their own learning pathways and choices (Facer and Green, 2007). Thus,
      growing numbers of authors are now discussing the likely nature and form
      of “curriculum 2.0” – what Edson (2007) terms as “user-driven education”,
      allowing learners to take an active role in what they learn as well as how and
      when they learn it. Of course, this “pick and mix approach” to curricular con-
      tent and form are also seen to present a fundamental challenge to the profes-
      sional roles and cultures of educators (Swain, 2009b). As McLoughlin and Lee
      (2008, p. 647) conclude, all of these proposals therefore centre on the need for
      educators to also change their practices and expand their vision of pedagogy,
      “where learners are active participants or co-producers of knowledge rather
      than passive consumers of content and learning is seen as a participatory,
      social process supporting personal life goals and needs”.
          All of these arguments reflect a growing belief that technology-based
      practices of collaboration, publication and inquiry should be foregrounded
      within schools’ approaches to teaching and learning. The mass collaboration
      seen to be at the heart of Web 2.0 applications has been touted by some com-
      mentators as having the potential to “change everything” – even allowing
      students to rewrite and edit school textbooks (Tapscott and Williams, 2008).
      For instance, calls continue to be made for the rebuilding of schools to fit
      with the needs and demands of modern technology. From continuing calls
      for a “recombinant architecture” to proposals for the re-design of the school
      environment into “collaboration-friendly”, “really cool spaces” (e.g. Dittoe,
      2006), the notion of redesigning and rebuilding the physical environment of
      the school continue to gain popularity. Underpinning many of these sugges-
      tions is the belief that children should be given more control of their interac-
      tions with information and knowledge. For instance, Marc Prensky (2008)
      argues for a “new pedagogy of kids teaching themselves with the teacher’s
      guidance”. This sense of allowing young people opportunities to influence
      the direction of institutional change is reflected in Donald Tapscott’s (1999)
      advice to “give students the tools, and they will be the single most important
      source of guidance on how to make their schools relevant and effective places
      to learn” (p. 11). While none of these authors are suggesting the complete
      abolishment of school, they are pointing towards a substantial alteration and
      refocusing of what schools are and what they do.



                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                  1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY – 33



Towards a more reasoned response to Web 2.0 and the school of the future

            At first glance, many of these responses and arguments appear perfectly
       well-reasoned and sensible. There is an undoubted need to reconcile school-
       ing with the challenges of digital technologies, and it makes sense to sketch
       out ideas for how systems of schooling that have not fundamentally changed
       since the beginning of the twentieth century can be brought up to date with
       twenty-first century life. Yet while compelling, there are a number of incon-
       sistencies to these current debates surrounding schools and Web 2.0 that
       merit further scrutiny and challenging. In particular, it should be observed
       that current discussions of Web 2.0 and schools repeat a long-standing ten-
       dency in education for exaggerated and extreme reactions to technology that
       are centred around matters of learning and teaching rather than the wider
       social, political, economic and cultural contexts of education. Specifically,
       most educational thinking concerning Web 2.0 reflects an implicit “technol-
       ogy-first” way of thinking, where Web 2.0 technologies are imbued with a
       range of inherent qualities that are then seen to “impact” (for better or worse)
       on learners, teachers and schools in ways that are consistent regardless of cir-
       cumstance or context. In this way, current debates over Web 2.0 and schools
       are perpetuating a long lineage in educational thinking about technology
       based around a crude but compelling “technologically determinist” perspec-
       tive that “social progress is driven by technological innovation, which in turn
       follows an ‘inevitable’ course” (Smith, 1994, p. 38).
           One of the key weaknesses of a technologically determinist reading of
       schools and Web 2.0 is the tendency to approach technology-based processes
       as a closed “black box”. As such it is important to recognise the ideological
       underpinnings of the current Web 2.0 drive in education. Indeed, it should
       be clear from the brief examples in this article that the current discussions
       over Web 2.0 and schools reflect a number of ongoing debates about educa-
       tion and society that are highly ideological in nature. As such, the forms of
       Web 2.0-based changes being proposed from within the education technology
       community are not merely benign technical re-adjustments to school-based
       learning and teaching. Whether they realise it or not, these proposals are
       highly political in nature.
            For example, much of the current debates about Web 2.0 and the re-inven-
       tion of schools (what can be recognised as a set of arguments concerning
       the re-schooling of society), position Web 2.0 technologies as a “technical
       fix” for addressing wider concerns about schools and schooling. Over the
       last forty years at least, schools have been seen by many commentators as a
       cause for concern rather than celebration, with accounts persisting in many
       developed countries of school systems somehow “failing” to perform as well
       they should. For many policy makers and other commentators, the under-
       performance of schools has led to what Stephen Gorard (2001) has termed


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
34 – 1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY

      a prevailing “crisis account” of schooling where educational opportunities
      are seen to be increasingly polarised, and schools are characterised by poor
      overall educational standards. In this sense, some sections of the educational
      community appear to be all too keen to seize upon Web 2.0 technologies
      as offering a ready “technical fix” to the problem of the failing – or at least
      underperforming – school. As such, many of the arguments being advanced
      for Web 2.0 are not driven by a deep belief about the educative power of tech-
      nology, rather they are driven by a deep concern about the state of schooling
      in contemporary society. As such, Web 2.0 technologies are being used as a
      vehicle through which to express a long-standing tendency in western socie-
      ties to view digital technology as a “technical fix” for wider social problems.
           The ideological underpinnings of the replacement arguments surround-
      ing Web 2.0 are even more diverse and hidden. In particular, proposals for
      the Web 2.0 replacement of the school should be seen as feeding into a wider
      anti-schooling sentiment that has long been implicit in discussion of educa-
      tion and technology, often based upon a range of anti-establishment ideals
      (see Bigum and Kenway, 1998). In this sense it is evident how much of the
      current calls outlined above for the discontinuation of schooling in favour of
      technological means advocate the comprehensive “deschooling” of society
      along digital lines – consciously updating the arguments of Ivan Illich. Illich’s
      (1971) condemnation of institutionalised learning centred on a set of concerns
      that educational institutions prevent individual growth and community-based
      learning. This logic has a direct lineage with contemporary rhetoric of digital
      technologies and education. As Charles Leadbeater (2008a, p. 44) reasoned
      recently, “In 1971 [deschooling] must have sounded mad. In the era of eBay
      and MySpace it sounds like self-evident wisdom”. Indeed, the tendency of
      educationalists to celebrate individuals’ self-determination of their learning
      via Web 2.0 tools feeds into a wider enthusiasm shared amongst many in
      education for the inherent benefits of forms of “informal learning” that take
      place outside the control of formal education organisations and settings (see
      Sefton-Green, 2004). This in turn can be seen as part of a wider societal ide-
      alisation of the informal (Misztal, 2000), and the networked individualism of
      everyday life (see Beck and Beck-Gernsheim, 2002).
          In one sense, these arguments stem from a continuation of the counter-
      cultural, Californian “anti-establishment” ideals that have underpinned much
      of the development of information technology since the 1970s. Care should
      be taken within educational debates to recognise the wider ideological con-
      notations and underpinnings of Web 2.0 discourse. As Danah Boyd (2007,
      p. 17) points out, for many technologists the notions of “Web 2.0” and “social
      software” are not used merely as neutral labels, but also as a rallying call for
      a new age of activities which are made “by the people, for the people” rather
      than centred around official, institutional interests. Yet while the intentions of
      many technologists may well be rooted in such relatively benign sensibilities,


                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                  1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY – 35



       it is noticeable that the spirit of these arguments is now being used to support
       a removal of the state from the provision of public education by a range of
       more neo-conservative and neo-liberal interests (see Kovacs, 2007; Apple,
       2004). For example, it is noticeable how new Internet technologies are begin-
       ning to be enrolled into recent neo-liberal arguments for the “end of school”
       and realising the “dream of education without the state” (Tooley, 2006). Here
       technology is valorised as an ideal vehicle for the establishment of “a genuine
       market in education, where there was no state intervention of any kind, in
       funding, provision or regulation” (Tooley, 2006, p. 26). For example, Tooley
       (2006, p. 22) talks of “the technological capability to allow inspiring teachers
       to reach millions of young people [rather than] forc[ing] all teachers into an
       egalitarian straight-jacket”.
            From this perspective, many of the arguments for the Web 2.0 replace-
       ment of schools could be said to feed into the wider libertarian discourses
       that have long pervaded societal and political discussion of digital technol-
       ogy – what writers such as Langdon Winner (1997) have termed “cyber-
       libertarianism”. Here the power of technology and the power of the individual
       (what Kelemen and Smith [2001, p. 371] term “two ideas which lie at the
       heart of modern civilisation”) converge into an argument for the creation of
       new forms of action and organisation that do not require the appropriation of
       traditional space or structures. In this sense digital technology is positioned
       as nothing less than “a moral enterprise set to rescue the world” (Kelemen
       and Smith, 2001, p. 370), underpinned by an ideological faith in the power
       of radical individualism, market forces and pursuit of rational self-interest
       (Winner, 1997). All of these sentiments seem a world away from the hopes
       of more social and communal forms of learning outlined at the beginning of
       this article.

Conclusion: Towards a more critical understanding of Web 2.0, schools
and schooling
            Any reader of this article should now be clear about the political nature
       and the political importance of schools and Web 2.0 technology. Debates
       about schools and Web 2.0 are not simply about matters of Internet bandwidth
       or the pedagogic affordances of wikis. They are also debates about questions
       of benefit and power, equality and empowerment, structure and agency and
       social justice. In this sense, it is crucial to recognise that Web 2.0 is a con-
       tradictory, inconsistent and polemic notion – there is no neat, unproblematic
       “Web 2.0” solution to the deficiencies of twenty-first century education.
       Instead Web 2.0 is a site of intense ideological competition and struggle along
       many lines and involving many different interests. For technologists, there is
       a feeling that Web 2.0 offers a second chance to get the Internet “right” – to
       correct the wrongs of the dot.com boom and bust and counter the increased


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
36 – 1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY

      commercial proprietary control of Internet services. For those in business
      and commerce, there is a feeling that Web 2.0 represents a new struggle for
      harnessing technology in the pursuit of profit and the harnessing of consumer
      desires. Similarly in educational terms there is a feeling amongst many learn-
      ing technologists that Web 2.0 represents the “killer-app” for bringing more
      desirable forms of socio-constructivist learning to the masses. For some
      radically minded educators and proponents of democratic education there is
      a feeling that Web 2.0 represents a ready means to reinvent the provision of
      education along more expansive, equitable and easy lines. Conversely, for
      neo-liberal educators there is a feeling that Web 2.0 represents a ready means
      to wrestle schooling and education away from the clutches of “big govern-
      ment” and the state.
           Thus, there is a pressing need to refocus current educational discussion
      and debate away from how Web 2.0 technologies may be best used to revital-
      ise learning and education in the twenty-first century. As with any other edu-
      cational technology, Web 2.0 applications do not present a ready “technical
      fix” to the many social problems faced by contemporary education systems
      around the world. As Scholz (2008) observes, Web 2.0 “is not and cannot be
      the all-mighty teacher, healer, and redeemer for everything that went astray in
      society”. Instead, if Web 2.0 is already being used as a ready site for rehears-
      ing many of the wider debates, controversies and tensions about the future
      of schools and schooling in the twenty-first century, then it makes sense that
      more meaningful lines of debate can be opened up and pursued. As Michael
      Apple (2002, p. 442) has reasoned:
          the debate about the role of the new technology in society and in
          schools is not and must not be just about the technical correctness of
          what computers can and cannot do. These may be the least important
          kinds of questions, in fact. Instead, at the very core of the debate are
          the ideological and ethical issues concerning what schools should be
          about and whose interests they should serve.
          In particular, current thinking about Web 2.0 and schools therefore
      contains a number of silences and gaps that require recognising and con-
      fronting – not least the portrayal of new technology as capable of enacting
      new arrangements and forms of education. For all its intuitive appeal, the
      widespread valorisation of informal learning and the technology-empowered
      individual learner dangerously depoliticises the act of learning (Gorman,
      2007), placing far too much emphasis on the disembodied individual learner.
      Such arguments could be said to present an overly simplistic view of success-
      ful education relying merely on groups of like-minded individuals, failing
      to consider the wider social, economic, political and cultural contexts of the
      societal act of schooling. A number of critical questions therefore remain
      unasked and unanswered. For example, if the state is no longer responsible



                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                  1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY – 37



       for the provision of education through school systems, then who is to assume
       responsibility? What is the role of the private sector and corporate capital-
       ism in the libertarian take on Web 2.0 based schooling? What inequalities
       of access, skills, resourcing or know-how will remain, and who will be con-
       cerned with correcting them?
            All of these questions and silences point to the dangers of educational
       technologists using Web 2.0 as a justification for giving up on the notion of
       the school with some form of pedagogic authority and responsibility. Instead
       of rejecting the entire notion of the industrial-era school as it currently exists,
       it may be more productive to set about addressing the “problem” of schools
       and technology in subtler and less disruptive ways that work “with” the
       micro-politics of the school rather than against them. As Wilhelm (2004,
       p. xii) puts it, “meet[ing] people where they are, not where they would like
       them to be?” From this perspective, the argument should be considered that
       it perhaps makes little sense – and is of little practical help – to argue that
       the only way that Web 2.0 technologies can be properly used in education is
       by radically altering the school. Perhaps it makes more sense to seek ways of
       reducing the imperative for engineering widespread Web 2.0-driven changes
       to the schools of today and, instead, adopt more organic, “bottom-up”
       approaches to the adjustment of schools and schooling. More attention could
       be paid, for example, to exploring ways of “loosening up” the use of digital
       technologies within school settings and introducing a degree of Web 2.0-led
       informality to schools’ digital practice without undermining the overall insti-
       tutionalised social order of the school. Thus instead of calling for a complete
       learner-driven free-for-all mode of school technology use, careful thought
       now needs to be given as to exactly how the relationships between formal-
       ity and informality within schools may be adjusted and altered in ways that
       can shift the frames of in-school technology use without undermining basic
       institutional structures and interests.
           Above all it would seem crucial that the voices, opinions and desires of
       learners and teachers are paid more attention to in the further discussion and
       debates that are now required to advance ways in which such beneficial loos-
       ening of school technology use may be achieved.
           Indeed, as Daanen and Facer (2007) argue, one of the key issues under-
       pinning any use of technology in education is the simple question of “who
       decides?” At present it would seem that more often than not, Web 2.0 tech-
       nology use is something that is being “handed down” to those involved in
       schools as a fait accompli rather than something that is negotiable and malle-
       able. Instead, Web 2.0 technologies are too more important and significant an
       issue to be simply “handed down” to education. As such a far more inclusive
       and user-driven debate about Web 2.0 and school now needs to be initiated




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
38 – 1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY

      – involving all of the “publics” of education, not least teachers, learners, par-
      ents and other people in the currently “silent majority” of end-users:
          When we look at the capacity emerging technologies may offer to
          reorganise the institutions, practices and people of education, the
          issues raised are broader than those raised by the needs of future
          employers…. As such, we cannot leave discussions of the future
          role of technology in education only to the technology industry….
          Instead, we need to develop the mechanisms for an open and public
          debate on the nature and purpose of education in the digital age
          which goes beyond safe slogans such as “meeting the needs of every
          child” (who can disagree with that?). Instead, we need to confront
          the fact that longstanding assumptions about what education is for,
          who conducts it, and how it is assessed, may need to be challenged
          (Daanen and Facer, 2007, p. 28).




                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                  1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY – 39




                                           References

       Allen, M. (2008), “Web 2.0: An argument against convergence”, First
          Monday, Vol. 13, No. 3.
       Apple, M. (2002), “Is the new technology part of the problem or part of the
         solution in education?”, in Darder, A., M. Baltodano and R. Torres (eds.),
         The Critical Pedagogy Reader, Routledge, London.
       Apple, M. (2004), “Are we wasting money on computers in schools?”,
         Educational Policy, Vol. 18, No. 3, pp. 513-522.
       Beck, U. and E. Beck-Gernsheim (2002), Individualization, Sage, London.
       Bigum, C. and J. Kenway (1998), “New information technologies and the
          ambiguous future of schooling: Some possible scenarios”, in International
          Handbook of Educational Change, Kluwer, Springer, New York, NY.
       Bigum, C. and L. Rowan (2008), “Landscaping on shifting ground: Teacher
          education in a digitally transforming world”, Asia-Pacific Journal of
          Teacher Education, Vol. 36, No. 3, pp. 245-255.
       Boyd, D. (2007), “The significance of social software”, in Burg, T. and
         J. Schmidt (eds.), BlogTalks Reloaded: Social Software Research and
         Cases, Books on Demand, Norderstedt.
       Brusilovsky, P. (2008), “Social information access: The other side of the
          social web”, Lecture Notes in Computer Science, No. 4910, pp. 5-22.
       Buschman, J. (2009), “Information literacy, ‘new’ literacies and literacy”, The
          Library Quarterly, Vol. 79, No. 1, pp. 95–118.
       Castells, M. (2008), “Internet beyond myths: The record of scholarly
          research”, presentation to London School of Economics, 24 October.
       Code, J. and N. Zaparyniuk (2009), “The emergence of agency in online
         social networks”, in Hatzipanagos, S. and S. Warburton (eds.), Handbook
         of Research on Social Software and Developing Community Ontologies,
         IGI Global, Hershey, PA.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
40 – 1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY

      Crook, C. (2008), “Theories of formal and informal learning in the world
         of Web 2.0”, in Livingstone, S. (ed.), Theorising the Benefits of New
         Technology for Youth, University of Oxford/London School of Economics.
      Crook, C. and C. Harrison (2008), Web 2.0 Use for Learning at Key Stage
         Three and Four: Final Report, Becta, Coventry.
      Dale R., S. Robertson and T. Shortis (2004), “You can’t not go with the
         technological flow, can you?” Journal of Computer Assisted Learning,
         Vol. 20, pp. 456–470.
      Daanen, H. and K. Facer (2007), 2020 and Beyond: Educational Futures,
        Futurelab, Bristol.
      Davies, J. and G. Merchant (2009), Web 2.0 for Schools: Learning and Social
        participation, Peter Lang, New York, NY.
      Dede, C. (2008), “A seismic shift in epistemology”, Educause Review, May,
        pp. 80-81.
      Dittoe, W. (2006), “Seriously cool places: The future of learning-centred
         built environments”, in Oblinger, D. (ed.) Learning Spaces, EDUCAUSE,
         Washington, DC.
      Edson, J. (2007), “Curriculum 2.0: User-driven education”, The Huffington
         Post, 25 June, www.huffingtonpost.com/jonathan-edson/curriculum-20-
         userdri_b_53690.html.
      Evens, A. (2009), “Dreams of a new medium”, Fiberculture, Vol. 14 http://
         journal.fibreculture.org/issue14/index.html.
      Facer, K. and H. Green (2007), “Curriculum 2.0 educating the digital generation”,
         Demos Collection, No. 24, pp. 47-58.
      Fisher, M. and D. Baird (2009), “Pedagogical mashup: Gen Y, social media,
         and digital learning styles”, in Hin, L. and R. Subramaniam (eds.)
         Handbook of Research on New Media Literacy at the K-12 Level,
         IGI Global, Hershey, PA.
      Gorard, S. (2001), “International comparisons of school effectiveness: The
        second component of the ‘crisis account’ in England?”, Comparative
        Education, Vol. 37, No. 3, pp. 279–296.
      Gorman, R. (2007), “The Feminist standpoint and the trouble with ‘informal
        learning’: A way forward for Marxist-Feminist educational research”, in
        Green, A., G. Rikowski and H. Raduntz (eds.), Renewing Dialogues in
        Marxism and Education, Palgrave Macmillan, London.




                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                  1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY – 41



       Grant, L. (2009), “HEATHER I DON’T CARE DO UR OWN PAGE! A case
          study of using wikis for collaborative inquiry in school”, Learning, Media
          and Technology, Vol. 34, No. 2, pp. 105-117.
       Greenhow, C., B. Robelia and J. Hughes (2009), “Web 2.0 and classroom
          research: What path should we take now?” Educational Researcher,
          Vol. 38, No. 4, pp. 246-259.
       Hargadon, S. (2008), Web 2.0 is the future of learning, 04 March, www.
         stevehargadon.com/2008/03/web-20-is-future-of-education.html.
       Illich. I. (1971), Deschooling Society, Penguin Books, Harmonsworth.
       Jenkins, H. (2004), “Why Heather can write”, Technology Review, BizTech,
          6 February, www.technologyreview.com.
       Johnson, N. (2009), “Teenage technological experts’ views of schooling”,
          Australian Educational Researcher, Vol. 36, No. 1, pp. 59-72.
       Kelemen, M. and W. Smith (2001), “Community and its ‘virtual’ promises:
          A critique of cyberlibertarian rhetoric”, Information, Communication &
          Society, Vol. 4, No. 3, pp. 370-388.
       Knobel, M. and C. Lankshear (2006), “Weblog worlds and constructions
         of effective and powerful writing: cross with care and only where signs
         permit”, in Pahl, K. and J. Roswell (eds.), Travel Notes From the New
         Literacy Studies: Instances of Practice, Multilingual Matters, Clevedon.
       Kovacs, P. (2007), “The anti-school movement”, in Gabbard, D. (ed.),
         Knowledge and Power in the Global Economy: The Effects of School
         Reform in a Neoliberal/Neoconservative Age, Routledge, London.
       Lameras, P., I. Paraskakis and P. Levy (2009), “Using social software for
         teaching and learning in higher education” in Hatzipanagos, S. and
         S. Warburton (eds.), Handbook of Research on Social Software and
         Developing Community Ontologies, IGI Publishing, Hershey, PA.
       Leadbeater, C. (2008a), We-Think, Profile, London.
       Leadbeater, C. (2008b), “People power transforms the web in next online
          revolution”, The Observer, 09 March, p. 26.
       Luckin, R. et al. (2009), “Do Web 2.0 tools really open the door to learning:
          practices, perceptions and profiles of 11-16 year-old learners”, Learning,
          Media and Technology, Vol. 34, No. 2, pp. 87-114.
       Luke, C. (2003), “Pedagogy, connectivity, multimodality, and interdisciplinarity”,
          Reading Research Quarterly, Vol. 38, No. 3, pp. 397-413.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
42 – 1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY

      Lund, A. and O. Smørdal (2006), “Is there a space for the teacher in a wiki?”,
        in Proceedings of the 2006 International Symposium on Wikis, pp. 37-46,
        Odense, Denmark.
      Mason, R. and F. Rennie (2007), “Using Web 2.0 for learning in the
        community”, Internet and Higher Education, Vol. 10, pp. 196-203.
      McLoughlin, C. and M. Lee (2008), “Mapping the digital terrain: New media
        and social software as catalysts for pedagogical change” in Hello! Where
        are You in the Landscape of Educational Technology? Proceedings,
        Ascilite Melbourne 2008, www.ascilite.org.au/conferences/melbourne08/
        procs/mcloughlin.html.
      Negroponte, N. (1995), Being Digital, Coronet, London.
      O’Reilly, T. (2005), What is Web 2.0? Design Patterns and Business Models
         for the Next Generation of Software, www.oreillynet.com/pub/a/oreilly/
         tim/news/2005/09/30/what-is-web-20.html.
      Orton-Johnson, K. (2007), “The online student: Lurking, chatting, flaming
         and joking”, Sociological Research Online, Vol. 12, No. 6, www.
         socresonline.org.uk/12/6/3.html.
      Prensky, M. (2008), “The role of technology in teaching and the classroom”,
         Educational Technology, Vol. 48, No. 6, November/December.
      Roberts, B. (2009), “Beyond the networked public sphere: Politics,
        participation and technics in Web 2.0”, Fiberculture, Vol. 14, http://
        journal.fibreculture.org/issue14/index.html.
      Scholz, T. (2008), “Market ideology and the myths of Web 2.0”, First
         Monday, Vol. 13, No. 3.
      Shaffer, D. (2008), “Education in the digital age”, The Nordic Journal of
         Digital Literacy, Vol. 4, No. 1, p. 39-51.
      Shirky, C. (2008), Here Comes Everybody: The Power of Organizing Without
         Organizations, Allen Lane, London.
      Smith, M. (1994), “Recourse of empire”, in Smith, M. and L. Marx (eds.)
        Does Technology Drive History? The Dilemma of Technological
        Determinism, MIT Press, Cambridge, MA.
      Solomon, G. and L. Schrum (2007), “Web 2.0: New tools, new schools”,
         International Society for Technology in Education, Washington, DC.
      Somekh, B. (2007), “How education systems are emasculating technology”,
        paper presented to CAL 07 Conference, March, Dublin.




                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                  1. WEB 2.0 AND THE SCHOOL OF THE FUTURE, TODAY – 43



       Swain, H. (2009a), “Look! No fees”, The Guardian, Education Supplement,
         Tuesday, 06 October, p. 7.
       Swain, H. (2009b), “Dawn of the cyberstudent”, The Guardian, 20 January,
         University Challenge Supplement, p. 1.
       Tapscott, D. (1999), “Educating the Net generation”, Educational Leadership,
          Vol. 56, No. 5, pp. 6-11.
       Tapscott, D. and A. Williams (2008), Wikinomics: How Mass Collaboration
          Changes Everything, Atlantic.
       Tooley, J. (2006), “Education reclaimed”, in Booth, P. (ed.), Towards a Liberal
          Utopia?, Continuum, London.
       Wang, S. and J. Chern (2008), “The new era of ‘school 2.0’ – teaching with
         pleasure, not pressure”, in Proceedings of World Conference on Educational
         Multimedia, Hypermedia and Telecommunications, Association for the
         Advancement of Computing in Education, Chesapeake, VA.
       Wilhelm, A. (2004), Digital Nation: Toward an Inclusive Information Society,
          MIT Press, Cambridge, MA.
       Winner, L. (1997), “Cyberlibertarian myths and the prospects for community”,
         Computers and Society, Vol. 27, No. 3, pp. 14-19.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                            2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION? – 45




                                             Chapter 2

             Can digital learning resources spur innovation?


                                           Jan Hylén
                                       Metamatrix, Sweden




    This chapter looks at the results of a study carried out in the five Nordic
    countries[i] that analysed recent developments in the area of digital learning
    resources (DLR) from the perspective of systemic innovation. Three different types
    of innovation are examined: government-initiated innovations; innovations initi-
    ated by commercial actors; bottom-up innovations (user-generated). The authors
    point out how technology makes the conditions for DLR innovation different from
    many other fields of education and present five “embryonic scenarios” illustrating
    the ways that DLR might be strengthened, promoted, developed and incorporated
    are presented. The chapter concludes with recommendations directed towards to
    the production and use of DLR, and to the more general issue of systemic innova-
    tion in education.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
46 – 2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION?

Background, objectives and methodological approach

           Change is taking place at various speeds in different parts of most OECD
      countries’ education systems, with varying drivers and different degrees of
      premeditation. Although the management of change within complex systems
      is a key challenge to educational policy makers, the dynamics of innovation
      in education are not fully understood. So far, not much comparative analysis
      has been carried out on the policies related to educational innovation, the
      knowledge base on which they draw, and their effectiveness.
          The report forms part of the OECD studies on systemic innovation,
      including the work on systemic innovation in vocational education and train-
      ing (VET). The research draws on lessons learned from previous CERI work
      on Open Educational Resources (OER) in the broader field of digital learning
      resources (OECD, 2007), and provides a better understanding of the process
      of systemic innovation regarding ICT in schools. The lessons learned from
      the OER project include the strength of bottom-up innovations, the impor-
      tance for the education sector of new business models emerging around free
      content and new copyright licenses, such as Creative Commons. It is still
      unknown how these recent developments impact on the production and use
      of DLR in schools. That institutions and individuals sometimes give away
      their knowledge for free as OER also highlights the need for countries to take
      a global view on the production and distribution of DLR – be it commercial
      or non-commercial resources. Finally, it relates closely to CERI work within
      the New Millennium Learner project.
          The broad aim of this activity has been to review and evaluate the pro-
      cess of systemic innovation in policy making and in public and private initia-
      tives designed to promote the development, distribution and use of DLR for
      the school sector. In so doing, the activity has brought together evidence on:
          1. How countries 1 go about initiating ICT-based educational innova-
             tions related to DLR, the players and processes involved, the knowl-
             edge base which is drawn on, and the procedures and criteria for
             assessing progress and outcomes;
          2. Factors that influence the success of policies aimed at promoting
             ICT-based educational innovations, particularly those related to the
             production, distribution and use of DLR, including user involvement
             in the production process and new actors such as the gaming industry
             and media companies;
          3. User-driven innovations related to DLR, carried out by teachers and
             researchers, such as innovative production and use of DLR, and how
             the educational system responds to such innovations.




                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                            2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION? – 47



           Accordingly, instead of focusing on discrete institutional innovations,
       this activity has aimed at a better understanding of how the process of sys-
       temic innovation works best in relation to DLR. The innovation process, as
       defined in the analytical framework of this study, is composed of several
       phases, such as initiation, implementation, scale up, monitoring, and evalua-
       tion. These have been examined in the study together with other factors, such
       as governance and financing, that influence the development of the innova-
       tion process.
            In a literature review on innovation in education, OECD (2009a) con-
       cludes that the extent to which something is new to a given social context is
       crucial to identifying innovation. A reform, on the other hand, could be, for
       example, an official legitimisation of well-known teaching practices. Thus, in
       line with the already stated pragmatic perspective, the concept of innovation
       that was used in this study was deliberately open: innovation is change that
       is introduced with the aim of improving the operation of education systems,
       their performance, the perceived satisfaction of the main stakeholders, or
       all of them at the same time. The use of such an open definition allows for
       diversity.
            DLR used on a large scale in schools are considered as instances of sys-
       temic innovation. One important related question is why education systems
       should be innovative. This has to do with the possibility of the system to
       contribute to a societal need of innovation, which is seen not only as a key
       factor to economic growth but also to social welfare. Innovation relies heav-
       ily on the creation of basic knowledge, through both education and science.
       A well-performing education system facilitates the adoption and diffusion
       of innovation, by providing human capital for innovation and by innovation
       within education and training. A well-performing education system also
       means a system in tune with changes taking place in society, such as glo-
       balisation, technological developments and the growing amount of informal
       learning that is taking place outside the education system. It also needs to
       take into account individual needs of children, differences in learning styles,
       special needs, and special talents. To meet these demands our education
       systems need to improve their operations, performance, and the satisfaction
       among the stakeholders – hence to innovate.
           The methodological approach in the study consisted of two different
       strands, analytical and empirical. Building on the parallel OECD project
       on systemic innovation in VET, an analytical framework was developed by
       the Secretariat. The framework has also made use of the three classic pillars
       of ICT policy development: investments in ICT infrastructure in schools;
       investments in in-service training or competence development for teachers
       (and head teachers); and investments in development of content and software
       tools. The empirical strand was based around a series of country visits and



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
48 – 2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION?

      case studies. Rather than aiming for full country reviews, the project built
      on case studies developed by a team of international experts on the basis of
      a Country Background Report. The cases were proposed by each country,
      discussed with the experts and chosen by the Secretariat.
          The study used the term “digital learning resources” (DLR). It was not
      the aim of the study to do any innovative work related to the definition of
      the concept, only to state the position of the DLR project regarding some
      of the issues that arose in the discussion on the concept of digital learning
      resources. This study has only considered learning resources that are digital
      – either digitised or digital by origin. By a digital resource we understand
      a resource that exists in binary numeric form, as in digital audio or digital
      images, videos or software.

Main findings

          The examination of the different stages of the innovation process has
      been of central importance throughout this study. Particularly important is
      the role of different stakeholders and how they have used various kinds of
      knowledge during the five phases of innovation: initiation, implementation,
      scale-up, monitoring and evaluation. These phases need to be understood as
      a cycle with knowledge playing a central role feeding into all stages and also
      informing future scale-ups and innovations.
          Table 2.1 highlights the kinds of issues that arise when looking at the
      context, output and role of stakeholders during the different stages of the
      innovation process. The questions in the boxes guided the exploration of the
      cases of innovation investigated in this study.
          Looking at the five phases of innovation, the following main findings
      can be reported:
           The initiation phase can be looked at in terms of the stakeholders that
      initiated the innovation, for example, whether it was driven by governments
      or by government agencies, local or regional authorities, commercial play-
      ers or users. When looking at target groups for the innovations and funding
      models used to foster DLR, no salient pattern arises. With a few exceptions it
      is clear that the use of academic research has so far been very limited regard-
      less of who initiated the innovation. This is also the case of the involvement
      of stakeholders in the initiation phase. Almost all innovations in the study are
      initiated on the basis of “build it and they will come”.
          The implementation phase regarding DLR innovations is somewhat dif-
      ferent compared to innovations in other fields of education, including VET.
      The DLR cases in this study cover a range of resources, from new websites
      built by a small group of teachers and government initiated campaigns, to


                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                              2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION? – 49



     Table 2.1. The process of innovation related to context, output and stakeholders

                                                          Process of innovation

                                                                                                  Monitoring and
                             Initiation          Implementation              Scale-up              evaluation

Context                Who initiated the     Who funded the             Who is running          Does the funding
                       innovation and with   implementation?            and scaling-up the      mechanism affect the
     Funding
                       what kind of funding?                            innovation and with     model of monitoring
                                                                        what kind of funding?   and evaluation?

Context                Who initiated the       Are there different      Is it easier to         Does the monitoring
                       innovation and          knowledge bases          scale-up when           and/or evaluation
     Target groups
                       towards which target    used by different        targeting particular    depend on the target
                       group?                  target groups?           user groups?            group?

Output                 Who initiated the       Are there different      Is it easier to         Will the monitoring
                       innovation and          knowledge bases          scale-up if the         and evaluation
     Radical or
                       was it radical or       used if the innovation   innovation is radical   look different if
     incremental DLR
                       incremental?            is radical or            or incremental?         the innovation is
                                               incremental?                                     radical compared to
                                                                                                incremental?

Role of stakeholders   Any role for            Any use of               Any role for            Any role for
                       stakeholders in the     stakeholders’            stakeholders in the     stakeholders in the
                       initiation process?     knowledge?               scale-up process?       evaluation process?


         novel ways of organising market offers from companies. In none of the cases
         in this study are organisational issues, e.g. re-organisation of the workflow
         or workload of a large number of people, of high importance. Since there is
         no use of pilots before launching an innovation, incremental developments
         are common.
             The idea of scaling up a digital innovation is rather peculiar: given that
         the host organisation has enough bandwidth and server capacity, any number
         of users can use a digital artefact at the same time. The marginal cost for one
         new user is close to zero. When talking about the production process, as for a
         publishing house, scaling up might also mean enlarging the number of DLR
         they offer.
             Issues related to scale-up looks at funding models associated with the
         sustainability of an innovation. Many have experienced that the ease with
         which one can initiate or start up a project contrasts with the difficulties of
         keeping it going in the long run. Sustainability is a key issue particularly
         for user-generated innovations. Included in this category are development


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
50 – 2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION?

      projects that started with government or EU funding and that turned into
      commercial companies – sometimes intentionally so by the innovators, some-
      times rather unwillingly. A few of the user-generated innovations have found
      ways of capitalising on user-created content (UCC) in order to scale up their
      activities. So far publisher and government-initiated innovations have had
      difficulties in doing the same.
           Monitoring and evaluation are essential stages of the innovation cycle.
      When talking about web-based innovations, two methods are normally used
      for gathering information on user profiles, frequency of use and feedback on it:
          1. Web statistics – an easy way to check the number of downloads or
             users, how long they spend with the DLR, which parts most people
             use, which web pages they spend most time on, etc.
          2. User feedback, usually collected in a non-systematic way.
          Web statistics and user feedback are used by all actors together with dif-
      ferent kinds of monitoring. Publishers and other commercial actors comple-
      ment these methods with market statistics. Overall, formal evaluations are
      rare, except for EU funded projects where they are usually mandatory either
      as part of the project or via an external review by experts.
          Looking at the knowledge base used in the innovation process, it seems
      clear that, in the mid-1990s when most government portals were initiated, the
      knowledge base was weak. Little effort has been made by private or public
      players to strengthen this knowledge base or to make use of existing research
      and knowledge in the innovation process.
          There is a wide range of stakeholders involved in the process of innova-
      tion in education, including students, parents, teachers, researchers, schools,
      local or regional educational authorities, private companies, not-for-profit
      organisations and charitable foundations, public innovation agencies, gov-
      ernment (including state and sub-state agencies), and international organisa-
      tions. The stakeholders have diverse viewpoints and incentives to innovate or
      promote innovation, such as increased effectiveness of teaching and learning,
      opportunities of cutting costs, identification of best practise to improve the
      system and, in the cases of commercial players, the creation of new markets
      and enhancing emerging business opportunities.
           The issue of incentives is related to the reasons why innovations are
      initiated. Most government initiated innovations come as a result of either a
      long-term interest to improve the educational system or an immediate need to
      respond to criticism. Innovations initiated by the private sector are assumed to
      be initiated by reason of profit, which of course does not per se exclude a will
      to improve the system. A second reason might be a need to innovate in order
      to meet the competition coming from other players, although no immediate



                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                            2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION? – 51



       revenue can be expected from the innovation. The motive for individual teach-
       ers or researchers seems to be a mix of a need to improve their working condi-
       tions and an aspiration for professional development and school improvement.

Government-initiated innovations

            Four of the five Nordic countries instigated national educational portals
       in the mid 1990s. In Iceland at first, a private company performed the func-
       tions of a national portal before it was bought by the Ministry of Education
       and turned into a national portal. All the portals have grown incrementally,
       undergoing several stages of development. When the portals were launched,
       they all targeted both students and teachers. Although they have chosen
       somewhat different strategies they all offer similar services (such as the-
       matic DLR and activities for teachers to use, in-service training, links to
       relevant websites, etc.). Web statistics and non-systematic gathering of user
       feedback seem to be the most common knowledge base, although at least
       one (SE1)2 has used a formal evaluation and another (FI1) academic knowl-
       edge on different stages of their development. In addition, informal sharing
       of knowledge and experience among countries seems to have been another
       way of informing their development. Looking at other government initiated
       innovations, the pattern is similar. Only rarely are stakeholders involved and
       only rarely is academic research used before launch or evaluations during the
       implementation and scale-up. This contrasts with a much more systematic
       use of evaluation and research of national programmes and policies of ICT
       in education.
           At the time the national portals were initiated, there were several barriers
       that posed challenges to their development and implementation:
                The knowledge base for this kind of innovations was weak. There was
                not much academic, or other codified professional knowledge to build
                on. This is probably one reason why peer learning among experts
                from ministries and national agencies, for example under the auspices
                of the Nordic Council of Ministers and the European Schoolnet, has
                been so important.
                The involvement of stakeholders seems also to have been weak.
                None of the Nordic countries seem to have had regular meetings with
                groups of teachers, principals, representatives of local authorities,
                educational publishers or researchers before launching their educa-
                tional portals. Again, the lack of existing models and lessons to learn
                from at the time should be kept in mind.
                There is no evidence of a demand from teachers or students in the mid
                or late 1990s to have a national educational platform.



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
52 – 2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION?

           However, there seems to have been one strong driver – a sense of urgency
      among educational decision makers that ICT would change our societies
      dramatically. Consequently the schools needed to change as well. This was
      coupled with the belief that ICT could advance educational reform. As vis-
      ible from the Nordic countries’ national ICT strategies from that time, these
      arguments, together with the ambition to support economic growth by devel-
      oping human capital and promoting social development and enhancing social
      cohesion, were the rationales for justifying investments on ICT in education.
          Closely related to this is whether digital competence is considered a key
      competence for the future or not. Some countries have taken into account
      the European Commission and European Parliament declaration of digital
      competence as a key competence for the future (EC, 2006) and this in itself
      often functions as a driver. Among the Nordic countries this is most explicitly
      used by Norway which has adopted digital competence as a basic skill in the
      curriculum, integrated in all subjects. Denmark, Finland and Iceland have, to
      various degrees, implemented policies to the same effect. In Sweden it is still
      being discussed how and to what extent this should be done.
          A few years after the launch of the national portals, and with the burst of
      the ICT bubble around the turn of the millennium, political interest was less
      evident in some countries. In the absence of political leadership, one driving
      force behind the continuous development and implementation work seem to
      be senior officials, “intrapreneurs” 3, on the technical side within ministries or
      government agencies.
          In other countries, like Denmark and Norway, the political interest for
      ICT in education has remained strong, resulting in new government initia-
      tives like ITMF, The Virtual Gymnasium and ITIF in Denmark (Dalsgaard,
      2008) and the programme for digital literacy with the inclusion of digital
      competence as a core competence in all subjects in Norway (Erstad et al.,
      2008).
           One more potentially important factor for facilitating the development of
      DLR is to exploit the concept of a national digital commons,4 the opportunity
      for individuals and companies to share publicly funded digital resources for
      non-commercial purposes for free.

Innovations initiated by commercial actors

          Looking at commercially initiated DLR it should be noted that most
      Nordic publishers lack confidence that there is a viable market for DLR. They
      experience the market as incipient and there is an economic risk involved that
      should not be underestimated. But, at the same time, it should also be pointed
      out that there are examples in the Nordic countries of companies making a



                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                            2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION? – 53



       living out of producing and selling DLR. It could be said that publishers, who
       for many years have profited from selling textbooks to schools, have a social
       responsibility to help develop a country’s digital competence. Governments
       may look at publishers and textbooks in different ways but, to stimulate inno-
       vation in the education sector, they should create frameworks to encourage
       publishers to respond to the concept of digital competence.
            Publishers often rightly state that teachers appreciate materials that
       help them to implement the curriculum. But this does not mean that teach-
       ers want textbooks or textbook-related materials only. They also want easy
       access to pre-sorted information, to modules they can process and apply in
       ways that fit their own needs and ambitions, to flexible testing tools, practi-
       cal tips, examples of good practice, and to communities with other teachers.
       Publishers could provide a new and different range of services and thereby
       remain as crucial in the education market as they have traditionally been.
           Looking at drivers and barriers the most important driver is, of course, an
       effective demand from schools (i.e. that schools are actually prepared to buy
       DLR at market price). In the absence of an effective demand – caused either
       by a lack of resources in schools or by lack of interest – it is debatable to what
       extent publishers can be expected to create a demand and thus a new market.
       Would it be in their long-term interest to do so? It can be assumed that pub-
       lishers might have a medium or long-term interest in introducing DLR, given
       that the demand is slowly increasing and existing textbooks gradually are
       getting out of date.
            A central driver for innovations on an incipient market seems to be gov-
       ernment seed money and public tenders to publishers. Seed money lowers
       the threshold for publishers to innovate by reducing the commercial risk they
       are taking.
           Furthermore, a key driver is to provide schools and teachers with infor-
       mation about available DLR. The Danish repository Materialeplatformen
       and the Norwegian DigLib are examples of this. Such repositories could be
       complemented with ways to facilitate the evaluation of DLR for teachers by
       providing user-feedback and number of downloads.
           If DLR are cannibalising an already profitable textbook market then this
       can also act as an important barrier for publishers with commercially success-
       ful printed textbooks. The difficulty of localising DLR is another possible
       barrier, although it could also act as a driver for local publishers.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
54 – 2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION?

Bottom-up innovations

         The user-generated innovations presented in the study are all classic
      examples of a small group of enthusiastic and skilled teachers or researchers,
      working hard to make their idea successful.
           Although several of them have turned into at least partially commercial
      companies (e.g. School Web [IC3] and Katla Web [IC5], Peda.net [FI2],
      Lektion.se [SE4]), this seems not to have been the driving force behind the
      innovation. At least some of the innovators stated during interviews that they
      would have been more comfortable to continuing their innovation with public
      funding. They did not look upon themselves as business entrepreneurs. Still,
      the fact that they succeeded in transforming their initiatives into businesses
      might become a driver for others. Thus a barrier to user-generated innova-
      tions would be created if the education system was not prepared to support
      or accept such a transformation for financial or other reasons. Education sys-
      tems with publicly funded clearing houses, rubberstamping teaching materi-
      als for schools, might be less flexible in this matter.
          From the policy point of view, questions of interest relate to what can be
      done to promote, nurse and nourish user-generated innovations. There seem
      to be a number of drivers that can be used in governmental strategies, such
      as:
              Provide funding for development projects. The drawback of project
              funding is that a lot of projects happen only because the funding is
              available, not because there is genuine demand for them. An alterna-
              tive strategy could be to cluster funding offers like the European
              Commission sometimes does.
              Provide seed money, i.e. small amounts to develop a project idea,
              write a proposal and pitch this to existing funding agencies.
              Provide transition funding, to help keep innovations afloat once the
              initial project funding has ended but while people still need time and
              resources to experiment with different business models. An example
              of this kind of funding is what the European Commission used to call
              Accompanying Measures.
              Promote or develop national or international platforms for sharing
              results and findings partners. The French organisation PrimTICE,
              which has been set up to enable the identification, description, index-
              ing and pooling of ICTE uses in primary education, is one example.
              The EU-funded project eTwinning is another example of a service for
              partner finding.




                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                            2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION? – 55



                Foster and encourage research and evaluation projects so that govern-
                ments and government agencies as well as development projects or inno-
                vative business people can learn from others’ successes and mistakes.
           Moreover, the opportunity for innovators and entrepreneurs to launch
       disruptive innovations (Christensen and Horn, 2008) could be a driver of a
       slightly different kind. Opportunities to initiate disruptive innovations occur
       when established actors (in this case governments, government agencies and
       publishers) fail to see that there is a “market” for a different kind of DLR – a
       kind no one is offering at the moment. At least Lektion.se (SE4) and School
       Web (IC3) seem to be examples of disruptive innovations, offering products
       and services of a new and simpler kind than publishers or government agen-
       cies. Both present teacher initiated materials – often not as sophisticated or
       well designed as materials from publishers. The School Web offers about
       30% of its DLR for free to anyone, not only subscribers. The business model
       of Lektion.se also builds on revenues from advertisements instead of sales to
       teachers or schools. A similar Swedish case is Skolporten.com, a company
       offering not DLR but school related information and news for free on their
       website and through weekly newsletters. According to official statistics, from
       an independent statistical company, Skolporten.com has some 80 000 sub-
       scribers mostly from the school sector.5 Subscription to the newsletter is free
       and the business model is similar to Lektion.se – school-related advertise-
       ments complemented by other activities, such as organisation of conferences.
       In terms of establishing themselves as players with impact on a systemic
       level, these three examples are truly successful.
            A number of barriers to bottom-up innovations has been identified in
       this study:
                A possible barrier to bottom-up innovations is the unwillingness of
                teachers, schools, local or national educational authorities to accept
                and use bottom-up innovations, e.g. innovations lacking a qual-
                ity assurance from the government or a government agency. Such
                unwillingness has not been detected in the Nordic countries. There
                are examples of bottom-up innovations from all five countries which
                play an important role in respective countries.
                One existing and harmful barrier seems to be a lack of overview of
                developments and mechanisms to help build synergies between them. A
                quote from the Finnish Country Case Report illustrates this point. The
                team of experts conclude that “[i]nnovation is certainly evident but is
                characterised by small, local projects and initiatives some of which fail
                to see the value in sharing their results via the available national portal.
                In a decentralised education system, better co-ordination is needed to
                enable cross-fertilisation and “mash-ups” of innovations (increasingly
                necessary in a Web 2.0 world)” (Country Case Report: Finland).


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
56 – 2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION?

              Low use of existing DLR, or low interest in new ICT developments
              by teachers, could be expected to be another barrier to innovation.
              Although most of the cases investigated in this study were developed
              without much previous demand from teachers or students, lack of
              demand would surely be a hurdle to innovation in the long run.
           As already noted, the Empirica study (2006) investigates barriers to the
      use of ICT in terms of lack of access to computers and the Internet, lack of
      adequate content and lack of motivation. According to Empirica (2006),
      Denmark scores highest among the Nordic countries regarding the propen-
      sity among its teachers to take up ICT in their teaching. This study looks at
      three components affecting the use of ICT in teaching – access, competence
      and motivation. When looked at individually, the largest differences among
      the countries is found in motivation. Teachers in Iceland and Sweden, and to
      some extent in Finland, are much less motivated to use ICT than Danish and
      Norwegian teachers as well as European teachers in general (see Table 2.2).
      Reasons for this lack of motivation are not known but, irrespective of them,
      this deficiency can be expected to affect the use of DLR.
         One might speculate if there are virtuous and vicious circles in play – in
      countries where governments have shown a long-term interest in promoting

           Table 2.2. Teachers’ access, competence, and motivation to use ICT

                                Accessa             Competenceb             Motivationc
          Denmark                 71.3                  93.3                   70.9
          Finland                63.3                   84.9                   57.8
          Iceland                58.8                   88.2                   29.4
          Norway                  68.1                  90.9                   72.8
          Sweden                  67.9                  93.3                   41.4
          EU 25+2                60.7                   82.0                   68.4

          a. The higher the value, the greater the percentage of teachers who agree with the
             statement that their school is well-equipped.

          b. The higher the value the greater percentage of teachers feels themselves skilled
          in using ICT.

          c. The higher the value the greater percentage of teachers are motivated to use
             ICT.
          Source: Benchmark Access and Use of ICT in European Schools 2006, Empirica
          (2006).




                                                 INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                            2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION? – 57



       the use of ICT in education (e.g. in terms of government policies and pro-
       grammes, and participation in international studies in ICT), there seems to be
       a high interest or motivation in using ICT in general and DLR in particular on
       the part of teachers. It could also be expected that there is a growing demand
       from teachers for continuous political support and for more – and better –
       DLR. Hence, a virtuous circle is created. On the other hand, in countries
       with weak political interest which might manifests itself in terms of unclear
       policies and few programmes, teachers might be expected to have less com-
       petence and less motivation to use DLR. The vicious circle means that the
       demand for new ICT policies and programmes, as well as for DLR, would be
       weaker than in other countries. The recommendations look at ways to break
       such vicious circles.

Looking at the future of DLR

           In the information society it is important that people can use ICT and
       digital media in working life as well as in their role as citizens and during
       leisure time. Technological development creates new opportunities for learn-
       ing, both inside and outside of schools. Young people need to be digitally
       competent and most often it is expected that the school will furnish young
       people with the skills needed. To do this, schools need to use and work with
       different kind of digital tools, not least in the form of DLR.
           Furthermore, what used to be a rather stable setting with fixed roles
       – educational policy makers setting the scene for learning through curricula;
       educational publishers developing the learning materials building on the cur-
       ricula; and schools implementing the curricula issued by policy makers and
       using the textbooks produced by publishers – is now changing. New actors
       such as media companies, broadcasters, computer game developers, interna-
       tional publishing houses, and software developers are moving in. Teachers
       are producing and sharing DLR on an unforeseen level. Students are using
       DLR and digital tools they find for free on the Internet both during and after
       school hours, often challenging what the teacher and the school offers. At the
       same time, new digital divides are emerging, this time dividing those who
       can master the flow of information, sift, digest and use it, and those who are
       unable to protect their integrity on the Internet and get lost in the new digital
       landscape. Education policy makers need to respond to these challenges.
           The annual Horizon Report (2009) describes “the personal web” as one
       of the strong trends in higher education within the next two to three years.
       The personal web means that “computer users are assembling collections
       of tools, widgets, and services that make it easy to develop and organise
       dynamic online content. Armed with tools for tagging, aggregating, updat-
       ing, and keeping track of content, today’s learners create and navigate a web



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
58 – 2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION?

      that is increasingly tailored to their own needs and interests”. In compulsory
      schooling this trend is probably more related to teachers. But it is clearly
      challenging the way teachers, learners and publishers are working today. So
      far, this report has described how governments, publishers and groups of
      teachers and researchers are producing DLR at the moment. But the changing
      landscape makes new scenarios for the production and use of DLR possible.
      In such scenarios, new models of production, new business models and new
      ways of distributing and using DLR should be taken into account. Below are
      five embryonic scenarios on novel ways of producing, distributing and use
      DLR.
           The first builds on the Norwegian initiative NDLA which describes an
      interesting case of how teachers are more closely involved in production. A
      number of regional educational authorities has teamed up and decided to
      produce some DLR on their own instead of spending all their money on DLR
      produced by publishers. They ask some of their teachers to do the authoring,
      with the same kind of salary as before. Since the teachers are producing DLR
      on behalf of their employer, using the tools of the school, all the intellectual
      rights to the materials belong to the local educational authority. These mate-
      rials are mixed with professionally produced ones, bought by the authorities
      from publishers and media companies. All materials are published in digital
      format. The authorities have decided not only to share the materials among
      themselves but to publish all materials using Creative Commons licenses,
      which means that other teachers cannot only use the materials in their teach-
      ing but also adapt and reuse them. This is in many ways challenging the role
      of publishers in the educational market.
           The second scenario is intended for local educational authorities. They
      could ask a teacher or a consultant to gather Open Educational Resources,
      i.e. materials already free for schools and materials they have the right to
      use in schools (usually because a Creative Commons license is used). The
      focus in this case is on gathering existing materials, not on their production.
      The work would be to compile materials to fit the local needs of the schools.
      Since the materials are open, local educational authorities could share these
      materials among each other, given that they also spend resources on tagging
      the DLR with metadata making it possible to search for them and find them
      on the Internet.
          The third embryonic scenario is directed towards publishers who need
      to respond to these challenges. One way of doing this could be to disag-
      gregate content and offer smaller chunks of learning materials rather than
      fully fledged productions. Individual teachers, schools or local educational
      authorities could then subscribe to the repository and authoring tools and use
      these learning objects as they choose. On top of offering the content, publish-
      ers could provide the service of putting it together in a way that fits the local



                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                            2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION? – 59



       needs. This is similar to the OER model described above, but it would have
       an extra quality stamp both on the content and on the compilation process.
       The important thing is that again the one-size-fits-all model is abandoned.
            The fourth scenario is also intended for publishers. They could work in
       close co-operation with one or several local educational authorities and the
       local teachers, taking much the same role as NDLA in the Norwegian case.
       The role of the publisher would be to offer some of the content, to lead the
       compilation process putting its knowledge and quality stamp on the materi-
       als. The business model is that schools or local authorities would pay for the
       service as well as the content from the publisher.
           The final scenario focuses on teachers. Teachers could very well work
       without both local educational authorities and publishers. The Internet opens
       up new opportunities for teacher associations or similar organisations to
       play a role related to educational materials. One example could be a national
       association of teachers in mathematics or science starting an OER commu-
       nity and repository, inspiring and promoting teachers to develop and share
       resources among each other. The success of Lektion.se (SE4) is an example
       of the possible success of a teacher initiated and driven community. And, as
       described by the Horizon Report (2009), technological developments make it
       increasingly easy to find, sift and keep track of content.
           It is hoped that the ideas presented in this chapter, as well as the findings
       and analysis of this report, will help to move forward the research agenda on
       the use and effects of DLR and ICT on learning, given the growing impor-
       tance of new technologies and digital media in modern societies.

Conclusions and policy implications

            There are two kinds of conclusions and recommendations emerging from
       the report. The first relates to the production and use of DLR, and the second
       to the more general issue of systemic innovation in education.
           Governments can take different roles in innovation, from creating favour-
       able conditions to fostering or being leaders of innovation. They often take
       several roles at the same time, depending on their needs and the political
       interest in promoting innovation in the area in question. The policy recom-
       mendations cover all of these potential roles.
          In order to create enabling conditions for innovation in the area of DLR,
       governments could:
              establish a coherent vision on digital competence;
              make publicly funded information freely available for commercial and
              other use;


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
60 – 2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION?

            join up innovation initiatives making researchers and entrepreneurs
            visible;
            establish a forum for dialogue between innovators and stakeholders;
            and
            support the building up of a formal knowledge base for DLR develop-
            ment.
           Furthermore, it is recommended that governments federate existing
      educational portals to provide support services of different kinds in order
      to facilitate access and use of DLR, both commercial and non-commercial,
      and to promote DLR design and use by teacher training institutions both for
      initial and in-service training. Local authorities are recommended to increase
      teacher awareness on the existence of Open Educational Resources and to
      invest in training on fair use for teachers and school managers as well as to
      value the use of DLR for teacher professional development.
          To foster innovation governments are recommended to supplement seed
      money with development funds and transition funding for development pro-
      jects; and to promote co-operation between public and private players for
      DLR development.
          To be leaders of innovation governments are recommended to consider
      the relative circumstances of their country when deciding about initiating
      new DLR or instead supporting initiatives of others. In the case of smaller
      countries for example it may be preferable to identify DLR at European level
      and to focus more on localisation. They also need to rethink their role in rela-
      tion to communities using more of an “engagement” and less of a “delivery”
      strategy.
           One conclusion related to the discussion of innovation in education con-
      sists in recognising that technology makes the conditions for DLR innovation
      different from many other fields of education. It seems clear that successful
      ICT-based innovations spread fast and that small, user-generated innovation
      may have a systemic impact. Moreover it is difficult to plan for scaling-up
      ICT-based innovations since the end-users decide the success of innovations.




                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                            2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION? – 61




                                                 Notes

1.     “Countries” are not necessarily to be equated with “governments”. In this field in
       particular, a range of significant agents and institutions are likely to be involved,
       with much of the impetus coming from the bottom up.
2.     A list of the case studies can be found in Appendix 2.A.
3.     Wiktionary defines ”intrapreneurship” as the practice of applying entrepreneurial
       skills and approaches within an established company (http://en.wiktionary.org/
       wiki/intrapreneurship).
4.     The Nordic countries have long traditions in the use of common land. It may be
       common usage of natural resources in forests and on mountains, such as hunting,
       fishing and berry picking. The idea of digital commons, which originated in a
       Norwegian governmental white paper, is to build on the sharing culture and reuse
       of resources on the Internet. The digital commons should be as large as possible
       and contain information and material of high quality. It should focus on the needs
       of the users. It should be free for the individual user and permit reuse of digital
       resources for non-commercial purposes.
5.     See http://ts.se/Public/CirculationNumbers/EmailCertificateList.aspx.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
62 – 2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION?




                                     References

      Christensen C.M. and M.B. Horn (2008), “How do we transform our schools?”
         Education Next, Summer 2008, pp. 13-19, Hoover Institution, Stanford
         University, Palo Alto.
      Empirica (2006), “Benchmarking access and use of ICT in European
        schools 2006”, final report from Head Teacher and Classroom Teacher
        Surveys in 27 European Countries, August 2006, Bonn.
      European Commission (2006), “Recommendation of the European
         Parliament and of the Council of 18 December 2006 on key competences
         for lifelong learning” (2006/962/EC), Official Journal of the European
         Union, 30 December 2006.
      Johnson, L., A. Levine and R. Smith (2009), The 2009 Horizon Report, The
         New Media Consortium, Austin, TX.
      OECD (2007), Giving Knowledge for Free: The Emergence of Open
        Educational Resources, OECD Publishing.
      OECD (2009), Working Out Change: Systemic Innovation in Vocational
        Education and Training, Educational Research and Innovation, OECD
        Publishing.




                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                               2. CAN DIGITAL LEARNING RESOURCES SPUR INNOVATION? – 63




                                              Appendix 2.A

                           Cases studied in the DLR project

Country                  Name                                          Category                          Designation
            EMU                               The national educational portal                              DK 1
            Subscription to DLRs              Publishers selling packages of DLRs to schools               DK 2
Denmark
            ITIF (ICT in the public school)   Government programme with, among other things,               DK 3
                                              resources for private companies to produce DLRs
            Virtual School including EDU.fi The national educational portal                                FI 1
            Peda.net                          Research and development project providing schools           FI 2
                                              with DLRs
Finland
            Areena                            The digital extension of YLE´s televised production          FI 3
            Abitreenit                        Practice material for students preparing themselves for      FI 4
                                              the matriculation examination produced by YLE
            The Educational Gateway           The national educational portal                              IC 1
            The National Centre for           National agency developing and translating educational       IC 2
            Educational Materials (NCEM)      materials which are sold to schools.
            The School Web                    Private company developing and selling DLRs to schools       IC 3
            The Language Studio               Support and materials for distance teaching of Nordic        IC 4
Iceland                                       languages, supported by the city of Reykjavik
            The Katla Web                     Support and materials for teaching Icelandic as a second     IC 5
                                              language. School subscriptions
            IceKids                           Provide Icelandic families living abroad with learning       IC 6
                                              resources for studying their mother tongue. Family
                                              subscription
Norway      Utdanning.no                      The national educational portal                              NO 1
            Aschehough                        Publishing house with a web portal called Lokus.no           NO 2
            You Decide                        Government initiated campaign on the subject of data         NO 3
                                              protection
            IT for Teachers                   The national educational portal                              SE 1
            The Course Hub                    Government initiated DLRs repository for teachers            SE 2
            UR and the Media Bank             Radio and TV clips from the education broadcasting           SE 3
Sweden
                                              company
            Lektion.se                        Teacher created website and community for teachers           SE 4
                                              exchange of lesson plans




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
       PART II. HOW TECHNOLOGY-BASED INNOVATIONS ARE MONITORED, ASSESSED AND SCALED UP – 65




                                                Part II

   How technology-based innovations are monitored, assessed
                       and scaled up




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
             3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA – 67




                                             Chapter 3

       Monitoring and assessing the use of ICT in education:
                      The case of Australia


                                      John Ainley
                       Australian Council for Educational Research




    This chapter documents how Australia has had an increasingly complex per-
    spective on the broad issue of monitoring and assessing the use of technology in
    education. In particular, the Australian experience documents how to monitor
    technology use in schools in the context of a complex governance system, as a
    true recognition of the variance in scope and depth that technology-based innova-
    tions have across schools and territories. In addition, the chapter elaborates how
    Australia is addressing the need for substantial progress in the collection of evi-
    dence concerning how young people become equipped with digital literacy skills
    and, in a broader sense, with 21st century skills.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
68 – 3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA

Context

      Population characteristics
           Australia has a population of 21 million in an area of 8 million square kil-
      ometres. Although the overall population density is low, it is a highly urban-
      ised society. Outside the cities, the country is sparsely populated; 30% of
      primary schools have fewer than 100 students and 30% of secondary schools
      have fewer than 500 students. Australia is classified as a high-income coun-
      try: literacy among adults is nearly universal, nearly half of the population
      has completed secondary school, and 32% hold a university qualification.
      Although the Australian population is mainly of European background,
      immigration has produced greater ethnic and cultural diversity. One-fifth
      of the population (22%) was born overseas and a similar percentage (21%)
      speaks a language other than English at home. About 4% of Australian school
      students are indigenous, and some of them live in isolated communities. In
      recent years, Australian people have embraced the use of information and
      communication technologies (ICT). According to recent surveys conducted
      by the Australian Bureau of Statistics, more than three quarters (78%) of
      Australian households have access to a computer at home (up from 48% in
      1998), 72% have home Internet access (up from 16% in 1998), and 62% have
      broadband Internet access (ABS, 2009).

      Education systems
           Australia does not have a single national education system. Eight juris-
      dictions (states and territories) are each responsible for their own educational
      administrations although the overall structures are similar. Collaboration on
      matters of policy takes place in a council of ministers of education, and the
      role of the federal government has increased during the past two decades.
      Over the past two decades there has been a shift towards a greater role for
      national approaches, and this is reflected in the support for ICT in education.
      The Australian Information and Communications Technology in Education
      Committee (AICTEC): a national, cross-sectoral committee responsible for
      providing advice to all Australian Ministers of Education and Training on
      the utilisation of ICT in education and training (MCEETYA, 2008). It has
      recently enlarged its responsibilities to include advising on implementation
      of the Digital Education Revolution (DER).
           In addition to the fact that states and territories are responsible for the
      provision of school education one-third of school students are enrolled in non-
      government schools. In 2009, non-government schools enrolled 34% of the
      students (31% of primary and 39% of secondary school students), a propor-
      tion that has risen steadily since 1970 (ABS, 2010). The uptake of computers
      in teaching during the 1990s was very strong in independent non-government


                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
             3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA – 69



       schools, some of which introduced programmes in which each student had
       their own laptop computer (Shears, 1995).

       ICT use at home and school
           In general Australian students are frequent users of computer technology.
       According to data from PISA, 77% of 15-year-olds in 2006 used a computer at
       home “almost every day”, and 24% used a computer at school “almost every
       day” (Anderson and Ainley, 2010). These data indicate that Australia is one of a
       cluster of seven OECD countries with significantly high levels of student com-
       puter use at school (16% or more using computers at school daily): Australia,
       Austria, Canada, Denmark, the Netherlands, New Zealand, and Norway.
           National surveys conducted as part of the national assessment programme
       indicate that nearly half of the students in Grade 10 (48%) used a computer
       at home at least once every day and a further 25% used a computer at home
       almost every day (MCEECDYA, 2010). The corresponding figures for
       Grade 6 students were 30% and 25%. School use of computers is less frequent.
       Ten per cent of Grade 10 students used a computer at school at least once per
       day with a further 22% using a computer at school almost every day. The corre-
       sponding figures for Grade 6 students were 8% and 13% (MCEECDYA, 2010).
           Australian schools are well provided with computer technology. Data from
       PISA conducted in 2006 indicate that Australia has one of the highest levels
       of computer availability in secondary schools among the OECD countries,
       with an average of 2.9 students per computer (OECD, 2007). This ratio had
       3.3 students in 2003 and 4.5 students per computer in 2000 and indicates a
       substantial improvement in school computing resources in Australia over
       six years. Other countries that have ratios of three to four students per com-
       puter are Austria, Canada, Hong Kong, Iceland, Japan, Korea, Luxembourg,
       New Zealand, Norway, Sweden, United Kingdom and United States (Anderson
       and Ainley, 2010). Findings from the Trends in International Mathematics and
       Science Study (TIMSS) suggest that 78% of Grade 4 students also had high
       levels of access to computer technology in their classrooms. This is behind
       only Denmark (91%), New Zealand (89%), Scotland (89%) and Japan (84%)
       and is similar to Singapore (80%), England (77%), Sweden (77%) and the
       United States (77%) (Martin, Mullis and Foy, 2008).

Building interest in the educational use of ICT: 1990 to 2000

           There were several initiatives at state level designed to promote the uptake
       of ICT in schools. The following initiatives were introduced in Victoria.
           A Navigator Schools Project highlighted ICT practices and served as
       a focus for professional development and visits from neighbouring schools.


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
70 – 3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA

      A two-year study of the navigator schools project suggested that the intro-
      duction of ICT to those schools had effects on staff and students. Teachers
      changed in their behaviour and outlook when ICT was introduced with an
      accompanying reform and collaborated in using the technology to refor-
      mulate their approaches to teaching and curriculum (Clarkson, Dunbar and
      Toomey, 1999).
          A Notebooks for Teachers and Principals Program which supported
      principals and teachers to integrate learning technologies into school
      classroom and administrative practices. In return for leasing a notebook at
      subsidised rates teachers were expected to demonstrate a commitment to
      ongoing professional development in the use of learning technologies. An
      evaluation indicated that by June 2002, 91% of all teachers and principals had
      a notebook computer and 85% of those participants reported routine use of
      computers at school (McDougall, Nicholson and Marshall, 2001). A similar
      programme was introduced in other states.
          One of the ventures that featured in the 1990s was the introduction of
      programmes in which each child had a notebook computer. These received
      greatest attention in independent non-government schools that introduced
      them on a school-wide basis (Shears, 1995), but similar programmes were
      also tried in selected government schools (Rowe, 1993) and selected classes
      in government schools (Ainley et al., 2000).

Educational goals and plans for ICT in education

           The importance of ICT in education was given formal recognition in
      the emphasised in the Adelaide Declaration of Australia’s National Goals
      for Schooling which stated that when students left school they should be:
      “confident, creative and productive users of new technologies, particularly
      information and communication technologies, and understand the impact of
      those technologies on society” (MCEETYA, 1999). This theme continued in
      the later Melbourne Declaration on Educational Goals for Young Australians
      which was released in December 2008 (MCEETYA, 2008). The Melbourne
      Declaration asserted that “in this digital age young people need to be highly
      skilled in the use of ICT”.
         In 2000, the MCEETYA adopted a school education action plan titled
      Learning in an Online World (MCEETYA, 2000), which was updated as
      Contemporary Learning: Learning in an Online World (MCEETYA, 2005a).
       Overall, the plan established areas in which strategies were to be imple-
    mented by:
               developing teacher competence in using learning technologies effec-
               tively;


                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
             3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA – 71



                 implementing an advanced ICT infrastructure for education;
                 developing online resources for curriculum, teaching, and adminis-
                 tration;
                 facilitating the uptake and use of ICT in schools; and
                 establishing a framework to support the use of ICT to enhance learning.

Supporting the use of ICT in education

       Networked resources and information gateways
           Education Network Australia (EdNA) was established (by Education.
       au) to enable Australian educational institutions (schools, universities and
       technical institutes) to adopt new ICT services and to disseminate and pro-
       duce content and services relevant to the Australian experience. The EdNA
       Directory Service provided free access to quality education resources on the
       Internet for all sectors of Australian education. EdNA operates an informa-
       tion gateway—EdNA Online (www.edna.edu.au) – which provided access to
       resources, online networks, the EdNA sandpit, personal learning space, web
       conferencing, and access to education news through bulletin boards and RSS
       feeds.
           The intention behind Education Network Australia (EdNA) was to enable
       Australian educational institutions to adopt new information and communica-
       tion services and technologies, and to disseminate and produce content and
       services relevant to the Australian experience. The EdNA Directory Service
       provided free access to quality education resources on the Internet, for all
       sectors of Australian education. Effects of geographic isolation are alleviated
       through this initiative, with the opportunity it affords for collaborative access
       to national and international curriculum materials.
           Education.au supported collaborative projects such as the OzProjects
       website (which supported teachers in joining projects, creating their own
       projects, and choosing appropriate online tools for students), the International
       Learning Quest Challenge (which provided opportunities for teachers to inte-
       grate use of Internet into existing curriculum programmes), and the Netd@
       ys International Online Project (which promoted student use of new media in
       the areas of youth and culture).
           Most state authorities have established networks that link schools and
       education agencies. Through these networks, teachers have access to online
       resources provided by the state as well as resources provided through
       EdNA. In Victoria, networking was a central priority, with all schools
       connected to a Wide Area Network and a range of services including the
       Internet. A digital resource centre was established as a means of delivering


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
72 – 3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA

      multimedia curriculum resources. A Curriculum Development and Learning
      Technologies division developed materials and programmes to support
      state curriculum frameworks. These are accessible through the Education
      Channels and the Department of Education’s website. A website (SOFWeb)
      could be accessed across the state, nationally or internationally via Internet.

      Digital resources
          A joint venture called The Le@rning Federation, was developed as a
      major digital content project for Australian and New Zealand schools. The
      Le@rning Federation developed learning objects for schools as well as learn-
      ing and content management systems. Some initiatives involved the develop-
      ment of content to meet the curriculum, professional development, and other
      educational priorities of education systems.
          The Le@rning Federation, which was jointly managed by the Curriculum
      Corporation and education.au, began as a major digital content project for
      Australian and New Zealand schools. The Le@rning Federation developed
      specifications for educational soundness and new delivery systems such
      as web portals, learning management systems, and content management
      systems. A number of schools implemented major software packages to
      support these functions. The Le@rning Federation also developed a “Basic
      E-Learning Tool Set” to provide schools with the basic functionality for man-
      aging learning objectives, until comprehensive learning content management
      systems could be implemented within jurisdictions. State and territory educa-
      tion authorities also operated various initiatives for providing their schools
      with digital content. Some initiatives endeavoured to identify existing content
      and provide cost-effective access for schools. Others involved the develop-
      ment of content to meet the curriculum, professional development, and other
      educational priorities of education systems. Some schools established pro-
      grammes to support the development of new content by their own teachers.

      ICT competencies of teachers and leaders
          Data collected by state educational authorities suggest that around 90%
      of Australian teachers had at least basic competencies in ICT, and 50%
      of Australian teachers regarded their competency as “intermediate” or
      “advanced.” In terms of the self-reported competencies of secondary school
      mathematics and science teachers Australian teachers compare favourably
      with other countries (Ainley, Eveleigh and O’Malley, 2009). Three quar-
      ters (76%) were confident that they could “produce a letter using a word-
      processing program”, 58% were confident that they knew “which teaching/
      learning situations are suitable for ICT use” and 57% were confident that
      they “could find useful curriculum resources on the Internet”. However



                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
             3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA – 73



       only 46% were confident that they could “produce presentations with simple
       animation functions”, 42% were confident that they could “use ICT for moni-
       toring students’ progress and evaluating learning outcomes” and 37% were
       confident that they could “share knowledge and experiences with others in
       a discussion forum/user group on the Internet”. Australian secondary school
       mathematics and science teachers expressed relatively high levels of confi-
       dence compared to their peers in other countries. Year 8 science teachers in
       Australia were not significantly different in ICT confidence from those in
       Singapore, Hong Kong, Alberta, Ontario, Chile, and Norway and were sig-
       nificantly more confident than other countries in the SITES survey. Year 8
       mathematics teachers in Australia were not significantly different from those
       in Hong Kong, Singapore, Ontario, Alberta, Denmark, Chile and Norway
       but were significantly more confident than those in other countries (Ainley,
       Eveleigh and O’Malley, 2009). There was a positive association between there
       is a positive association between teacher confidence in using ICT and actual
       use of ICT in all countries (the median correlation coefficient was 0.3).
           A national review of teaching and teacher education published in 2003
       (Lee Dow, 2003) argued that ICT should be used widely in schools, and form
       part of the repertoire of all teachers. It recommended that teacher education
       programmes prepare prospective teachers to use ICT as a knowledge man-
       agement tool, and to support student learning. The review also argued that
       opportunities should be created for teachers to upgrade their ICT expertise.
       Australian school systems operate professional learning programmes to
       extend the ICT competences of teachers. Some school systems use in-school
       professional learning models based on mentors. Others have developed vari-
       ous forms of “centres of excellence,” that is, lighthouse or navigator schools,
       which serve as models of good practice and providers of professional devel-
       opment for teachers in surrounding schools. Some school systems reached
       agreement with local universities to ensure that pre-service ICT course
       content aligns with the needs and actual situations new teachers encounter
       in schools.
            A review conducted by Downes and colleagues (2001) of a national pro-
       ject offering models of teacher professional development designed to facilitate
       integration of ICT into classroom practice identified barriers and critical
       success factors, and provided advice and recommendations for programmes.
       The reviewers identified the principal barriers as funding, time, and a lack
       of linkages. They saw lack of funding as limiting the scope and form of
       teacher development, and identified time as one of the greatest challenges to
       effective professional development. They also pointed to a lack of connec-
       tion and linkage between pre-service and in-service teacher education, in
       areas associated with ICT. Downes and her co-reviewers argued for ongoing
       support for the integration of ICT, and a portal for resources and research by
       extending the existing database website. In particular, they maintained that


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
74 – 3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA

      EdNA Online (the Education Network Australia; see reference below to the
      website) should act as a professional development resource so as to support
      the integration of ICT in teaching and to facilitate connections between the
      professional development site and other EdNA Online-related sites. The
      reviewers argued also for the formation of a support network, linked to the
      online facility and targeted at professional development in relation to the inte-
      gration of ICT in pedagogy. Finally, they emphasised that integration of ICT
      in education required targeting staff responsible for professional development
      programmes, school leaders, in-school ICT coaches, leaders of professional
      associations, and teacher educators.

      Digital education revolution
           The federal government that was elected at the end of 2007 proposed a
      “digital educational revolution” as a centrepiece of its education platform.
      It aims to support change to teaching and learning in Australian schools. It
      will provide for new ICT equipment for all secondary schools with students
      in Years 9 to 12 and deploy high-speed broadband connections (fibre to the
      school) to Australian schools.
           The DER intends to ensure that new and continuing teachers have access
      to training in the use of ICT that enables them to enrich student learning. It
      will require competency in ICT as a graduation requirement and ongoing
      progressive development of the capacity of existing teachers. It will provide
      online curriculum tools and resources that support the national curriculum.
          It plans to enable parents to participate in their child’s education through
      a parent portal and to provide assistance for schools in the deployment of ICT
      (AICTEC, 2008). There is a detailed strategic plan and roadmap to guide its
      implementation.
          The stated goals of the DER include ensuring that students undertake
      stimulating learning activities that are supported by access to global infor-
      mation resources and tools for information processing, communication and
      collaboration. The DER intends that teachers should devise student-centric
      learning programmes of learning based on established curriculum standards
      and supported by contemporary learning resources.

Monitoring and evaluation

      Monitoring implementation
           It is recognised that an evaluation and monitoring plan will be required
      to track progress in implementing the DER and to assess its effectiveness and
      efficiency. It is based on a documents developed by the council of ministers



                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
             3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA – 75



       entitles “Digital education – making change happen” (MCEETYA, 2008).
       This document provides guidance about how the integration of ICT in teach-
       ing, learning and administration can be gauged at the school and system
       level. It consists of ten elements that are each characterised through descrip-
       tors grouped in three levels: the developing school, the accomplished school
       and the leading school. In addition there is a plan to develop an overall evalu-
       ation strategy for the initiative as a whole.
            The ten elements are as follows:
            1. Personalising and extending learning
            2. Enabling leadership
            3. Supporting professional learning
            4. Connecting learning beyond the school
            5. Improving assessment and reporting
            6. Developing, measuring and monitoring digital literacies
            7. Accessing and utilising student information
            8. Providing, accessing and managing teaching and learning resources
            9. Automating business processes
            10. Providing reliable infrastructure

       Monitoring the impact on teaching and learning
           ICT is applied in schools in a wide range of ways that reflect different
       purposes of learning. These include: using computers as an instructional
       delivery system to increase skills and knowledge; using technology as a tool
       for accessing resources, communication, analysis or simulations; using ICT
       to change teaching and learning processes in classrooms or school organisa-
       tion; acquiring ICT knowledge and skills and understanding the role of ICT
       in society; developing ICT skills and knowledge for use in learning, work or
       more general social transactions (Kozma and McGhee, 2003). The need for
       the educational community to consider the impact of ICT on teaching and
       learning is undisputed and a variety of approaches has been adopted to inves-
       tigate this. In Australia there has been a variety of studies that have investi-
       gated the impact of ICT on approaches to learning. These have included case
       studies of innovative practice (some of which use instruments as part of the
       study), quasi-experimental surveys of particular settings and some large-scale
       surveys.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
76 – 3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA

      Qualitative studies of innovative practice
           Qualitative research approaches can provide detail about people, pro-
      grammes and events in a natural context and describe the interactions among
      them. Typically qualitative methods have been based on small numbers of
      cases and with little capacity to generalise findings from those cases to wider
      contexts. The Second Information Technology in Education Study Module 2
      (SITES M2) sought to identify innovative educational practices using ICT
      (Kozma, 2003). The study was based on 174 case study reports drawn from
      28 participating countries, each of which described an innovative use of tech-
      nology to enhance pedagogy. Through mainly qualitative methods, the study
      examined the similarities across cases and across countries to identify pat-
      terns of innovative pedagogical practices. In SITES M2 rigorous procedures
      were enacted in terms of the selection of cases (using national panels or ref-
      erence groups and clearly articulated criteria for selection), common instru-
      mentation (in terms of who was interviewed in each case and the structure of
      the interview schedules), common data collection procedures (in terms of the
      duration and number of visits to each school) and the structure of each case
      report. Cluster analyses techniques were used integrate the findings from the
      large number of case reports.
           In Australia five examples of innovative use of ICT were investigated
      (Ainley, Banks and Fleming, 2002). One of these involved using ICT as
      part of a study of the novel Chinese Cinderella in junior secondary school.
      Students used a wide variety of ICT tools including Access Tool Box,
      Microsoft Front Page, MediaGram and digital cameras to develop personal
      e-portfolios that they stored on the school’s intranet. Another involved using
      Multi-media development tools to foster learning styles in a primary school.
      Curriculum topics were addressed through critical questions that students
      are asked to investigate. Students were organised into learning teams and
      the technologies used demonstrated an evolving sophistication. For example,
      in Grades 1/2, students made use of web sites to conduct research and email
      extensively. By Grades 6/7, students produced iMovies, preparing scripts
      and making and editing videos. The use of peer tutoring in ICT and a “skill
      register” meant that there was a great deal of interactive learning between
      students. A third example was the development of electronic distance educa-
      tion Virtual Schooling Service (VSS) to provide teaching in senior second-
      ary school subjects where the numbers in a school (e.g. in a rural location)
      were not viable for conventional teaching. A Virtual Private Network linked
      government schools utilised use a range of software to provide course
      materials and conduct the learning sessions (NetMeeting Whiteboards,
      PowerPoint, interactive spreadsheets, and WebQuest). A fourth example
      involved an extensive integration of ICT into teaching programmes. Students
      were grouped into four multi-age bands, and the curriculum was organised
      around themes. Teachers worked in teams and students were involved in


                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
             3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA – 77



       collaborative learning. Students share ideas about what they would like to
       learn within the themes and teachers take these ideas to flesh out the learn-
       ing programmes each term, assisted by two ICT experts at the school who
       suggest how ICT might best be incorporated into the proposed programmes.
       Students work in groups but develop electronic portfolios of their own work
       that are the basis of assessment by teachers and used to showcase their work
       to parents. A fifth example used a process of sustained electronic com-
       munication (called the Virtual Bridge) as part of its orientation programme
       for Grade 6 students from three small, remote feeder primary schools. The
       Virtual Bridge relies on a VPN for secure transmissions between students and
       the Grade 7 co-ordinator with the fundamental software being WebQuest.
            Another example of systematic case studies was the Innovation and Best
       Practice Project (IBPP) that focused on innovation in 107 schools (Cuttance,
       2001). Each had developed and implemented an innovation intended to
       improve learning outcomes for students. Schools were asked to research and
       assess the magnitude of the impact of the innovation on learning outcomes
       for different groups of students. Twenty schools provided opportunities for
       students to develop ICT-based skills and knowledge. Most provided evidence
       that the innovations were impacting on learning and learning outcomes
       beyond the standard curriculum outcomes. There was considerable variation
       in the way in which schools introduced ICT into their learning environments.
       Many innovations used computers and associated hardware to enhance the
       learning environment for students. Some used the Internet as a source of
       educational resources although bandwidth was a limitation. One school had
       developed on-line programmes for students to study from home and another
       was producing multi-media learning resources. Five of the schools were
       based on notebook computers used by all students in the group.

       Development of survey instruments
           There have been several ventures that have involved the development
       and application of self-report survey instruments on a limited scale to study
       the impact of ICT on teaching and learning. One is the Classroom Computer
       Climate Survey (CCCS) which was developed by Robertson, Fluck, Webb,
       and Loechel (2004) to gauge ICT usage and practices in Tasmanian schools.
       The CCCS was distributed once a year for the years 2002-04 to teachers
       of Years 3, 5 and 7. The survey contained several categorical items, which
       gather information regarding teacher characteristics (e.g. estimations of own
       IT skills, professional development in IT), student characteristics (e.g. student
       use of IT at school, and access at home) and school characteristics (e.g. the
       IT resources possessed by the school). In addition, the survey contains two
       open-ended items: “How does the computer help students achieve their




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
78 – 3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA

      learning outcomes across the curriculum?” and “How do you see computers
      affecting the future of classroom teaching?”
          Another survey instrument is the Technology-Rich Outcomes-Focused
      Learning Environment Inventory (TROFLEI): an 80-item inventory designed
      to assess classroom environments on ten dimensions. The TROFLEI built
      upon existing learning environment instrumentation through the use of the
      “What is happening in this class” (WIHIC) questionnaire as a basis for the
      development of a comprehensive instrument that includes a focus on technol-
      ogy and outcomes in secondary school classrooms. The TROFLEI has been
      validated with Tasmanian and Western Australian secondary school students
      (Dorman, Aldridge and Fraser, 2004).
           The Learning with ICT: Measuring ICT Use in the Curriculum instru-
      ment (Jamieson Proctor et al., 2006) was used to investigate both the quantity
      and quality of student use of ICT for learning. The instrument contained
      20 items, with response categories on a 4-point scale ranging from never to
      very often. Two sets of response categories are used to capture “current” and
      “preferred” teacher perceptions of ICT use by their students. The instrument
      was found to contain two strong factors: 14-items that define ICT as a tool
      for the development of ICT related skills and the enhancement of curriculum
      learning outcomes; and six items that define ICT as an integral component
      of reforms that change what students learn and how school is structured and
      organised. It has been found that male teachers report higher levels of con-
      fidence in using ICT with students for teaching and learning and students
      confident teachers use ICT more frequently to enhance the curriculum.

      Records and logs of activities in teaching
          Although much of the literature concerned with the impact of ICT
      on pedagogy is based on self-report questionnaire methods there other
      approaches to the study of pedagogy that make use of ICT and have been
      applied to the study of other aspects of pedagogy. These include the use of
      diaries or logs by students and or teachers and analysing the patterns that
      students use when working with computers.
          One evaluation of a class programme using laptop computers made exten-
      sive use of student diaries (M. Ainley et al., 2000). The impact of using laptop
      computers was monitored in a variety of ways. Diary formats completed by
      the students were used to identify what students were thinking about what
      was happening in their schooling as they carried their laptop from class to
      class each day. The basic procedure involved taking a sample of student diary
      entries and developing codes that reflected the range of comments being
      made. The coding system was then applied to a variety of diary formats to
      build up a picture of what the laptops and often computers in general meant for



                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
             3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA – 79



       student learning. Using spreadsheets and some simple graphing functions it
       was then possible to display the overall results and make comparisons between
       groups of students. It concluded that a range of tool functions were served by
       having a laptop computer and that although there was variation in response
       among both students and teachers to the programme and most of it was very
       positive. Students saw the computers as a tool for getting work done, and at
       the same time a tool that required them to learn how it operated (M. Ainley
       et al., 2000).
            Allan and Ainley (2002) focused on the use of ICT in teaching and learn-
       ing in a specific field of study: Studies of Society and Environment (SOSE).
       The study involved 400 students and their teachers from 28 Grade 5 classes
       in ten Victorian primary schools. It used questionnaires completed by teach-
       ers and students at the beginning and end of the topics, “Time, Continuity and
       Change” or “Place and Space”. In addition teachers completed a summary of
       their SOSE unit planners which provided details of learning objectives and
       outcomes, tools and resources, methods of assessment and the grouping con-
       figuration of the children for each of the activities. Each activity described
       in the teachers’ SOSE unit planners was then coded according to the nature
       of use of ICT as a tool for learning, the knowledge and cognitive processing
       learning objectives underpinning the activity, and the domains of learning
       involved. In total, more than 500 activities were planned by the 28 teachers
       across the ten different schools in the study. On inspection of the unit plan-
       ners, 58% of the teacher-planned activities incorporated some form of ICT
       use and just four per cent of all activities were conducted using ICT exclu-
       sively. Most commonly, teachers planned activities in the SOSE unit which
       used ICT to research for information and to process responses (i.e. work with
       ideas or construct response using information); 61% and 51% of activities
       respectively. Fewer activities involved presenting information (26%), learning
       ICT skills (19%), learning interactively (5%) and communicating (4%). The
       types of knowledge objective (following Anderson et al.’s taxonomy) used in
       the planning of SOSE activities (with or without using ICT) most commonly
       involved factual and conceptual knowledge. Very few procedural and meta-
       cognitive knowledge objectives were planned. In terms of planned objectives
       involving cognitive processing categories, remembering and understanding
       were most common. The results indicated that the potential of ICT to facili-
       tate higher order thinking skills was not being fully utilised by these teachers.
           M. Ainley and Hidi (2002) have investigated the dynamics of interest in
       student learning by focusing tasks that could occur in any classroom. Those
       tasks are presented in an interactive computer package that incorporates
       probes to monitor interest and emotions as they progress through the task
       and relate those to measures of learning at the conclusion of the task. Using
       the computer-based approach it has been possible to log students’ feelings,
       reactions and decisions across the course of the learning task. Sequences in


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
80 – 3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA

      the data record can be analysed using time logs that record students’ paths
      through the task, levels of interest, choices of which material to access, and
      decisions about when to start writing an answer. It is evident that interactive
      software such as this provides great potential for educational research to
      probe more deeply the way students interact with tasks. These studies demon-
      strate the capacity of ICT to provide insights into the sequences that students
      follow when they complete a task.

      Large scale surveys of teaching practice
           Australia participated in SITES 2006 (Law, Pelgrum and Plomp, 2008) as
      a benchmarking country (Ainley, Eveleigh and O’Malley, 2009). The results
      indicated that Australian science and mathematics teachers are relatively high
      users of ICT compared to their counterparts in other countries. A higher per-
      centage of Year 8 science teachers in Australian secondary schools used ICT
      in the past year than in most other countries surveyed (similar to Singapore,
      Hong Kong SAR, and Alberta). In addition, Australia was one of a group of
      countries in which a high percentage of Year 8 mathematics teachers used
      ICT (behind only Norway). Compared with their peers in other countries,
      Year 8 science and mathematics teachers in Australia are confident users
      of ICT. Computers in Australian secondary schools are less often located in
      classrooms (and more often in computer laboratories) than in countries such
      as Canada, Finland, Hong Kong and Norway. Australia is also a moderately
      high user of other ICT resources such as smart boards but is relatively low
      in terms of providing email facilities for students and data logging technolo-
      gies for use in science classes. Despite their confidence in being able to use
      ICT, fewer Australian science and mathematics teachers than their peers in
      countries such as Chinese Taipei, Denmark, Estonia, Hong Kong, and Israel
      participate in ICT-related professional development. The data from SITES
      suggest that there remains much to be done in extending professional devel-
      opment for teachers but that this should not be at the level of introductory
      courses.
           The use of ICT is greater when teachers have a higher level of or con-
      fidence in ICT, when teachers have participated in ICT-related professional
      development, and when there are fewer contextual obstacles (infrastructure,
      digital learning resources, access). In Australia, as in most countries, the
      percentage of teachers reporting ICT use is significantly higher for science
      teachers than for mathematics teachers. One inference to be drawn from this
      is that the subject (or discipline) context is an important aspect of the uptake
      of ICT in teaching. It may be that some subjects lend themselves more readily
      to the pedagogical use of ICT, that there are stronger traditions of innovation
      in some subjects or that digital resources are more available in some subjects
      than others.



                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
             3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA – 81



            The most frequently cited obstacle to incorporating ICT in teaching was
       the time required to develop and implement activities. Another factor men-
       tioned was the availability of digital learning resources in schools and student
       access to ICT tools. Infrastructure was seen an obstacle to ICT use by only
       about one quarter of Australian teachers and a similar number cited their own
       knowledge of using ICT in pedagogy as a limiting factor. These patterns were
       similar for science and mathematics teachers. School principals also indicated
       that a lack of time for teachers to use ICT was an obstacle to incorporating it
       in teaching. Three of the four top priorities nominated by school principals
       for enhancing the use of ICT in their schools, involved teachers: improving
       the ability of teachers to make good pedagogical use of ICT, improving the
       technical skills of teachers and increasing the number of teachers using ICT
       for teaching/learning purposes.

       Evaluating progress in ICT literacy
           In 2005, Australia began a cycle of thrice-yearly national surveys of
       the ICT literacy of students (Ainley and Fraillon, 2007; MCEETYA, 2007;
       MCEECDYA, 2010) using an innovative computer-based assessment. The
       Australian national assessment of ICT Literacy involved developing a com-
       puter-based assessment that combined automated skills assessments and the
       production of complex work products involving the evaluation and integration
       of information. The assessment instrument combined multiple item types
       within a single, consistently administered assessment. Within each assess-
       ment module students were asked multiple-choice questions to assess knowl-
       edge, to perform specific functions within simulations of software products
       to assess skill with common applications, to provide constructed responses
       to specific questions and to create work products using live applications.
       The work products, created under controlled, consistent conditions for every
       student, were evaluated using standardised rubrics by trained assessors. The
       assessment instrument consisted of seven thematic modules of which each
       student completed three. One reason for conducting the assessment with a
       number of modules was to ensure that the assessment instrument accessed
       what is common to the ICT construct across a sufficient breadth of authentic
       contexts.
           Completing authentic tasks in real contexts was seen as fundamental to
       the design of this ICT literacy assessment. The assessment model defined
       a single variable, ICT literacy, which integrated three related strands. One
       strand involved identifying required information; formulating and execut-
       ing a strategy to find information; making judgements about the integrity
       of the source and content of the information; and organising and storing
       information for retrieval and reuse. The second strand involved: adapting and
       authoring information; making choices about the nature of the information



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
82 – 3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA

      product; reframing and expanding existing information to develop new
      understandings; and collaborating and communicating with others. A third
      strand included understanding the capacity of ICT to impact on individuals
      and society, and the consequent responsibility to use and communicate infor-
      mation ethically.
          In the 2005 cycle the assessment was administered using sets of six net-
      worked laptop computers with all necessary software installed. Test adminis-
      trators travelled to each school with the networked computers to manage the
      process. The assessment was conducted with approximately 7 400 students
      in 520 schools. The 2008 cycle made more extensive use of school computers
      with delivery being based on a plug-in server (68% of schools), connection to
      a server farm (19% of schools) and sets of networked computers brought to
      the school (14% of schools). It was conducted with just fewer than 11 000 stu-
      dents in just fewer than 600 schools.
          In addition to conducting analyses of patterns among states and groups of
      students using ICT Literacy scale scores a set of six proficiency levels were
      defined as shown in Table 3.1. The boundary between level 2 and level 3 was
      defined as the proficient standard for students in Grade 6 and between level 3
      and level 4 was defined as the proficient standard for students in Grade 10.
      The results indicated an improvement between 2005 and 2006 for Grade 6
      students and a (not statistically significant) tendency to improve for Grade 10
      students.
          ICT literacy was associated with socioeconomic background and indig-
      enous status. ICT literacy also differed among geographic locations: met-
      ropolitan students to recorded higher ICT literacy scores than students in
      provincial areas who, in turn recorded higher scores than those in remote
      areas. It was also noted that there was an increase in the use of computers at
      home and at school and this increase may well be a reason for the increased
      level of ICT literacy. Social communication was the group of most frequently
      used applications followed by entertainment and school utilities with com-
      puter technology being the least frequently used group of applications.




                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
               3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA – 83



   Table 3.1. ICT literacy proficiency level descriptions and percentage distributions,
                                      2005 and 2008

                                                                                                      %          %
                                                                                                    Grade 6   Grade 10
Level                                Proficiency level description                                 2005 2008 2005 2008
  6     Students working at level 6 create information products that show evidence of technical     0    0    0     1
        proficiency, and careful planning and review. They use software features to organise
        information and to synthesise and represent data as integrated complete information
        products. They design information products consistent with the conventions of specific
        communication modes and audiences and use available software features to enhance
        the communicative effect of their work.
  5     Students working at level 5 evaluate the credibility of information from electronic         0    1    12   18
        sources and select the most relevant information to use for a specific communicative
        purpose. They create information products that show evidence of planning and
        technical competence. They use software features to reshape and present information
        graphically consistent with presentation conventions. They design information products
        that combine different elements and accurately represent their source data. They use
        available software features to enhance the appearance of their information products.
  4     Students working at level 4 generate well targeted searches for electronic information      8    15   49   47
        sources and select relevant information from within sources to meet a specific purpose.
        They create information products with simple linear structures and use software
        commands to edit and reformat information products in ways that demonstrate some
        consideration of audience and communicative purpose. They recognise situations in
        which ICT misuse may occur and explain how specific protocols can prevent this.
  3     Students working at level 3 generate simple general search questions and select the        41    41   32   26
        best information source to meet a specific purpose. They retrieve information from given
        electronic sources to answer specific, concrete questions. They assemble information in
        a provided simple linear order to create information products. They use conventionally
        recognised software commands to edit and reformat information products. They recognise
        common examples in which ICT misuse may occur and suggest ways of avoiding them.
  2     Students working at level 2 locate simple, explicit information from within a given        39    30   6     7
        electronic source. They add content to and make simple changes to existing information
        products when instructed. They edit information products to create products that show
        limited consistency of design and information management. They recognise and identify
        basic ICT electronic security and health and safety usage issues and practices.
  1     Students working at level 1 perform basic tasks using computers and software. They         13    13   0     1
        implement the most commonly used file management and software commands when
        instructed. They recognise the most commonly used ICT terminology and functions.

Source: MCEECDYA (2010).



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
84 – 3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA




                                       References

      Ainley, M. and S. Hidi (2002), “Dynamic measures for studying interest and
         learning”, in P. R. Pintrich and M. L. Maehr (eds.), Advances in Motivation
         and Achievement: New Directions in Measures and Methods, Vol. 12,
         pp. 43-76, JAI, New York, NY.
      Ainley, J., D. Banks and M. Fleming (2002), “The influence of IT: Perspectives
         from five Australian schools”, Journal of Computer Assisted Learning,
         Vol. 18, No. 4, pp. 395-405.
      Ainley, J., F. Eveleigh and K. O’Malley (2009), ICT in the Teaching of Science
         and Mathematics in Year 8 in Australia: a Report from the Sites Survey,
         Department of Education, Employment and Workplace Relations, Canberra.
      Ainley, M. et al. (2000), Computers, Laptops and Tools, Australian Council for
         Educational Research, Melbourne.
      Aldridge, J. M., J. P. Dorman and B. J. Fraser (2004), “Use of multi-trait – multi-
         method modelling to validate actual and preferred forms of the Technology-
         rich Outcomes-focused Learning Environment Inventory (TROFLEI)”,
         Australian Journal of Educational and Developmental Psychology, Vol. 4,
         pp. 110-125.
      Allan, A. and J. Ainley (2002), “Engaging the child or engaging the computer?
         A study of how teachers and students use and experience information and
         communication technology in the upper primary classroom”, Australian
         Council for Educational Research, Melbourne.
      Anderson, R. and J. Ainley (2010, in press), Technology and learning: Access
        in schools around the world, in B. McGaw, E. Baker, and P. Peterson,
        International Encyclopedia of Education, 3rd edition, Elsevier, Amsterdam.
      Australian Bureau of Statistics (ABS) (2009), Household Use of Information
        Technology, Australia, 2008-09 (Catalogue No. 8146.0), Australian Bureau
        of Statistics, Canberra.




                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
             3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA – 85



       Australian Bureau of Statistics (ABS) (2010), Schools Preliminary
         Australia: 2009 (Catalogue No. 4220.0), Australian Bureau of Statistics,
         Canberra.
       Australian ICT in Education Committee (2008), Success through Partnership:
         Achieving a National Vision for ICT in Schools, Canberra.
       Clarkson, P., A. Dunbar and R. Toomey (1999), Whole School Reform and the
          Use of ICT: An Evaluation of the Navigator Schools Project, Department of
          Education, Employment and Training, Melbourne.
       Committee for the Review of Teaching and Teacher Education (Kwong
         Lee Dow, Chair) (2003), Australia’s Teachers: Australia’s Future – Advancing
         Innovation, Science, Technology and Mathematics, Department of Education,
         Science and Training, Canberra.
       Cuttance, P. (2001), School Innovation: Pathway to the Knowledge Society,
          Department of Education Science and Training, Canberra.
       Dorman, J. P., J. M. Aldrige and B. J. Fraser (2006), “Using students’ assessment
         of classroom environment to develop a typology of secondary school
         classrooms”, International Education Journal, Vol. 7, No. 7, pp. 906-915.
       Downes, T. et al. (2001), Making Better Connections: Models of Teacher
         Professional Development for the Integration of Information and
         Communication Technology into Classroom practice, Department of
         Education Science and Training, Canberra.
       Jamieson Proctor, R. M. et al. (2006), “ICT integration and teachers’ confidence
          in using ICT for teaching and learning in Queensland state schools”,
          Australasian Journal of Educational Technology, Vol. 22, No. 4, pp. 511-530.
       Kozma, R. (ed.) (2003), Technology, Innovation, and Education Change: A
         Global Perspective: A Report of the Second Information Technology in
         Education Study (SITES) Module 2, International Society for Technology in
         Education, Eugene, OR.
       Kozma, R., and R. McGhee (2003), “ICT and innovative classroom practices”,
         in R. Kozma (ed.), Technology, Innovation, and Education Change: A Global
         Perspective: A Report of the Second Information Technology in Education
         Study (SITES) Module 2, International Society for Technology in Education,
         Eugene, OR.
       Law, N., W. Pelgrum and T. Plomp (2008), Pedagogy and ICT Use in Schools
          Around the World: Findings From the IEA SITES 2006 Study, Comparative
          Education Research Centre, University of Hong Kong, Hong Kong.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
86 – 3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA

      McDougall, A., P. Nicholson and A. Marshall (2001), “Context and outline of the
        DEET’s ‘Notebook computers for teachers and principals’ initiative: Initial
        evaluation findings”, ICT in Education, Vol. 24, No. 1, pp. 17-22.
      Martin, M., I. Mullis and P. Foy (2008), TIMSS 2007 International Science
        Report: Findings from IEA’s Trends in International Mathematics and
        Science Study at the Fourth and Eighth Grades, Lynch School of Education,
        Boston College, Chestnut Hill, MA.
      Ministerial Council for Education, Early Childhood Development and Youth
        Affairs (MCEECDYA) (2008), Melbourne Declaration on Educational Goals
        for Young Australians, Curriculum Corporation, Carlton.
      Ministerial Council for Education, Early Childhood Development and Youth
        Affairs (MCEECDYA) (2010), National Assessment Program – ICT Literacy
        Years 6 and 10 Report 2008, Curriculum Corporation, Carlton.
      Ministerial Council for Education, Employment, Training and Youth Affairs
        (MCEETYA) (2007), National Assessment Program – ICT Literacy Years 6
        and 10 Report 2005, Curriculum Corporation, Carlton.
      Ministerial Council for Education, Employment, Training and Youth Affairs
        (MCEETYA) (ICT in Schools Task Force) (2008), Digital Education: Making
        Change Happen, Curriculum Corporation, Carlton.
      Ministerial Council on Education, Employment, Training and Youth Affairs
        (MCEETYA) (1999), National Goals for Schooling in the Twenty First
        Century, Curriculum Corporation, Melbourne.
      Ministerial Council on Education Employment Training and Youth Affairs
        (MCEETYA) (2005), Contemporary Learning: Learning in an Online World,
        Curriculum Corporation, Melbourne.
      Ministerial Council on Education, Employment, Training and Youth Affairs
        (MCEETYA) (2000), Learning in an Online World: The School Education
        Action Plan for the Information Economy, Education Network Australia,
        Adelaide.
      Ministerial Council on Education, Employment, Training and Youth Affairs
        (MCEETYA) (2005), Contemporary Learning: Learning in an Online World,
        Curriculum Corporation, Carlton.
      Ministerial Council on Education, Employment, Training and Youth
        Affairs (MCEETYA) (2008), Australian ICT in Education Committee
        Workplan 2008, Curriculum Corporation, Carlton.
      Rowe, H. (1993), Learning with Personal Computers, Australian Council for
        Educational Research, Melbourne.




                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
             3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA – 87



       Robertson, M. et al. (2004), “Classroom computer climate, teacher reflections
         and ‘re-envisioning’ pedagogy in Australian schools”, Australasian Journal
         of Educational Technology, Vol. 20, No. 3, pp. 351-370.
       Shears, L. (ed.) (1995), Computers and Schools, Australian Council for
          Educational Research, Melbourne.
       McDougall, A., P. Nicholson and A. Marshall (2001), “Context and outline of the
         DEET’s ‘Notebook computers for teachers and principals’ initiative: Initial
         evaluation findings”, ICT in Education, Vol. 24, No. 1, pp. 17-22.
       Martin, M., I. Mullis and P. Foy (2008), TIMSS 2007 International Science
         Report: Findings from IEA’s Trends in International Mathematics and
         Science Study at the Fourth and Eighth Grades, Lynch School of Education,
         Boston College, Chestnut Hill, MA.
       Ministerial Council for Education, Early Childhood Development and Youth
         Affairs (MCEECDYA) (2008), Melbourne Declaration on Educational Goals
         for Young Australians, Curriculum Corporation, Carlton.
       Ministerial Council for Education, Early Childhood Development and Youth
         Affairs (MCEECDYA) (2010), National Assessment Program – ICT Literacy
         Years 6 and 10 Report 2008, Curriculum Corporation, Carlton.
       Ministerial Council for Education, Employment, Training and Youth Affairs
         (MCEETYA) (2007), National Assessment Program – ICT Literacy Years 6
         and 10 Report 2005, Curriculum Corporation, Carlton.
       Ministerial Council for Education, Employment, Training and Youth Affairs
         (MCEETYA) (ICT in Schools Task Force) (2008), Digital Education: Making
         Change Happen, Curriculum Corporation, Carlton.
       Ministerial Council on Education, Employment, Training and Youth Affairs
         (MCEETYA) (1999), National Goals for Schooling in the Twenty First
         Century, Curriculum Corporation, Melbourne.
       Ministerial Council on Education Employment Training and Youth Affairs
         (MCEETYA) (2005), Contemporary Learning: Learning in an Online World,
         Curriculum Corporation, Melbourne.
       Ministerial Council on Education, Employment, Training and Youth Affairs
         (MCEETYA) (2000), Learning in an Online World: The School Education
         Action Plan for the Information Economy, Education Network Australia,
         Adelaide.
       Ministerial Council on Education, Employment, Training and Youth Affairs
         (MCEETYA) (2005), Contemporary Learning: Learning in an Online World,
         Curriculum Corporation, Carlton.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
88 – 3. MONITORING AND ASSESSING THE USE OF ICT IN EDUCATION : THE CASE OF AUSTRALIA

      Ministerial Council on Education, Employment, Training and Youth
        Affairs (MCEETYA) (2008), Australian ICT in Education Committee
        Workplan 2008, Curriculum Corporation, Carlton.
      Rowe, H. (1993), Learning with Personal Computers, Australian Council for
        Educational Research, Melbourne.
      Robertson, M. et al. (2004), “Classroom computer climate, teacher reflections
        and ‘re-envisioning’ pedagogy in Australian schools”, Australasian Journal
        of Educational Technology, Vol. 20, No. 3, pp. 351-370.
      Shears, L. (ed.) (1995), Computers and Schools, Australian Council for Educational
         Research, Melbourne.




                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
   4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE – 89




                                             Chapter 4

         Extending and scaling technology-based innovations
              through research: The case of Singapore


                 David Hung, Kenneth Lim, and David Huang
National Institute of Education, Nanyang Technological University, Singapore




    This chapter examines the question of 21st century skills through the prism of a
    case study on Singapore. Certainly, this case is quite particular in many respects,
    including its emphasis on the design, implementation and evaluation of national
    master plans. The support to technology-based innovations, as well as their
    monitoring and assessment, have been playing an important role in these plans.
    This contribution discusses the different ways in which practitioners, researchers
    and policy makers have been involved in the process of documenting successful
    innovations and planning for scaling up. The authors suggest that careful attention
    should be paid to the translation process from the initiation of innovation to the
    implementation of innovation.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
90 – 4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE

Introduction

           Traditionally, the extending and scaling of research innovations in educa-
      tion has been viewed through the lens of multiplication (increasing numbers)
      and spreading (increasing areal reach). In Diffusion of Innovations (Rogers,
      1964), innovation is defined as any new idea, practice or object, and innova-
      tion diffusion is measured in terms of the number of innovation adopters over
      temporal, social and spatial dimensions. Such a view of innovation-scaling is
      therefore product-oriented, in that the deliverables (performance indicators)
      of successful “scaling” are defined according to strict numeric constructs
      (e.g. the number of teachers, the number of schools, the number of school
      clusters, etc). We would extend Rogers’ definition of innovation that the inno-
      vation idea should add value to the current status quo, and in this case, to the
      learning and educational process as mediated by technology.
          Another limitation of product-oriented innovation-scaling theories such
      as “Diffusion of Innovations” is that the process (rather than factors) in which
      pre-adopters make transitions through their participatory involvements in
      the community has not generally been the focus of investigation. An often
      implicit assumption of this view is that these innovation “products” are
      replicable en masse without undergoing (and, without the need to undergo)
      significant change from the original.
          Such a view of scaling stems from a twentieth-century Fordist-production
      paradigm (as applied to traditional notions of Instructional Design), and there
      are numerous examples of such an interpretation which can be cited from
      within the Singapore school system. Such views are challenged by alterna-
      tive paradigms such as “Communities of Practice”, in which social and con-
      textual dimensions and people-activity interdependencies are foregrounded,
      thus supporting a process-oriented perspective. Such processes are enacted
      through shared codes of conduct, histories and cultures (Wenger, 1998).
          The purpose of this paper is to further our understanding of a process-
      oriented and situative view of extending and scaling innovations, recognis-
      ing the contextual fidelity of each translative iteration. Moreover, we hope
      to develop a locally oriented translation-scaling framework for extending
      technological innovations. We worked on the premise that an innovation with
      a view to extension or scaling would be premised on the assumption that the
      normative population through which the “spread” is intended is generally not
      ready for the innovation. Hence the need for translation and reaching out to
      the targeted population.
          It is our contention that 21st century learning and literacies demand a
      fundamental rethinking of such a framing of innovation-scaling. We echo
      Latour (1993) in proposing a more nuanced, situative view of innovation-
      scaling – one which explicitly foregrounds the local contextual factors and


                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
   4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE – 91



       interplays within which all iterations from the original are embedded. To
       elaborate, a shift needs to be made from the strict multiplicative metaphor to
       what we term a “resemblance” metaphor. We argue that inherent in such a
       resemblance metaphor is the explicit recognition that the extension and scal-
       ing of innovations arising from education-research is just as much a process
       as it is a product; and because innovation-scaling is a process, it is by defini-
       tion not processes to be replicated, but instead to be re-created / re-instanti-
       ated / re-enacted. Such instantiations and enactments take place in the milieu
       of the products of the innovation, namely artefacts and boundary objects.
       The latter form the substrate from which the dialectical interactions between
       product, process and participant-practitioner are lived and therefore reified.
       Going forward, it is our strong conviction that such a framing of the exten-
       sion and scaling of innovations will inform the direction of many education-
       research interventions in Singapore. Hence, subsequent re-instantiations from
       the original are not reproductions but re-creations which have resemblances
       to the original. We posit that such a view is critical for scaling technologi-
       cal innovations in particular because the affordances inherent in supporting
       pedagogy as recreated in subsequent instantiations are inquiry-inclined.
       Teachers have to appropriate not just the products, but they must re-contex-
       tualise the (inquiry) processes – usually related to curricular and assessment
       resources – in their respective settings and with a view to being consistent to
       the conceptual underpinnings of the original innovation.
           In the translation literature, it is generally accepted that there are two
       broad thrusts of translation (Institute of Medicine’s Clinical Research
       Roundtable, Sung et al., 2003). These are, namely, translational research
       (T1) and translation research (T2). T1 refers to deductively-derived research
       interventions, which have been enacted within relatively homogenous and
       resource-rich contexts. T2 refers to the dissemination, implementation and
       diffusion of T1 research into community-practice and policy (e.g. Narayan
       et al., 2000; Schillinger, 2007). In turn, dissemination refers to how the
       targeted distribution of information and intervention materials can be suc-
       cessfully executed; implementation can be thought of as referring to the
       implementation of content (i.e. the interpretation by practitioners of the
       research evidence and of the codified intervention) within a given (political/
       professional/socio-economic/organisational/attitudinal) context, through the
       process of enacting and engaging in strategies for change in management
       practices. As for diffusion, the lens is turned on the factors for successful
       adoption of the intervention which results in widespread use by the target
       population. Such “successful adoption” can be further analysed in terms of
       the uptake of the practice and/or innovation, as well as in terms of the pen-
       etration of broad-scale recommendations through dissemination.
          Unlike translation science, diffusion research can trace its roots into the
       mid-twentieth century. Kroeber (1940) and Hägerstrand (1967) have written


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
92 – 4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE

      extensively on the matter, and of particular relevance to the present discussion
      is the reminder that diffusion need not necessarily be assumed to originate
      continually from a single, authoritative source (expansion diffusion) but can
      also take place through other models, such as those of contagion and hierarchy.
      In this regard, consideration should therefore be placed on the dispositions of
      the change agents, particularly according to Rogers’s (1964) characterisations
      of innovators, early adopters, the early majority, the late majority and laggards.
           What might some key performance indicators of T2 be? In the translation
      literature, Glasgow’s (1999) articulation of the so-called RE-AIM framework
      has been frequently cited and speaks authoritatively to this question. The
      RE-AIM framework was explicitly designed from the outset to be an evalu-
      ation framework for translation. It describes five dimensions, which operate
      at either (or both) the individual and organisational levels. At the individual
      level, the success of any given translation effort can be evaluated against
      the criteria of Reach (into members of the target population), Effectiveness
      (ability of the intervention to do more good than harm in a real-world set-
      ting (contrasted with Efficacy)), and Maintenance (in individuals over time).
      At the organisational level, the success of any given translation effort can
      be evaluated against the criteria of Adoption (by target settings and target
      institutions), Implementation (consistency across programme components
      and members in terms of the aforementioned aspects of content, context and
      process) and Maintenance (in populations over time; implicit in this notion
      of maintenance are both sustainability of the innovation/intervention and the
      adaptation thereof). The RE-AIM framework is extremely helpful because it
      is accommodative enough to provide policy makers and programme evalua-
      tors with an evaluative structure against which to compare (in a fairly objec-
      tive manner) potentially very different interventions. Potentially, weights
      could be applied to the five dimensions in order to reflect the priorities and
      imperatives of policy and funding. To the present author’s mind, this point
      cannot be over-emphasised.

A way forward

           Summarising from the above issues, a process-oriented translation
      framework does not preclude the role of products and other codified forms
      of reifications. Based on our experiences in Singapore, innovations – be they
      pedagogical and/or technological – usually begin as research initiatives/
      projects. These research efforts (from the Learning Sciences Lab, Singapore)
      can be broadly construed as T1 efforts. Many of the technology-mediated
      research projects from the Learning Sciences Lab (Singapore) adopt design-
      based research methodologies, implying that researchers and practitioners in
      school-based innovations co-construct the agenda of the entire intervention.
      Arising from these interventions, there is an emergent recognition that for


                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
   4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE – 93



       T2 to occur, people-preparation and foundational tenets described through
       a sufficiently well-specified and articulated set of codifications in the form
       of design principles, norms and procedures are necessary (see Figure 4.1).
       Alongside these codified representations of design principles, technologi-
       cal innovations include prototypes (products). It is acknowledged that such
       prototypes run the risk of being extended or scaled without the necessary
       understandings of how they are to be appropriated in ways that deeply
       embrace the philosophical underpinnings of the design principles arising
       from their respective theories of action. It is an understandable apprehension
       of researchers who purport process-orientations that their codified products
       get lethally mutated from their original intentions. Worse still is the possible
       ex-appropriation (through questionable legal means and motives) of products
       in ways that undermine the intellectual property rights of the originators.
       How might some of these issues be reconciled, particularly the thorny issue
       of managing mutations?

         Figure 4.1. Framework for translation and extension/scaling innovations
                                                             New Beginnings


              Research                    Design
                                                               Recreation




               Study                     Principle/
                                         Artifacts

                                                 boundary
           People embodied
                                                 objects
           with philosophical
           understandings                                                      Legitimate
                                                                              Sound Principle



                                                      Mutations                  Lethal



            Consistent with the re-creation and “resemblance” metaphor, rather than
       re-production, we conjecture that mutations are inevitable; and, indeed, desir-
       able and healthy. To be philosophically situative, every recontextualisation
       effort cannot be identical to previous instantiations since any social context
       is chaotic (Poincaré, 1890) and unpredictable. The issue is hence not about
       mutations per se but is instead the extent to which these mutations might be
       considered lethal! To probe more deeply into this notion of “lethality”, lethal-
       ity is often considered relative to the original as a frame of reference. We



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
94 – 4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE

      would, however, wish to suggest that since mutations are inevitable, the cri-
      teria for judging mutations should be whether they are legitimate rather than
      lethal. By legitimate, we suggest that the mutations are within sound learning
      principles broadly specified rather than very specific design principles nearly
      consistent with the original design specifications of the research project.
           Central to the translation design of our framework (see Figure 4.1) are the
      following principles:
          1. Embodiment
          2. Reification/codification
          3. Dialogue
          4. Brokering
          By embodiment we refer to the intentional design for people (researchers,
      practitioners, brokers) to take an active part in the actual social participation
      of the research project and the subsequent instantiations. Through embodi-
      ment, participants develop both explicit reifiable knowledge and the implicit
      understandings that may not be made explicit through language (a notion
      similar to the map not being the territory).
          Consistent with our design research methodologies, we acknowledge the
      need to describe a set of codified design principles that arise from the research
      study. The specification-details vary across research projects but a general rule
      of thumb might be that someone expressing interest in applying the design
      principles could be confident in staying true to them when complemented
      with exemplars (such as, but not limited to, video cases), and curricular and
      assessment resources. Importantly, to avoid lethal mutations, the people who
      take these products forward need to have a sensible and consistent understand-
      ing of the philosophical underpinnings of the original project. This cannot be
      achieved through traditional notions of “sharing sessions” but should instead
      be structured through exchange programmes/mentorship programmes.
           Such programmes segue well into “dialogue”, by which is meant the need
      for participants, whether in the research project or in subsequent instantiations,
      to constantly dialogue around the reifications constructed by the research
      project and in the translation efforts. Through the process of dialogue, miscon-
      ceptions can be explicated and understandings advanced. Participants newer to
      the process can also be gradually enculturated through dialogue between dif-
      ferent members of the research-translation community (researchers, practition-
      ers, brokers). The key thrust is in involving stakeholders from the start with a
      view to establishing shared language and understandings through boundary
      objects (such as design artefacts). Since translation must be locally relevant,
      outcomes must be important to practitioners and seen as feasible, and they
      must address issues of local concern (Glasgow and Emmons, 2007).


                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
   4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE – 95



           Finally, brokers are those who mediate and enable others who are able to
       deeply articulate the goals and philosophies of the original research project
       (on the one hand) and the subsequent uptake by individuals who appropri-
       ate the design principles and resources (on the other). Brokering is needed
       to bring diverse groups of people together who – because of their differing
       philosophical trajectories – may not necessarily be readily thought of by the
       other as holding potentially similar levels of specificity with regard to their
       respective epistemological stances.

From research projects to extension and scaling

            To reiterate, our starting point is research projects. This seeding process
       of research projects is an intentional stage where different interventions are
       trialled in schools. Importantly, it is recognised that design-based research
       involves practitioners at an early stage. Care needs to be taken that such practi-
       tioner-involvement is not enacted in a monologic way but through true dialogue
       arising from mutual trust and respect. Even with embedded translation and the
       design principles reified, there is still a need to extend the dialogic conversa-
       tions to subsequent stakeholders on the potential challenges and issues (such as
       IP rights) so as to build towards future extension and scaling. In the extension
       and scaling efforts, deliberate efforts should be made to seed for a research-
       practice community as a structure through which participants may engage with
       the various context-instantiations. All stakeholders bring their respective disci-
       plinary understandings and values to the table. Brokers are present to mediate
       these axiologies such that they are mutually honoured by all parties.
            The projects conducted by the researchers at the Learning Sciences Lab
       usually adopt the Design Research methodology (van den Akker, Gravemeijer,
       McKenney and Nieveen, 2006). Inherent in this methodology is the joint
       partnership in the research design and implementation with practitioners
       (e.g. teachers and school leaders) throughout the process. Teachers co-design
       the tasks and the curricular and assessment resources together with research-
       ers, and the overall design process is iterative and refined as it is implemented
       with students. Through a collaborative journey, stakeholders co-create the
       innovation by designing the process in an iterative, evaluative and reflec-
       tive fashion through dialogue and experimentation. The entire process is
       “researched” through constant monitoring and evaluation.
           In the same vein, the principles inherent in design research are appro-
       priated into our proposal of the three-staged process of translating research
       into practice. Early adopters are sought and enculturated, including policy
       makers who can influence the diffusion process. Design specifications from
       the research project(s) are further tested and elaborated in (more homogene-
       ous) settings where the designs are to be potentially implemented. Dialogue



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
96 – 4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE

      is the key strategy to enable new stakeholders to understand the conceptual
      underpinnings of the innovation. The entire three-stage translation process
      is “researched” – monitored and evaluated in an iterative fashion. Before
      the third stage of “extension and scaling”, lessons need to be learned by a
      clear articulation of design principles and establishment of the boundaries of
      immutable features of the intervention – this is where the design specifica-
      tions cannot be compromised. In other words, the stakeholders involved need
      to clearly understand the extent through which mutations are legitimate,
      beyond which lethal mutations will likely occur.
           As an example, one of the flagship projects from the Learning Sciences
      Lab is the NRF-funded citizenship education videogame Space Station
      Leonis. This game was piloted in schools during 2007 and extended in 2008.
      The theoretical foundation for the game-enactment is Gee’s (2007) notions
      of projective identity and dialectical phronesis, which itself has its roots in
      Aristotlean noesism. The game allows players to adopt role performances and,
      in the (inquiry) process, to make decisions. These games are not instructional
      games where content learning is the focus but rather are more consistent
      with the MMPOGs that are popular among youths, where players experience
      embodiment as avatars.
           The Leonis programme has now run its course in terms of the initial
      funding and is presently being considered for release to schools in Singapore.
      Because of its inherently axiologic design trajectory, it is acknowledged that the
      Leonis programme may not necessarily be easily appropriated in a high degree
      of fidelity to its original design parameters by practitioners who do not share a
      strong Bakhtinian/Losskyian philosophical tradition. At the same time, other
      voices in this heteroglossic contestation about translating the Leonis programme
      into contexts and settings which it was not originally designed for (such as the
      primary school environment) have resulted in programmatic offshoots-in-
      development inspired by (but having no affiliation with) the Leonis programme,
      which have attempted to re-contextualise what are perceived to be its key design
      principles into a socio-cultural developmental milieu for younger learners.
          In terms of the three-stage framework proposed above, this deliberate
      structuring for extension and scaling is a necessary follow-through upon the
      completion of the initial research project. In this stage, issues of licensing and
      IP rights have to be seriously considered. Deliberate plans should be made
      to expand the pool of stakeholders in order to complement efforts to foster a
      deeper sense of Bakhtinian dialogism as a lens for critical inquiry.
          In the example of the Leonis programme, seeding a community in game-
      based learning would be needed. Key community members would include
      research faculty, teachers from participating schools, and officers from the
      local ministry of education involved in assisting schools with game-based
      pedagogies.


                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
   4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE – 97



            Table 4.1. Three stages from research projects to extension and scaling

 Stages                            Tenets                                  Key translation issues
                 Research projects need to perturb the       – How do we identify research projects which have
 Seeding         status quo of current pedagogical and         the potentials for extension and scaling?
 research        learning practices                          – What are the criteria?
 projects
                 Researchers engage with practitioners in    – How do we design research projects with
 (T1)
                 co-designs and implementations throughout     translation, extension and scaling in view?

                 Before extension and scaling can take place, – How can research designs be extended out and
                 there needs to be an intentional phase          scaled up in different contextual situations?
                 needed to dialogue and think through issues – What or where are the boundaries beyond which
                 that would surface in extension and scaling     the design cannot be compromised?
                 efforts. These issues include: IP matters,    – What are the key strategies for re-iterating?
                 adequate preparation of the people involved
                 in extension and scaling, testing out the
 Deliberate      adequacy of the specifications of design
 structuring     principles and related resources, and others.
 for                                                         – What are the inter-relating factors leading to
 extension                                                     extension and scaling?
 and                                                         – What should we take into consideration when
 scaling                                                       preparing for extension and scaling?
                                                             – What product issues need to be put in place?
                 Seeding a community of stakeholders who
                                                             – What process preparations are needed?
                 understands the translation efforts
                                                             – What criteria do we employ to determine
                                                               readiness for extension and/or scaling?
                                                             – Who should these initial stakeholders be
                                                               (e.g. innovators) in the seeding process of the
                                                               community?

                 Scaling begins when translation issues have – How can we engage in extensions and scalings
                 been piloted and design specifications tested which are legitimate?
                 and diverse populations would likely interpret
                 these resources in appropriate ways.
 Extension
                                                             – How do we evaluate the successful efforts in
 and
                                                               extension and scaling (RE-AIM)?
 scaling
                                                             – What do we do with lethal mutations?
 (T2)
                 Sustaining the community                    – How do we plan for sustaining the community?
                                                             – Who (e.g. early adopters)? Should we be
                                                               reaching out to in order to encourage new
                                                               membership in the community?




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
98 – 4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE

          Most of the research projects undertaken at the Learning Sciences Lab
      have pedagogical innovations that can be characterised as “distant from
      current school practice”. Another project referred to as knowledge-building
      community attempts to enculturate a knowledge-building epistemology to
      the science inquiry process where students think like scientists. Although
      students in Singapore generally do well for national science stakes examina-
      tions, knowledge-building is only nascent to school practices. For knowledge-
      building to scale up, teachers’ epistemology of science needs to be changed to
      one of inquiry and meaning-making.
           Gradually changing current school practices is key to the translation
      process, and this is indeed a daunting challenge. One of the key strategies is
      to show, through research evidence, that students do not necessarily perform
      worse in traditional exams despite spending a considerable amount of time
      in these inquiry-based innovations. Enlisting the support of parents in these
      efforts has also been useful to the infusion of such innovations.

Conclusions

          This paper has been framed as a positional statement of present under-
      standings of translation, extension and scaling, as held by its authors in their
      respective institutional contexts. It is acknowledged that the nascent nature of
      translation science renders the expectation of any definitive conclusions both
      unhelpful and unrealistic. Instead, it is hoped that the framework and ideas
      presented here will form a common substrate upon which conversations about
      these very issues may emerge and be sustained.
           By way of a summary and less by way of a conclusion, therefore, we believe
      that the key to successful translation is people. Effective strategies to enculturate
      people include the following.
              Involving stakeholders in co-analysis and co-design from the start
              through:
              -    iterations of the design-development programme;
              -    planning for curricular enactments;
              -    planning for professional development;
              -    working towards mutual benefit for all stakeholders (including
                   cognizance of local issues and concerns); and
              -    building towards mutual trust and respect (no single member
                   should have ownership of the data, nor of its interpretation.




                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
   4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE – 99



                 Establishing shared language and understandings through boundary
                 objects (design artefacts) via:
                 -    clear articulation of design principles; and
                 -    clear establishment of the boundaries of immutable features of
                      the intervention (addressing “flexibility versus fidelity” through
                      “rigour without rigidity”).
                 Seeding and incubating a community of researchers, practitioners
                 and brokers to augment translation by:
                 -    building on strengths and resources within the community (Israel
                      et al., 1998); and
                 -    bearing in mind that while the community may be a global col-
                      lective, translation always needs to be locally relevant, by which
                      is meant that outcomes must be important to practitioners and
                      must be seen as feasible and addressing issues of local concern
                      (Glasgow and Emmons, 2007).
            Taken together, it is envisaged that the preceding strategies will go some
       way towards illuminating what is as yet an amorphous and evolving ecologi-
       cal space that draws its impetus for growth from the research-practice nexus.
       It is also hoped that this paper builds on Rogers’s (1964) work by drawing
       attention to the product-process dialectic, foregrounding the latter while
       not discounting the former. In this way, we are informed by the increasing
       number of post-modern socio-cultural milieux in which scaling is far more a
       trans-contextual phenomenon than that which was afforded to Rogers in the
       1960s.
            We have chosen to stress the importance of people and stakeholders as a
       key dimension in successful translation efforts. By arguing for a community
       framework to augment translation efforts, we have proposed a social par-
       ticipatory process complementing the traditional product-oriented scaling
       models. We see this as another significant contribution of this paper to the
       literature. We frame such a community as central to the success of re-creat-
       ing resemblances and legitimate mutations relative to the original research
       innovation.
           Finally, consistent to the principles of the design research methodol-
       ogy, studying and refining the translation trajectory is our process-oriented
       approach for monitoring and evaluating the research innovations in a sys-
       temic fashion. Instead of a product-oriented scaling emphasis that usually
       measures quantitative outcomes of technological adoption, we design for
       a systemic change process involving qualitative measure of change and
       outcomes. We argue for such a “situative” need in order to complement the
       many qualitative studies that measure spread in school systems but offer little


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
100 – 4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE

      understanding of the process and nature of change involved. Consistent to the
      situative stance, our process-oriented and contextually sensitive design meth-
      odology for translation enables knowledge and understandings to be socially
      constructed within the community of stakeholders. Through such a lived
      experience, knowledge is “managed” and understandings deepened through
      the production of artefacts and as mediated by dialogue.




                                           INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
  4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE – 101




                                           References

       Coburn, C. (2003), “Rethinking scale: Moving beyond numbers to deep and
         lasting change”, Educational Researcher, Vol. 32, No. 6, pp. 3-12.
       Gee, J. P. (2007), What Video Games Have to Teach us About Learning and
         Literacy, Palgrave Macmillan.
       Glasgow, R. E. (1999), “Evaluating the public health impact of health promotion
          interventions: The RE-AIM framework”, American Journal of Public Health,
          Vol. 89, No. 9, pp. 1322-1327.
       Glasgow, R. E. and K. M. Emmons (2007), “How can we increase translation of
          research into practice? types of evidence needed”, Annual Review of Public
          Health, Vol. 28, pp. 413-433.
       Hägerstrand, T. (1967), Innovation Diffusion as a Spatial Process, University of
         Chicago Press, Chicago, IL.
       Israel, B.A. et al. (1998), “Review of community-based research: assessing
           partnership approaches to improve public health”, Annual Review of Public
           Health, Vol. 19, pp. 173-202.
       Kroeber, A. L. (1940), “Stimulus diffusion”, American Anthropologist, Vol. 42,
          pp. 1-20.
       Latour, B. (1993), We Have Never Been Modern, Harvard University Press,
          Cambridge, MA.
       Lewis, C., R. Perry and A. Murata (2006), “How should research contribute to
         instructional improvement?” Educational Researcher, Vol. 35, No. 3, pp. 3-14.
       Narayan, K. M. et al. (2000), “Translation research for chronic disease: The case
          of diabetes”, Diabetes Care, Vol. 23, pp. 1794-1798.
       Poincaré, J. H. (1890), “Sur le probleme des trios corps et les equations de la
          dynamique” Acta Mathematica, Vol. 13, pp. 1-270.
       Rogers, E. M. (1964), Diffusion of Innovations, Free Press, New York, NY.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
102 – 4. EXTENDING AND SCALING TECHNOLOGY-BASED INNOVATIONS THROUGH RESEARCH: SINGAPORE

      Rose, G. (1992), The Strategy of Preventive Medicine, Oxford University Press,
         New York, NY.
      Schillinger, D. (2007), An Introduction to Effectiveness, Dissemination
         and Implementation Research, University of California San Francisco,
         San Francisco, CA.
      Senge, P. (1990), The Fifth Discipline: The Art and Practice of the Learning
         Organisation, Broadway Business.
      Sung, N. S. et al. (2003), “Central challenges facing the national clinical research
         enterprise”, Journal of the American Medical Association (JAMA), Vol. 289,
         pp. 1278-1287.
      Thompson, M. and D. Wiliam (2008), “Tight but loose: A conceptual framework
         for scaling up school reforms”, in E. Caroline Wylie (ed.), Tight But Loose:
         Scaling Up Teacher Professional Development in Diverse Contexts, ETS,
         New Jersey.
      van den Akker, J. et al. (eds.) (2006), Educational Design Research, Routledge,
         London.
      Wenger, E. (1998), Communities of Practice: Learning, Meaning and Identity,
        Cambridge University Press, Cambridge.
      Woolf, S. H. (2008), “The meaning of translational research and why it matters”,
        Journal of the American Medical Association (JAMA), Vol. 299, pp. 211-213.




                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                                       PART III. PROMISING AVENUES FOR RESEARCH – 103




                                               Part III

                           Promising avenues for research




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                            5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH – 105




                                             Chapter 5

                              The third lever:
                 Innovative teaching and learning research


                     Maria Langworthy (Langworthy Research),
                  Linda Shear and Barbara Means (SRI International)




    The following chapter introduces an important international and comparative
    research effort to develop and contribute a set of tools to measure educators’
    adoption of innovative teaching practices. It looks at the degree to which those
    practices provide students with learning experiences that promote the skills they
    will need to live and work in the 21st century. Still in its initial phase, this major
    research effort represents an important challenge to existing assumptions about
    the lack of connection between teachers’ innovation practices involving technol-
    ogy and students’ achievements.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
106 – 5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH

Introduction

           Today’s ICT-enabled global economy requires a much higher percent-
      age of workers with advanced skills and learning capacities (Partnership for
      21st Century Skills, 2004; Scheuermann and Pedró, 2009). Countries that do
      not develop workers with the skills and capacities to meet new labour market
      dynamics risk falling behind (Wagner, 2008). Throughout history, education
      systems have evolved in relation to changes in the means of production and
      the needs of labour markets for different distributions of skills and knowledge
      (Cole, 2010). The challenges that confront education leaders today are simi-
      larly propelled by economic and social shifts that demand more individuals
      achieve more advanced skills and learning capacities than ever before.
           To address these challenges, there is growing consensus on the need to
      dramatically rethink how learning happens inside and outside schools. Much
      of this debate is centred on the potential for technology to play a more direct
      and central role in student-centred learning (Mitra et al., 2005; Christensen
      et al., 2008). Researchers and advocates suggest that the role of schools and of
      educators should change to meet the needs of new millennium learners today
      (Pedró, 2009; Dede, 2010) and to build on what has been learned through
      research about how people learn (Bransford et al., 1999). Policy makers and
      many education system leaders have supported these directions through their
      budget allocations (i.e. significant investments in ICT) and by supporting pro-
      fessional development programmes for educators aimed at using ICT in the
      classroom. However, these two levers of support do not appear to be enough

                      Figure 5.1. Educational transformation model

                                           Social
                                    and Economic Change

                                      Education System
                                          Change

                                      School Leadership


                                     Innovative Teaching
                                          Practices



                                          Student
                                          Centered
                                          Learning




                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                            5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH – 107



       to compel the widespread educational change and innovation needed to trans-
       form the majority of schools and teachers. In most countries and education
       systems around the world, real change in education is still happening in only
       a very few cases, driven by heroic individuals who innovate their teaching
       practices and their schools in relative isolation. Further change is required
       at the system level to enable more widespread change in how education is
       conceived and realised in the 21st century. Education leaders and researchers
       are beginning to reflect on what other kinds of levers at the system level will
       enable the broad majority of schools and educators to implement change in
       the near term (DeLorenzo et al., 2008; Fullan, 2010).
           Educational practice measures represent a third lever at the system level.
       The need for new student educational achievement assessments that measure
       the new types of skills and capacities needed in the 21st century has become
       well understood. Extensive work and significant progress is already under-
       way in this area, though adoption of new forms of student assessment may
       well take many years (Brinkley et al., 2010). However, less attention has been
       focused on measures that can be used to provide feedback to schools and
       educators, particularly measures of innovative teaching practices that support
       students’ acquisition of 21st century skills.
            This paper describes a new international research project called Innovative
       Teaching and Learning (ITL) Research which aims to develop and contribute a
       set of tools to measure educators’ adoption of innovative teaching practices and
       the degree to which those practices provide students with learning experiences
       that promote the skills they will need to live and work in the 21st century. The
       ITL Research project (www.itlresearch.com) studies what works at the system,
       school, educator, classroom and student levels. Through this investigation, the
       project is developing and testing research methods that measure innovative
       teaching practices and students’ 21st century skills. These methods will be con-
       tributed to the public domain, potentially providing the basis of several new key
       indicators for ICT in education and allowing more consistent evaluations and
       comparable data internationally of progress in effectively integrating ICT into
       teaching and learning by educational systems and by individual schools. The
       paper begins with a description the project’s background, design and methods. It
       then describes the initial policy implications of this project. Throughout the dis-
       cussion, a few of the findings from the pilot year data collection are described,
       but the full report of the pilot year analysis will be published separately.
           ITL Research began with pilot work in Finland, Indonesia, Russia and
       Senegal in 2009, and it is expanding to additional countries for the period
       from 2010 to 2012. The project will provide education policy makers with both
       new measurement tools and descriptions of how technology can be integrated
       into teaching and learning to achieve desired student learning outcomes.
       Microsoft’s Partners in Learning has provided the initial sponsorship for ITL



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
108 – 5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH

      Research, which uses a multi-stakeholder partnership model including each
      participating country’s education policy makers and highly respected local
      research organisations as well as a group of international advisors.

ITL research background

          Educational change is complex and takes place within existing complex
      ecosystems of influences that include national policies and programmes,
      support from local communities, school-specific cultures and leadership,
      and the inexhaustible diversity of individual educators and learners. From an
      investigation standpoint, ITL Research takes a broad look at education eco-
      systems, seeking to contribute to the current understanding of how effective
      transformation of teaching and learning supported by technology is taking
      place. The project is fundamentally focused on the need to measure system
      and school elements in relation to what happens in the classroom. It begins
      with this premise that teaching and learning take place within highly com-
      plex ecosystems where causality for learning outcomes is unlikely to lie in a
      single or even a small set of variables. ITL Research examines the patterns of
      relationships between many variables that have been identified as related to
      learning outcomes in previous research (see logic model, below).
          Within educational ecosystems, ITL’s core focus of investigation is on
      “Innovative Teaching Practices” that provide students with learning experi-
      ences that promote 21st century skills. This focus on teaching practices is

                        Figure 5.2. Innovative Teaching Practices



                                                     Student-Centered Pedagogy




              Innovative                              Extending learning beyond
              Teaching                                      the classroom
              Practices




                                                       ICT used for teaching and
                                                               learning




                                           INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                            5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH – 109



       informed by many studies that suggest having technology in schools does
       not by itself lead to changes in learning outcomes (Dynarski et al., 2007).
       First, having technology available does not mean that educators will use it
       or meaningfully integrate it in teaching and learning (Cuban, Kirkpatrick
       and Peck, 2001; Russell et al., 2003). Second, a significant body of research
       shows that how technology is used can determine whether or not its use
       affects learning outcomes (Wenglinsky, 2005). Third, ITL Research focuses
       on educators and their teaching practices as a key factor in influencing
       learning outcomes based on research that demonstrates the teacher’s effect
       (Darling-Hammond, L., 2010).
            The construct of “Innovative Teaching Practices” was developed based
       on these ideas and an extensive review of existing literature and previ-
       ous research, including leading multinational studies such as the Second
       Information Technology in Education Study (SITES; Law, Pelgrum and
       Plomp, 2006) and the Programme for International Student Assessment
       (PISA; OECD, 2006); frameworks for 21st century teaching and learning
       (e.g. UNESCO, 2008; Government of South Australia, 2008; ISTE, 2007,
       2008); and research on specific constructs related to teaching practices that
       are associated with positive student outcomes (e.g. Bryk, Camburn and Louis,
       1999; Groff and Mouza, 2008).
           The construct of “Innovative Teaching Practices” in the ITL model
       includes dimensions beyond technology use that speak to the larger peda-
       gogical philosophies that inform individual educators’ decisions. More
       specifically, “Innovative Teaching Practices” are characterised by student-
       centred pedagogy, learning opportunities that transcend the school walls,
       and the integration of ICT into teaching and learning. These dimensions are
       explained in more detail below.

ITL research design
           ITL Research will conduct research for three years, developing parallel
       country case studies for a deep investigation of the national and school-level
       factors that shape teaching practices within particular country contexts. At
       the global level, ITL Research will look across the country cases to provide
       information and informed recommendations on how to most effectively
       reshape teaching and learning for students’ acquisition of 21st century skills.
           In 2009-10, the pilot year of ITL Research was carried out in four partici-
       pating countries (Finland, Indonesia, Russia and Senegal) that were selected
       in part to reflect the range of economic, cultural and educational conditions
       around the world. In 2010, Australia, Mexico, the United Kingdom and the
       United States are joining the study.



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
110 – 5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH

          ITL Research’s Global research questions are as follows:
          1. To what extent do innovative teaching practices contribute to
             21st century learning outcomes? 1
          2. What school-level conditions contribute to innovative teaching
             practices?
          3. How are national or regional programme supports associated with
             increases in innovative teaching practices? 2
          These questions contributed to the logic model shown in Figure 5.3
      below. While this model is not a comprehensive picture of all of the influ-
      ences that shape education ecosystems, it represents a range of key constructs
      and dimensions that research and policy directions suggest are particularly
      important, and it seeks to look at these dimensions in relation to one another
      from a systems perspective.

                            Figure 5.3. ITL research logic model

           CONTEXT & INPUTS                            PRACTICES                   OUTCOMES
      National              School & Teacher              Classroom                    Student


                            School Culture
                             and Supports
     Education
       Policy                                           Innovative
                              ICT Access                                            Students’ 21 st
                                                         Teaching
                             and Supports                                           Century Skills
                                                         Practices
      Program
      Supports                 Educator
                               Attitudes




         Each of these constructs and dimensions carries specific descriptions and
      meanings that are used throughout the ITL Research programme and methods:
                 Innovative teaching practices are the focal point of the investiga-
                 tion. As noted above, the construct of innovative teaching practices
                 is defined as including three primary dimensions:
                 -   Student-centred pedagogies. In the ITL model, student-centred
                     pedagogies include practices of teaching and learning that are
                     project-based, collaborative, foster knowledge building, require
                     self-regulation and assessment, and are both personalised



                                               INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                            5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH – 111



                      (allowing for student choice and relevance to the individual
                      student) and individualised (allowing students to work at their
                      own pace and according to their particular learning needs). Each
                      of these elements has a strong base of prior research linking it
                      to positive outcomes for students in terms of development of
                      21st century skills (for example, Bransford, Brown and Cocking,
                      1999; Darling-Hammond et al., 2008).
                 -    Extension of learning outside the classroom. This construct refers
                      to learning activities that reflect the nature of high perform-
                      ing work groups in the 21st century. Learning activities extend
                      beyond the traditional boundaries of the classroom, for example,
                      by including individuals from beyond the classroom community
                      (for example, parents, experts, community members), by provid-
                      ing opportunities for 24/7 learning (for example, research outside
                      the classroom), fostering cross-disciplinary connections, and pro-
                      moting global awareness and cultural understanding.
                 -    ICT integration in teaching and learning. This construct relates
                      to uses of technology by teachers and by students. Because the
                      impact of information and communication technologies (ICT)
                      can vary widely depending on its pedagogical application
                      (Myndigheten For Skölutveckling, 2008), this construct includes
                      a focus on how ICT is used and not simply whether it is used.
                      For example, ITL distinguishes in its measures between basic or
                      rote use of technology and higher-level technology use that takes
                      better advantage of the potential technology offers for learning.
                 Innovative teaching practices are shaped by a host of school and
                 educator-level factors. The model focuses on three overarching
                 constructs:
                 -    School culture and supports that shape teaching practice. For
                      example, research has shown that teacher communities of prac-
                      tice can provide a powerful force for change (Little, 2006) and
                      that school leadership is a critical factor in providing the con-
                      text for innovative practices to emerge and extend throughout a
                      school (Shear et al., 2010).
                 -    ICT access and support, including such factors as location,
                      availability, and functionality of ICT tools. Educators cite lack
                      of technology access and support as a primary barrier to the
                      integration of ICT in teaching and learning (Law, Pelgrum and
                      Plomp, 2006).
                      Educator attitudes shape educational reform in powerful ways.
                      In ITL Research, this construct includes teacher beliefs about


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
112 – 5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH

                  teaching and learning (for example, their beliefs about new vs.
                  traditional pedagogies; Becker and Reil, 2000), teacher motiva-
                  tion and self-efficacy (Gibson and Dembo, 1984), and teacher
                  attitudes towards ICT’s value in teaching and learning.
              School and classroom factors operate in the context of national and
              regional education systems (including structure, policy, vision, and
              capacity) and programme supports (such as professional develop-
              ment) which may be available through programmes sponsored by
              government, NGOs, or the private sector.
              Finally, students’ 21st century skills include broad skills that are
              seen as important goals of innovative teaching practices. The ITL
              Research model characterises these skills as knowledge building,
              problem solving and innovation, skilled communication, collabora-
              tion, self-regulation, and use of ICT for learning.
         These key constructs are described in more detail in the ITL Research
      Design (available at www.itlresearch.com), including specific definitions that
      operationalise each of the constructs for the purposes of ITL Research.
           ITL Research uses a global research design with common methods that
      are carried out by country research teams. SRI International is the global
      research partner responsible for overall research design, methods develop-
      ment, data collection co-ordination, and global results synthesis. This role
      ensures that methods and overall design parameters are developed centrally
      and implemented consistently across countries. At the same time, a research
      partner in each country is engaged to carry out the local research and par-
      ticipates in the global network. Research partners are selected through a
      competitive proposal process, and each represents a leading independent or
      university-based research organisation in its country (see list of local partners
      below). The country partners create local design plans and adapt instruments
      as required to ensure that the research is appropriate to the country context
      and serves local as well as global needs.
          The target student age range for this research is 11 to 14, so the design
      in each country involves the level of schooling that serves most students at
      these ages. In different countries, this is either upper elementary or lower
      secondary school.
         Each of these constructs and dimensions carries specific descriptions
      and meanings that are used throughout the ITL Research programme and
      methods:
              Innovative teaching practices are the focal point of the investigation.
              As noted above, the construct of innovative teaching practices is
              defined as including three primary dimensions:



                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                            5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH – 113



                 Student-centred pedagogies. In the ITL model, student-centred
                 pedagogies include practices of teaching and learning that are
                 project-based, collaborative, foster knowledge building, require self-
                 regulation and assessment, and are both personalised (allowing for
                 student choice and relevance to the individual student) and individu-
                 alised (allowing students to work at their own pace and according to
                 their particular learning needs). Each of these elements has a strong
                 base of prior research linking it to positive outcomes for students in
                 terms of development of 21st century skills (for example, Bransford,
                 Brown and Cocking, 1999; Darling-Hammond et al., 2008).
                 Extension of learning outside the classroom. This construct refers
                 to learning activities that reflect the nature of high-performing work
                 groups in the 21st century. Learning activities extend beyond the
                 traditional boundaries of the classroom, for example, by including
                 individuals from beyond the classroom community (for example, par-
                 ents, experts, community members), by providing opportunities for
                 24/7 learning (for example, research outside the classroom), fostering
                 cross-disciplinary connections, and promoting global awareness and
                 cultural understanding.
                 ICT integration in teaching and learning. This construct relates to
                 uses of technology by teachers and by students. Because the impact
                 of information and communication technologies (ICT) can vary
                 widely depending on its pedagogical application (Myndigheten For
                 Skölutveckling, 2008), this construct includes a focus on how ICT
                 is used and not simply whether it is used. For example, ITL distin-
                 guishes in its measures between basic or rote use of technology and
                 higher-level technology use that takes better advantage of the poten-
                 tial technology offers for learning.
                 Innovative teaching practices are shaped by a host of school and edu-
                 cator-level factors. The model focuses on three overarching constructs:
                 -    School culture and supports that shape teaching practice. For
                      example, research has shown that teacher communities of prac-
                      tice can provide a powerful force for change (Little, 2006) and
                      that school leadership is a critical factor in providing the con-
                      text for innovative practices to emerge and extend throughout
                      a school (Shear et al., 2010). ICT access and support, including
                      such factors as location, availability, and functionality of ICT
                      tools. Educators cite lack of technology access and support as a
                      primary barrier to the integration of ICT in teaching and learn-
                      ing (Law, Pelgrum and Plomp, 2006). Educator attitudes shape
                      educational reform in powerful ways. In ITL Research, this
                      construct includes teacher beliefs about teaching and learning


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
114 – 5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH

                   (for example, their beliefs about new vs. traditional pedagogies;
                   Becker and Reil, 2000), teacher motivation and self-efficacy
                   (Gibson and Dembo,1984), and teacher attitudes towards ICT’s
                   value in teaching and learning.
              -    School and classroom factors operate in the context of national
                   and regional education systems (including structure, policy,
                   vision, and capacity) and programme supports (such as profes-
                   sional development) which may be available through programmes
                   sponsored by government, NGOs, or the private sector.
              -    Finally, students’ 21st century skills include broad skills that are
                   seen as important goals of innovative teaching practices. The ITL
                   Research model characterises these skills as knowledge building,
                   problem-solving and innovation, skilled communication, collabo-
                   ration, self-regulation and use of ICT for learning.
         These key constructs are described in more detail in the ITL Research
      Design (available at www.itlresearch.com), including specific definitions that
      operationalise each of the constructs for the purposes of ITL Research.
           ITL Research uses a global research design with common methods that
      are carried out by country research teams. SRI International is the global
      research partner responsible for overall research design, methods develop-
      ment, data collection co-ordination, and global results synthesis. This role
      ensures that methods and overall design parameters are developed centrally
      and implemented consistently across countries. At the same time, a research
      partner in each country is engaged to carry out the local research and par-
      ticipates in the global network. Research partners are selected through a
      competitive proposal process, and each represents a leading independent or
      university-based research organisation in its country (see list of local partners
      below). The country partners create local design plans and adapt instruments
      as required to ensure that the research is appropriate to the country context
      and serves local as well as global needs.
          The target student age range for this research is 11 to 14, so the design in each
      country involves the level of schooling that serves most students at these ages.
      In different countries, this is either upper elementary or lower secondary school.
           International workshops are conducted annually to train the local research
      teams on the constructs and research methods of ITL Research. Year 1 and
      Year 2 of the research include researcher collaboration and refinement of the
      methods based on prior year experience and findings in participating countries.
      Face-to-face researcher workshops are supplemented by regular telephone con-
      ferences and ongoing electronic communications between the global and coun-
      try research partners in order to support and monitor ongoing consistency and
      research quality.


                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                 5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH – 115



                                     Table 5.1. Global ITL research team

Country                      Role                                           Organisation

Global        Project sponsor                    Microsoft Partners in Learning

USA           Project management and oversight   Langworthy Research

              Global research lead               Center for Technology in Learning, SRI International

Finland       Country research partner           Agora Center and Finnish Institute for Educational Research,
                                                 University of Jyväskylä

              Government partner                 National Board of Education

Indonesia     Country research partner           Centre for Strategic and International Studies

              Government partner                 Ministry of National Education (MONE)

Russia        Country research partner           Institute of New Technologies, Moscow

              Government partner                 The Academy for Teachers Training and Professional Retraining
                                                 for Educators (APKiPPRO)

Senegal       Country research partner           Association of Teachers and Researchers of ICT in Education and
                                                 Training

              Government partner                 National Ministry of Education

Mexico        Country research partner           Proyecto Educativo SC

              Government partner                 Secretaría de Educación Pública

UK            Country research partner           London Knowledge Lab

              National partner                   Specialist Schools and Academies Trust


Methods, sample and outputs

              For a systemic understanding of education ecosystems following the ITL
          Research logic model, it is necessary to collect data at multiple levels within
          the system, from the national or regional context to the school, educator,
          classroom and student levels. ITL Research uses a mixed methods approach,
          with an integrated design that provides consistent definitions and measures of
          essential constructs to ensure consistency across instruments. This multi-year
          study will collect data from each source annually to offer analysis of change
          over time.
               ITL Research integrates these methods to provide data on the constructs
          in the logic model. Some constructs are measured through multiple methods.


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
116 – 5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH

       For example, “Innovative Teaching Practices” are investigated through
       teacher surveys and interviews, classroom observations by trained research-
       ers, and artifacts of classroom practice in the form of the learning activities
       that teachers ask students to carry out.
           In each country, the samples of schools and educators are designed to
       reflect both what is considered innovative practice in that national context
       and “school as usual.” Approximately 25 schools are selected in each country
       to participate in this research, to achieve a sample size of 650+ responses to
       the ITL teacher survey with a 70% to 80% response rate within each school.
       Schools in the sample are generally clustered in two or three geographic areas
       to make school site visits economically feasible.
           ITL Research will also borrow from the model of action research. It sees
       research not only as a means of externally studying a situation, but as a direct

                             Table 5.2. ITL research methods summary

Method                                              Purpose                               Levels of System Addressed

Surveys of teachers       Provides quantitative data from a large sample of respond-      National/regional, school,
and school leaders        ents to describe teacher and school leader experiences of       educator, classroom
                          national context and programmes, school culture and sup-
                          ports, and self-reported beliefs and practices.

Interviews with school    Provides richer and more contextualised data from a smaller National/regional, school,
leaders and teachers      sample of respondents on participants’ experience of reform educator, classroom
                          and the factors that shape it.

Analysis of Learning      Uses artefacts of actual classroom practice to measure          Classroom, student
Activities and Student    opportunities for 21st century learning according to a set of
Work (LASW)               dimensions that are defined consistently across countries
                          and classrooms.

Classroom observations Allows researchers to observe and describe classroom               Classroom
                       environments and learning activities in common ways
                       across country and school contexts.

Student focus groups3     Elicits data on students’ experience of teaching and learning Classroom, student
                          from a small sample of students.

Interviews with national/ Provides system-level data on education goals, national         National/regional
regional education        programmes and strategies, and challenges within the
leaders                   country.

Achievement data          Where available, provides data on student academic              Student
                          achievement based on national exams.




                                                         INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                            5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH – 117



       tool of engagement with teachers and schools. They become both partners in
       developing the methods and constituents of the research results who have an
       interest in “moving their numbers”. ITL’s initial plan was modified after the
       pilot year data collection to more directly incorporate the idea of the schools
       and educators as research partners. This direction came in part due to the
       difficulty of engaging schools and teachers in research projects in which
       they were not true stakeholders. It was also informed by one of the findings
       of the pilot year data analysis. Analysis of the ITL Research Teacher survey
       with teachers across the four countries showed that among seven different
       types of professional development, participation in “individual or collabora-
       tive research on a topic of interest to you professionally” had the strongest
       correlation with innovative teaching.4 This suggests that educators who
       engage directly in research appear to also practice innovative teaching more
       frequently. Research, after all, is simply learning by another name. This led to
       the refinement of the ITL Research design after the pilot year to more directly
       engage schools and teachers in the research project by sharing the data results
       from their school with them and engaging them in professional development
       discussions related to the research methods and concepts behind them.
           Finally, ITL Research will produce three categories of outputs based on
       the data collection efforts worldwide:
            1. Research findings. Reports, journal publications and conference
               presentations at the global and country levels. These will begin to
               be published in the autumn of each year of the project based on the
               previous year’s data collection and analysis.
            2. Quantitative data that can be used by other researchers. Data from
               the ITL Teacher and School Leader surveys will be made available
               to interested educational researchers for further analysis (request at
               www.itlresearch.com). Over time, the project hopes to establish a
               community of international researchers and practitioners participat-
               ing in the project, using the data and methods for further investiga-
               tions and to inform professional development efforts.
            3. Methods. As mentioned above, one of the major contributions of this
               project is the development of a set of tested and reliable methods for
               measuring innovative teaching and learning that can be used interna-
               tionally. The instruments from each method and associated training
               and analysis materials will be made publicly available, though they
               will not be formally published until the conclusion of the project
               as each year refinements to each method are expected. Pilot year
               instruments are available online at www.itlresearch.com. Microsoft’s
               Innovative Schools Program has already adopted the teacher and
               school leader surveys from ITL to establish a school-level evaluation
               system for the 40+ schools in the programme internationally. This


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
118 – 5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH

                 school-level evaluation system based on ITL methods will be made
                 an online service available at no cost to interested schools interna-
                 tionally in 2011 through Microsoft’s Partners in Learning Network
                 (www.partnersinlearningnetwork.com). The surveys and reports will
                 be available in local languages.

            Figure 5.4. ITL research timeline – annual milestones for 2010-2012

     Jan          April         July/Aug       Aug/Sept             Sept            Oct             Nov/Dec



   Data       Data collection   Analysis    Research methods     International   Final reports   Revised methods
   collection ends/analysis     reporting   revised,             workshop for    published,      published.
   in schools begins            begins      Policy stakeholder   all research    data made       Data collection
   under way                                meetings             partners        available       begins




Policy implications

           At conferences around the world, education researchers and government
      policy-makers have been discussing and examining what teachers and schools
      need to do to capitalise on ICT to help students attain 21st century learning
      goals. These leaders have been encouraging more innovative teaching and
      learning through their communications, their budget allocations for technol-
      ogy and their support for professional development focused on integrating
      technology in teaching and learning. In most cases, governments have not
      been measuring the degree to which teachers and schools implement changes
      in teaching practices. And very few governments have been positioned to
      measure the impact of changes in teaching practices on students’ acquisition
      of new skills and capacities. Research supports the adage “you get what you
      measure”. Lack of access to measures of innovative teaching and of students’
      21st century skills hampers school improvement efforts and government ini-
      tiatives to drive change.
           One of the key findings of the OECD’s Teaching and Learning International
      Survey (TALIS) was the significant positive impact that school evaluations and
      teacher appraisals have on both teachers’ job satisfaction and their professional
      development. This study, which took place in 23 countries, highlighted the sig-
      nificance of the specific content of teacher appraisals and feedback:
            The greater the emphasis on specific aspects of teacher appraisal and
            feedback, the greater the change in teachers’ practices to improve
            their teaching. In some instances, more emphasis in school evalua-
            tions on certain aspects of teaching is linked to an emphasis on these



                                                       INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                            5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH – 119



            aspects in teacher appraisal and feedback, which, in turn, leads to
            further changes in teachers’ reported teaching practices. In these
            instances, the framework for the evaluation of education appears to
            be operating effectively (OECD, 2009).
           However, roughly three quarters of teachers in the study reported they
       would receive no recognition for being more innovative in their teaching,
       suggesting that teacher appraisals are not currently aligned with claimed
       system goals for innovating teaching and learning. Qualitative findings from
       ITL Research’s pilot year data collection also suggest that teachers and school
       leaders believe that their schools and practices are currently evaluated based
       on traditional areas of focus and content. This evaluation structure serves as
       a kind of perceptual barrier that protects the majority of teachers as they con-
       tinue teaching practices in traditional ways. Even in an education system that
       allows a high degree of school autonomy, such as that of Finland, educators’
       practices are framed by the traditional learning goals articulated in policy:
            Not all teachers are willing to develop. […] Some teachers support
            strongly traditional teaching (ITL School Leader Interview, Finland).
            This is related to the teachers’ high “self-accountability” on the
            normative goals embedded in the school curriculum in national and
            local levels. All teachers said in their interviews that they have a full
            autonomy to teach in the ways they see appropriate. The goals of
            national curriculum focus on subject learning. It is understandable
            that teachers attempt to carefully follow the stated goals. In such a
            situation, it is most convenient and safe to stick to the well-proven
            methods (Norrena and Kankaanranta, 2010).
           Since most systems evaluate performance based on more traditional
       teaching and learning goals, education systems de facto support the inertia
       of existing practices that are designed to achieve traditional learning goals.
       Although education systems espouse education for the 21st century as a goal,
       many countries lack the kind of systemic alignment around that goal that will
       be needed for educational change.
            Certainly there is a need to introduce clearly defined learning goals and
       assessments for students on the skills and capacity areas associated with
       life and work in the 21st century.5 As described above, work on new student
       learning goals and assessments is ongoing in many different initiatives.
       However, systems also need to introduce innovative teaching practice goals,
       where the practices are directly aligned with the new learning goals and
       assessments. The achievement of these new teaching practice goals should
       be monitored and measured to ensure credibility in the eyes of teachers and
       school leaders. Thus, governments can support the transition to more inno-
       vative teaching practices by developing and promoting school and teacher
       evaluations that include measures of innovative teaching practices.


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
120 – 5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH

          ITL Research can serve to address this need for new emphases in teacher
      evaluations by providing a consistent and tested set of methods for measur-
      ing teachers’ practices with respect to dimensions of innovation (described
      above). By becoming involved in the ITL Research project, governments can
      have access to these methods and can further test and adapt them for profes-
      sional development purposes or to measure their teachers’ practices and how
      they are changing over time. If many governments choose to use common
      definitions and methods growing out of the ITL work, their data on teacher
      practice and student 21st century skills will have added meaning because
      they will be able to compare it to that of other countries. In this way, ITL-
      based methods and measures could become the basis of new international
      indicators of the use and effect of ICT for learning (Johannessen, 2009).
      Finally, the ITL research model and research results should be observed by
      governments and education system leaders for their potential to inform policy
      directions on ICT in education. This research can bring new insights to gov-
      ernments and schools’ efforts to innovate in education. ITL can also provide
      a common language that supports international dialogue on what is working,
      what needs work, and how much progress is being made toward educational
      transformation.
           To expand the scale of innovative educational transformation, govern-
      ments, systems and schools need ways of measuring progress in innovative
      teaching and its impact on students’ learning. If educational systems and
      schools begin to define and measure educational progress in a consistent
      way that includes innovative practice, it will signal a clear structural shift
      to school leaders and teachers that the system has a new set of goals more
      aligned with the needs and requirements of 21st century economies and socie-
      ties. The goal of ITL Research is to contribute a set of consistent and tested
      methods to enable governments, education systems and schools worldwide to
      begin on this path.




                                           INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                            5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH – 121




                                                 Notes

1.   For purposes of this study, “21st century learning outcomes” are defined as the fol-
     lowing set of skills: knowledge building, problem-solving and innovation, skilled
     communication, collaboration, self-regulation, and use of technology for learning.
     These skills are defined more explicitly and in relation to the research methods
     in the ITL Research Design document and through the Learning Activities and
     Student Work Coding Guides, both available at www.itlresearch.com.
2.   In the pilot year, this research question is being considered at a high level, and
     pilot data collected will be used to determine how much depth is appropriate for
     research on this question in subsequent years.
3.   Student focus groups were not conducted in the pilot year but are planned for 2010
     and 2011.
4.   Because results cited in this report are based on pilot data, they should be consid-
     ered preliminary and subject to confirmation in the main study.
5.   ITL Research is currently in discussions with the ATC21S project about partner-
     ing the two projects in countries in which both projects are underway, such as
     Australia, Finland, the United Kingdom and the United States.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
122 – 5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH




                                      References

      Baker, E. L. and J. L. Herman (2003), “A distributed evaluation model”, in
         G. Haertel and B. Means (eds.), Evaluating Educational Technology, Teachers
         College Press, New York, NY.
      Balanskat, A., R. Blamire and S. Kefala (2006), The ICT Impact Report: A
         Review of studies of ICT Impact on Schools in Europe, European Schoolnet,
         retrieved 15 August 2009 from http://insight.eun.org/ww/en/pub/insight/misc/
         specialreports/impact_study.htm.
      Bransford, J. D., A. L. Brown and R. R. Cocking (1999), How People Learn:
         Brain, Mind, and Experience, National Academy Press, Washington, DC.
      Brinkley, M. et al. (2010), “Whitepaper: Developing 21st century skills and
         assessments”, draft copy, Assessment and Teaching of 21st century Skills,
         retrieved from http://atc21s.org/white-papers/, January 2010.
      Christensen, C., C. Johnson, and M. Horn (2008), Disrupting Class: How
         Disruptive Innovation Will Change the Way the World Learns, McGraw Hill,
         New York, NY.
      Cole, Michael (2010), “What’s culture got to do with It? Educational research as a
         necessarily interdisciplinary exercise”, AERA Distinguished Lecture, AERA
         Annual Meeting, 01 May 2010, Denver, CO.
      Cuban, L., H. Kirkpatrick and C. Peck (2001), “High access and low use of
        technologies in high school classrooms: Explaining an apparent paradox”,
        American Educational Research Journal, Vol. 38, No. 4, pp. 813-834.
      Darling-Hammond, L. (2010), The Flat World and Education, Teachers College
         Press, New York, NY.
      Darling-Hammond, L. et al. (2008), Powerful Learning: What We Know About
         Teaching for Understanding, Jossey-Bass, San Francisco, CA.
      Dede, Chris (2010), “Transforming schooling via the 2010 national educational
        technology plan”, Teachers College Record, date published: 10 June 2010,
        retrieved on 28 June 2010 from www.tcrecord.org, ID Number 15998.




                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                            5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH – 123



       DeLorenzo, R. et al. (2008), Delivering on the Promise: The Education
         Revolution, Solution Tree.
       Dynarski, M. et al. (2007), Effectiveness of Reading and Mathematics Software
         Products: Findings from the First Student Cohort, report to Congress,
         NCEE 2007-4006, U.S. Department of Education, Washington, DC.
       Fullan, Michael (2010), All Systems Go: The Change Imperative for Whole
          System Reform, Corwin Press.
       Gibson, S. and M. Dembo (1984), “Teacher efficacy: A construct validation”,
          Journal of Educational Psychology, Vol. 76, No. 4, pp. 569-582.
       Government of South Australia (2008), eStrategy Framework, Adelaide, The
         State of South Australia, Department of Education and Children’s Services,
         retrieved 15 July 2009 from www.decs.sa.gov.au/learningtechnologies/files/
         links/eStrategy_Framework_screen.pdf.
       ISTE (2007), National Educational Technology Standards for Students,
          2nd edition, International Society for Technology in Education, Eugene, OR.
       ISTE (2008), National Educational Technology Standards for Teachers,
          2nd edition, International Society for Technology in Education, Eugene, OR.

           Multi-channel and multi-method?” Assessing the Effects of ICT in Education:
           Indicators, Criteria and Benchmarks for International Comparisons,
           European Union and OECD, Brussels and Paris.
       Kozma, R. B. (ed.), 2003, Technology, Innovation, and Educational Change:
         A Global Perspective, International Society for Technology in Education,
         Eugene, OR.
       Law, N., W. Pellgrum and T. Plomp (2006), Pedagogy and ICT Use in Schools
          Around the World: Findings from the IEA SITES 2006 Study, IEA,
          Hong Kong.
       Little, J. W. (2006), “Inside teacher community: Representations of classroom
           practice”, Teachers College Record, Vol. 105, No. 6, pp. 913-945.
       Mitra, S. et al. (2005), “Acquisition of computing literacy on shared public
          computers: Children and the ‘hole in the wall’”, Australasian Journal of
          Educational Technology, Vol. 21, No. 3, pp. 407-426, retrieved 28 June 2010,
          www.ascilite.org.au/ajet/ajet21/mitra.html.
       Means, B. et al. (2003), “Studying the cumulative impacts of educational
         technology”, in G. Haertel and B. Means (eds.), Evaluating Educational
         Technology, Teachers College Press, New York, NY.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
124 – 5. THE THIRD LEVER: INNOVATIVE TEACHING AND LEARNING RESEARCH

      Myndigheten For Skölutveckling (2008), Effective Use of ICT in Schools:
        Analysis of International Research, The Swedish National Agency for School
        Improvement, Stockholm.
      Norrena, J. and M. Kankaanranta (2010), “Pilot year ITL qualitative report:
         Finland”, internal working document available upon request from authors.
      OECD (2006), Are Students Ready for a Technology-rich World? What PISA
        Studies Tell Us, OECD Publishing.
      OECD (2009), Creating Effective Teaching and Learning Environments: First
        Results from TALIS, OECD Publishing.
      Partnership for 21st Century Skills (2004), Framework for 21st Century
         Learning, retrieved 15 July 2009 from www.21stcenturyskills.org.
      Pedró, F. (2009), Reframing the Policy Expectations About Technology in
         Education, OECD Publishing.
      Russell, M. et al. (2003), “Examining teacher technology use: implications
         for pre-service and in-service teacher preparation”, Journal of Teacher
         Education, Vol. 54, No. 4, pp. 297-310.
      Scheuermann, F. and F. Pedró (eds.) (2009), Assessing the Effects of ICT
         in Education: Indicators, Criteria and Benchmarks for International
         Comparisons, JRC/European Commission and OECD.
      Shear, L. et al. (2010), The Microsoft Innovative Schools Program Year 2
         Evaluation Report, Microsoft, Seattle, WA.
      UNESCO (2008), UNESCO’s ICT Competency Standards for Teachers, retrieved
        15 July 2009 from http://cst.unesco-ci.org/sites/projects/cst/default.aspx.
      Wagner, T. (2008), The Global Achievement Gap: Why Even Our Best Schools
        Don’t Teach the New Survival Skills our Children Need – And What We Can
        Do About It, Basic Books, New York, NY.
      Wenglinsky, H. (2005), Using Technology Wisely: The Keys to Success in
        Schools, Teachers College Press, New York, NY.




                                           INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                        6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS – 125




                                             Chapter 6

           Design research on technology-based innovations


                            Jan van den Akker
     SLO (Netherlands Expertise Centre for Curriculum Development) and
                   University of Twente, the Netherlands




    The curriculum is, along with assessment, a key driver for education because
    curricula define goals, content and, in some cases, also the methods of teaching
    and learning. This chapter suggests that we regard curricula as a roadmap for
    education. In particular, the author looks at the benefits and limitations of cur-
    riculum design research and how its results have the potential to make an impor-
    tant contribution to curriculum policies and development. Rather than attempting
    to implement elaborate and complete interventions, a process whereby one comes
    to (successive) prototypes that increasingly meet the innovative aspirations and
    requirements is suggested. The process is often iterative, cyclic or spiral: analysis,
    design, evaluation and revision activities are iterated until a satisfying balance
    between ideals and realisation has been achieved. The author concludes with
    some specific research characteristics that would strengthen the growth of knowl-
    edge through design research.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
126 – 6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS

Introduction

          Over the last decades and building upon experiences worldwide, a lot
      of research-based knowledge has grown about the potential added value of
      Information and Communication Technology (ICT) for education (see Voogt
      and Knezek, 2008). Although it is not easy to come to general conclusions
      about the uptake and impact of ICT on educational practices, the overall pic-
      ture is rather sobering. There are scattered promising pockets of success, but
      the potential of ICT is still rarely realised. Integration of ICT use in schools
      remains slow and transfer of exemplary classroom practices appears to be
      limited. ICT seems a typical illustration of the statement by Hargreaves and
      Fink (2006) that change in education is easy to propose, hard to implement,
      and extraordinarily difficult to sustain. Thus, notwithstanding great ambi-
      tions and many investments, there is still a long way to go.
           How may educational research help in addressing these challenges?
      Those contributions vary over different research approaches. Plomp (2009)
      distinguishes various questions, aims and functions of research, such as:
      to describe, to compare, to evaluate, to explain, to predict, to design and
      to develop. One may also discern various primary orientations of research:
      theory, practice or policy. Much policy-oriented research on ICT occurs
      through surveys, monitoring and assessment, focusing on (descriptive)
      measures about actual practices and outcomes of ICT. However, the cen-
      tral orientation in this paper will be on research that aims for ICT-related
      improvement and innovation of education, under the label of design research.
      Design reserach in education is a relatively new approach, with its roots in
      the early nineties, and since then documented in a number of special issues
      of prominent journals (for example: Educational Researcher, Vol. 32, No. 1,
      2003; Educational Psychologist, Vol. 39, No. 4, 2004; Journal of the Learning
      Sciences, Vol. 13, No. 1, 2004; Educational Technology, Vol. 45, No. 1, 2005)
      and a number of books (for example: van den Akker et al., 2006; and Kelly,
      Lesh and Baek, 2008).
           Moreover, this paper takes a broad curricular perspective as that appears
      very helpful for systematically addressing the challenge of strengthening ICT
      in education because it transcends the often too narrow and isolated approach
      of studying the use and effects of ICT in teaching and learning.
           Curricular Design Research intentionally combines two fields: “cur-
      riculum design” and “design research”. In particular, it focuses on how design
      research can increase the quality of curriculum design and development (also
      in relation to ICT potential). Moreover, it illustrates how the relevance of edu-
      cational research – a widely debated issue – can benefit from a connection to
      curriculum policies and practices.




                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                        6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS – 127



            Given this aim it helps to have a number of basic concepts and ana-
       lytical perspectives available that can structure curricular deliberations and
       reduce the complexity of curriculum tasks. The initial focus in this chapter
       (building on van den Akker, 2003; and Thijs and van den Akker, 2009) is
       on summarising a set of concepts and perspectives that help to increase the
       transparency and balance of curriculum analysis, development and discourse.
       Then, the focus will shift towards (curricular) design research (building on
       van den Akker, 1999, and van den Akker et al., 2006).

Curriculum: What’s in a name?

            When there is a myriad of definitions of a concept in the literature (as
       with curriculum), it is often difficult to keep a clear focus on its essence. In
       those cases it often helps to search for the etymological origin of the concept.
       The Latin word “curriculum” (related to the verb currere, i.e. running) refers
       to a “course” or “track” to be followed. In the context of education, where
       learning is the central activity, the most obvious interpretation of the word
       “curriculum” is then to view it as a course, trajectory or “plan for learning”
       (cf. Taba, 1962). This very short definition (reflected in related terms in many
       languages) limits itself to the core of all other definitions, permitting all sorts
       of elaborations for specific educational levels, contexts, and representations.
       Obviously, contextual specification is always needed in curriculum conversa-
       tions to clarify the perspective.
            Given this simple definition, a differentiation between various levels of
       the curriculum has proven to be very useful when talking about curricular
       activities (policy making; design and development; evaluation and implemen-
       tation). The following distinctions appear to be helpful:
                 International/comparative (or supra level)
                 System/society/nation/state (or macro) level (e.g. national syllabi or
                 core objectives)
                 School/institution (or meso) level (e.g. school-specific curriculum)
                 Classroom (or micro) level (e.g. textbooks, instructional materials)
                 Individual/personal (or nano) level
           The supra level usually refers to international debates or agreements
       on aims and quality of education, sometimes fuelled by outcomes of inter-
       nationally comparative studies. Curriculum development at the supra level
       is usually of a “generic” nature, while “site-specific” approaches are more
       applicable for the levels closer to school and classroom practice. Moreover,
       the process of curriculum development can be seen as narrow (developing
       a specific curricular product) or broad (a long-term, ongoing process of


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
128 – 6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS

      curriculum improvement, often including many related aspects of educational
      change, e.g. teacher education, school development, and examinations). In
      order to understand problems of curriculum decision-making and enactment,
      a broader description of curriculum development is often most appropriate:
      usually a long and cyclic process with many stakeholders and participants; in
      which motives and needs for changing the curriculum are formulated; ideas
      are specified in programmes and materials; and efforts are made to realise the
      intended changes in practice.
          Moreover, curricula can be represented in various forms. Clarification of
      those forms is especially useful when trying to understand the problematic
      efforts to change the curriculum. A common broad distinction is between the
      three levels of the “intended”, “implemented” and “attained” curriculum. A
      more refined typology (van den Akker, 2003) is outlined in Table 6.1.

                    Table 6.1. Typology of curriculum representations

      INTENDED       Ideal            Vision (rationale or basic philosophy underlying a curriculum)
                     Formal/Written   Intentions as specified in curriculum documents and/or materials

      IMPLEMENTED    Perceived        Curriculum as interpreted by its users (especially teachers)
                     Operational      Actual process of teaching and learning (also:
                                      curriculum-in-action)

      ATTAINED       Experiential     Learning experiences as perceived by learners
                     Learned          Resulting learning outcomes of learners


           Traditionally, the intended domain refers predominantly to the influence
      of curriculum policy makers and curriculum developers (in various roles), the
      implemented curriculum relates especially to the world of schools and teach-
      ers, and the attained curriculum has to do with the students.
          Besides this differentiation in representations, curriculum problems can
      be approached from various analytical angles. For example, Goodlad (1994)
      distinguishes the following three different perspectives:
          1. Substantive, focusing on the classical curriculum question about what
             knowledge is of most worth for inclusion in teaching and learning.
          2. Technical-professional, referring to how to address tasks of curricu-
             lum development.
          3. Socio-political, referring to curriculum decision-making processes,
             where values and interests of different individual and agencies are
             at stake.


                                                 INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                          6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS – 129



           Some might argue that this list is too limited as it refers especially to cur-
       riculum issues for “traditional” planning for learning in schools, and does not
       include the more “critical” perspectives that are amply present in curriculum
       theory literature (e.g. Pinar et al., 1995). However, from a primary interest in
       curriculum improvement, the three perspectives seem useful and appropriate.
           Moreover, for the sake of technology-based innovations, it is important
       to notice that the various concepts, levels, perspectives and arguments in this
       section all have meaningful implications for a comprehensive approach.

The vulnerable curriculum spider web

            One of the major challenges for curriculum improvement is creating
       balance and consistency between the various components of a curriculum
       (i.e. plan for learning). What are those components? The relatively simple
       curriculum definition by Walker (2003) includes three major planning ele-
       ments: content, purpose and organisation of learning. However, curriculum
       design and implementation problems have taught us that it is wise to pay
       explicit attention to a more elaborated list of components. Elaborating on
       various typologies, we have come to adhere to a framework (see Table 6.2)
       of ten components that address ten specific questions about the planning of
       student learning.


                                  Table 6.2. Curriculum components

                Rationale or Vision          Why are they learning?

                Aims and Objectves           Toward which goals are they learning?

                Content                       What are they learning?

                Learning Activities          How are they learning?

                Teacher Role                  How is the teacher facilitating learning?

                Materials and Resources      With what are they learning?

                Grouping                      With whom are they learning?

                Location                     Where are they learning?

                Time                          When are they learning?

                Assessment                    How to measure how far learning has progressed?




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
130 – 6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS

          The “rationale” (referring to overall principles or central mission of
      the plan) serves as major orientation point, and the nine other components
      are ideally linked to that rationale and preferably also consistent with each
      other. For each of the components many sub-questions are possible, not only
      on substantive issues (see the next section) but also, for example, on such
      “organisational” aspects as following:
              Grouping
              -   How are students allocated to various learning trajectories?
              -   Are students learning individually, in small groups or whole-
                  class?
              Location
              -   Are students learning in class, in the library, at home or elsewhere?
              -   What are the social/physical characteristics of the learning envi-
                  ronment?
              Time
              -   How much time is available for various subject matter domains?
              -   How much time can be spent on specific learning tasks?

                              Figure 6.1. Curricular spider web
                                                                                   nt
                                                                                 me
                                                                              ess
                                                     Time



                                                                           Ass




                                                                                                          s
                                                                                                      tive
                              Lo                                                                  jec
                                 c   ati                                                      & Ob
                                                                                       s
                                         o   n                                      Aim

                                                            Rationale
                                                                                        Con
                               Grouping                                                    ten
                                                                                              t
                                                                               Le
                                                                                 arn
                                                      s
                                                   ce



                                                                  le


                                                                                     ing
                                                 ur



                                                              Teacher ro
                                             so




                                                                                        act
                                       Re
                                     &




                                                                                           ivi
                                  ls
                              ria




                                                                                            tie
                              e




                                                                                              s
                           at
                          M




                                                                    INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                        6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS – 131



            Our preferential visualisation of the ten components is to arrange them as
       a spider web (Figure 6.1), not only illustrating its many interconnections but
       also underlining its vulnerability. Thus, although the emphasis of curriculum
       design on specific components may vary over time, eventually some kind of
       alignment has to occur to maintain coherence. A striking example is the trend
       toward integration of ICT in the curriculum with (usually) initial attention to
       changes in materials and resources. Many implementation studies have exem-
       plified the need for a more comprehensive approach and systematic attention
       to the other components before one can expect robust changes.
            The spider web also illustrates a familiar expression: every chain is as
       strong as its weakest link. That seems another very appropriate metaphor for
       a curriculum, pointing to the complexity of efforts to improve the curriculum
       in a balanced, consistent and sustainable manner.

Perspectives on substantive choices

           A classic approach to the eternal question of what to include in the cur-
       riculum (or even more difficult as well as urgent: what to exclude from it) is
       to search for a balance between three major sources or orientations for selec-
       tion and priority setting:
                 Knowledge: what is the academic and cultural heritage that seems
                 essential for learning and future development?
                 Society: which problems and issues seem relevant for inclusion from
                 the perspective of societal trends and needs?
                 Learner: which elements seem of vital importance for learning from
                 the personal and educational needs and interests of the learners
                 themselves?
           Answers to these questions usually constitute the rationale of a cur-
       riculum. Inevitably, choices have to be made, usually involving compromises
       between the various orientations (and their respective proponents and pressure
       groups). Oftentimes, efforts fail to arrive at generally acceptable, clear and
       practical solutions. The result of adding up all kinds of wishes is that curricula
       tend to get overloaded and fragmented. Implementation of such incoherent
       curricula eventually tends to lead to student frustrations, failure and dropout.
           How to create a better curriculum balance? Easy answers are not avail-
       able, but a few alternatives seem to have some promise. First, in view of the
       multitude of (academic) knowledge claims, it sometimes helps to reduce the
       big number of separate subject domains to a more limited number of broader
       learning areas, combined with sharper priorities in aims for learning (focus-
       ing on basic concepts and skills).



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
132 – 6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS

           Second, referring to the avalanche of societal claims, more interac-
      tion between learning inside and outside the school may reduce the burden.
      However, the most effective response is probably to be more selective in react-
      ing to all sorts of societal problems. As Cuban (1992) phrased it clearly: schools
      should not feel obliged to scratch the back of society every time society has an
      itch.
          And third, about the learners’ perspective: worldwide, many interest-
      ing efforts are ongoing to make learning more challenging and intrinsically
      motivating by moving from traditional, teacher and textbook-dominated
      instruction towards more meaningful and activity-based learning approaches.
          Obviously, ICT creates new challenges but also offers new opportunities
      for addressing the substantive dilemmas described.

Development strategies

           To sketch curriculum development as a problematic domain is actually
      an understatement. From a socio-political stance, it often seems more appro-
      priate to describe it as a war zone, full of conflicts and battlefields between
      stakeholders with different values and interests. Problems manifest themselves
      in the (sometimes spectacular and persistent) gaps between the intended cur-
      riculum (as expressed in policy rhetoric), the implemented curriculum (real life
      in school and classroom practices), and the attained curriculum (as manifested
      in learner experiences and outcomes). A typical consequence of those tensions
      is that various frustrated groups of participants blame each other for the fail-
      ure of reform or improvement activities. Although such blaming games often
      seem rather unproductive, there are some serious critical remarks to be made
      on many curriculum development approaches worldwide. First of all, many
      curriculum reform efforts can be characterised by overly big innovation ambi-
      tions (especially of politicians) within unrealistically short timelines and with
      very limited investment in people, especially teachers. Unfortunately, many
      ICT-related reforms have suffered from this problem. Second, oftentimes there
      is a lack of coherence between the intended curriculum changes with other
      system components (especially teacher education and assessment/examination
      programmes). And last but not least, timely and authentic involvement of all
      relevant stakeholders is often neglected.
          From a strategic point of view, the literature has offered us many
      (technical-professional) models and strategies for curriculum development.
      Three prominent approaches are Tyler’s rational-linear approach, Walker’s
      deliberative approach, and Eisner’s artistic approach. As it does not fit within
      the purpose of this chapter to explain those models in particular, the reader is
      referred to educative texts from Marsh and Willis (2003) or to the overview
      of prominent approaches in Thijs and van den Akker (2009).


                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                        6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS – 133



            Obviously, the context and nature of the curriculum development task
       at hand will determine to a large extent what kind of strategy is indicated.
       It is noteworthy that we are beginning to see more blended approaches that
       integrate various trends and characteristics of recent design and development
       approaches in the field of education and training (for an overview and a series
       of examples, see van den Akker et al., 1999; or van den Akker and Kuiper,
       2009). Some key characteristics:
                 Pragmatism: recognition that there is not a single perspective, over-
                 arching rationale or higher authority that can resolve all dilemmas for
                 curriculum choices to be made. The practical context and its users
                 are in the forefront of curriculum design and enactment.
                 Prototyping: evolutionary prototyping of curricular products and their
                 subsequent representations in practice is viewed as more productive than
                 quasi-rational and linear development approaches. Gradual, iterative
                 approximation of curricular dreams into realities may prevent paralysis
                 and frustrations. Formative evaluation of tentative, subsequent curricu-
                 lum versions is essential to such curriculum improvement approaches.
                 Communication: a communicative-relational style is desirable in
                 order to arrive at the inevitable compromises between stakeholders
                 with various roles and interests and to create external consistency
                 between all parties involved.
                 Professional development: In order to improve chances on successful
                 implementation, there is a trend towards more integration of cur-
                 riculum change and professional learning and development of all
                 individuals and organisations involved.
           Design or development(al) research is a research approach that incorpo-
       rates some of these characteristics, and it becomes even more promising by
       adding the element of knowledge growth to it. Such research can strengthen
       the knowledge base in the form of design principles that offer heuristic advice
       to curriculum development teams, when (more than in common development
       practices) deliberate attention is paid to theoretical embedding of design
       issues and empirical evidence is offered about the practicality and effective-
       ness of the curricular interventions in real user settings.
           However, there are several persistent dilemmas for curriculum develop-
       ment that cannot easily be resolved, let alone through generic strategies. For
       example: how to combine aspirations for large-scale curriculum change and
       system accountability with the need for local variations and ownership? The
       tension between these conflicting wishes can be somewhat reduced when
       one avoids the all too common “one size fits all” approach. More adaptive
       and flexible strategies will avoid detailed elaboration and over-specification
       of central curriculum frameworks. Instead, they offer substantial options


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
134 – 6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS

      and flexibility to schools, teachers, and learners. Although struggles about
      priorities in aims and content will remain inevitable, the principle of “less is
      more” should be pursued. However, what is incorporated in a core curriculum
      should be clearly reflected in examination and assessment approaches.
           The “enactment” perspective (teachers and learners together create their
      own curriculum realities) is increasingly replacing the “fidelity” perspec-
      tive on implementation (teachers faithfully follow curricular prescriptions
      from external sources). That trend puts even more emphasis on teachers as
      key people in curriculum change. Both individual as well as team learning
      is essential (Fullan, 2007). Teachers need to get out of their customary isola-
      tion. Collaborative design and piloting of curricular alternatives can be very
      productive, especially when experiences are exchanged and reflected upon in
      a structured curriculum discourse. Interaction with external facilitators can
      contribute to careful explorations of the “zone of proximal development” of
      teachers and their schools. Cross-fertilisation between curriculum, teacher,
      and school development is a conditio sine qua non for effective and sustain-
      able curriculum improvement. The increasingly popular mission statements
      of schools to become attractive and inspiring environments for students and
      teachers can only be realised when such integrated scenarios are practised.

The potential of curriculum design research

           Various motives for initiating and conducting curriculum design research
      can be mentioned. A basic motive stems from the experience that many
      research approaches (e.g. experiments, surveys, correlational analyses), with
      their focus on descriptive knowledge, hardly provide prescriptions with useful
      solutions for a variety of design and development problems in education.
      Probably the greatest challenge for professional designers is how to cope with
      the manifold uncertainties in their complex tasks in very dynamic contexts. If
      they do seek support from research to reduce those uncertainties, several frus-
      trations often arise: answers are too narrow to be meaningful, too superficial
      to be instrumental, too artificial to be relevant, and, on top of that, they usu-
      ally come too late to be of any use. Curriculum designers do appreciate more
      adequate information to create a solid ground for their choices and more timely
      feedback to improve their products. Moreover, the professional community of
      developers as a whole would be helped by a growing body of knowledge of the-
      oretically underpinned and empirically tested design principles and methods.
          Another reason for curricular design research stems from the highly
      ambitious and complex nature of many curriculum reform policies in edu-
      cation worldwide. These reform endeavours usually affect many system
      components, are often multi-layered, including both large-scale policies
      and small-scale realisation, and are very comprehensive in terms of factors



                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                        6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS – 135



       included and people involved. Those radical “revolutions”, if promising at all,
       cannot be realised on the drawing table. The scope of diverse needs is often
       very wide, the problems to be addressed are usually ill-specified, the effec-
       tiveness of proposed interventions is mostly unknown beforehand, and the
       eventual success is highly dependent on implementation processes in a broad
       variety of contexts. Therefore, such curriculum reform efforts would profit
       from more evolutionary (interactive, cyclic, spiral) approaches, with inte-
       grated research activities to feed the process (both forward and backward).
       Such an approach would provide more opportunities for “successive approxi-
       mation” of the ideals and for more strategic learning in general. In conclu-
       sion: curriculum design research seems a wise and productive approach for
       curriculum development. And innovative efforts to integrate ICT in education
       is an excellent example of such challenges.

Features of curriculum design research

            Curriculum design research is often initiated for complex, innovative
       interventions for which only very few validated principles are available to
       structure and support the design and development activities. Since in those
       situations the image and impact of the intervention to be developed is often
       still unclear, the research focuses on realising limited but promising examples
       of those interventions. The aim is not to elaborate and implement complete
       interventions but to come to (successive) prototypes that increasingly meet
       the innovative aspirations and requirements. The process is often iterative,
       cyclic or spiral: analysis, design, evaluation and revision activities are iterated
       until a satisfying balance between ideals and realisation has been achieved.
            To what extent do these design research activities differ from what is
       typical for design and development approaches in professional practices?
       What are the implications of the accountability of researchers to the “sci-
       entific forum”? At the risk of exaggerating the differences, let us outline
       some of them, based on what is known about routinised standard-patterns
       in curriculum development practices. Of course, a lot of activities are more
       or less common for both approaches, so the focus will be on those additional
       elements that are more prominent in design research than in common design
       and development practices.

       Preliminary investigation
            A more intensive and systematic preliminary investigation of curriculum
       tasks, problems and context is made, including searching for more accurate
       and explicit connections of that analysis with state-of-the-art knowledge from
       literature. Some typical activities include: literature review; consultation of
       experts; analysis of available promising examples for related purposes; case


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
136 – 6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS

      studies of current practices to specify and better understand needs and prob-
      lems in intended user contexts.

      Theoretical embedding
           More systematic efforts are made to apply state-of-the-art knowledge in
      articulating the theoretical rationale for curriculum design choices. Moreover,
      explicit feedback to assertions in the design rationale about essential char-
      acteristics of the intervention (substantive design principles) is made after
      empirical testing of its quality. This theoretical articulation can increase the
      “transparency” and “plausibility” of the rationale. Because of their specific
      focus, these theoretical notions are usually referred to as “mini” or “local”
      theories although sometimes connections can also be made to “middle-range”
      theories with a somewhat broader scope.

      Empirical testing
          Clear empirical evidence is delivered about the practicality and effective-
      ness of the curriculum for the intended target group in real user settings. In
      view of the wide variation of possible interventions and contexts, a broad
      range of (direct/indirect; intermediate/ultimate) indicators for “success”
      should be considered.

      Documentation, analysis and reflection on process and outcomes
          Much attention is paid to systematic documentation, analysis and reflec-
      tion on the entire design, development, evaluation and implementation
      process and on its outcomes in order to contribute to the expansion and speci-
      fication of the methodology of curriculum design and development.
          Typical questions for design research are:
              What are essential features of successful interventions? (For compli-
              cated challenges or “wicked” problems, cf. Kelly, 2009.)
              How do those interventions operate in real life?
              How can they be developed and implemented?
           In view of its aim, design research is never a “quick fix” operation, but
      it usually takes a long trajectory, where the research is intertwined with con-
      tinuous development activities – from problem formulation up to and includ-
      ing implementation.
         More than most other research approaches, design research aims at
      making both practical and scientific contributions. In the search for innovative



                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                        6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS – 137



       “solutions” for curriculum problems, interaction with practitioners (in various
       professional roles: teachers, policy makers, developers, and the like) is essen-
       tial. The ultimate aim is not to test whether theory, when applied to practice,
       is a good predictor of events. The interrelation between theory and practice
       is more complex and dynamic: is it possible to create a practical and effective
       curriculum for an existing problem or intended change in the real world? The
       innovative challenge is usually quite substantial, otherwise the research would
       not be initiated at all. Interaction with practitioners is needed to gradually
       clarify both the problem at stake and the characteristics of its potential solu-
       tion. An iterative process of “successive approximation” or “evolutionary pro-
       totyping” of the “ideal” intervention is desirable. Direct application of theory
       is not sufficient to solve those complicated problems. One might state that a
       more “constructivist” development approach is preferable: researchers and
       practitioners co-operatively construct workable interventions and articulate
       principles that underpin the effects of those interventions.
            Another reason for co-operation is that without involvement of practition-
       ers it is impossible to gain clear insight in potential curriculum implemen-
       tation problems and to generate measures to reduce those problems. New
       interventions, however imaginative their design, require continuous antici-
       pation at implementation issues. Not only for “social” reasons (to build com-
       mitment and ownership of users) but also for “technical” benefits: to improve
       their fitness for survival in real life contexts. Therefore, rigorous testing of
       practicality is a conditio sine qua non in design research.

Emphasis on formative evaluation

             As has become clear in the previous sections, formative evaluation holds
       a prominent place in curricular design research. The main reason for this cen-
       tral role is that formative evaluation provides the information that feeds the
       optimisation of the intervention and the cyclic learning process of curriculum
       developers during the subsequent loops of a design and development trajectory.
       It is most useful when fully integrated in a cycle of analysis, design, evaluation,
       revision, etc., and when contributing to improvement of the curriculum.
           Thus, the basic contribution of formative evaluation is to quality
       improvement of the curriculum under development. Quality, however, is an
       abstract concept that requires specification. During development processes,
       the emphasis in criteria for quality usually shifts from relevance to consist-
       ency, to practicality, to effectiveness.
           Relevance refers to the extent that the intended curriculum is perceived to
       be a relevant improvement to practice, as seen from the varied perspectives of
       policy makers, practitioners and researchers. Consistency refers to the extent
       that the design of the curriculum is based on state-of-the-art knowledge and


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
138 – 6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS

      that the various components of the intervention are consistently linked to
      each other (cf. the curricular spider web). Practicality refers to the extent
      that users (and other experts) consider the intervention as clear, usable and
      cost-effective in “normal” conditions. Effectiveness refers to the extent that
      the experiences and outcomes with the intervention are congruent with the
      intended aims. Also, issues of scalability and sustainability may be included
      in a broad interpretation of effectiveness.
           The methods and techniques for evaluation will usually be attuned to that
      shift in criteria (cf. the various stages of alpha, beta and gamma testing in the
      IT sector). For example, adequate evaluation of validity can start with com-
      ments of critical friends on initial drafts and then move over to more systematic
      expert appraisal. Practicality is often tested via micro-evaluations and try-outs
      in real classroom practices. Evaluation of effectiveness usually requires (more
      large-scale) field tests. In later stages of formative evaluation, methods of data col-
      lection will usually be less intensive but with an increasing number of respondents
      (e.g. achievement test for many students at the end compared to in-depth interview
      with a few experts in the beginning). See Nieveen (2009) for more elaborate expla-
      nations and suggestions for these shifts in formative evaluation.
          Formative evaluation within design research should not only concentrate
      on locating shortcomings of the intervention in its current (draft) version
      but especially generate suggestions on how to improve those weak points.
      Richness of information, notably salience and meaningfulness of suggestions
      on how to make an intervention stronger, is therefore more productive than
      standardisation of methods to collect and analyse data. Also, efficiency of
      procedures is crucial. The lower the costs in time and energy for data collec-
      tion, processing, analysis and communication will be, the bigger the chances
      of actual use and impact on the development process. For example, samples
      of respondents and situations for data collection will usually be relatively
      small and purposive compared to sampling procedures for other research
      purposes. The added value of getting “productive” information from more
      sources tends to decrease because the opportunities for “rich” data collection
      methods (such as interviews and observations) are limited with big numbers.
      To avoid an overdose of uncertainty in data interpretation, often triangula-
      tion (of methods, instruments, sources and sites) is applied. These arguments
      especially hold true for early stages of formative evaluation when the inter-
      vention is still poorly crystallised.

Generalisation of curricular design research findings

           In practical terms, the most relevant outcome of curricular design research
      is its contribution towards optimisation of the curricular product and its actual
      use, leading to better instructional processes and learning results. However, a



                                               INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                        6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS – 139



       major contribution to knowledge to be gained from design research is in the
       form of “design principles” (both substantive and methodological) to support
       developers in their task. Those principles are usually incorporated within (a
       growing set of) heuristic statements, following a format such as:
                 If you want to design intervention X [for purpose/function Y in con-
                 text Z]
                 then you are best advised to give that intervention the characteristics
                 C1, C2,…, Cm [substantive emphasis]
                 and to do that via procedures P1, P2, …, Pn [procedural emphasis]
                 because of theoretical arguments T1, T2, …, Tp
                 and empirical arguments E1, E2, …, Eq.
           Obviously those principles cannot guarantee success, but they are
       intended to select and apply the most appropriate (substantive and procedural)
       knowledge for specific design and development tasks.
            It is not uncommon in design research that such knowledge, especially the
       substantive knowledge about essential curriculum characteristics, can partly be
       extracted from a resulting prototype itself. That is one of the reasons that makes
       it so profitable to search for and carefully analyse already available curricula
       to generate ideas for new design tasks. However, the value of that knowledge
       will strongly increase when justified by theoretical arguments, well-articulated
       in providing directions, and convincingly backed-up with empirical evidence
       about the impact of those principles. Moreover, those heuristic principles will
       be additionally powerful if they have been validated in successful design of
       more interventions in more contexts. Chances for such knowledge growth will
       increase when design research is conducted in the framework of research pro-
       grammes, because projects can then build upon one another.
           Since data collection in design research is often limited to small (and pur-
       posive) samples, efforts to generalise findings cannot be based on statistical
       techniques, focusing on generalisations from sample to population. Instead
       one has to invest in “analytical” forms of generalisation (cf. Yin, 2003): read-
       ers/users need to be supported to make their own attempts to explore the
       potential transfer of the research findings to theoretical propositions in rela-
       tion to their own context. Reports on design research can facilitate that task of
       analogy reasoning by a clear theoretical articulation of the design principles
       applied and by reflection on the results afterwards. Moreover, it is helpful to
       offer a careful description of both the evaluation procedures as well as the
       implementation context. Especially a “thick” description of the process-in con-
       text may increase the “ecological” validity of the findings, so that others can
       estimate in what respects and to what extent transfer from the reported situ-
       ation to their own is possible. Another option that may stimulate exploration


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
140 – 6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS

      of possibilities for (virtual) generalisation and transfer to various settings is to
      organise interactive (network) meetings with experts and practitioners from
      related contexts to discuss the plausibility of the research findings and recom-
      mendations for related tasks and contexts.
          Last but not least, curricular design research may offer drafts of various
      relevant curriculum versions (with proven consistency and practicality) that
      can be compared in more quantitative, large-scale (quasi-)experimental studies.

Conclusions

           In view of the (rapidly growing but still relatively) modest familiarity of
      educational design research to the wider audience, it is wise to invest a lot in
      the transparency, plausibility, “trustworthiness”, and “reconstructiveness”
      of its arguments and findings. Also at stake is the “credibility” (expertise in
      depth and breadth; track record) of research team and partners.
          For closure, we briefly mention some research characteristics that strengthen
      the growth of knowledge through design research:
              “Standing on shoulders”: less isolated studies, but deliberate efforts
              to build on previous research and development work through a pro-
              grammatic approach.
              Variation and adaptability in intervention and methodology.
              Deliberate triangulation in sites and sources of data gathering.
              Increasing insight in essence of successful interventions versus vari-
              ations in other features.
              Preventing premature closure to seemingly effective but limited
              interventions.
              Knowledge sharing and distribution through systemic partnerships
              and networks.
              Involvement and joint professional development of many partners:
              teachers, school leaders, teacher educators, researchers, curriculum
              and textbook developers, assessment specialists, etc.
          The more these characteristics receive attention, design research is also a
      very promising avenue for technology-based innovation.




                                              INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                         6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS – 141




                                           References

       Cuban, L. (1992), “Curriculum stability and change”, in P. Jackson (ed.),
         Handbook of Research on Curriculum, pp. 216-247, Macmillan,
         New York, NY.
       Fullan, M. (2007), The New Meaning of Educational Change, 4th edition,
          Teachers College Press, New York, NY.
       Goodlad, J. (1994), “Curriculum as a field of study”, in T. Husén and
         T. Postlethwaite (eds.), The International Encyclopedia of Education,
         pp. 1262-1276, Pergamon Press, Oxford.
       Hargreaves, A. and D. Fink (2006), Sustainable Leadership, Jossey-Bass,
          San Francisco, CA.
       Kelly, A. (2009), “When is design research appropriate?” in T. Plomp and
          N. Nieveen (eds.), An Introduction to Educational Design Research,
          pp. 73-87, SLO, Enschede, the Netherlands.
       Kelly, A., R. Lesh and J. Baek (eds.) (2008), Handbook of Design Research
          Methods in Education Innovations in Science, Technology, Engineering,
          and Mathematics Learning and Teaching, Lawrence Erlbaum Associates,
          Mahwah, NJ.
       Marsh, C. and P. Willis (2003), Curriculum: Alternative Approaches, Ongoing
         Issues, 3rd edition, Merrill/Prentice Hall, Upper Saddle River, NJ.
       Nieveen, N. (2009), “Formative evaluation in educational design research”, in
          T. Plomp and N. Nieveen (eds.), An Introduction to Educational Design
          Research, pp. 89-101, SLO, Enschede, the Netherlands.
       Pinar, W. et al. (1995), Understanding Curriculum: An Introduction to the
          Study of Historical and Contemporary Curriculum Discourses, Peter Lang,
          New York, NY.
       Plomp, T. (2009), “Educational design research: an introduction”, in T. Plomp and
          N. Nieveen (eds.), An Introduction to Educational Design Research, pp. 9-35,
          SLO, Enschede, the Netherlands.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
142 – 6. DESIGN RESEARCH ON TECHNOLOGY-BASED INNOVATIONS

      Taba, H. (1962), Curriculum Development: Theory and Practice, Harcourt, Brace
         and World, New York, NY.
      Thijs, A. and J. van den Akker (2009), Curriculum in Development, SLO,
         Enschede, the Netherlands.
      van den Akker, J. (1999), “Principles and methods of development research”, in
         J. van den Akker et al. (eds.), Design Approaches and Tools in Education and
         Training, pp. 1-14, Kluwer Academic Publishers, Dordrecht.
      Van den Akker, J. (2003), “Curriculum perspectives: An introduction”, in
         J. van den Akker, W. Kuiper and U. Hameyer (eds.), Curriculum Landscapes
         and Trends, pp. 1-10, Kluwer Academic Publishers, Dordrecht.
      van den Akker, J. et al. (eds.) (1999), Design Approaches and Tools in Education
         and Training, Kluwer Academic Publishers, Dordrecht.
      van den Akker, J. et al. (eds.) (2006), Educational Design Research, Routledge,
         London.
      Voogt, J. and G. Knezek (eds.) (2008), International Handbook of Information
         Technology in Primary and Secondary Education, Springer, New York, NY.
      Walker, D. (2003), Fundamentals of Curriculum: Passion and Professionalism,
        Lawrence Erlbaum Associates, Mahwah, NJ.
      Yin, R. K. (2003), Case Study Research: Design and Methods, Sage, London.




                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                         CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS – 143




                                            Conclusion

                     Lessons learnt and policy implications


                        Oystein Johannessen and Francesc Pedró
                       Norwegian Ministry of Education and CERI




    This final chapter summarises the lessons learnt from the OECD expert meeting
    held in Florianopolis (Brazil) in November 2009 as well as the policy implications.
    On the whole, it addresses the issue of how a systemic approach can improve our
    understanding of how technology-based school innovations work and how local
    innovations can be scaled up successfully. In times of economic crisis, a systemic
    approach to technology-based innovation in education is even more urgently
    needed. Most countries are now facing difficult times, and OECD member states
    are no exception to this. The immediate programmes that many governments have
    launched – sometimes in a co-ordinated way, with the aim of facing the financial
    crisis – have also been coupled in many cases with in-depth reflection about the
    way in which our economies work and with strategies to promote longer-term
    development and vision. In the context of this reflection, it becomes apparent that in
    the medium and long-term, innovation will increasingly be a key factor not only to
    economic growth but also to social welfare. The efforts to sustain technology-based
    innovations in education should be no exception to this. In the light of the financial
    crisis, each educational system should improve its ability to scale up technology-
    based innovation for improved learning outcome and learning strategies.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
144 – CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS

Introduction

          This closing chapter summarises the main lessons learnt from the pre-
      vious sections. Drawing on these lessons, some policy principles are sug-
      gested. However, in order to translate them into policy recommendations it is
      important to consider three important analytical dimensions: the pedagogical,
      the technological and the social, i.e. that which is related to the very nature
      of social innovations. The chapter ends with a discussion of the pending
      research agenda, particularly with regard to new analytical needs that the
      application of a systemic approach to the understanding of technology-based
      innovations can instill in policy makers, researchers and practitioners.

Lessons learnt

           The expert meeting in Florianopolis discussed a range of topics related to
      innovation, technology and learning over the course of the two-day meeting.
      The conclusions that are presented here are some of the preliminary conclusions
      that were drawn at the end of the meeting. A note of caution is pertinent here,
      because these conclusions do not constitute the full ptechnologyure of the meet-
      ing, but they give an overview over some critical issues that were discussed.
          1. There is an imbalance between investments in infrastructure,
             content, support and teacher training and the required efforts
             to build a sustainable knowledge base regarding technology-
             based school innovations. What does this mean? The answer is
             quite simple. Many countries have been investing heavily in the core
             strategic components for technology in education in order to foster
             its use in teaching and learning, but evidence from, for example, the
             OECD study on systemic innovation in digital learning resources in
             the Nordic countries suggests that a sufficient knowledge base is not
             available, and the existing knowledge has barely been tapped into.
          2. There is a tension between technology and pedagogy. There are
             more cases of technology-driven innovations than of pedagogy-
             driven innovations. This conclusion suggests that the availability
             and, in some cases, even the fascination for technology is the main
             driver behind innovations in this area. The link between technology
             and pedagogy is too weak or in the worst case non-existent, which
             seriously inhibits the potential of the educational system to be at the
             forefront of the utilisation of technologies for learning.
          3. There is an axis between radical and incremental innovation.
             This finding is aligned with the discussion about whether innovations
             only can be radical, or whether innovations can be undertaken in
             small steps. Some of the innovations mentioned at the expert meeting


                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                         CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS – 145



                 were well inside the box of an educational system, while other inno-
                 vations where “out-of-the-box” innovations taking a truly different
                 view of education from the outset.
            4. There is a need to balance the expectations of the power of tech-
               nology with the reality of what is feasible. Since the first half of the
               20th century, technological innovations have been said to have revo-
               lutionary possibilities for education. Over the decades, we have seen
               various expressions of this. There is a need to find a good balancing
               point between what technologies have to offer and the financial,
               organisational and pedagogical realities of education.
            5. The complexity of the issues at stake requires a multi-layered, multi-
               disciplinary and multi-methodological approach. Technology-based
               innovations address all aspects and all stakeholder groups of education.
               The phases of innovation and the need for good M&E systems underpin
               this conclusion.
            6. The potential of “new” research disciplines such as brain research
               and research on learning environments must be explored. This
               conclusion highlights the need to engage research disciplines that
               in the past have not been involved in issues related to technology in
               learning. Such an approach could be useful in order to embed technol-
               ogy in core issues related to teaching and learning.
            7. Current work on assessment seems to be limited to digital lit-
               eracy. Further research should be grounded in classroom practice.
               Some of the examples mentioned at the expert meeting show that
               the foremost use of technology related to the assessment of digital
               literacy, cf. the Australian case. There is a need to expand the use of
               technology for literacy to other areas, and a focus on both formative
               and summative assessment should be upheld.
            8. There is a need for a social dialogue on educational innovations
               involving all stakeholders. This conclusion supports the view taken
               in the OECD Innovation Strategy that innovation processes today
               are considered to be social, multi-stakeholder oriented. Without suf-
               ficient stakeholder involvement, the risk of innovation failure is likely
               to increase.
            9. Research must be translated into meaningful guidelines for
               improving practice. Innovations will never become true innovations
               if they are not put into practice. This conclusion points both to the
               importance of a solid knowledge base, in which empirical evidence
               from pedagogical practice is also included, as well as a fruitful dia-
               logue with practitioners in order to facilitate good implementation
               strategies for technology in teaching and learning.


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
146 – CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS

          10. To what extent is research evidence phased into the education
              and practice of teachers? This open question raises two important
              issues. Firstly, educational systems need to ensure that research
              evidence related to technology and learning is made available in a
              “teacher-friendly” format. Secondly, the professional culture and
              professional development of teachers should address an orientation
              towards research evidence as a part of the professional ethos among
              teachers. A third issue is related to the question about whether
              teacher training institutions manage to be at the forefront of the
              development of the discourse on technology in teaching and learning.
              This issue is addressed by the ongoing OECD project on technology
              use in initial teacher training.

Policy principles

          These conclusions drawn from the discussions at the meeting can be
      combined with the main results of previous CERI work on systemic innova-
      tion and on technology in education. The result suggests a number of impor-
      tant principles for policy actions.

      Systemic innovation is a useful analytical framework for the
      assessment of innovation policies.
          The main benefit of the systemic innovation approach is that it can help
      governments and other stakeholders to have a comprehensive evaluation
      of how the system works and how they can enhance their own innovation
      capacity. It is thus relevant from a policy perspective because it clarifies what
      information gaps exist, and particularly where, in the lifecycle of the innova-
      tion, a good evidence base might be more useful. In the end, the systemic
      approach to innovation contributes to the assessment of how the innovation
      system works and to the identification of policies that are capable of boosting
      the innovative potential of technology in education.
           Although efforts to develop a systemic approach to innovation in this
      field are still rare, they have the potential to develop better processes and to
      contribute to an incremental improvement of the education system.

      A coherent and targeted system should be in place to promote and
      support successful innovations and to induce system-wide change.
      Such systems are still infrequent at the country level.
          The need to respond in a timely manner to the socio-economic and techno-
      logical challenges that all education systems are facing in an increasingly glo-
      balised and rapidly changing world seem to be driving most technology-based


                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                         CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS – 147



       innovations. Political leadership and capacity to steer and manage the innova-
       tion, the availability of resources, and/or the existence of regulatory mecha-
       nisms supporting the process seem to play a crucial enabling role in most
       systemic innovations. Equally, the availability of evidence and a good level
       of consensus among stakeholders also seem to play a crucial role during the
       design and implementation of the innovations.
           Nevertheless, innovation enablers and barriers are not universal but
       rather context specific. While it is true that their presence or absence will
       facilitate or hinder the innovation processes in any education system, their
       importance seems to vary depending on the cases and the context. This is
       particularly true of the role of consensus among stakeholders, of evidence and
       of political leadership. In particular, evidence can facilitate the adoption of
       innovation and inform the process – although the existing evidence suggests
       that innovations primarily draw upon tacit knowledge and beliefs or a sense
       of urgency to change the status quo.

       In the domain of technology-based innovations, education systems
       need a formalised, coherent, well-sustained and up-to-date
       knowledge base to increase their innovation capacity, to address
       knowledge gaps and to fully benefit from systemic innovations.
           Technology-based school innovations are rarely the result of an embodied
       set of knowledge or empirical evidence accumulated over the years, knowl-
       edge or evidence from which stakeholders nourish their decisions and to which
       they contribute with their feedback. Moreover, countries do not seem to pay
       enough attention to monitoring and evaluating how technology-based innova-
       tions, particularly those whose realisation requires a large amount of policy
       commitment and financial investment, evolve in the context of the system. In
       addition, little has been done to assess when a particular innovation can be
       said to be a success or a failure and what lessons can be learned as a result.
           Although there has not been empirical validation of the assumption that a
       better knowledge base results in more successful innovations in our case studies,
       the existing lack of a link between research and innovation efforts is remark-
       able. This is reflected mostly at the government level, with a generalised lack
       of attention given to the issue of merging both activities to result in a coherent
       knowledge base. It is also clear that innovation, on the one hand, and research,
       on the other, seem to appeal to different profiles of education professionals.
           Finally, it is particularly perplexing to see the lack of research evidence
       and gaps in the feedback loop of the evaluation process in conjunction with
       the push for greater accountability and increased assessment of the system,
       teachers and students. This is a clear incoherence in the system that needs to
       be addressed.



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
148 – CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS

      Education systems may be losing innovation opportunities due to a
      lack of evaluations and knowledge feedback into the system.
          Despite its potential, the evaluation of innovations seems to be a miss-
      ing feature of education systems. This applies equally to local and discrete
      innovations as well as to top-down innovations, including those aiming for
      system-wide impact. There may be many explanations for this, ranging from
      the lack of sustained research efforts in the particular domain of technology
      in education, the disconnection between practitioners, researchers and policy
      makers, the lack of dedicated mechanisms to gather relevant information or
      even the prevalent culture of the sector.
           A particular situation where the relevance of evaluation becomes even
      clearer is piloting. Pilots fulfil a very important role in those systemic innova-
      tions that aim at having a deep impact on the system. While they are costly
      in terms of time and resources, they play an important role in the prevention
      of implementation gaps and innovation fatigue. Piloting may be useful for
      technical and organisational purposes, but unless a monitoring and evalua-
      tion procedure is carefully implemented, the benefits of piloting may be lost.

The analysis of technology-based innovation in education

          The work done during the expert meeting led to the identification of four
      axes for the analysis of technology-based innovations in education. These are:
              The policy axis
              The pedagogical axis
              The technology axis
              The knowledge axis
          The policy axis is necessary in order to highlight a systemic approach as a
      prerequisite for long-term horizon in technology-based innovation in education.
      The pedagogical and technological axes are the main domains when we talk
      about technology in learning. The knowledge axis is selected because of the
      key importance attached to having an available knowledge base for innovation.

      Policy axis
           The policy axis emphasises the need to approach innovation in educa-
      tion in a systemic fashion – i.e. looking at innovations that the educational
      system in a country, a state or a region can benefit from – and this axis links
      innovation to policy making and policy choices that need to be made in order
      to facilitate innovation, its impact and its knowledge base.



                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                         CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS – 149



           One of the most important challenges for innovation policies for technol-
       ogy in education is to ensure sufficient policy coherence. The various policy
       elements cannot be regarded in isolation; they are elements that are interre-
       lated, and they are often necessary for other policy elements to be effective.
       Policy coherence lies at the heart of the systemic approach to innovation
       through its focus on policy elements and their internal relations.
           In order to give a deeper understanding of policy coherence, we can
       make the simple distinction between horizontal and vertical coherence in an
       educational system. The horizontal coherence refers to the links between the
       key elements in a policy or strategy whereas the vertical coherence refers to
       the links or channels through various levels of the educational system. Both
       types of coherence will be elaborated in the following text.
           With regard to horizontal coherence, this approach is necessary in order to
       understand how different policy elements interplay. The following figure shows
       a simplified model of coherence between key policy elements. This simplified
       model is not intended to be prescriptive with regard to the most important
       policy elements. In most cases, curriculum, professional development for teach-
       ers and school leaders and assessment are key elements in any context.

          Figure 7.1. Simplified model of coherence between key policy elements

                                                  Curriculum




                                   Professional                Assessment




       Coherence between key elements in the promotion of innovations in
       education
            To give an example of the interrelations we are talking about here, we
       can look at the relation between curriculum and assessment. In many coun-
       tries, curriculum is the key driver for education, and the curriculum can be
       regarded as the contract between education and society at large. In some cases,
       the curriculum is changed in order for education to keep up with changes in
       society and changes in the demands on the future workforce. However, if this
       change occurs without similar changes in the system of assessment, the lack


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
150 – CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS

      of assessment change may act as an inhibitor for educational change because
      assessment is, in many countries, one of the most important drivers for edu-
      cational change.
          In a similar way, changes in the curriculum should be followed by pro-
      fessional development among teachers because teachers need to be enabled
      to implement the changes in the curriculum (and in the assessment system).
          The simplified model can be said to represent a core set of horizontal
      policy coherence. There are, of course, other important areas to consider,
      such as infrastructure, policies on open standards and interoperability, digital
      learning resources, etc.
          The horizontal coherence is important in order to secure a robust imple-
      mentation of innovations and educational change. This cuts both ways, of
      course. Bottom-up innovations require links between different levels in
      the educational system in order to be communicated to national authorities,
      which can be important, e.g. when it comes to the scaling up of innovation.
          A common experience across countries is that the road from the Ministry
      of Education to each classroom seems very long. Sufficient vertical policy
      coherence can be a means to reduce this distance.
           In considering the issue of policy coherence, countries and regions should
      also take into consideration how policies for technology-based innovation
      are related to the overall policies and goals of regional or national education.
      Across countries, there are variations as to how countries design their strate-
      gies. Some countries have dedicated policies for technology in education.
      Some of these policies cover a wide spectre of issues, i.e. they encompass all
      key policy elements for technology in education. Other countries have more
      focused strategies. This seems to be the case in a number of countries that
      have developed strategies for what is commonly known as “1 to 1 computing”,
      i.e. ensuring that most learners are equipped with laptops, notebooks etc. These
      strategies normally have elements of professional developments embedded, but
      the core of these strategies is to ensure access and equity among learners with
      regard to technology in education. Other countries do not have separate tech-
      nology strategies because technology is embedded in overall national strategies
      and policies. For example, this is the case in Norway and Sweden.
           Making policy coherence work is an important factor when it comes to
      linking, e.g. research and classroom practice. As Hung et al. have pointed out in
      their chapter; we need to pay careful attention to the translation process from the
      initiation of innovation to the implementation of innovation. Some countries have
      established national centres or agencies for technology in education such as Becta
      in the United Kingdom or the Centre for Technology in Education in Norway.
      Such agencies are important mediators between policy, practice and research, and
      they play an important role in the acquisition and dissemination of knowledge.


                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                         CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS – 151



       Pedagogical axis
            The pedagogical axis is largely about how technology can contribute to
       improved learning outcomes and learning strategies among learners. This
       is the key challenge when it comes to technology and learning, and we will
       focus on some critical areas in this chapter.
            The curriculum is, along with assessment, a key driver for education,
       because curricula define goals, content and, in some cases, also the methods
       of teaching and learning. To paraphrase Professor van den Akker’s contribu-
       tion, we can regard curricula as the roadmap for education.
           The first issue is how we can approach the curriculum with regard to
       innovation. As the curriculum enables practitioners to navigate through the
       key areas of each subject or discipline, the curriculum can offer what we may
       label as “zones of innovation”. Let us take a look at a couple of examples.
       Firstly, in a subject like English (as a first language), it is important to look at
       and work with different text genres, in some cases we refer to what is called
       multimodal text (Multimodal texts are those that combine, for example, print
       text, visual images and spoken word as in film or computer presentation
       media). A strategy sometimes preferred by teachers is to let learners work
       collaboratively on multimodal texts, and this is an area where some teachers
       prove to be innovators in how they use, for example, social media such as
       blogs and wikis to facilitate collaborative writing.
           Another example of how technology is used to improve teaching and
       learning is to use simulations and visualisation in science. This highlights
       how technology can improve the teaching and learning process by enabling
       pedagogical approaches that are impossible or more difficult to facilitate
       without the use of technology. Key here is that teachers are able to identify
       those parts of the curriculum – the curricular zones of innovation – where
       technology can add value. This focus in innovation can be said to belong to
       what is often referred to as user-generated or bottom-up innovation, and in
       many cases the innovations are incremental in nature. The climate for this
       kind of innovation is dependent on the culture in schools and on the profes-
       sional culture of teachers. An important factor in order to enable such inno-
       vations to scale up is the availability of a culture of sharing among teachers.
       Development of professional communities of practice aiming at sharing
       resources and experiences can be important facilitating tools for this.
           The pedagogical axis is heavily dependent on a coherent and available
       knowledge base on how technology can add value to teaching and learning.
       Further, the pedagogical axis can benefit from a research-based and profes-
       sional discourse on whether and how technology can transform and enrich
       learning processes.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
152 – CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS

      Technological axis
          The technological axis reflects the strong importance most countries place
      on access to laptops, broadband internet connection, learning management
      systems etc. Infrastructure becomes an enabler for access and equity with
      regard to technology in education, as the many 1 to 1 computing initiatives
      around the world demonstrate. As such, infrastructure is a necessary, but not
      sufficient prerequisite for the advancement of technology in education.
           The key and probably most important issue with regard to technology
      in education is that technology must not only be accessible, it must also be
      robust and scalable enough to tackle the needs of teachers and learners. If the
      infrastructure is neither available nor working, good intentions of integrating
      technology into teaching cannot be fulfilled. Neither available nor working,
      good intentions of integrating technology into teaching cannot be fulfilled.
      Another dimension in the domain of technology is that the need for of support
      services for technology and pedagogy must be catered for. A single school
      is in most cases not equipped with the necessary human resources to ensure
      a professional and cost-effective procurement. Local authorities can address
      this issue at some aggregated level or through collaboration between local
      authorities in a region. National authorities can contribute through national
      procurement frameworks, of which Becta is a good example.
          The use of technology in schools can benefit with collaboration with
      other parts of education. Although the evidence is scarce, there is reason to
      believe that some countries have utilised some kind of trickle-down model for
      innovation in learning technologies through collaboration with higher edu-
      cation institutions and actors. In the state of Maine in the United States, the
      network was hosted by the University of Maine. In Norway, the solution for
      federated identity management now being deployed in primary and second-
      ary education was originally developed in a higher education setting. Such
      innovation models should be the subject of further study.
          Technology in education has centred much on laptops and netbooks. The
      computer has for a long time been the dominant technological artefact. That
      is now changing, and the educational system should prepare for a time of
      greater technological diversity.
           The 2010 K-12 edition of the Horizon (Johnson et al., 2010) report points
      at six emerging technologies that will impact education over the next four to
      five years. The technologies are:
              Cloud computing
              Collaborative environments
              Mobiles



                                           INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                         CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS – 153



                 Educational games
                 Augmented reality
                 Flexible thin displays
           Even if these predictions should prove to impact education in other
       ways and at other times than predicted, they clearly indicate some important
       trends related to net-based resources, the emergence of mobile and handheld
       devices, the incremental integration of game-based applications and the
       development of enriched forms of learning resources – all in all trends that
       will require a more robust infrastructure than most schools have today.

       Knowledge axis
           The knowledge axis is linked to the important role knowledge plays in
       innovation processes. As the background paper for the expert meeting points
       out, education systems need “… a formalised, coherent, well-sustained and
       up-to-date knowledge base to increase their innovation capacity …” (OECD,
       2009a).
            The knowledge challenge can in many ways be said to be a double or
       even a triple challenge. The first part of this challenge is to secure that a suf-
       ficient knowledge base is established. A knowledge base can partly be based
       on research and partly on experience and evidence from pedagogical prac-
       tice. Some countries may still have cultural barriers towards an orientation
       towards what works, so-called clearinghouses, but the general impression is
       that the educational community sees the benefits of an updated knowledge
       base. In an international context, we may ask whether a knowledge base is
       a national or an international issue. The answer lies probably somewhere
       between “yes” and “no”. Parts of a knowledge base on technology in educa-
       tion will be rooted in the context of each educational system at regional or
       national level and must thus be a national responsibility. Other issues are not
       limited to context and can be subject of international collaboration.
           The second layer of the knowledge base challenge is to secure effective
       dissemination of knowledge. This can be regarded as a part of the peda-
       gogical support services that should be available to teachers, i.e. a knowledge
       infrastructure for practitioners. Different solutions may be preferred, but a
       key issue is to ensure that the knowledge base is structured in a meaningful
       way, e.g. aligned with the curriculum, and that it is easily accessible. This is
       an area where mediating agents such as national agencies can play an impor-
       tant role along with bottom-up initiatives such as communities of practice
       among teachers and school leaders.
          A third layer of the knowledge challenge is the necessity of utilising the
       knowledge base. This layer is largely about the demand for the knowledge


INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
154 – CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS

      base and the professional culture among practitioners in education, which is
      a prerequisite for integrating research-based evidence into teaching and lead-
      ership. A Norwegian study from 2007 (Jensen, 2007) suggests that teachers
      have access to resources tailored to their needs to a lesser degree than some
      other professions. If this is an expression of lack of resources or lack of an
      orientation towards such resources within teachers’ professional culture is an
      open question. However, if teachers are not inclined to use resources derived
      from research and innovation, public investments in dissemination strategies
      run the risk of being under-utilised.
          An interesting finding from the TALIS survey (OECD, 2009c) that
      deserves attention is that the respondents in the survey identify a high need for
      professional development in technology for instruction. There may be several
      reasons for that related to the pace of technological change, but it can also indi-
      cate a difficulty in developing the necessary capacity in technology in schools.

Policy implications

          In times of economic crisis, a systemic approach to technology-based
      innovation in education is even more urgently needed. Most countries are
      now facing difficult times and OECD member states are no exception to this.
      The immediate programmes that many governments have launched – some-
      times in a co-ordinated way, with the aim of facing the financial crisis – have
      also been coupled in many cases with in-depth reflection about the way in
      which our economies work and with strategies to promote longer-term devel-
      opment and vision. In the context of this reflection it becomes apparent that
      in the medium and long-term, innovation will increasingly be a key factor
      not only to economic growth but also to social welfare. The efforts to sustain
      technology-based innovations in education should be no exception to this. In
      the light of the financial crisis, each educational system should improve its
      ability to scale up technology-based innovation for improved learning out-
      come and learning strategies.
          With the support of the remaining stakeholders, governments in particu-
      lar may need to do the following in order to set up the conditions for such a
      system:

      Develop a systemic approach to innovation as a guiding principle for
      innovation-related policies.
          Such a systemic approach includes at least five basic elements:
              A clear policy intended to support research on technology in edu-
              cation in light of national priorities, both at policy and practitioner
              levels.


                                             INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                         CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS – 155



                 An evolving framework for sustaining both top-down and bottom-up
                 technology-based innovations, including monitoring and evaluation
                 mechanisms which can contribute to the generation of new knowl-
                 edge about policies and practices regarding technology in education.
                 A unified knowledge-base which includes both research evidence
                 and the new knowledge emerging from the assessment of innova-
                 tions, including links to international knowledge bases on these
                 topics.
                 Regular efforts to synthesise and disseminate new knowledge on effec-
                 tive policies and practices regarding the use of technology(e.g. obser-
                 vatories or clearinghouses) as an enabler of educational innovations, in
                 order to challenge the status quo of the system, set new horizons and
                 contribute to incremental change.
                 Capacity building (structural, personal) to enable all the elements above.

       Promote a continuous and evidence-informed dialogue about
       innovation with the stakeholders in the field.
            Often, policy discussions about technology in education are particu-
       larly prone to biased uses of the knowledge base, particularly in view of the
       absence of solid empirical evidence. However, the engagement of stakehold-
       ers in policy dialogue is a requisite for successful policy interventions in
       view of reaching a consensus and sharing a vision which guides action. It is
       therefore of highest importance to inform the policy debate with evidence,
       provided that all stakeholders share a minimal capacity level to be able to
       benefit from it.

       Build a well-organised, formalised, easily accessible and updated
       knowledge base about technology in education, as a prerequisite
       for successfully internalising the benefits of innovation.
            In many countries the usual mechanisms (such as dedicated journals,
       academic journals, conferences, national reference and research centres, and
       the like) that would contribute to the articulation of a knowledge base are not
       in place. Some countries may want to address this need by using existing
       facilities or mechanisms while others may prefer to set up new measures as
       an indication of the increased priority allotted to technology-based innova-
       tion in education, such as the creation of dedicated research centres, networks
       or prioritised calls. The benefits of investments made in technology-based
       innovations will hardly be recognised and of any relevant use unless the
       appropriate tools for knowledge management are in place: to gather knowl-
       edge which might be usually dispersed (for instance, by different stakeholders



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
156 – CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS

      but also from diverse sources of innovation), to cumulate in a consistent and
      coherent way, to articulate it in order to generate clear messages, and finally
      to disseminate results in decision-oriented terms both for practitioners and
      policy makers.

      Supplement investments in technology-based innovations with the
      necessary efforts in monitoring and evaluation.
          It is in the best interest of public governance and accountability to gen-
      erate the mechanisms and procedures required to critically approach both
      bottom-up and top-down innovations. An empirical assessment can contrib-
      ute decisively to:
              Informing decisions about the scaling-up or the diffusion of innova-
              tions.
              Instilling in the main actors involved the culture of output-oriented
              innovation: innovations aimed at measurable improvements that can
              help when coping with innovation fatigue or resistance.
              Getting value for money.
              Obtaining feedback on the results of particular policy measures intended
              to foster innovation.

      Support relevant research on technology in education according to
      national priorities and link these efforts to innovation.
          Research on technology in education is, compared to other areas of
      research in education, ill-served for a number of reasons. Education systems
      could greatly benefit from a national system of educational research on tech-
      nology which combines the following elements:
              Funding opportunities for researchers according to national priorities
              with international standards of quality.
              Capacity building with the co-operation of research centres and univer-
              sities, if possible in view of cooperation with international networks.
              Dissemination activities, particularly by means of tailored publica-
              tions, intended to engage a large range of stakeholders in the discus-
              sion of the implications of research evidence, who in some cases may
              require some additional capacity building.
              Setting up mechanisms for the involvement of those institutions or
              programmes responsible for initial and continuous teacher training.




                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                         CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS – 157



       Ensure that technology-based innovations do not reinforce existing
       digital divides or create new ones.
           The socio-economic background of the students is an important determi-
       nant for how they succeed in education. As the report on technology use and
       educational performance in PISA 2006 shows (OECD, 2010a), the analysis
       of PISA data demonstrates that for educational performance, computer use
       amplifies a student’s academic skills and competences, and these compe-
       tences are related to the student’s social, cultural and economic capital. The
       educational system should take this challenge seriously as access to comput-
       ers and broadband internet connection has become almost universal.

       Align or embed strategies for technology-based innovations with
       national policies for educational quality and equity.
            Having a separate technology strategy for education can serve several
       purposes. One purpose is the signalling effect a strategy can have – a strat-
       egy signals that the domain the strategy covers is an important priority and
       should get the attention it deserves. On the other hand, technology in educa-
       tion should in order to be sustainable be well aligned with national policies
       and priorities for quality and equity in education. One of the reasons for this
       is that it will underpin the pedagogical function of technology, where technol-
       ogy becomes a means to an end and not an end in itself.

The road ahead

            Technology has come to stay in our schools and access step by step
       becomes universal in most countries. What lies ahead? Rigid predtechnolo-
       gyions cannot be made, but a possible route ahead can be a “hybrid” phase,
       in which countries and systems still will focus on technology-related issues.
       The pace and dynamics of technological change will probably be drivers for
       such a focus. At the same time the need to embed technology in policies and
       strategies will remain and probably grow stronger. Previous and future work
       of the OECD will help inform the international educational community with
       important insights on what constitutes effective teaching environments and
       strategies as well insights into the building blocks of innovative learning
       environments. The TALIS report (OECD, 2009c) conveys important findings
       with regard to the building blocks of an effective teaching environment, and
       a future challenge is to increase our efforts to analyse how technology can be
       a major contribution to such environments, thus lowering the threshold some
       teachers experience with regard to using technology in their teaching.
           Another area that deserves attention is the relationship between public ser-
       vice and the formal system of schooling on the one hand and other stakeholders,



INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
158 – CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS

      such as non-formal schooling and private sector actors on the other hand. The
      Lumiar case from Brazil is a powerful expression of how the traditional system
      of schooling is challenges, and the formal education system should be open to
      innovations coming from “outside of the family”. Public service and government
      should be aware of the important role it has to play as an engine for innovation.
      The OECD report on measurement of innovation (OECD, 2010b) points to for
      instance a high level of readiness to develop and implement e-government ser-
      vices as a factor for an innovative public sector.
          A third area for further debate is the need to analyse the drivers of emerg-
      ing technological artefacts, especially with regard to underlying motives of
      such phenomena. Selwyn (Selwyn, 2009) offers a critical perspective to the
      emergence of Web 2.0 applications and the hype surrounding their uptake in
      education. As Selwyn points out, one of the drivers for this attention seems
      to be actors with a deschooling agenda.
           It has been said that no man is an island. This is also true when it comes
      to technology-based innovation in education. No actor or group of stakehold-
      ers can secure success and sustainability in this domain by themselves. Only
      through collaboration within education, between education and industry, and
      between education and other parts of public service, can we secure success
      for the benefit of all learners.




                                            INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
                                         CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS – 159




                                           References

       Johnson, L., R. Smith, A. Levine and K Haywood (2010), 2010 Horizon Report:
          K-12 Edition, The New Media Consortium, Austin, Texas.
       Johannessen, Ø. (2009), “In Search of the Sustainable Knowledge Base: Multi-
          channel and Multi-method?” in Scheuermann, F. and F. Pedró, Assessing the
          Effects of Technology in Education: Indicators, Criteria and Benchmarks for
          International Comparisons, European Union/OECD.
       Leadbeater, C., Open Innovation (Powerpoint), www.charlesleadbeater.net/cms/
          site/docs/Open%20Innovation.ppt.
       Lippoldt, D. and P. Stryszowski (2009), Innovation in the Software Sector, OECD
          Publishing.
       OECD/Eurostat (2005), Oslo Manual: Guidelines for Collecting and Interpreting
         Innovation Data, 3rd Edition, The Measurement of Scientific and
         Technological Activities, OECD Publishing.
       OECD (2009a), A Systemic Approach to Technology-Based School Innovations,
         background paper, OECD, Paris.
       OECD (2009b), Beyond Textbooks: Digital Learning Resources as Systemic
         Innovation in the Nordic Countries, Educational Research and Innovation,
         OECD Publishing.
       OECD (2009c), Creating Effective Teaching and Learning Environments: First
         Results from TALIS, OECD Publishing.
       OECD (2009d), Working Out Change: Systemic Innovation in Vocational
         Education and Training, OECD Publishing.
       OECD (2010a), Are the New Millennium Learners Making the Grade?
         Technology Use and Educational Performance in PISA, OECD Publishing.
       OECD (2010b), Measuring Innovation: A New Perspective, OECD Publishing.




INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
160 – CONCLUSION. LESSONS LEARNT AND POLICY IMPLICATIONS

      OECD (2010c), The OECD Innovation Strategy: Getting A Head Start on
        Tomorrow, OECD Publishing.
      Pedró, F. (2008), New Millennium Learners: A Project in Progress, OECD Centre
         for Educational Research and Innovation, Paris.
      Selwyn, N. (2009), Web 2.0 and The School of the Future, Today, University of
         London, London.




                                           INSPIRED BY TECHNOLOGY, DRIVEN BY PEDAGOGY – © OECD 2010
OECD PUBLISHING, 2, rue André-Pascal, 75775 PARIS CEDEX 16
                     PRINTED IN FRANCE
  (96 2010 12 1 P) ISBN 978-92-64-09478-9 – No. 57613 2010
inspired by Technology, Driven by Pedagogy
A SySTEmiC APPrOACh TO TEChnOlOgy-bASED SChOOl
innOvATiOnS
This report highlights key issues to facilitate understanding of how a systemic
approach to technology-based school innovations can contribute to quality
education for all while promoting a more equal and effective education system.
It focuses on the novel concept of systemic innovation, as well as presenting
the emerging opportunities to generate innovations that stem from Web 2.0 and
the important investments and efforts that have gone into the development and
promotion of digital resources. It also shows alternative ways to monitor, assess and
scale up technology-based innovations. Some country cases, as well as fresh and
alternative research frameworks, are presented.
Today, sufficient return on public investments in education and the ability to
innovate are more important than ever. This was the conclusion of the international
conference on “The School of Tomorrow, Today” organised by the OECD Centre
for Educational Research and Innovation with the support of the Secretariat of
Education of the State Santa Catarina (Brazil), in November 2009. The conference
and this resulting report share the overall goal of addressing the issue of how
education systems achieve technology-based innovations.




  The full text of this book is available on line via this link:
  	 www.sourceoecd.org/education/9789264094789
  Those with access to all OECD books on line should use this link:
  	 www.sourceoecd.org/9789264094789
  SourceOECD is the OECD’s online library of books, periodicals and statistical databases.
  For more information about this award-winning service and free trials, ask your librarian, or write to
  us at SourceOECD@oecd.org.




                                                      iSbn 978-92-64-09478-9

www.oecd.org/publishing
                                                               96 2010 12 1 P       -:HSTCQE=U^Y\]^:

								
To top