Andreas Schleicher

Document Sample
Andreas Schleicher Powered By Docstoc
					           1
           1
          International Benchmarking




                                                             International Benchmarking
                                                            What it means – what it takes
     11 September 2009
                                       Andreas Schleicher
American Diploma Project




                                                                   Washington, September 11, 2009

                                                                            Andreas Schleicher
                                                                    Head, Indicators and Analysis Division
                                                                      OECD Directorate for Education
Dimensions for educational benchmarking
                              Domain 1              Domain 2            Domain 3
                          Outputs and          Policy Levers        Antecedents
                           Outcomes             shape educational    contextualise or
                          impact of learning        outcomes        constrain ed policy

                           Quality and         Individ attitudes,   Socio-economic
Level     Individual      distribution of       engagement and      background of
  A        learner       knowledge & skills        behaviour            learners
                             Quality of        Teaching, learning Student learning,
Level   Instructional      instructional         practices and    teacher working
  B       settings            delivery         classroom climate       conditions
                            Output and           The learning          Community
Level   Schools, other    performance of        environment at         and school
  C      institutions       institutions             school          characteristics
                         Social & economic        Structures,        National educ,
Level    Country or        outcomes of           resource alloc        social and
  D       system             education            and policies      economic context
Dimensions for educational benchmarking
                              Domain 1              Domain 2            Domain 3
                          Outputs and          Policy Levers        Antecedents
                           Outcomes             shape educational    contextualise or
                          impact of learning        outcomes        constrain ed policy

                           Quality and         Individ attitudes,   Socio-economic
Level     Individual      distribution of       engagement and      background of
  A        learner       knowledge & skills        behaviour            learners
                             Quality of        Teaching, learning Student learning,
Level   Instructional      instructional         practices and    teacher working
  B       settings            delivery         classroom climate       conditions
                            Output and           The learning          Community
Level   Schools, other    performance of        environment at         and school
  C      institutions       institutions             school          characteristics
                         Social & economic        Structures,        National educ,
Level    Country or        outcomes of           resource alloc        social and
  D       system             education            and policies      economic context
Australia
Austria
                                                          A world of change – higher education
Czech Republic
Denmark
Finland
Germany
Greece
        Expenditure per student at tertiary level (USD)




Hungary
Iceland
Ireland
Italy
                                                            Cost per student




Japan
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Republic
Spain
Sweden
United Kingdom                                                                 Graduate supply
United States                                                                                    Tertiary-type A graduation rate
                                                  A world of change – higher education
Expenditure per student at tertiary level (USD)




                                                    Cost per student




                                                                                         United States


                                                                             Finland




                                                                       Graduate supply
                                                                                                         Tertiary-type A graduation rate
                                                  A world of change – higher education
Expenditure per student at tertiary level (USD)




                                                                                 Australia
                                                                United Kingdom         Finland




                                                                                                 Tertiary-type A graduation rate
                                                  A world of change – higher education
Expenditure per student at tertiary level (USD)




                                                                               Tertiary-type A graduation rate
                                                  A world of change – higher education
Expenditure per student at tertiary level (USD)




                                                                               Tertiary-type A graduation rate
                                                  A world of change – higher education
Expenditure per student at tertiary level (USD)




                                                                               Tertiary-type A graduation rate
                                                  A world of change – higher education
Expenditure per student at tertiary level (USD)




                                                                               Tertiary-type A graduation rate
                                                  A world of change – higher education
Expenditure per student at tertiary level (USD)




                                                                               Tertiary-type A graduation rate
                                                  A world of change – higher education
Expenditure per student at tertiary level (USD)




                                                                          United States




                                                                                           Australia


                                                                                          Finland




                                                                                      Tertiary-type A graduation rate
Dimensions for educational benchmarking
                              Domain 1              Domain 2            Domain 3
                          Outputs and          Policy Levers        Antecedents
                           Outcomes             shape educational    contextualise or
                          impact of learning        outcomes        constrain ed policy

                           Quality and         Individ attitudes,   Socio-economic
Level     Individual      distribution of       engagement and      background of
  A        learner       knowledge & skills        behaviour            learners
                             Quality of        Teaching, learning Student learning,
Level   Instructional      instructional         practices and    teacher working
  B       settings            delivery         classroom climate       conditions
                            Output and           The learning          Community
Level   Schools, other    performance of        environment at         and school
  C      institutions       institutions             school          characteristics
                         Social & economic        Structures,        National educ,
Level    Country or        outcomes of           resource alloc        social and
  D       system             education            and policies      economic context
                    Education Indicators   2009 edition of
                             Programme     Education at a Glance                                                                                                        14
                                                                                                                                                                        14




                                   0
                                           5
                                                   10
                                                          15




                    -10
                           -5
                                                                                  points




 B7.1
        Portugal                                                                Percentage

           Spain
   Switzerland
   Belgium (Fl.)
           Korea
  Belgium (Fr.)
          Greece
   Luxembourg
                                                                                           Salary as % of GDP/capita




        Denmark
           Japan
        Germany
          France
United Kingdom
  New Zealand
                                                                                           Instruction time




          Turkey
   Netherlands
            Italy
         Austria
                                                            Difference with OECD average




        Australia
Czech Republic
                                                                                           1/ Teaching time
                                                                                                                       as a percentage of GDP per capita (2006)




         Iceland
          Finland
                                                                                                                                                               Contribution of various factors




         Ireland
        Hungary
         Norway
 United States
                                                                                                                                                        to instructional cost per high school student




          Poland
                                                                                           1/ Estimated class size




Slovak Republic
Dimensions for educational benchmarking
                              Domain 1              Domain 2            Domain 3
                          Outputs and          Policy Levers        Antecedents
                           Outcomes             shape educational    contextualise or
                          impact of learning        outcomes        constrain ed policy

                           Quality and         Individ attitudes,   Socio-economic
Level     Individual      distribution of       engagement and      background of
  A        learner       knowledge & skills        behaviour            learners
                             Quality of        Teaching, learning Student learning,
Level   Instructional      instructional         practices and    teacher working
  B       settings            delivery         classroom climate       conditions
                            Output and           The learning          Community
Level   Schools, other    performance of        environment at         and school
  C      institutions       institutions             school          characteristics
                         Social & economic        Structures,        National educ,
Level    Country or        outcomes of           resource alloc        social and
  D       system             education            and policies      economic context
                                                                                     565    High science performance
                                                                                                          565                                   565
                                                                                                              Finland
   17
   17                                                                                                                       Average performance
                                                                                                                            of 15-year-olds on
      for developing assessments


                                                                                     545                      545
                                                                                                                            PISA in science
                                                                                                                                         545
                                                                                                              Hong Kong-China
      Lessons from PISA




                                                                                           Chinese Taipei  Canada
                                                                                 Italy           Estonia   Japan                          Spain
                                                                                                            New Zealand
                                                                                                            Australia
                                                                                     525                   525
                                                                                                           Netherlands                        La
                                                                                                                                             525 Rioja
                                                                                            Liechtenstein  Korea
                                                                                                            Slovenia                          Castile and Leon
                                                        Autonoma of Bolzano                United Kingdom Germany                             Navarre
                                                         Friuli Venezia Giulia             Czech Republic Switzerland                         Aragon
                                                                      Veneto                 Macao-China Austria
                                                                      Trento                      Ireland  Belgium
                                                                                     505                   505
                                                                                                           Hungary                           505
                                                                                                           Sweden                             Cantabria
                                                                                                                                              Basque Country
                                                                                                           Poland                             Asturias
                                                              Emilia Romagna                        France Denmark                            Galicia
                                                                                                  Iceland Croatia
               11 September 2009
                                   Andreas Schleicher




                                                                      Piemonte
International Assessment Seminar




                                                                                           United States Latvia
                                                                    Lombardia                              Slovak Republic, Spain, Lithuania Catalonia
                                                                                                   Norway 485
                                                                                     485                   Luxembourg                        485
                                                                                                              Russian Federation
                                                                                                 Portugal     Italy
                                                                      Liguria                                 Greece

                                                                                     465                      465                               465
                                                                                                                                                Andalusia


                                                                                                              Israel

                                                                                     445                      445                               445
                                                                     16                          16         … 18 countries perform below this line
                                                                                                                             6       16                     6

                                                                                             Low science performance
International Assessment Seminar
               11 September 2009            Lessons from PISA
                  Andreas Schleicher
                                            for developing assessments                                                  19
                                                                                                                        19



                                                   100
                                                         120
                                                               140




                  0
                      20
                             40
                                       60
                                              80
       Turkey
      Hungary
        Japan
       Belgium
         Italy
     Germany
      Austria
  Netherlands
Czech Republic
         Korea
Slovak Republic
       Greece
  Switzerland
  Luxembourg
      Portugal
       Mexico
United States
     Australia
 New Zealand
        Spain
       Canada
       Ireland
     Denmark
        Poland
      Sweden
                                                                                                                      Consistency in quality standards




      Norway
                                                                     Variation in the performance of 15-year-olds in mathematics




       Finland
       Iceland
                                                                                                International Assessment Seminar
                                                                                                               11 September 2009                     Lessons from PISA
                                                                                                                       Andreas Schleicher
                                                                                                                                                     for developing assessments                                                                   20
                                                                                                                                                                                                                                                  20




                                                                                                                 -80
                                                                                                                            -60
                                                                                                                                         -40
                                                                                                                                               -20
                                                                                                                                                                                         100




                                                                                                                                                     0
                                                                                                                                                         20
                                                                                                                                                              40
                                                                                                                                                                   60
                                                                                                                                                                            80
                                                                                                      Turkey
                                                                                                     Hungary
                                                                                                       Japan
                                                                                                      Belgium
                                                                                                        Italy
                                                                                                                                                                          schools



                                                                                                    Germany
                                                                                                                                                                       Variation of




                                                                                                     Austria
                                                                                                                                                                    performance within




                                                                                                 Netherlands
                                                                                               Czech Republic
                                                                                                        Korea
                                                                                               Slovak Republic
                                                                                                      Greece
                                                                                                 Switzerland
                                                                                                 Luxembourg
                                                                                                     Portugal
                                                                                                      Mexico
                                                                                               United States
                                                                                                    Australia
                                                                                                New Zealand
                                                                                                                              schools




                                                                                                       Spain
                                                                                                                           Variation of




                                                                                                      Canada
                                                                                                      Ireland
                                                                                                                       performance between




                                                                                                    Denmark
                                                                                                       Poland
                                                                                                     Sweden
                                                                                                     Norway
                                                                                                                                                                                                                                                Consistency in quality standards
                                                                                                                                                                                               Variation in the performance of 15-year-olds in mathematics




                                                                                                      Finland
                                                                                                      Iceland
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 4.1a, p.383.
   21
   21 for developing assessments
                                                          Strengths and weaknesses of countries in science
                                                                             relative to their overall performance
                                                                                                 France
      Lessons from PISA




                                                                                                        Overall science score
                                                                                                        France=495


                                                              Science                                    Identifying scientific issues
                                                              competencies
                                                                                                         Explaining phenomena scientifically


                                                                                                         Using scientific evidence
               11 September 2009
                                   Andreas Schleicher
International Assessment Seminar




                                                                                                         Knowledge about science


                                                                                                         Earth and space
                                                              Science
                                                              knowledge                                  Living systems


                                                                                                         Physical systems



                                                        -35        -25          -15            -5             5              15            25         35




                                                                             OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 2.13
   22
   22 for developing assessments
                                                          Strengths and weaknesses of countries in science
                                                                             relative to their overall performance
                                                                                          Czech Republic
      Lessons from PISA




                                                                                                        Overall science score
                                                                                            France=495 Czech Republic=512


                                                              Scientific                                 Identifying scientific issues
                                                              competencies
                                                                                                         Explaining phenomena scientifically


                                                                                                         Using scientific evidence
               11 September 2009
                                   Andreas Schleicher
International Assessment Seminar




                                                                                                         Knowledge about science


                                                                                                         Earth and space
                                                              Scientific
                                                              knowledge                                  Living systems


                                                                                                         Physical systems



                                                        -35         -25         -15            -5             5              15            25         35




                                                                             OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 2.13
Dimensions for educational benchmarking
                              Domain 1              Domain 2            Domain 3
                          Outputs and          Policy Levers        Antecedents
                           Outcomes             shape educational    contextualise or
                          impact of learning        outcomes        constrain ed policy

                           Quality and         Individ attitudes,   Socio-economic
Level     Individual      distribution of       engagement and      background of
  A        learner       knowledge & skills        behaviour            learners
                             Quality of        Teaching, learning Student learning,
Level   Instructional      instructional         practices and    teacher working
  B       settings            delivery         classroom climate       conditions
                            Output and           The learning          Community
Level   Schools, other    performance of        environment at         and school
  C      institutions       institutions             school          characteristics
                         Social & economic        Structures,        National educ,
Level    Country or        outcomes of           resource alloc        social and
  D       system             education            and policies      economic context
                                                                                       High policy value
   24A real-time assessment
   24 for developing assessments
                               Must haves                                                                     Quick wins
               environment that bridges the
                gap between formative and
                                                       Examine individual, institutional
                  summative assessment .
                                                      and systemic factors associated
      Lessons from PISA




                       Extending the range of              with high performance
                     competencies through which
                       outcomes are assessed
                                              Monitor educational progress
                      Measuring growth in learning
                                                        Low feasibility                                                    High feasibility
               11 September 2009
                                   Andreas Schleicher
International Assessment Seminar




                                                             Assuming that every new               Establish the relative standing
                                                             skill domain is orthogonal              of states on international
                                                                    to all others                            standards



                                                                          Money pits                       Low-hanging fruits

                                                                                       Low policy value
American Diploma Project
     11 September 2009       International Benchmarking
        Andreas Schleicher                                25
                                                          25



                               What it takes…
  26
  26                                                               OECD’s PISA assessment of the
                                                                 knowledge and skills of 15-year-olds
          International Benchmarking




                                                                       Coverage of world economy 83%
                                                                                                 86%
                                                                                                 85%
                                                                                                 81%
                                                                                                 77%
                                                                                                 87%


                                                                Subnational/regional PISA assessments in
                                                                 Country          Coordinated by           Reported at
                                                                Australia        National authorities     National level
                                                                Belgium          Regions                  OECD level
     11 September 2009
                                       Andreas Schleicher
American Diploma Project




                                                                Brazil           Regions                  National level
                                                                Canada           National authorities     National level
                                                                Germany          National authorities     National level
                                                                Italy            Regions                  OECD level
                                                                Mexico           National authorities     National level
                                                                Spain            Regions                  OECD level
                                                                Switzerland      Regions                  National level
                                                                United Kingdom   Regions                  OECD level
  27
                                                                                   What it takes
  27      International Benchmarking



                                                            r   Implementing PISA
                                                                volume of the tests, e.g.
                                                                     – 3½ hours of main assessment area
                                                                     – 1 hour for each of the minor assessment areas
                                                                each student
                                                                     – 2 hours on paper-and-pencil tasks (subset of all questions)
                                                                     – ½ hour for questionnaire on background, learning habits,
                                                                       learning environment, engagement and motivation
                                                                school principals
     11 September 2009
                                       Andreas Schleicher
American Diploma Project




                                                                     – questionnaire (school demography, learning environment)
                                                            r   Alternatives
                                                                   Suitable if state performance is main interest
                                                                     – Curriculum match / assessment match
                                                                     – Embedding PISA items in state tests
                                                                     – Embedding state items in PISA tests
                                                                   Requires coherence between assessment
                                                                    frameworks .
  28
                                                                                      Test Items
  28      International Benchmarking




                                                            r   A unit structure
                                                            r   Authentic stimuli/contexts
                                                            r   High proportion of constructed response items
                                                                   Multiple-choice
                                                                    Short constructed responses
     11 September 2009
                                       Andreas Schleicher
American Diploma Project




                                                                


                                                                   Open constructed responses .
  29
  29
                                                                           Main products
          International Benchmarking




                                                            r   A set of basic indicators that provide policy
                                                                makers with a baseline profile of the
                                                                knowledge, skills and competencies of
                                                                students in their state relative to those in
                                                                other countries
                                                            r   A set of contextual indicators that provide
                                                                insight into how such skills relate to
                                                                important demographic, social, economic and
     11 September 2009
                                       Andreas Schleicher
American Diploma Project




                                                                educational variables
                                                            r   Trend indicators that become available
                                                                because of the on-going, cyclical nature of
                                                                the data collections
                                                            r   A knowledge base for further focused
                                                                policy analysis.
  30
  30      International Benchmarking



                                                               www.oecd.org; www.pisa.oecd.org
                                                                 – All national and international publications
                                                                 – The complete micro-level database
                                                               email: pisa@oecd.org

                                                            
                                                                           Thank you !
                                                                Andreas.Schleicher@OECD.org
     11 September 2009
                                       Andreas Schleicher
American Diploma Project




                                                            … and remember:
                                                                Without data, you are just another person with an
                                                                opinion

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:10
posted:11/30/2011
language:English
pages:28