CfE Presentations2009 GudrunErickson

Document Sample
CfE Presentations2009 GudrunErickson Powered By Docstoc
					              SQA Seminar
           Glasgow, 8 April 2009


Sweden’s experiences in curriculum
  and assessment development

              Gudrun Erickson
      University of Gothenburg, Sweden
          Department of Education



                                         www.gu.se
                          Outline
                 • The Swedish school system

• Curriculum: current system – discussions – (proposed) changes

• Assessment: current system – discussions – (proposed) changes

       • Collaborative approaches to national assessment

            • National and international experiences

           • Concluding remarks – Common concerns

                        • DISCUSSION


                                                           www.gu.se
                     The Swedish school system
                            http://www.skolverket.se/sb/d/190

                    • Highly decentralized; main responsibilities at local level
• National level (e.g.): Education Act; Curricula; Syllabi for subjects, including grading criteria;
                                             National tests
                             • A goal and criterion referenced system
           • Preschool; 9-year compulsory school; upper secondary education; adult
       adult education; universities… (http://skolnet.skolverket.se/polopoly/utbsys-eng/)
             • > 98% of students continue to three-year upper secondary education
        • Development dialogues; individual development plans; written reports; formal
                           grading (3/4 point scale), from school year 8
                                • Teachers responsible for grading
                 • Extensive system of advisory national testing and assessment



                                                                                          www.gu.se
The Swedish National Agency for Education
 Advisory
  council

                                                                        Director-general´s
              Information                                                     office
                                        Director-general
                                        Deputy director general         Legal secretariat
             Internal audit                                            Information service




                                  National                   National
    Education                                                                           Administration
                                 evaluation                development

   Preschool and
                              Education statistics
                                                         Quality development   .       Human resources
 Compulsory school

  Upper secondary                                         School improvement                 Finance
                                Result analysis
      school

                                 Analysis and             Government funding                   ICT
  Adult education
                                   research

  Assessment and
      testing

                                                                                                www.gu.se
                 Syllabi and local planning



National      Why           What                   How?          When?
 level     this subject?   direction?            (Very limited    (Limited
                                                  guidance)      guidance)

 Local      Why this                               How           When
                               What (exact)
             to reach          subject matter?    will we        to teach
 level        goals?                              work?           what?




                                                                 www.gu.se
                  Balancing act




    Flexibility              Quality assurance
Local independence            Equity / Equality




                                                  www.gu.se
       Discussions & (Proposed) Changes
• Increasing clarity of curricula and syllabi, including indications of
                       essential content areas

                     • Increased quality control

    • More grade levels (from 3/4 to 6 + “no basis for grading”)

  • Earlier start of formal grading (from school year 8 down to 6)

• Earlier national tests; national tests in a wider range of subjects

     • Individual written documentation from school year one

         Ongoing revision of national curricula and syllabi


                                                                www.gu.se
   Aims of the current national testing/assessment
                       system
       • Enhance individual educational achievement;
• Clarify curricular subject goals and indicate clearly strengths
         and weaknesses in individual learner profiles;
  • Concretize curricular subject goals and grading criteria;
 • Enhance equity and fairness in assessment and grading;
     • Provide statistics for local and national analyses of
                   educational achievement

                       Advisory function

                                                          www.gu.se
       The national assessment system
                 should not:
• Determine the precise choice of content and methods
  To be done in collaboration between teachers and students

• Function as examinations
  Teachers to award grades based on the assessment of students’
  accumulated work;
  Consequently, the national tests have an advisory function;
  however, to what extent is not defined




                                                                www.gu.se
         National assessment materials
           • Formative materials / Diagnostic materials,
          including models for self and peer assessment
                   A widening range of subjects

      • National tests: Summative subject tests (mandatory)
            Extensive materials, including guidelines
      An increasing number of ‘core’ subjects (Sw, Maths, Eng,
                         Science…)

• Electronic assessment bank (formative & summative materials)
                     A widening range of subjects

The Swedish National Agency for Education commissions different
university departments in the country to take responsibility for test
                  development (and research)

                                                              www.gu.se
                Reactions to national tests
• Students usually positive to varied, authentic, “different” tasks/tests
• > 95 % of teachers positive to the national tests
 function, content, level of difficulty, guidelines, individual proficiency
 profiles, layout…
• [Teacher] discussions, e.g., about workload; weight/importance of
 test results; adaptation/accommodation; standards…

• [Political] discussions, e.g., about number of tests; aim(s) of national
 tests; role of national test; stability over time; reliability…




                                                                   www.gu.se
          Test grade: national subject tests (2008)
            year 9 – upper secondary education
             %   “Not pass”/   Pass   Pass with     Pass with
                 Fail                 distinction   special
                                                    distinction

Swedish
9                3%            41%      45%         11%
B                11%           46%      34%         9%
Mathematics
9                17%           48%      26%         10%
A                23%           45%      23%         9%

English
9                4%            35%      43%         18%
A                6%            39%      43%         11%



                                                              www.gu.se
     Final Grade & national Test Grade
   – end of compulsory school (year 9) – 2008

                FG<TG      FG=TG       FG>TG


Swedish         6%         79 %        15 %


Mathematics     2%         74 %        24 %


English         7%         84 %        9%


                                              www.gu.se
              Can teachers’ ratings be trusted?
• Joint study (2008) by the Swedish NAE and four university departments
   responsible for test development; to be published shortly on the NAE website

• Random sampling: 100 teacher-rated grade 9 tests of Swedish, Mathematics and
   English (oral subtests not included, since recording is not a requirement)

• Re-rating by three independent raters; 10-point scale

• To some extent, varying results between subjects (higher correlations for
   Mathematics and English than for Swedish)

• Example: English
   Listening & Reading comprehension (50/50 constructed and selected response):
   r = > .99; Written production (“essay”): r = .86-.93; Generalizability coefficient: .85
   [Analyses of oral test results in similar studies show comparable results]




                                                                                www.gu.se
Collaborative approaches to national assessment
            • Political level (NA) – Universities
       • University level (national and international)
• Test development (researchers from different disciplines,
           teacher trainers, teachers, students…)
                • Local test administration
                   • Rating by teachers
     • Reporting by teachers (results and reactions)
         • Data collection – Analyses – Research
            • Public reporting / Dissemination

                                                         www.gu.se
A collaborative test development process (FL)
• Analyses of literature, research, curricula, examples of tests
• Development of items and tasks based on common, explicit
  principles and specifications
• Piloting with a limited number of students > Adjustment of tasks
• Large scale pre-testing in randomly selected classes in the country
  (n ≈ 400 students/task); systematic collection of feedback from all
  participating teachers and students
• Analyses of test results and feedback (a "q+q” approach)
• Selection/Sequencing of tasks; Standard setting and Benchmarking
• Nation-wide test administration
• Collection of data / Analyses / Research > Reporting (publicly
  available on the web)


                                                                www.gu.se
                Contributors and contributions
• Teachers and Teacher trainers
  Development groups; Task/Item construction; Mini trials – Pre-testing
  (administrating, observing, analysing, discussing, reporting): Selection,
  composition, sequencing, standard setting; Rating and benchmarking;
  Reporting and responding after administration of tests…
• Students of ‘all’ ages
  Providing information and sharing their views in interviews and (regular)
  questionnaires in connection with pre-testing; comments on, e.g, relevance;
  perceived level of difficulty; content; vocabulary; clarity of instructions; time,
  speed of speech; Retrospective, task related self-assessment…
• Researchers
  Aspects of language; Gender; Quantitative properties of item and test data;
  Teachers’ handling of constructed response; Progression in tasks and tests;
  Dimensionality; Rater introspection; Test-taker and teacher feedback…


                                                                             www.gu.se
                     Current discussions
         • Number/Type of aims for national assessment system?

           • Clarifying/Strengthening the role of national tests?

  • Formative and Diagnostic materials – where, how and from/by whom?

• Effects and implementation of computer assisted testing and assessment?
               Aspects of validity? Linear – Sequential – CAT..?

     • Increasing standardization - if yes, concerning what, and how?

           • Clearer emphasis on aspects of reliability (equity)?

   • How to avoid narrowing the curriculum - negative impact/washback?

   • Emphasizing common basic principles for all types of assessment !

                                                                    www.gu.se
 Good [language] testing and assessment
according to approximately 1 400 teenagers in 10 European countries
                     Breadth and Variation
          Focus on ”usefulness” / communication
                       Learning potential
                              Clarity
                             Fairness
                          Enough time
                            Challenge

                    (Erickson & Gustafsson, 2005)
                   www.ealta.eu.org/resources.htm


                                                            www.gu.se
                                 EALTA
                          www.ealta.eu.org
                   European Association for
               Language Testing and Assessment
                           • Broad membership
Teachers, Teacher trainers, Developers of large scale testing systems
                           • Non-commercial
                     • Individual membership free
                 • Founded with support from the EU
         • Resources page; Discussion list for members
           Annual conferences (June 2009: Turku, Finland)

      797 individual members (41 European countries); 81 associate members
     (27 countries, 5 continents); 51 institutional members; 16 expert members

                                                                           www.gu.se
      EALTA Guidelines for Good Practice in
       Language Testing and Assessment
         Adopted in 2006; Currently translated into 34 languages
                Translations freely available on the web

        Address the activities of the three membership categories
                               via questions

      Initial emphasis on basic principles for all types of assessment:

                            TRANSPARENCY
Respect for students/examinees, responsibility, fairness, reliability, validity,
                collaboration among the parties involved



                                                                      www.gu.se
  Concluding remarks – Common concerns
 • Creating a reasonable balance between local and central initiative,
                      interpretation and responsibility

      • Maintaining high standards of validity as well as reliability
         • Aiming for positive impact on learning and teaching
• Creating a reasonable balance, and maintaining a close cooperation,
   between R & D – research and development / researchers and test
                             developers

• Further developing and elaborating methods of collaboration with wide
                          groups of stakeholders

  • Bridging the gap between formative and summative assessment –
                  emphasizing common, basic principles

                                                                   www.gu.se

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:9/14/2012
language:Unknown
pages:22