CfE Presentations2009 GudrunErickson

Document Sample
CfE Presentations2009 GudrunErickson Powered By Docstoc
					              SQA Seminar
           Glasgow, 8 April 2009

Sweden’s experiences in curriculum
  and assessment development

              Gudrun Erickson
      University of Gothenburg, Sweden
          Department of Education

                 • The Swedish school system

• Curriculum: current system – discussions – (proposed) changes

• Assessment: current system – discussions – (proposed) changes

       • Collaborative approaches to national assessment

            • National and international experiences

           • Concluding remarks – Common concerns

                        • DISCUSSION

                     The Swedish school system

                    • Highly decentralized; main responsibilities at local level
• National level (e.g.): Education Act; Curricula; Syllabi for subjects, including grading criteria;
                                             National tests
                             • A goal and criterion referenced system
           • Preschool; 9-year compulsory school; upper secondary education; adult
       adult education; universities… (
             • > 98% of students continue to three-year upper secondary education
        • Development dialogues; individual development plans; written reports; formal
                           grading (3/4 point scale), from school year 8
                                • Teachers responsible for grading
                 • Extensive system of advisory national testing and assessment

The Swedish National Agency for Education

              Information                                                     office
                                        Deputy director general         Legal secretariat
             Internal audit                                            Information service

                                  National                   National
    Education                                                                           Administration
                                 evaluation                development

   Preschool and
                              Education statistics
                                                         Quality development   .       Human resources
 Compulsory school

  Upper secondary                                         School improvement                 Finance
                                Result analysis

                                 Analysis and             Government funding                   ICT
  Adult education

  Assessment and

                 Syllabi and local planning

National      Why           What                   How?          When?
 level     this subject?   direction?            (Very limited    (Limited
                                                  guidance)      guidance)

 Local      Why this                               How           When
                               What (exact)
             to reach          subject matter?    will we        to teach
 level        goals?                              work?           what?

                  Balancing act

    Flexibility              Quality assurance
Local independence            Equity / Equality

       Discussions & (Proposed) Changes
• Increasing clarity of curricula and syllabi, including indications of
                       essential content areas

                     • Increased quality control

    • More grade levels (from 3/4 to 6 + “no basis for grading”)

  • Earlier start of formal grading (from school year 8 down to 6)

• Earlier national tests; national tests in a wider range of subjects

     • Individual written documentation from school year one

         Ongoing revision of national curricula and syllabi

   Aims of the current national testing/assessment
       • Enhance individual educational achievement;
• Clarify curricular subject goals and indicate clearly strengths
         and weaknesses in individual learner profiles;
  • Concretize curricular subject goals and grading criteria;
 • Enhance equity and fairness in assessment and grading;
     • Provide statistics for local and national analyses of
                   educational achievement

                       Advisory function

       The national assessment system
                 should not:
• Determine the precise choice of content and methods
  To be done in collaboration between teachers and students

• Function as examinations
  Teachers to award grades based on the assessment of students’
  accumulated work;
  Consequently, the national tests have an advisory function;
  however, to what extent is not defined

         National assessment materials
           • Formative materials / Diagnostic materials,
          including models for self and peer assessment
                   A widening range of subjects

      • National tests: Summative subject tests (mandatory)
            Extensive materials, including guidelines
      An increasing number of ‘core’ subjects (Sw, Maths, Eng,

• Electronic assessment bank (formative & summative materials)
                     A widening range of subjects

The Swedish National Agency for Education commissions different
university departments in the country to take responsibility for test
                  development (and research)

                Reactions to national tests
• Students usually positive to varied, authentic, “different” tasks/tests
• > 95 % of teachers positive to the national tests
 function, content, level of difficulty, guidelines, individual proficiency
 profiles, layout…
• [Teacher] discussions, e.g., about workload; weight/importance of
 test results; adaptation/accommodation; standards…

• [Political] discussions, e.g., about number of tests; aim(s) of national
 tests; role of national test; stability over time; reliability…

          Test grade: national subject tests (2008)
            year 9 – upper secondary education
             %   “Not pass”/   Pass   Pass with     Pass with
                 Fail                 distinction   special

9                3%            41%      45%         11%
B                11%           46%      34%         9%
9                17%           48%      26%         10%
A                23%           45%      23%         9%

9                4%            35%      43%         18%
A                6%            39%      43%         11%

     Final Grade & national Test Grade
   – end of compulsory school (year 9) – 2008

                FG<TG      FG=TG       FG>TG

Swedish         6%         79 %        15 %

Mathematics     2%         74 %        24 %

English         7%         84 %        9%

              Can teachers’ ratings be trusted?
• Joint study (2008) by the Swedish NAE and four university departments
   responsible for test development; to be published shortly on the NAE website

• Random sampling: 100 teacher-rated grade 9 tests of Swedish, Mathematics and
   English (oral subtests not included, since recording is not a requirement)

• Re-rating by three independent raters; 10-point scale

• To some extent, varying results between subjects (higher correlations for
   Mathematics and English than for Swedish)

• Example: English
   Listening & Reading comprehension (50/50 constructed and selected response):
   r = > .99; Written production (“essay”): r = .86-.93; Generalizability coefficient: .85
   [Analyses of oral test results in similar studies show comparable results]

Collaborative approaches to national assessment
            • Political level (NA) – Universities
       • University level (national and international)
• Test development (researchers from different disciplines,
           teacher trainers, teachers, students…)
                • Local test administration
                   • Rating by teachers
     • Reporting by teachers (results and reactions)
         • Data collection – Analyses – Research
            • Public reporting / Dissemination

A collaborative test development process (FL)
• Analyses of literature, research, curricula, examples of tests
• Development of items and tasks based on common, explicit
  principles and specifications
• Piloting with a limited number of students > Adjustment of tasks
• Large scale pre-testing in randomly selected classes in the country
  (n ≈ 400 students/task); systematic collection of feedback from all
  participating teachers and students
• Analyses of test results and feedback (a "q+q” approach)
• Selection/Sequencing of tasks; Standard setting and Benchmarking
• Nation-wide test administration
• Collection of data / Analyses / Research > Reporting (publicly
  available on the web)

                Contributors and contributions
• Teachers and Teacher trainers
  Development groups; Task/Item construction; Mini trials – Pre-testing
  (administrating, observing, analysing, discussing, reporting): Selection,
  composition, sequencing, standard setting; Rating and benchmarking;
  Reporting and responding after administration of tests…
• Students of ‘all’ ages
  Providing information and sharing their views in interviews and (regular)
  questionnaires in connection with pre-testing; comments on, e.g, relevance;
  perceived level of difficulty; content; vocabulary; clarity of instructions; time,
  speed of speech; Retrospective, task related self-assessment…
• Researchers
  Aspects of language; Gender; Quantitative properties of item and test data;
  Teachers’ handling of constructed response; Progression in tasks and tests;
  Dimensionality; Rater introspection; Test-taker and teacher feedback…

                     Current discussions
         • Number/Type of aims for national assessment system?

           • Clarifying/Strengthening the role of national tests?

  • Formative and Diagnostic materials – where, how and from/by whom?

• Effects and implementation of computer assisted testing and assessment?
               Aspects of validity? Linear – Sequential – CAT..?

     • Increasing standardization - if yes, concerning what, and how?

           • Clearer emphasis on aspects of reliability (equity)?

   • How to avoid narrowing the curriculum - negative impact/washback?

   • Emphasizing common basic principles for all types of assessment !

 Good [language] testing and assessment
according to approximately 1 400 teenagers in 10 European countries
                     Breadth and Variation
          Focus on ”usefulness” / communication
                       Learning potential
                          Enough time

                    (Erickson & Gustafsson, 2005)

                   European Association for
               Language Testing and Assessment
                           • Broad membership
Teachers, Teacher trainers, Developers of large scale testing systems
                           • Non-commercial
                     • Individual membership free
                 • Founded with support from the EU
         • Resources page; Discussion list for members
           Annual conferences (June 2009: Turku, Finland)

      797 individual members (41 European countries); 81 associate members
     (27 countries, 5 continents); 51 institutional members; 16 expert members

      EALTA Guidelines for Good Practice in
       Language Testing and Assessment
         Adopted in 2006; Currently translated into 34 languages
                Translations freely available on the web

        Address the activities of the three membership categories
                               via questions

      Initial emphasis on basic principles for all types of assessment:

Respect for students/examinees, responsibility, fairness, reliability, validity,
                collaboration among the parties involved

  Concluding remarks – Common concerns
 • Creating a reasonable balance between local and central initiative,
                      interpretation and responsibility

      • Maintaining high standards of validity as well as reliability
         • Aiming for positive impact on learning and teaching
• Creating a reasonable balance, and maintaining a close cooperation,
   between R & D – research and development / researchers and test

• Further developing and elaborating methods of collaboration with wide
                          groups of stakeholders

  • Bridging the gap between formative and summative assessment –
                  emphasizing common, basic principles


Shared By: