Canadian Language Benchmarks Assessment

Document Sample
Canadian Language Benchmarks Assessment Powered By Docstoc
					       Canadian Language
       Benchmarks (CLB)
                 Presented by
Elena Durette, Kerry Louw and Bonnie Nicholas
                  EDPY 593
                March 3, 2008
L2 Assessment in Canada before
the CLBA
                           I have a student who
      At our school, if       just moved from
  students know the past   Ontario. He was in an
   tense, we put them in     intermediate class
  an intermediate class.   there, but here he‟s a
                               high beginner.
What are the Canadian Language
Benchmarks (CLB)?
 1991: Immigration Canada report to Parliament
 1992: Consultation workshops
 1993: National Working Group on Language
   1995: Draft CLB document field-tested
   2000: CLB 2000 published
   Ongoing: CLB are used for Language Instruction
    for Newcomers to Canada (LINC) classes, Stage
    I, Benchmarks 1 through 4
   Stages II and III are used at higher levels
Schematic Structure of the
Canadian Language Benchmarks
 12 benchmarks, divided into three proficiency stages
The benchmarks are . . .
 a descriptive scale of communicative proficiency
 a framework of reference
 a national standard
 competency-based
 criterion-referenced
 learner-centred
 on a continuum
 task-based
 subjective
 analytical
  The competency-based approach
 Focuses on „what‟ learners should be able to do

 Is concerned with attainment of specific standards
               criterion-referenced not norm-referenced
 Was first adopted in vocationally-oriented education and later
               accepted by policymakers and leaders in curriculum
               development as a “state-of-the-art” approach to ESL

 A CLB competency statement contains a task and a number of
     of „how well‟ or standard statements (example of Writing Benchmark 3)
               Task: Write a short text about a personal or familiar situation.
               Performance Indicators: Conveys the message, reader can follow, uses
                 appropriate language and content, describes time and location with precision,
                 makes few errors in grammar, punctuation and spelling

Source: Integrating CLB Assessment into your ESL Classroom (2005).
Development of the CLB
Assessment Tools
 1995: Peel Board of Education (Ontario)
  contracted to develop assessment instruments
  (Pierce and Stewart)
 1996: CLBA kit developed
 1998: Federal funding for the CCLB
 1998 - present: ongoing development of new
  assessment tools by CCLB
 2002: CLBPT developed
 Now: free online assessment PD offered
  from CCLB
   CLB Tests and Assessment Tools
   for Adult ESL
             CCLB Centre for Canadian
              Language Benchmarks                         Centre for
                                                        Education and
                                                        Training (CET)
          CLBPT                CLB Classroom
    (Canadian Language
      Placement Test)                                         CLBA
                                CLB Summative          (Canadian Language
                                 Assessment                Benchmarks
                                    (SAM)                 Assessment)
     LPT (Literacy
    Placement Tool)
                               (Workplace Language
      CELBAN                                           (Canadian Language
  (Canadian English                                    Benchmarks Literacy
      Language                     CanTEST                 Assessment)
Benchmarks Assessment      Canadian Test of English
     for Nurses)           for Scholars and Trainees
      CLB Assessment in Edmonton
              LARCC (Language Assessment, Referral and
              Counseling Centre) at Catholic Social Services

                Pre-screening by intake workers

         Less than 6 years                                                         CLBA
                                                    More than 6 years
          of education in                                                    (non-LINC client;
                                                     of education in
         home country (L1)                                                     higher stakes)
                                                    home country (L1)

     Literacy Placement Tool                           CLB Placement Test
               LPT                                           CLBPT
        (LINC; low-stakes)                              (LINC; low-stakes)
Source: Annette Kreider, Catholic Social Services
An example of the streaming process

CLBA was designed to be:
   Accountable to stakeholders
   User-friendly (efficient, reliable, cost-effective)
   CLB-compatible
   Able to place learners on a continuum
   Culturally accessible (not culture-free)

Sources: Norton Pierce, B. & Stewart, G. (1997). The Development of the Canadian Language Benchmarks
            Assessment. TESL Canada Journal, 14, 2.
         Norton, B. & Stewart, G. (1999). Accountability in Language Assessment of Adult Immigrants in
            Canada. The Canadian Modern Language Review, 56, 2, 223-244
What is the CLBA?
 “a task-based assessment instrument
 assesses English proficiency from Benchmark 1 to
    Benchmark 8
   tests English Listening / Speaking through one-on-
    one interview
   tests Reading by means of passage and response to
    short answer / multiple choice questions
   tests Writing by means of authentic, life-applicable
   The CLBA is recognized by language and adult
    training providers cross Canada and has
    attracted international interest.”
  CLBA materials

•CLBA kit is task-based
•Tests all four skills in
three separate
•Administered individually
(writing component can be
done in a group)
•Can take up to 4 hours to
•No cost to client at
federally-funded centres
 streamlined placement test; maximum 80 min
 can be administered in a group setting
 used to place adult learners in ESL programs
 4 parallel versions
 task-based
 has been “validated
to the CLBA scores”
The CLBLA and the LPT
 Assessment tools for literacy learners
 Both measure the degree of literacy in the
  learner's first language, and are used to
  place clients in literacy classes
 CLBLA is available in 26 languages
Original use of the CLBA
“. . . there has never been
   any suggested purpose for
   the CLBA other than ESL
   classroom placement.”

Source: Stewart, G. & Cohen, C. (2001). A response to a “Critical Review of the
   CLBA”. Contact, Spring 2001, p. 24
Current uses of the CLBA
The following are uses for the CLBA endorsed by the CCLB:
 to allow learners to understand their progress in learning ESL
 As a proficiency test for:
    eligibility determination for language training programs,
    needs assessment,
    referral to appropriate training;
 facilitating initial placement into programs or classes,
 development of programming to meet specific language skills
    to provide teachers, program administrators and funders with a
       potential program effectiveness indicator (Note: applicable
       only if the program has a CLB-based curriculum or curriculum
    to provide assistance to employers in the development of
       appropriate training programs for their ESL learner employees

Reliability and Validity Measures

 “The final (CLBA) tests are sufficiently
    reliable. . . In a low-stakes placement
    test, these findings were deemed
    satisfactory. If this had been a high-
    stakes, gatekeeping test for college
    entrance. . . We could not have been

   Source: Norton Pierce, B. & Stewart, G. (1997). The Development of the Canadian Language Benchmarks
            Assessment. TESL Canada Journal, 14, 2.
Assessing the CLB Assessment
 CLBA takes up to 4 hours and measures what a
  client can do
 CLBPT takes less time and measures what a client
  cannot do well¹
 CLBPT is “not as reliable as the CLBA. It has not
  been sufficiently validated.”²
 CLBPT “is validated to be reliable within 1 benchmark
  95% of the time” but “the study on the test is not a
  public document”. ³

   ¹
    ³private correspondence with Jennifer McKay, Project Manager, Assessment, CCLB
CLB Assessor Qualifications
 Knowledge of second language acquisition:
        300 hours ESL teaching experience and
        ongoing PD and
        formal training or professional portfolio
   Experience in adult ESL assessment
   Cultural sensitivity
   Interview skills
   Decision-making skills
   Proficiency in English

CLB Assessor Training
 CLBA certification requires 5 days of
  “intensive instruction, skill application and
 CLBLA (Literacy Assessment) requires CLBA
  certification, and is an additional 1.5 days of
 CLBPT requires a one-day training
 Cost: $750 for a license and material for 1
  assessor; $325 for each assessor
Summative Assessment Manual for
Teachers (SAM) at CLB 4
 Measures student achievement only after exposure to
    thematical and functional instruction
   For ESL teachers with adult learners in publicly funded
    programs (LINC and non-LINC)
   Based on the CLB framework, standards and curricula
   Piloted for reliability, validity and practicality
   Easy to use and administer
   Learner and context-sensitive
   Photocopiable

NOT TO BE USED FOR: literacy students, placement purposes or
  general proficiency testing, or for testing without prior instruction
  in the theme
SAM is not for:
 Students with literacy needs in their first language

 ESL learners who are literate in a non-Roman
  alphabet/script (i.e. Russian, Chinese, Arabic,
  Punjabi) who are in the early stages of applying their
  literacy skills to English

 “Administering SAM to literacy learners is not a fair
  assessment practice.” They should be assessed
  separately with a literacy-oriented tool.

                                                         p. 3
Key Concepts in SAM Assessment
 Reliable: designed to produce consistent
  results if all the teacher-assessors use the
  scoring criteria on the same tasks in the same
 Valid: designed to measure what it claims to
  measure - the achievement of selected CLB
  standards demonstrated through familiar
  topics and discourse/text formats.
SAM‟s Security Considerations
 “Validity depends on the security of its formal
    (standardized) assessment tools.”
   All teachers must ensure the security.
   “Care must be taken that the test is not compromised
    by rote memorization.”
   Not to be viewed by unauthorized people
   Should be stored in designated locked units
   Integrity rests on a firm commitment of teachers to
    securely handling and disposing of task copies and to
    safe record-keeping.
   (BUT . . . if you have $100, you can purchase a copy
    from the CCLB.)
  Administering a formal SAM
       5-task sequence
 Task 1: Speak-Listen – Student/teacher
     interview       (7 – 8 minutes)
 Task 2: Listen    (15 minutes)
 Task 3: Speak      (2 – 3 minutes)
 Task 4: Write      (20 minutes)
 Task 5: Read       (20 minutes)
                     65 minutes
You be the ESL test-taker:
    You be the ESL assessor. . .
SAM Reporting
 Language Competency Report: reports what
  the learner can do at Benchmark level and is
  issued at the student‟s request for reference
  purposes outside of ESL programs.

 Teacher Recommendations: A form for office
  use to record the assessing teacher‟s
The greatest challenge using the
CLB is assessment

It is important to me that what we do in the classroom is
   relevant and meaningful to learners. I don’t want to teach to
   a test. How can I assess learners based on the kinds of
   activities we do in class on an everyday basis?

I teach in an academic bridging program that has begun to use
   the CLB. We’ve always had end of semester tests that
   include reading, writing, and grammar. How can I develop
   ongoing assessment related to the CLB that is relevant to
   learners’ academic goals?

I’ve just started teaching in a program that uses the CLB
   standards. This CLB document looks overwhelming. How will
   I know whether learners are achieving what is expected at
   Benchmark 2? Where do I start?         (Integrating CLB Assessment)
Integrating CLB Assessment into
your ESL Classroom (2005)

   Helps teachers with assessment practices
   Integrates teaching and learning activities into the classroom
   Relates to the CLB standards (CLBPT and CLBA)
   Meets program goals and the needs and interests of learners.

Formative assessment
    - ongoing feedback about how learners are doing
    - identifies what they need to work on
    - assists teachers with planning activities

Summative assessment
    - measures achievement of specific CLB outcomes
Assessment for Learning Principles
 Clear outcomes and assessment criteria
 „How‟ students learn along with „what‟ they
  learn is embedded in teaching
 Emphasizes progress and achievement
 Shared understanding of goals (CLB CanDO
  posters explain expectations)
 Constructive guidance on how to improve
 Self-assessment leads to self-management
Effective Assessment Practices
Lead to Significant Improvements in Learning

 Sharing expectations and criteria with learners
 Focusing on comments-only feedback
 Encouraging self assessment

CLB Assessment improves learning and motivation.
Teachers become focused in giving feedback.
An Assessment Process Model

                      Planning for Learning and Assessment

 Using Information for Planning            Setting Up Tasks and Collecting Information

                   Analyzing Information and giving Feedback
Back to: Who uses the CLB 2000?

 TESL professionals in educational institutions

 Adult ESL learners are the primary intended

 Learners become aware of the CLB mostly through
  their ESL instructors who share and interpret the
  main ideas and can help them achieve a CLB
                                        ( CLB 2000 )
ESL schools use the CLB 2000
   for teaching and testing
 12 levels of English listening, speaking, reading and writing.
       Level 1 students have little or no English
       Level 12 students are fluent speakers of English
       Level 9 or above usually have enough English for university,
        college or professional communication.

    Post-secondary institutions and employers refer to CLB levels.

 The CLB tells what people can do in English in daily communication.
       Can you answer questions about your name, address and phone
       Can you fill out an application form or write a memo?
       Can you listen to a news report on the radio?
       Can you write a report for your boss?

   The CLB summary checklists at each level help ESL learners mark
   language-learning progress and articulate goals.
CLB Resources for the Teacher

The CLB framework helps teachers plan units of work that support
  learners to develop their communicative language skills through
  real world tasks and activities.

Classroom teaching resources referenced to the CLB:

CLB 2000 A Guide to Implementation (2001)
CLB 2000 Additional Sample Task Ideas (2002)
LINC Classroom Activities      Books 1 – 4   NorQuest College
Canadian Snapshots             Level 3 and 6 Longman
CLB 2000 ESL for Literacy Learners (2001)

CLB publications and resources
       CLB / CLB Assessments I
         Pros          Cons
national standardization              teaching to the test
accountability to funders             restricts the language learning to
“survival-English” skills             functional
Teacher-friendly assessment tool      teachers lack training and power
in SAMs                                to administer CLBPT
Functions can be expressed in         There are no objective means for
                                       deciding that one functional item is
many different ways and at many
                                       more complex than another
different levels of complexity
                                       many LINC classes have non-
CBLT (competency-based
language teaching) is a rational       literate in L1 as L2 learners
approach to adult education             CBLT supports our social order
CBLT is geared towards adult               and need for a working class &
learners successfully functioning as        is designed to teach compliance
equal members in a changing
society                                 Has become high-stakes
                                          testing for some
    CLB / CLB Assessments II
        Pros                              Cons
Readily-available resources, and   CLB drives the market
more being published (books and     Complexities of funding issues
online resources)                   Reliability and validity issues
Thematically-based, context-       Themes can be restrictive for
embedded                            teachers
CLB gives a framework for          CLB can be overwhelming for
beginning teachers                  beginning teachers
                                    Few testing centres; assessment
                                    training is expensive
                                    Reliability and validity issues
                                    ESL schools often retest students
                                    who have been assessed at testing
                                    Disconnect from ESL learners in
                                    the K-12 system
Transformative Education
 While not perfect, it is possible that
  “competencies become tools in service of a
  transformative education, rather than a
  constraining framework for instruction.”
                         Auerbach (1988)
   Auerbach, E. R. (1988). Competency-based ESL: one step forward or two
    steps back? TESOL Quarterly, 20.
   Centre for Canadian Language Benchmarks.
   CLB 2000. Centre for Canadian Language Benchmarks. Citizenship and
    Immigration Canada
   Centre for Education & Training. (CLBA website)
   Cohen, Carolyn. (2003). From Art to Science with Art: LINC Assessor as ESL
    Professional. Association of Teachers of English as a Second Language of
    Ontario, Contact, Vol. 29, No. 2, Spring 2003.
   Fox, J. & Courchêne, R. (2005). The Canadian Language Benchmarks (CLB): A
    Critical Appraisal. Contact, Vol. 31, No. 2, Special Issue 2005.
   Hudson, T. (2005). Trends in Assessment Scales and Criterion-Referenced
    Language Assessment. Annual Review of Applied Linguistics (2005) 25, 205–
   Integrating CLB Assessment into your ESL Classroom (2005). Centre for
    Canadian Language Benchmarks. The Government of Alberta and
    Citizenship and Immigration Canada
   Norton Pierce, B. & Stewart, G. (1997). The Development of the Canadian
    Language Benchmarks Assessment. TESL Canada Journal, 14, 2.
   Norton, B. & Stewart, G. (1999). Accountability in Language Assessment
    of Adult Immigrants in Canada. The Canadian Modern Language Review,
    56, 2, 223-244
   Rossiter, M. J., & Pawlikowska-Smith, G. (1999). The use of CLBA scores
    in LINC program placement practices in western Canada. TESL Canada
    Journal, 16, 39-52.
   SAM (2004). Centre for Canadian Language Benchmarks. The
    Government of Alberta and Citizenship and Immigration Canada
   Stewart, G. & Cohen, C. (2001). A response to a “Critical Review of the CLBA”.
    Contact, Spring 2001, p. 24. (Note: we were unable to locate a copy of the
    original article.)
   Watt, D.L.E. & Lake, D.M. (2001). The CLBA-TOEFL Comparison
    Study.Contact, 27(2), 10-13.

     Thanks to Annette Kreider, LINC team Leader, Catholic Social Services
       and Jennifer McKay, Project Manager, Assessment, Centre for
       Canadian Language Benchmarks.
Selected online resources
    This is the home page of the Canadian Language Benchmarks. The site can be a bit confusing to navigate,
    but there is a wealth of information here.
    This is the homepage of the CLBA and the CLBLA.
    This link will take you to the CLBA Projects, Benchmarks Competency Assessment training: A Guide to
    Assigning Benchmarks.
    This is the CLBA information package published by SAIT for its students.
    A comparison of which proficiency tests the U of Calgary accepts and the cost of each test.
    This page shows equivalencies among CLBA, TOEFL and other standardized tests accepted by NorQuest
    Citizenship and Immigration Canada (CIC) was the impetus behind the development of the CLB. This is a
    summary of a 2004 government report on linc, including a very interesting section on “The Adequacy of the
    Assessment Tools.”
    Proceedings of the Immigration, Integration and Language Public Policy Conference, held in Calgary in
    This is an annotated bibliography on assesment alternatives.
    This is the website for CAELA, the Center for Adult English Language Acquisition, which seems to be the
    U.S. equivalent of our CLB.