Fall 2009-2010 Steve Pearse, Writing Specialist for CAA Options "Identifying Problems with Writing COEs Before Submission"
The Writing Collection of Evidence
Identifying Problems with Writing COEs Before Submission…
• • • •
Reviewing Key Writing COE Guidelines Addressing content-related issues Considering prompt-related issues Reminding teachers & their student-writers about process-related issues Throughout the presentation: Sharing examples of strong, helpful support for writers
Essential Webinar Goals:
Presenting key guidelines for the Writing COE as they pertain to appropriate educator support Identifying examples of inappropriate educator assistance Identifying & explaining the significance of missing, weak, & misplaced process materials to student-writer performance Providing exemplars & explanations of helpful, appropriate student-writer support
The Writing COE in Context: Guidelines…
Let's first consider key Writing COE requirements, as published in the new 'Red Book'—the 2009-10 Comprehensive Guide—available for download on the COE Web site (http://coe.k12.wa.us).
Student-writer knowledge profile…designed for students who likely
possess the writing skills and knowledge assessed on the writing state high school assessment, but who have not successfully demonstrated writing proficiency under large-scale testing conditions.
COE content: process & product…work samples that demonstrate an
understanding of the writing process and the application of skills in idea development, organization, style and the use of conventions as described in the high school writing standards.
Regarding content-related issues…
Work samples placed in opposite (expository vs. persuasive) purposes
The Work Sample Documentation Form (p. I-22, Guidelines) determines which work samples—listed under either "Titles of Expository work samples" or "Titles of Persuasive work samples"—are to be scored during a given scoring session. This is the scoring protocol, regardless of which purpose box has been checked on a given individual Writing Work Sample Task Form.
Assessing for an apparently unintended purpose…
According to one Collection's Work Sample Documentation Form, Work Sample #5—Senior Class Trip—is listed as a persuasive writing response. In contrast, the prompt clearly indicates the expository purpose: As the Senior Class Trip approaches, you have taken on the task of planning the trip. In a multiple-paragraph essay for your class adviser and classmates, explain where the class will go and why.
Even though the prompt and the Work Sample Documentation Form that precedes it indicate the response was written with the persuasive purpose in mind, scorers must assess this student's work according to the features (see Checklist for Expository Writing) and the levels of performance (established Anchor Papers) for expository text.
As even lower-scoring (for Content, Organization & Style) persuasive purpose Anchors incorporate some persuasive qualities, the obviously wrongly-categorized student work can be at a distinct disadvantage.
Work Samples Placed Out of Context
Conducting a Sufficiency Review (see Guidelines, p. V-3) will ensure that a given Writing COE meets Sufficiency requirements. Just as important is confirming that each work sample (final student response) immediately follows its aligned Writing Work Sample Task Form in the order listed on the Work Sample Documentation Form. This will ensure that the intended work samples for each purpose are scored appropriately (see p. I22).
Scoring problem: Prompt vs. Response
Example: a work sample makes its appearance immediately following a prompt that stipulates a topic & purpose not addressed by that response. Opening paragraphs of the student work: I agree that School Board want that all students who do not meet standard on the WASL will be required to enroll in summer school. I agree with this because many students are lazy and ignore school programs and then we can see result[s] of their study with the help of this test. Prompt as indicated on the preceding page (WS Task Form): In a multiple-paragraph letter to [an exchange] student new to your school, explain the important features or aspects of your community.
Prompts designed or modified for the COE
One of the COE's strengths is its flexibility, allowing teachers to select, create, or modify existing tasks/prompts for individual student-writers.
• Sample prompts appropriate for the Writing COE:
-Teacher-created prompts appearing within a submitted Collection: Write a letter to your parent(s) to persuade them as to whether they should buy you your own car when you graduate from ____ High School. -Prompts written by COE staff-trained educators and available on the COE Web site (email@example.com): At ____ High School, we require all juniors to complete a job shadow…by May of their junior year. Write an essay to your Advisory Teacher in which you explain the most important things you learned from your job shadow experience.
Example classroom task not modified for the Writing COE:
Write your reflection paper as an essay or friendly letter that fully explains your project. Use the following prompts as a guide for your reflection:
[Seven bullets follow, each directing the student-writer to address a specific feature of the project in a particular way…]
• Describe your overall project: summarize what was created or accomplished. • Explain the process you went through. • Describe new skills that you have learned or improved. • Write about any challenges or obstacles that you met or overcame. • Explain how your project has benefited you in terms of personal growth… RESULT: With a smorgasbørd of topics to be 'covered,' no reference to intended audience, and no single purpose indicated, the task is vast and potentially confusing. The resulting student response takes the form of a wide-ranging report as opposed to a focused explanation.
Inappropriate teacher scaffolding or assistance…
Do not provide direct, explicit teacher assistance that changes the outcome of the student work. —2009-10 COE: A Comprehensive Guide, p. I-20 Because it can affect student work to the point of changing the outcome, inappropriate teacher scaffolding or assistance is a contentrelated issue.
The teacher role for the Writing COE is limited to providing:
(1) appropriate instruction BEFORE students begin to work on a given task (work sample) (2) COE—appropriate prompts/tasks (3) sufficient time & opportunity to complete a COE (4) minimal teacher assistance, when permitted, that supports but does not affect the outcome of the student's writing process
Also regarding teacher assistance…
Teacher assistance on extended-time work samples must be limited to general advice ( 'Be sure to review your draft for ways to appeal to your audience') or reminders ('Don't forget to check for and correct those sentence formation errors'). Teachers are expected to refrain from providing directive, explicit assistance that could compromise the authenticity of student work and affect its outcome. No teacher assistance—orally or in writing—is allowed as students respond to on-demand prompts (please see Guidelines, p. I-19), as is the case for the HSPE in Writing.
Examples of overly-specific directions, scaffolding…
Inappropriate scaffolding as embedded within prompts:
After watching the Secondhand Lions, write a multiparagraph essay explaining Walter's (the main character's) conflicts. Also explain how Walter influenced his uncles and how his uncles influenced him. Then describe a problem you have faced and explain how someone helped you or helped someone else.
CONCERN: The directions embedded within this prompt make the student-writer's organizational and contentrelated decisions for her, including sequencing (then…). In addition, the prompt requires the student to write for an inappropriate COE purpose: describe.
Overly-specific directions, scaffolding…
A television network is looking for ideas for a new television show for teenagers. Write a letter to the president of the network explaining your idea for the new television show.
This prompt/task should have concluded at this point. Had that been the case, the student-writer would have had sufficient opportunity to determine what to include (ideas & content) and how to present (organization) that content for the stated purpose and audience.
But the directions continued beyond providing the prompt, explicitly scaffolding the writer's approach: Include all the information that will help the president evaluate your idea, including the show's title, what kind of show it is (such as reality, comedy, game show or sports), specific details or features that would be appealing to teenage viewers, and what viewers might see in a typical episode.
Indications of inappropriate teacher—and peer—assistance also sometimes appear within student-writer explanations of process:
For this one…I got help from some of my friends and one of my teachers helped me correct some mistakes. I had some trouble coming up with ideas so I asked some of [my] teachers for help.
A second example of inappropriate teacher and peer assistance appears within this work sample explanation:
Next, my paper was peer-edited. Peer-editing was done by using a modified State writing rubric. I had to make more revisions based on what my friend did. I gave my final paper to my teacher for grading. My teacher used a State writing rubric to grade. If I did not get almost perfect scores, I made more revisions.
Indications of inappropriate teacher assistance also appears within some Work Sample-specific process materials—prewriting documents, drafts, revisions, and/or evidence of editing. The following teacher notes appearing on studentwriters' drafts "cross the line" between general advice and reminders to explicit revising directions:
—Let's not do a listed intro. Instead, tell me how hiphop lyrics make you feel… —Why? What does the reader learn about you? —Describe…'It starts as a feeling'… —What does he do? How does he appeal to her? Do you agree with the way ___ treats women? Or is he just feeling this way for her?
PROMPT-related issues…Multiple Purposes Some Writing COEs submitted for scoring did not provide effective prompts that:
• • •
focus on a single, COE-appropriate purpose ask the writer to address a specific audience direct the writer to use a COE-appropriate form (essay or letter)
Sample prompt addressing multiple purposes:
In a multi-paragraph letter to the architects, describe your ideal classroom (you can consider things like size and shape of the classroom, wall and floor surfaces, technology, storage, furniture, etc.) and explain why the things you want to include will help the learning process.
PROMPT-related issues, continued No indicated purpose, audience, or form: What are the effects of youth drinking? Multiple purposes + no indicated audience or form: Describe a time when you felt at a disadvantage because of your family's education level, cultural background, and/or financial status. Explain in detail how you worked through the situation. How have these types of experiences influenced your day-to-day action and/or your goals for the future?
Process reflecting appropriate teacher support…
Whether an explanation of the process taken to produce a given work sample, or actual process documents, this required material tracks the process leading to final drafts. It also confirms the authenticity of the student work, as this explanation demonstrates:
How to Make a Bagel Pizza
To complete the assignment, I used the writing process. I made a thinking map. I used the 'flow map' to organize my ideas. Then I made a rough draft. Then I typed my paper. When I typed my paper, I used the computer to check my grammar and spelling. Then I made a final draft that I turned into my teacher. This was the on-demand paper. The best thing that I like about this assignment is that I had a pizza for a snack and do [sic] the rough drafts in Spanish. I would organize my thoughts better to improve my work. I learned to do the steps of the pizza systematically, in order.
Strong Exemplars: Work sample process documents
Idea List: What Have I Learned?
• How to read, watching a movie called Harry Potter taught me to read • Vocabulary! Reading has taught me new vocabulary words • Reading, TV and my mom taught me how to draw. Not school. • Reading comics taught me to read, draw and a l ot of vocabulary words
Strong Exemplars: Work sample process documents…
…generally takes the form of the student-writer's first attempt to produce complete, connected text. It may appear as a 'clean' text, free of revising notes or edits (corrections) or may include revisions and/or edits added at a later time in the process.
• Revising …or 're-seeing' an initial text for Content, Organization & Style (COS), may include any combination of adding, omitting, restating, or reordering text.
Process documents may feature text revisions either as a series of altered texts or as a single document featuring a variety of student-writer notations, inserts, cross-outs, and proofreading symbols.
Purposeful Revision: An Excerpted Paragraph
TOPIC: Mandatory "B" Grade Point Average DRAFT
They are going to be worried and they are going to be stress out. After they find out that they need a good grade point average, they don't know what to do. That's because they do not have a good grade point average. Also students are also not going to be able to think strait [sic], they are just going to be worried and they are going to feel helpless and think bad thoughts…
REVISION They are going to be worried and they are going to stress out. For example my cousin ____ needed to get a job because he needed to help his family. When ___ went to school he had a B grade average. When ___ got the job his grade went down because of his job and in his mind he needed to help his family with the bills and rent. But further on he had to drop out of school because it was getting stressful for him going to school…
Strong Exemplars: Work sample process documents… EDITING—'correcting' a text for Conventions (CONV)— may include attention paid to spelling, punctuation, capitalization, grammar & usage, sentence formation, and/or paragraphing indicators. Whereas some student-writers 'mark up' drafts or revisions with actual edits, others do so electronically, using standard word processing tools such as Spell Check and Grammar Check. In either case, final drafts— Work Samples—often reflect greater attention to the Conventions of Standard English.
Missing Process Materials…
Contrary to Writing COE Guidelines, many recently submitted COEs did not provide evidence of the use of the writing process or an explanation of the process followed for developing each work sample. In many cases, a lack of work sample-specific process materials correlated with poor final draft performance.
Scoring floor procedure:
Scorers are trained to locate, read, and assess final drafts only, as indicated by the Work Sample Documentation Form. Only the Director or Assistant Director locates process materials or explanations, and then as an authenticity check.
Weak Process Materials… Explanations
For all my writing, I did a brainstorm then wrote a draft. Then I checked for mistakes and retyped the final.
Such generic, one-size-fits-all statements simply summarizing the student-writer's general approach to writing tasks do not "…trace the development of a given work sample from beginning to end" (Guidelines, V-16). In addition, students are denied the opportunity to reflect upon specific elements of process and product, an important reflective skill, per the high school benchmarks, state standards.
Weak Process Materials… Documents
Weak process documents deny student-writers the opportunity to engage in a thorough, individualized process—from initial idea generation (pre-writing) to drafting, revising, and editing.
All Writing COE work samples—both on-demand and extended-time—benefit from evolving via a thoughtful, purposeful writing process.
Weak Process Materials… (continued)
With purposeful revising… layered elaboration—'saying more' through explanation, interpretation, clarification, and specification—often occurs when writers 're-see'—revise—initial drafts with a specific topic, audience, and purpose in mind.
When Washington state-assessed papers were analyzed, specific layered elaboration was the most critical element that differentiated between Content, Organization & Style (COS) scores of "2" and "3" and scores of "3" and "4."
—"Elaboration-Week 2" PPT, OSPI High School Instructional Support Materials for Writers (2006)
Misplaced Process Materials…
• Work sample explanations and/or process materials should be
placed in the back of the COE in the same order as the final drafts appear in the front of the COE (per Guidelines). • What to score? When process materials precede FINAL DRAFTS Example: A student wrote what turned out to be his/her initial draft handwritten just beneath the prompt as it appeared on the Work Sample Task Form. The actual final draft (as indicated) appears—revised, edited, and word-processed—on the following page of the Collection. Guidelines stipulate that process-related documents or explanations should be placed in the back of the COE in the same order as the final drafts appear in the front of the COE. This placement ensures that scorers will neither (1) inadvertently score text that does not represent the student's best work, nor (2) be in any way affected by early-draft performance prior to locating that final draft.
• Appropriate educator support + Student-Writer engagement Purposeful process = Best opportunity to meet standard _________ Thank you for supporting the Writing COE and Your Student-Writers! +
OSPI Lesley Klenk, CAA Options Administrator Lesley.firstname.lastname@example.org (360)725-6330, Fax(360)725-6332 Amanda Mount, COE Operations Specialist Amanda.email@example.com (360)725-6037, Fax (360)725-6332 Steve Pearse, COE Writing Specialist (360) 725-6037 firstname.lastname@example.org