Subject Leaders Questions – APP November 2008 Over 100 questions were posted during the three days of training. Many of the questions were similar in nature or subject matter. In order to ensure that this is a readable and usable document we have grouped the questions and, where a question was repeated, represented it as a summary question. Assessment Guidelines 1. Why does the prefixed phrase change from levels 1-3 “in most writing…” to levels 4+ “across a range of writing...”? This is simply intended to indicate that, as pupils obtain greater mastery of the secretarial aspects of writing, we would expect them to be applying their skills and understanding in a wider and wider range of contexts both in explicit literacy lessons and across the curriculum. 2. Why are the guidelines for maths and English not aligned? (e.g. pupil name and class, layout of judgement boxes at bottom of guidelines) We are currently reviewing the layout. As further material is developed, we will seek to make the layout as consistent as possible. 3. Are you planning on renaming the A3 criteria sheets „criteria‟? At the moment the Primary Framework also calls them guidelines. We have tried to be consistent and have always referred to the two level, A4 sheets as 'Assessment Guidelines'. In the MGP pilot, the A3 sheets are referred to as the assessment criteria. 4. Will there be any further exemplification of the criteria in writing i.e. further examples? No, it is not our intention to illustrate the guidelines with more examples nor to create additional standards files at this stage. However, ten new standards files for writing have been developed as part of the piloting of APP in KS1 and these are now available on the Framework website. 5. Will there be some support for teachers and teaching assistants to understand the terminology used on the guidelines? Some of the terms are unfamiliar and haven‟t been used commonly in the past. Yes, we are developing brief guidance which will be placed on the website. 6. Can we have assessment guidelines with space for notes at the bottom? We will look at the possibility of creating space. In the meantime, we suggest that you use sticky notes to annotate the criteria. 7. When will the L1 to L8 sheet be available? Following the completion of the KS1 APP pilot, the L1 criteria are finalized and have been added to the existing L2-8 sheets. 8. Fractions does not appear in the level descriptions at level 1 or level 2 but is in the AGs (rightly!) Are the level descriptions going to change? Are there other examples like this? There are no plans to change the level descriptions in advance of the recommendations of the current Primary Curriculum Review. In this case, the Guideline criteria simply reflect the fact that pupils are encountering the groundwork for later work on fractions e.g. through work on halving 9. Can we have L5/6 guidelines on the primary framework? There is a link on the primary framework to the full range of APP criteria (up to Level 8). Standards Files 10. There were a number of questions about the current Standards Files. Some minor updates to the APP Key Stage 2 standards files assessment guidelines on the National Strategies website have now been made to reflect the ‘making a judgement’ process more accurately. For example: the assessment guideline sheets in Jason and Trevor’s reading standards files indicated them as being both L3 and IE for AF7 and it is not effective practice to have two outcomes so these have now been updated some of the assessment guidelines used ticks to identify assessed judgements and this is too general. The use of colour highlighters to help identify progress is recommended as this more usefully helps to identify areas for improvement some of the highlighting has been accentuated to focus more clearly on where evidence is identified in the standards files and this includes some underlining to help exemplify transitional progress (see Harry Low 5 writing). The changes that have been made are: Subject Pupil Commentary Reading Peter AF5 and AF6 changed to IE = insufficient evidence Reading Charlotte AF5 changed to IE = insufficient evidence Reading Jason Change to guidance materials Reading Trevor Change to guidance materials: AF4 bullet 1 is not now highlighted AF5 L3 bullet is not now highlighted, both L4 bullets highlighted Reading Chas AF7 changed to IE = insufficient evidence Removed AF7 bullet 2 highlighting Writing Stephanie Update to guideline to indicate why low 2 Writing Jason AF8 and AF6 changed to BL L3 for HWP Writing Mikki Guidelines highlighted for greater visual clarity Writing Harry Highlighting slightly amended to show development at a bullet with an AF Writing Farida Highlighting on AG slightly amended to avoid confusion and identify bullets where there is development or growing evidence 11. Are there plans to include evidence from cross-curricular work in the current standards files or update them to include video and audio evidence? There are no immediate plans to update the existing files from KS2 because we will review the situation following consultation on the new primary curriculum during 2009. It is likely that QCA will need to develop a significant amount of new national exemplification material following the publication of the new Primary Curriculum but the current materials for KS1 and KS2 can be used for standardization until then. 12. When does high at one level become low at the next one, e.g. a high 3 become a low level 4? The obvious answer is, 'when the use of the assessment guidelines and associated guidance together with the teacher's professional judgement indicates this'. Assessment, whether by teachers or testing, is not a highly precise science and pupils have different profiles of strengths and areas for development. The critical issue is that, whether high 3 or low 4, pupil and teacher are both clear about what they need to do to ensure good progress in future learning. 13. When making an overall decision for mathematics, does the weighting between the attainment targets remain the same between all levels? We have tried to avoid any highly complicated weighting formulae in APP in order to emphasise the essentially 'best fit' nature of teachers' professional judgements. In the advice on reaching an overall judgement in mathematics, we have reminded teachers that AT2 (number and the number system) carries about twice the weighting of the other three attainment targets. In reading and writing, we have not recommended any specific weighting of the different assessment focuses but we have produced flowcharts for each aspect which take teachers through the suggested process of arriving at an overall level. 14. Why is data handling a separate element when it is part of number at KS1? Although AT4 (data handling) is indeed treated as part of AT2 in the current key stage 1 programme of study, feedback from teachers in the pilot schools reported that they found it useful to have the key criteria listed separately in the relevant column on the assessment guidelines. This supported their recognition of significant learning in children's data handling and helped them see continuity with future planning and learning. 15. Can you do level 2 reasoning about level 1 number? The idea of the assessment guidelines is that they encourage teachers to see the particular strengths and weaknesses of individual pupils and to make use of this information in ways that encourage further progress. Over time, you would expect to see evidence of pupils' learning in all assessment focuses although there is the option of indicating that, in any one period, there may be insufficient evidence to make a clear judgement. It is entirely possible for children to be operating at different levels in different assessment focuses – this is clearly illustrated in some of the Standards Files – but where evidence continues to be elusive in particular AFs, teachers might want to consider if they are providing children with enough opportunities to demonstrate their learning in this aspect. 16. Are we getting any more training files at KS1 or 2? There are no plans to develop more training files – the current Standards Files are designed for use in standardisation meetings within and across schools. 17. Are there any plans to give LAs electronic copies of files to be used for training? All files are downloadable as PDFs. For the new KS1 files, it is possible to download all video material separately if necessary. These can then be copied to CD or hard drives if that is the most useful way for them to be used. 18. We have had a number of complaints about the quality of the standards files. When they are printed from the site they are not as clear to see as they could be. Colleagues have found them difficult to read. It there anything that can be done? The files are designed to be viewed online. We have done our best to ensure the quality of the pdf versions is as good as possible, but we will review this again. 19. Can we be sure that levels 2 and 3 in KS1 standards files materials equate with the same levels in KS2 standards files? Yes, they have been extensively piloted to ensure that this is the case. 20. Will there be standards files for secure level 2 and low level 2 in both subjects? Yes, they are now available on the National Strategies Framework website. 21. What response should I give to teachers that disagree with the overall judgements in the standards files? Many of my pilot group would have assigned a lower level. They felt that the standards files awarded a „tick‟ for statements that they felt were not secure in the evidence. The judgements exemplify national standards and are based on the available evidence included in the files – examples of the pupils' work, observations and what else the teacher knew about their achievement. 22. Where is the evidence of guided work as a key opportunity for assessment in the standards files? For example, why is there no reference to guided reading or book band levels in Molly‟s info? The standards files are authentic collections of evidence for real pupils in real classrooms and reflect the curriculum they have been offered and the particular teaching approaches used in the school. There is considerable evidence from guided work across the range of files. Collecting evidence 23. Do you suggest that schools use the collection of evidence forms used with KS1 pilot schools? This may be appropriate for the one or two pupils selected for moderation purposes but 'collecting' evidence is not a general requirement of APP. As the DCSF pamphlet ‘Getting to Grips with Assessing Pupils’ Progress’ (DCSF-00129 -2009BKT-EN) states: ‘You don’t need to gather weighty collections or portfolios of pupils’ work.’ Periodic assessment using the APP criteria involves reviewing the available evidence which is already there from ongoing teaching and learning and day-to-day assessment to make a best-fit judgement. (The pamphlet can be found on the National Strategies website at http://nationalstrategies.standards.dcsf.gov.uk/app) Implementation of APP in KS1 24. KS1 materials – if we use all of these materials for 2010 are we expecting Y1 and Y2 teachers to be confident in using them too soon in comparison to KS2? Or will they pick it up quicker due to using similar methods before? KS1 materials have been produced in response to requests from schools to have the basis for a whole-school approach and for more exemplification material to support teacher assessment in Years 1 and 2. Evidence suggests that the APP approach is closer to existing practice in many KS1 classrooms and may therefore be easier to implement. Our advice continues to be to start small but across the school and work towards scaling up quickly to full implementation across reading, writing and mathematics. 25. Can you clarify “usually with support in the Level 1 writing guidelines?” Supporting learning is an important aspect of teaching at any stage. By scaffolding learning the teacher helps learners face the challenges of new ideas and skills, perhaps by breaking down the task into more manageable elements, leading the learner through an activity step by step, or by completing part of the task for them in order to allow the learner to concentrate on other aspects. As learners grow in confidence, teachers decrease the amount of support offered until children are completing a task independently and applying learning. In general, it is the independent skill that we assess, but if only independent achievement is recognised, in the early stages of learning, significant milestones of progress would be missed. For children working at or near level 1 their competence and confidence is developed through support. Assessment criteria need to recognise these aspects of performance. APP and Sub–levels 26. Is the diagram High/Secure/Low still current thinking? Are sub levels still referred to as low / secure / high or are they thirds? Are they applied similarly in the MGP pilot? Judgements for each Attainment Target are refined into 'Low', 'Secure' or 'High' within the level and are deemed to be equal. Teachers are asked to look across the AFs and decide whether the level is Low, Secure or High: Low, if there is highlighting above and below the line or the evidence is thin Secure, if across the AFs, the criteria for the level are mostly highlighted High, if across the AFs, all criteria for the level highlighted. There is guidance available to support teachers in arriving at these judgements on the National Strategies website. A good degree of professional judgement is also needed as there is no exact mathematical formula which has to be applied. This is also true in the MGP pilot. 27. How does the language of APP relate to the tracking language of a/b/c and how does this relate to APS? Some schools choose to record ‘high’, ‘secure’ and low’, broadly as a/b/c for the purpose of recording and tracking data. Rather than being based on a mathematical formula for sub-dividing a level, ‘high’, ‘secure’ and ‘low’ reflect the professional nature of the judgements that are made through APP, based on a teacher’s consideration of a range of evidence over time. Average point scores are simply a numerical representation of these sub-divisions and are useful for looking at the average performance of a cohort. 28. What do we say to headteachers who insist on tracking progress termly/half termly based on a system using a,b,c terminology? In terms of the frequency of tracking progress, our advice is that for pupils making expected progress, termly tracking is appropriate for the core subjects because it represents a period over which it is reasonable to find discernible progress. More frequent assessment against national standards is unlikely to yield useful information except where intervention is in place to ensure accelerated progress. A full guidance paper is available on the CD Leading and Managing APP (00650- 2008CDO-EN) and the tracking tutorial and tracking grid on the Primary area of the NS site shows how information from APP assessments can be used in tracking children’s progress (web link). Schools in the MGP pilot assess pupils as high, secure or low against the APP criteria as described above and report these to the DCSF as a, b and c. 29. How do we support teachers/leaders in separating APP standards/level descriptions from commercially produced criterion scales? The APP criteria are nationally agreed against national standards and as such are unique. They have been rigorously trialled in pilot schools. The process ensures teachers are taking information from a range of evidence and not just one piece of work. APP improves teaching and learning because it helps teachers (and pupils and their parents) understand both where their children are and how the outcomes of the assessment can inform planning and next steps at whole class, group and individual level. The process can be used throughout KS1, 2 and 3 and therefore provides a picture of progression and attainment throughout a child’s learning journey. KS1 Statutory Assessment 30. How will the APP language of low, secure and high impact on current judgements which are reported as a, b and c? Will statutory assessment reporting be changed and when? The Expert Group, set up after the withdrawal of end of KS3 tests to look at the future of testing and assessment at Key Stages 1, 2 and 3, is reviewing the current arrangements for statutory assessment to ascertain whether they are working well. We would not expect any changes ahead of their report, which is due in the spring. 31. Statutory assessment at the end of KS1 is based on teacher assessment. Why then do we report standards at the end of KS1 in the form of sub levels derived from the numerical division of a test mark scheme? Standards reported at the end of KS1 are based on teacher assessment. The Assessment and Reporting Arrangements (ARA) for KS1 for 2009 states that: ‘If teacher assessment and task and test results differ, the teacher assessment results should be reported, provided the judgement is based on an appropriate range of evidence from work done in class. Schools are not obliged to report task or test results ...’ The ARA also states that where teachers reach a level 2 judgement, they should then consider whether the performance is just into level 2, securely at level 2 or at the top end of level 2. This is consistent with judgements made using APP. These are then recorded as 2C, 2B or 2A. 32. Does the new ARA give equal status to teacher assessment and tests? Yes, as cited above, the ARA for 2009 states that where teacher assessment and test results differ, schools should report teacher assessment judgements. APP in KS3 33. If the guidelines are available to cover the range L1-L8 why is the process different? The KS3 model appears to be around one activity rather than the range of evidence, while in secondary English there are only low and secure. Why are the same messages not being given at primary and secondary? In the refreshed versions of APP for KS3 which were published on the Secondary National Strategy site on October 2008, we have ensured that the approach and messages, terminology and practice are as consistent as possible across phases, for example in the revised materials there are no longer suggested ‘tasks’ to assess English or Mathematics. The Assessment for Learning Strategy 34. Is the expectation that all teachers will be using APP for all children in reading, writing and mathematics by September 2009? Yes – but see below for further guidance 35. In a mature APP environment where teachers have worked with a sample and have a very good understanding of criteria and standards, does applying the criteria to all children involve filling in an APP grid for all children? Our advice at the conference was that starting with a sample group of children helps teachers to become familiar with and embed the assessment criteria and to ensure that there is a variety and range of evidence However, it is essential that teachers progress as rapidly as possible to using APP for every pupil. Once teachers are confident in their use of APP they will be able to use the assessment criteria in the guidelines to make teacher assessment judgements against national curriculum levels about all children in the class – i.e. use APP to make periodic assessments of all children in the class. The completion of the assessment guidelines aids this process at lots of levels – when teachers are learning, when schools are moderating judgements and when teachers are discussing particular children. By highlighting the criteria pupils have met on the assessment guidelines, teachers can see both strengths as well as gaps in learning for each pupil. It also helps teachers to track the progress of individual pupils over time. The completion of the guidelines is a tool to support the Assessing Pupils Progress system of teacher assessment; it is not the APP system itself. 36. What is the likelihood of APP becoming statutory? APP is not statutory. Teacher assessment is statutory – APP is offered to schools as a tool to support teachers' professional skills in assessment, improve planning and pedagogy and offer a more consistent and accurate basis for tracking progress, setting appropriate targets for improvement and sharing information with pupils and parents/carers. 37. Is the Standards Funding for AfL already in schools ring-fenced? If yes, what can we do to encourage schools who‟ve decided not to start on APP to use money to revisit this decision and start it? The funding is allocated through Standards Fund and is not ring-fenced. In 2009-10 it will be focused on helping schools who have not yet started using APP and on those that may need extra support to embed the process. Full details of the guidance are available in the standards fund letter to local authorities and further guidance was distributed at the Strategy Manager meetings in January 2009. 38. The AfL Strategy refers to having a trained „assessment specialist in every school‟. How does this fit in with crucial importance of all teachers being skilled in assessment? Having an assessment 'specialist' does not imply that other teachers in the school shouldn't be skilled in making professional judgements about the progress of the children in their care. The 'specialist' might be in a position to induct new staff in assessment policy and practice, coordinate standardisation and moderation activity, help ensure school-wide consistency of approach – in other words help maintain an efficient system for assessment across the school. We are currently working with the Chartered Institute of Educational Assessors and other partners to explore this role in greater depth. 39. Are end of KS2 tests likely to follow the path of end of KS3, or are they likely to be replaced by single level tests? If the latter, how will APP fit in with single level tests? National tests in KS2 are here to stay. It’s not an either/or choice between tests and classroom assessment; all good teachers track their pupils’ learning closely, and the evidence shows clear links to tracking and faster progress. The Making Good Progress pilot is also trying out strengthened teacher assessment using APP alongside ‘single level tests’ in primary schools and we may move to that model if the independent evaluation shows that this approach helps children do better at school. 40. Could you clarify the dates for online reporting to parents in primary and secondary? By September 2010 all secondary schools will be expected to provide information to parents covering achievement, progress, attendance, behaviour and special needs, on a timely and frequent basis. Primary schools must also report online to parents by September 2012. PNS Resources 41. Are there any plans to re-write the multi modal text document for use with APP? We are currently reviewing whether it would be appropriate to rewrite this document. 42. The Primary Framework includes writing targets – 2a, 2b, 2c – whereas APP uses low, secure and high. Can you explain this? The writing targets in the Support for Writing materials have been developed to support teachers who have identified specific gaps in learning, using information from day to day and periodic assessment practices, to teach specific writing skills, particularly through guided writing and 1:1 tuition sessions. A full guidance paper on the use of curricular targets is available on the CD Leading and Managing APP (00650-2008CDO-EN) and on the Primary Framework at http://nationalstrategies.standards.dcsf.gov.uk/node/102695 Please also see question 23 above. 43. Can we have L5 writing targets on the framework site? Yes, we are currently developing targets for Level 5 and these will be available on the Primary Framework. Miscellaneous 44. Are standards in teacher assessment the same as those in tests – can you clarify? The standards are the same but not all aspects of performance can be assessed through testing. For example, a test can’t tell you whether children read a wide range of texts but it can provide two or three types in a reading booklet and ask questions on them all in order to get a proxy measure on a single occasion. In mathematics, one or two questions are used to assess whether children know and can apply a particular operation. Tests provide a snapshot of pupils' achievements in a sample of activities which are drawn from the curriculum. Teacher assessment can draw on a much wider range of evidence representing the full curriculum. 45. Have we got a timeline for the speaking and listening materials? The Speaking and Listening materials are currently being piloted and will be published early in 2010. They will address levels 1 to 8 and the nature of the evidence will be cross curricular. 46. Are there going to be APP materials for foundation subjects in Primary? These are being developed for key stage 3 but there are no plans to do so for primary. This is principally because no one is expecting a primary teacher to assess all subjects in this way and we also need to see what the current Primary Curriculum Review recommends about subjects. 47. Are SIPS being trained on APP? Whose responsibility is this? National training for SIPs will ensure that the SIPs know about APP and understand its use and purpose. Local SIP CPD will need to ensure that all SIPs working within a local authority are able to ask schools the appropriate questions and then broker the support when schools need more help to develop APP. 48. How do schools demonstrate that their system is better than APP? APP is a nationally agreed assessment process against agreed national standards. It enables teachers to develop their understanding of progression through English and mathematics and to identify next steps in learning for individual pupils. We know that using APP to inform tracking systems works because teachers are confident in their own assessment and develop their confidence in their professional judgement. We also know that the process of learning and implementing APP is an extremely powerful form of CPD. A system that was ‘better’ than APP would have to be able to show the same level of scrutiny against national standards and the same outcomes in terms of impact on teaching and learning and on teachers’ professional development. 49. Are there plans to include P levels in APP? Please clarify when to start using P levels? Previous NAA conferences implied transfer should take place during/end of spring term in Y1. QCA is developing further exemplification of performance at p-levels. A full guidance paper on the appropriate use of p scales is available on the CD Leading and Managing APP (00650-2008CDO-EN). 50. How do you see the use of assessments from EAL learner in the context of APP? What is the place of an EAL / 1st language assessment? The Leading and Managing APP CD ((00650-2008CDO-EN) has a full guidance paper on using AP with children whose first language is not English. Information from assessments of children learning EAL will need to inform language development targets as well as ensuring progression in the core subject areas. For assessing reading, evidence should be gathered from across the curriculum and using a range of strategies through which children can demonstrate comprehension. When collecting evidence for mathematics, children who have used their first language to learn mathematics should be given opportunities to show their knowledge and understanding through the use of their first language as well as English. As with all learners, decisions about attainment levels need to be drawn from a range of evidence. The profiles of EAL learners are likely to be more uneven than those of other pupils and therefore it is essential that the widest range of evidence possible is used to make a periodic assessment. QCA will be developing further guidance materials on APP and EAL learners during 2009-10. 51. Will initial teacher training address APP so that NQTs come out well prepared to use APP? ITT providers have been introduced to APP via existing National Strategy networks and QCA has been liaising with the Teacher Development Agency. Further training for ITT providers is planned through the National Strategies this year and in 2009-10. 52. Is APP going to be shared at the NAA conferences in January – especially as it is mentioned in ARA RSI 2009? Where are Assessment Leads going to get their information and training? APP was shared at the January LA conferences organised by QCA. We would expect that the training and materials provided via the National Strategies will be shared with other relevant staff within your LA. 53. Is the APP agenda also a priority for the Greater Manchester Challenge? The introduction of APP as a means of providing robust and secure tracking data is a national priority. It will therefore be reflected in local agendas. 54. For schools that set, how will they get their cross curricular evidence for mathematics and English? The piloting of APP clearly showed that schools which did not set found it easier to draw on cross-curricular evidence for achievement in mathematics, reading and writing. The implication is not that schools should not set – that remains a school-level decision – but that if they do, they may need to work harder to ensure that evidence from the wider curriculum is taken fully into account. Teachers within year groups will need to liaise to ensure that all teachers can gather a range of evidence. It is useful to remember that observational evidence from another teacher may be a valuable form of evidence. Pupils also might be involved in the process of identifying work they have produced in other curricular areas that demonstrates good achievement. 55. When is the teacher‟s tv programme being shown? It can be freely watched at any time. The link is: http://www.teachers.tv/video/29809 56. Recent DCSF Raise online training emphasised question level analysis – is APP the priority for LAs being set by the DCSF or is it QCA optional tests? Question level data continues to be available for analysis in RAISE. Our suggestion would be for schools to explore the patterns emerging from completed APP guidelines to explore consistent strengths and areas for further development in the curriculum. RAISE does not have the functionality to analyse APP teacher assessments although there are plans to explore the feasibility of this.