Subject Leaders Questions APP by dfhrf555fcg


									Subject Leaders Questions – APP November 2008

Over 100 questions were posted during the three days of training. Many of the questions
were similar in nature or subject matter. In order to ensure that this is a readable and
usable document we have grouped the questions and, where a question was repeated,
represented it as a summary question.

Assessment Guidelines

1. Why does the prefixed phrase change from levels 1-3 “in most writing…” to
   levels 4+ “across a range of writing...”?

This is simply intended to indicate that, as pupils obtain greater mastery of the secretarial
aspects of writing, we would expect them to be applying their skills and understanding in a
wider and wider range of contexts both in explicit literacy lessons and across the

2. Why are the guidelines for maths and English not aligned? (e.g. pupil name and
   class, layout of judgement boxes at bottom of guidelines)

We are currently reviewing the layout. As further material is developed, we will seek to
make the layout as consistent as possible.

3. Are you planning on renaming the A3 criteria sheets „criteria‟? At the moment
   the Primary Framework also calls them guidelines.

We have tried to be consistent and have always referred to the two level, A4 sheets as
'Assessment Guidelines'. In the MGP pilot, the A3 sheets are referred to as the
assessment criteria.

4. Will there be any further exemplification of the criteria in writing i.e. further

No, it is not our intention to illustrate the guidelines with more examples nor to create
additional standards files at this stage. However, ten new standards files for writing have
been developed as part of the piloting of APP in KS1 and these are now available on the
Framework website.

5. Will there be some support for teachers and teaching assistants to understand
   the terminology used on the guidelines? Some of the terms are unfamiliar and
   haven‟t been used commonly in the past.

Yes, we are developing brief guidance which will be placed on the website.

6. Can we have assessment guidelines with space for notes at the bottom?

We will look at the possibility of creating space. In the meantime, we suggest that you use
sticky notes to annotate the criteria.

7. When will the L1 to L8 sheet be available?
Following the completion of the KS1 APP pilot, the L1 criteria are finalized and have been
added to the existing L2-8 sheets.

8. Fractions does not appear in the level descriptions at level 1 or level 2 but is in
   the AGs (rightly!) Are the level descriptions going to change? Are there other
   examples like this?

There are no plans to change the level descriptions in advance of the recommendations of
the current Primary Curriculum Review. In this case, the Guideline criteria simply reflect
the fact that pupils are encountering the groundwork for later work on fractions e.g.
through work on halving

9. Can we have L5/6 guidelines on the primary framework?

There is a link on the primary framework to the full range of APP criteria (up to Level 8).

Standards Files

10. There were a number of questions about the current Standards Files.

Some minor updates to the APP Key Stage 2 standards files assessment guidelines on
the National Strategies website have now been made to reflect the ‘making a judgement’
process more accurately. For example:

      the assessment guideline sheets in Jason and Trevor’s reading standards files
       indicated them as being both L3 and IE for AF7 and it is not effective practice to
       have two outcomes so these have now been updated
      some of the assessment guidelines used ticks to identify assessed judgements and
       this is too general. The use of colour highlighters to help identify progress is
       recommended as this more usefully helps to identify areas for improvement
      some of the highlighting has been accentuated to focus more clearly on where
       evidence is identified in the standards files and this includes some underlining to
       help exemplify transitional progress (see Harry Low 5 writing).

The changes that have been made are:

Subject    Pupil        Commentary
Reading    Peter        AF5 and AF6 changed to IE = insufficient evidence
Reading    Charlotte    AF5 changed to IE = insufficient evidence
Reading    Jason        Change to guidance materials
Reading    Trevor       Change to guidance materials:
                        AF4 bullet 1 is not now highlighted
                        AF5 L3 bullet is not now highlighted, both L4 bullets highlighted
Reading    Chas         AF7 changed to IE = insufficient evidence Removed AF7 bullet 2 highlighting
Writing    Stephanie    Update to guideline to indicate why low 2
Writing    Jason        AF8 and AF6 changed to BL L3 for HWP
Writing    Mikki        Guidelines highlighted for greater visual clarity
Writing    Harry        Highlighting slightly amended to show development at a bullet with an AF
Writing    Farida       Highlighting on AG slightly amended to avoid confusion and identify bullets
                        where there is development or growing evidence
11. Are there plans to include evidence from cross-curricular work in the current
    standards files or update them to include video and audio evidence?

There are no immediate plans to update the existing files from KS2 because we will review
the situation following consultation on the new primary curriculum during 2009. It is likely
that QCA will need to develop a significant amount of new national exemplification material
following the publication of the new Primary Curriculum but the current materials for KS1
and KS2 can be used for standardization until then.

12. When does high at one level become low at the next one, e.g. a high 3 become a
    low level 4?
The obvious answer is, 'when the use of the assessment guidelines and associated
guidance together with the teacher's professional judgement indicates this'. Assessment,
whether by teachers or testing, is not a highly precise science and pupils have different
profiles of strengths and areas for development. The critical issue is that, whether high 3 or
low 4, pupil and teacher are both clear about what they need to do to ensure good
progress in future learning.

13. When making an overall decision for mathematics, does the weighting between
    the attainment targets remain the same between all levels?
We have tried to avoid any highly complicated weighting formulae in APP in order to
emphasise the essentially 'best fit' nature of teachers' professional judgements. In the
advice on reaching an overall judgement in mathematics, we have reminded teachers that
AT2 (number and the number system) carries about twice the weighting of the other three
attainment targets. In reading and writing, we have not recommended any specific
weighting of the different assessment focuses but we have produced flowcharts for each
aspect which take teachers through the suggested process of arriving at an overall level.

14. Why is data handling a separate element when it is part of number at KS1?
Although AT4 (data handling) is indeed treated as part of AT2 in the current key stage 1
programme of study, feedback from teachers in the pilot schools reported that they found it
useful to have the key criteria listed separately in the relevant column on the assessment
guidelines. This supported their recognition of significant learning in children's data
handling and helped them see continuity with future planning and learning.

15. Can you do level 2 reasoning about level 1 number?
The idea of the assessment guidelines is that they encourage teachers to see the
particular strengths and weaknesses of individual pupils and to make use of this
information in ways that encourage further progress. Over time, you would expect to see
evidence of pupils' learning in all assessment focuses although there is the option of
indicating that, in any one period, there may be insufficient evidence to make a clear
judgement. It is entirely possible for children to be operating at different levels in different
assessment focuses – this is clearly illustrated in some of the Standards Files – but where
evidence continues to be elusive in particular AFs, teachers might want to consider if they
are providing children with enough opportunities to demonstrate their learning in this
16. Are we getting any more training files at KS1 or 2?

There are no plans to develop more training files – the current Standards Files are
designed for use in standardisation meetings within and across schools.

17. Are there any plans to give LAs electronic copies of files to be used for training?

All files are downloadable as PDFs. For the new KS1 files, it is possible to download all
video material separately if necessary. These can then be copied to CD or hard drives if
that is the most useful way for them to be used.

18. We have had a number of complaints about the quality of the standards files.
    When they are printed from the site they are not as clear to see as they could be.
    Colleagues have found them difficult to read. It there anything that can be done?

The files are designed to be viewed online. We have done our best to ensure the quality
of the pdf versions is as good as possible, but we will review this again.

19. Can we be sure that levels 2 and 3 in KS1 standards files materials equate with
    the same levels in KS2 standards files?

Yes, they have been extensively piloted to ensure that this is the case.

20. Will there be standards files for secure level 2 and low level 2 in both subjects?

Yes, they are now available on the National Strategies Framework website.

21. What response should I give to teachers that disagree with the overall
    judgements in the standards files? Many of my pilot group would have assigned
    a lower level. They felt that the standards files awarded a „tick‟ for statements
    that they felt were not secure in the evidence.

The judgements exemplify national standards and are based on the available evidence
included in the files – examples of the pupils' work, observations and what else the teacher
knew about their achievement.

22. Where is the evidence of guided work as a key opportunity for assessment in the
    standards files? For example, why is there no reference to guided reading or
    book band levels in Molly‟s info?

The standards files are authentic collections of evidence for real pupils in real classrooms
and reflect the curriculum they have been offered and the particular teaching approaches
used in the school. There is considerable evidence from guided work across the range of
Collecting evidence

23. Do you suggest that schools use the collection of evidence forms used with KS1
    pilot schools?

This may be appropriate for the one or two pupils selected for moderation purposes but
'collecting' evidence is not a general requirement of APP. As the DCSF pamphlet ‘Getting
to Grips with Assessing Pupils’ Progress’ (DCSF-00129 -2009BKT-EN) states:

       ‘You don’t need to gather weighty collections or portfolios of pupils’ work.’

Periodic assessment using the APP criteria involves reviewing the available evidence
which is already there from ongoing teaching and learning and day-to-day assessment to
make a best-fit judgement.

(The pamphlet can be found on the National Strategies website at

Implementation of APP in KS1

24. KS1 materials – if we use all of these materials for 2010 are we expecting Y1 and
    Y2 teachers to be confident in using them too soon in comparison to KS2? Or
    will they pick it up quicker due to using similar methods before?

KS1 materials have been produced in response to requests from schools to have the basis
for a whole-school approach and for more exemplification material to support teacher
assessment in Years 1 and 2. Evidence suggests that the APP approach is closer to
existing practice in many KS1 classrooms and may therefore be easier to implement. Our
advice continues to be to start small but across the school and work towards scaling up
quickly to full implementation across reading, writing and mathematics.

25. Can you clarify “usually with support in the Level 1 writing guidelines?”

Supporting learning is an important aspect of teaching at any stage. By scaffolding
learning the teacher helps learners face the challenges of new ideas and skills, perhaps by
breaking down the task into more manageable elements, leading the learner through an
activity step by step, or by completing part of the task for them in order to allow the learner
to concentrate on other aspects. As learners grow in confidence, teachers decrease the
amount of support offered until children are completing a task independently and applying
learning. In general, it is the independent skill that we assess, but if only independent
achievement is recognised, in the early stages of learning, significant milestones of
progress would be missed. For children working at or near level 1 their competence and
confidence is developed through support. Assessment criteria need to recognise these
aspects of performance.
APP and Sub–levels

26. Is the diagram High/Secure/Low still current thinking? Are sub levels still
    referred to as low / secure / high or are they thirds? Are they applied similarly in
    the MGP pilot?

Judgements for each Attainment Target are refined into 'Low', 'Secure' or 'High' within the
level and are deemed to be equal. Teachers are asked to look across the AFs and decide
whether the level is Low, Secure or High:
    Low, if there is highlighting above and below the line or the evidence is thin
    Secure, if across the AFs, the criteria for the level are mostly highlighted
    High, if across the AFs, all criteria for the level highlighted.
There is guidance available to support teachers in arriving at these judgements on the
National Strategies website. A good degree of professional judgement is also needed as
there is no exact mathematical formula which has to be applied. This is also true in the
MGP pilot.

27. How does the language of APP relate to the tracking language of a/b/c and how
    does this relate to APS?

Some schools choose to record ‘high’, ‘secure’ and low’, broadly as a/b/c for the purpose
of recording and tracking data. Rather than being based on a mathematical formula for
sub-dividing a level, ‘high’, ‘secure’ and ‘low’ reflect the professional nature of the
judgements that are made through APP, based on a teacher’s consideration of a range of
evidence over time. Average point scores are simply a numerical representation of these
sub-divisions and are useful for looking at the average performance of a cohort.

28. What do we say to headteachers who insist on tracking progress termly/half
    termly based on a system using a,b,c terminology?

In terms of the frequency of tracking progress, our advice is that for pupils making
expected progress, termly tracking is appropriate for the core subjects because it
represents a period over which it is reasonable to find discernible progress. More frequent
assessment against national standards is unlikely to yield useful information except where
intervention is in place to ensure accelerated progress.

A full guidance paper is available on the CD Leading and Managing APP (00650-
2008CDO-EN) and the tracking tutorial and tracking grid on the Primary area of the NS
site shows how information from APP assessments can be used in tracking children’s
progress (web link). Schools in the MGP pilot assess pupils as high, secure or low against
the APP criteria as described above and report these to the DCSF as a, b and c.

29. How do we support teachers/leaders in separating APP standards/level
    descriptions from commercially produced criterion scales?

The APP criteria are nationally agreed against national standards and as such are unique.
They have been rigorously trialled in pilot schools. The process ensures teachers are
taking information from a range of evidence and not just one piece of work. APP improves
teaching and learning because it helps teachers (and pupils and their parents) understand
both where their children are and how the outcomes of the assessment can inform
planning and next steps at whole class, group and individual level. The process can be
used throughout KS1, 2 and 3 and therefore provides a picture of progression and
attainment throughout a child’s learning journey.

KS1 Statutory Assessment

30. How will the APP language of low, secure and high impact on current
    judgements which are reported as a, b and c? Will statutory assessment
    reporting be changed and when?

The Expert Group, set up after the withdrawal of end of KS3 tests to look at the future of
testing and assessment at Key Stages 1, 2 and 3, is reviewing the current arrangements
for statutory assessment to ascertain whether they are working well. We would not expect
any changes ahead of their report, which is due in the spring.

31. Statutory assessment at the end of KS1 is based on teacher assessment. Why
    then do we report standards at the end of KS1 in the form of sub levels derived
    from the numerical division of a test mark scheme?

Standards reported at the end of KS1 are based on teacher assessment. The Assessment
and Reporting Arrangements (ARA) for KS1 for 2009 states that:

   ‘If teacher assessment and task and test results differ, the teacher assessment results
   should be reported, provided the judgement is based on an appropriate range of
   evidence from work done in class. Schools are not obliged to report task or test results

The ARA also states that where teachers reach a level 2 judgement, they should then
consider whether the performance is just into level 2, securely at level 2 or at the top end
of level 2. This is consistent with judgements made using APP. These are then recorded
as 2C, 2B or 2A.

32. Does the new ARA give equal status to teacher assessment and tests?

Yes, as cited above, the ARA for 2009 states that where teacher assessment and test
results differ, schools should report teacher assessment judgements.

APP in KS3

33. If the guidelines are available to cover the range L1-L8 why is the process
    different? The KS3 model appears to be around one activity rather than the range
    of evidence, while in secondary English there are only low and secure. Why are
    the same messages not being given at primary and secondary?

In the refreshed versions of APP for KS3 which were published on the Secondary National
Strategy site on October 2008, we have ensured that the approach and messages,
terminology and practice are as consistent as possible across phases, for example in the
revised materials there are no longer suggested ‘tasks’ to assess English or Mathematics.
The Assessment for Learning Strategy

34. Is the expectation that all teachers will be using APP for all children in reading,
    writing and mathematics by September 2009?

Yes – but see below for further guidance

35. In a mature APP environment where teachers have worked with a sample and
    have a very good understanding of criteria and standards, does applying the
    criteria to all children involve filling in an APP grid for all children?

Our advice at the conference was that starting with a sample group of children helps
teachers to become familiar with and embed the assessment criteria and to ensure that
there is a variety and range of evidence However, it is essential that teachers progress as
rapidly as possible to using APP for every pupil. Once teachers are confident in their use
of APP they will be able to use the assessment criteria in the guidelines to make teacher
assessment judgements against national curriculum levels about all children in the class –
i.e. use APP to make periodic assessments of all children in the class.

The completion of the assessment guidelines aids this process at lots of levels – when
teachers are learning, when schools are moderating judgements and when teachers are
discussing particular children. By highlighting the criteria pupils have met on the
assessment guidelines, teachers can see both strengths as well as gaps in learning for
each pupil. It also helps teachers to track the progress of individual pupils over time. The
completion of the guidelines is a tool to support the Assessing Pupils Progress system of
teacher assessment; it is not the APP system itself.

36. What is the likelihood of APP becoming statutory?

APP is not statutory. Teacher assessment is statutory – APP is offered to schools as a tool
to support teachers' professional skills in assessment, improve planning and pedagogy
and offer a more consistent and accurate basis for tracking progress, setting appropriate
targets for improvement and sharing information with pupils and parents/carers.

37. Is the Standards Funding for AfL already in schools ring-fenced? If yes, what
    can we do to encourage schools who‟ve decided not to start on APP to use
    money to revisit this decision and start it?

The funding is allocated through Standards Fund and is not ring-fenced. In 2009-10 it will
be focused on helping schools who have not yet started using APP and on those that may
need extra support to embed the process. Full details of the guidance are available in the
standards fund letter to local authorities and further guidance was distributed at the
Strategy Manager meetings in January 2009.

38. The AfL Strategy refers to having a trained „assessment specialist in every
    school‟. How does this fit in with crucial importance of all teachers being skilled
    in assessment?

Having an assessment 'specialist' does not imply that other teachers in the school
shouldn't be skilled in making professional judgements about the progress of the children
in their care. The 'specialist' might be in a position to induct new staff in assessment policy
and practice, coordinate standardisation and moderation activity, help ensure school-wide
consistency of approach – in other words help maintain an efficient system for assessment
across the school. We are currently working with the Chartered Institute of Educational
Assessors and other partners to explore this role in greater depth.

39. Are end of KS2 tests likely to follow the path of end of KS3, or are they likely to
    be replaced by single level tests? If the latter, how will APP fit in with single level

National tests in KS2 are here to stay. It’s not an either/or choice between tests and
classroom assessment; all good teachers track their pupils’ learning closely, and the
evidence shows clear links to tracking and faster progress. The Making Good Progress
pilot is also trying out strengthened teacher assessment using APP alongside ‘single level
tests’ in primary schools and we may move to that model if the independent evaluation
shows that this approach helps children do better at school.

40. Could you clarify the dates for online reporting to parents in primary and

By September 2010 all secondary schools will be expected to provide information to
parents covering achievement, progress, attendance, behaviour and special needs, on a
timely and frequent basis. Primary schools must also report online to parents by
September 2012.

PNS Resources

41. Are there any plans to re-write the multi modal text document for use with APP?

We are currently reviewing whether it would be appropriate to rewrite this document.

42. The Primary Framework includes writing targets – 2a, 2b, 2c – whereas APP uses
    low, secure and high. Can you explain this?

The writing targets in the Support for Writing materials have been developed to support
teachers who have identified specific gaps in learning, using information from day to day
and periodic assessment practices, to teach specific writing skills, particularly through
guided writing and 1:1 tuition sessions.

A full guidance paper on the use of curricular targets is available on the CD Leading and
Managing APP (00650-2008CDO-EN) and on the Primary Framework at
Please also see question 23 above.

43. Can we have L5 writing targets on the framework site?

Yes, we are currently developing targets for Level 5 and these will be available on the
Primary Framework.

44. Are standards in teacher assessment the same as those in tests – can you

The standards are the same but not all aspects of performance can be assessed through
testing. For example, a test can’t tell you whether children read a wide range of texts but it
can provide two or three types in a reading booklet and ask questions on them all in order
to get a proxy measure on a single occasion. In mathematics, one or two questions are
used to assess whether children know and can apply a particular operation. Tests provide
a snapshot of pupils' achievements in a sample of activities which are drawn from the
curriculum. Teacher assessment can draw on a much wider range of evidence
representing the full curriculum.

45. Have we got a timeline for the speaking and listening materials?

The Speaking and Listening materials are currently being piloted and will be published
early in 2010. They will address levels 1 to 8 and the nature of the evidence will be cross

46. Are there going to be APP materials for foundation subjects in Primary?

These are being developed for key stage 3 but there are no plans to do so for primary.
This is principally because no one is expecting a primary teacher to assess all subjects in
this way and we also need to see what the current Primary Curriculum Review
recommends about subjects.

47. Are SIPS being trained on APP? Whose responsibility is this?

National training for SIPs will ensure that the SIPs know about APP and understand its use
and purpose. Local SIP CPD will need to ensure that all SIPs working within a local
authority are able to ask schools the appropriate questions and then broker the support
when schools need more help to develop APP.

48. How do schools demonstrate that their system is better than APP?

APP is a nationally agreed assessment process against agreed national standards. It
enables teachers to develop their understanding of progression through English and
mathematics and to identify next steps in learning for individual pupils. We know that
using APP to inform tracking systems works because teachers are confident in their own
assessment and develop their confidence in their professional judgement. We also know
that the process of learning and implementing APP is an extremely powerful form of CPD.
A system that was ‘better’ than APP would have to be able to show the same level of
scrutiny against national standards and the same outcomes in terms of impact on teaching
and learning and on teachers’ professional development.
49. Are there plans to include P levels in APP? Please clarify when to start using P
    levels? Previous NAA conferences implied transfer should take place during/end
    of spring term in Y1.

QCA is developing further exemplification of performance at p-levels. A full guidance paper
on the appropriate use of p scales is available on the CD Leading and Managing APP

50. How do you see the use of assessments from EAL learner in the context of APP?
    What is the place of an EAL / 1st language assessment?

The Leading and Managing APP CD ((00650-2008CDO-EN) has a full guidance paper on
using AP with children whose first language is not English.
Information from assessments of children learning EAL will need to inform language
development targets as well as ensuring progression in the core subject areas.
For assessing reading, evidence should be gathered from across the curriculum and using
a range of strategies through which children can demonstrate comprehension.
When collecting evidence for mathematics, children who have used their first language to
learn mathematics should be given opportunities to show their knowledge and
understanding through the use of their first language as well as English.
As with all learners, decisions about attainment levels need to be drawn from a range of
evidence. The profiles of EAL learners are likely to be more uneven than those of other
pupils and therefore it is essential that the widest range of evidence possible is used to
make a periodic assessment. QCA will be developing further guidance materials on APP
and EAL learners during 2009-10.

51. Will initial teacher training address APP so that NQTs come out well prepared to
    use APP?

ITT providers have been introduced to APP via existing National Strategy networks and
QCA has been liaising with the Teacher Development Agency. Further training for ITT
providers is planned through the National Strategies this year and in 2009-10.

52. Is APP going to be shared at the NAA conferences in January – especially as it is
    mentioned in ARA RSI 2009? Where are Assessment Leads going to get their
    information and training?

APP was shared at the January LA conferences organised by QCA. We would expect that
the training and materials provided via the National Strategies will be shared with other
relevant staff within your LA.

53. Is the APP agenda also a priority for the Greater Manchester Challenge?

The introduction of APP as a means of providing robust and secure tracking data is a
national priority. It will therefore be reflected in local agendas.

54. For schools that set, how will they get their cross curricular evidence for
    mathematics and English?

The piloting of APP clearly showed that schools which did not set found it easier to draw
on cross-curricular evidence for achievement in mathematics, reading and writing. The
implication is not that schools should not set – that remains a school-level decision – but
that if they do, they may need to work harder to ensure that evidence from the wider
curriculum is taken fully into account. Teachers within year groups will need to liaise to
ensure that all teachers can gather a range of evidence. It is useful to remember that
observational evidence from another teacher may be a valuable form of evidence. Pupils
also might be involved in the process of identifying work they have produced in other
curricular areas that demonstrates good achievement.

55. When is the teacher‟s tv programme being shown?

It can be freely watched at any time. The link is:

56. Recent DCSF Raise online training emphasised question level analysis – is APP
   the priority for LAs being set by the DCSF or is it QCA optional tests?

Question level data continues to be available for analysis in RAISE. Our suggestion would
be for schools to explore the patterns emerging from completed APP guidelines to explore
consistent strengths and areas for further development in the curriculum. RAISE does not
have the functionality to analyse APP teacher assessments although there are plans to
explore the feasibility of this.

To top