Moderation of the FSA � best practices

Document Sample
Moderation of the FSA � best practices Powered By Docstoc
					Moderation of the FSA –
    Best Practice


     Linda Meyer
         INTEGRATED FSA
 The FSA for the qualification will
  focus on the extent to which a
  learner can demonstrate applied
  competence.
 Applied competence, in terms of
  the NQF is evidenced through
  learner’s ability to integrate
  concepts, ideas and actions in
  authentic, real-life contexts and is
  expressed as practical,
  foundational and reflexive
  competence
                   PURPOSE
   The final integrated summative
    assessment refers to the process of
    making judgments about achievement.
    This is carried out when a learner is
    ready to be assessed at the end of a
    learnership or full qualification. The FSA
    will be set by a registered assessor
    appointed by the
    Certification/Outsourced Partner after
    notification from the provider/assessor
    that the candidates are ready.
                     FSA
   Practical competence - demonstrated
    ability to perform a set of tasks and
    actions in authentic contexts
   Foundational competence -
    demonstrated understanding of what we
    are doing and why we are doing it
   Reflexive competence - demonstrated
    ability to integrate our performances with
    our understanding so that we are able to
    adapt to changed circumstances and
    explain the reason behind these
    adaptations.
       ASSESSMENT METHODS
   In setting the FSA, SETQAA would like
    Certification/ Outsourced Partners
    and/or assessors to take note of the
    different assessment methods and
    encourage the use of a wide variety of
    methods.
SETTING OF FINAL INTEGRATED
SUMMATIVE ASSESSMENT (FSA)
 • Function and role of the
   Certification Partner/Outsourced
   Partners
    The Certification/Outsourced
     Partner is responsible for setting of
     the FSA at NQF Level 4, with a
     memorandum of model answers
     and a grid showing its linkages
     with the assessment criteria of the
     qualification.
                   FSA
   All FSA tools and memoranda, even
    in draft form, are the property of the
    Services SETA ETQA and may not be
    revealed to, or discussed with any
    unauthorized person. The highly
    confidential nature of these
    assessment instruments must be
    safeguarded at all times.
                   FSA
   If the assessment instrument and
    memoranda are stored on computer,
    thorough security measures must be
    taken to prevent any illegal access to
    these files.
   The contents of the assessment
    instrument must reflect the relevant
    unit standards and qualification and
    must not be biased towards any
    particular textbook or learning
    material.
                 FSA

   A second assessment instrument
    should be set as standby assessment
    instrument. SETQAA have the right to
    propose the use of either assessment
    instruments for the FSA, if necessary.
             FSA
 The Certification/Outsourced
  Partner must ensure that the
  assessment instruments are not
  identical.
 The Certification/Outsourced
  Partner must also ensure that
  neither of these assessment
  instruments are the same as any
  previous assessment instruments.
       MODERATION CRITERIA
   The moderator must ensure that the
    assessment instrument is aligned with
    the purpose and rationale of the
    qualification and that it complies with
    the applicable SAQA level descriptor.
   Draft assessment instruments must fall
    within the ambit and in accordance
    with the guideline document.
     Functions and role of the
             moderator
   After the Certification/Outsourced
    Partner has set the FSA instrument,
    with a memorandum of model
    answers, an internal moderator,
    appointed by the
    Certification/Outsourced Partner,
    must moderate all draft assessment
    instruments and memoranda for the
    assessment instruments under his/her
    jurisdiction.
     Functions and role of the
            moderator
   During the moderation, the
    assessment instrument and
    memoranda are to be objectively
    assessed by the moderator with
    regard to the following:
       Functions and role of the
           moderator cont.
   The moderator must verify that
    papers set for the FSA`s are not
    identical.
   The moderator must verify that the
    Unit Standards are appropriate to the
    registered qualification.
   Functions and role of the
          moderator
The moderator must ensure that the
final summative integrated
assessment instrument is aligned
with the purpose and rationale of the
qualification and that it complies with
the applicable SAQA level descriptor.
        Functions and role of the
               moderator
   The moderator must ensure that the final
    integrated assessment instruments must
    fall within the ambit and in accordance
    with this guideline document.
   The extent to which the prescribed Unit
    Standards are covered as well as the
    assurance that nothing is asked that is
    outside the scope of the Unit
    Standard/Qualification must be verified.
  Functions and role of the
         moderator
 The spread and number of marks
allocated according to the weighting
  and credits as per the registered
  learnership/qualification must be
              assessed.
  Functions and role of the
         moderator

  The effectiveness of the type of
 assessment methods used, written
questions, essays, paragraphs, case
     studies, must be assessed.
     Functions and role of the
            moderator

   The questions, terminology,
    numbering, spelling, punctuation,
    sentence construction, units, and
    symbols must be clear,
    unambiguous and grammatically
    correct so that candidates will know
    exactly what is expected of them.
      Functions and role of the
             moderator
   The length of the assessment instrument
    must take into account the time
    allocated as well as reading time.
   The draft assessment instrument must
    be of such a length that a well-
    prepared learner will be able to answer
    it comfortably within the time
    allocated, with reasonable time
    remaining for revision.
     Functions and role of the
            moderator
   It should be borne in mind that it
    takes considerable time to read
    through certain assessment
    instruments.
   The memorandum must ensure that
    model answers give enough detail
    for the assessor of the FSA to conduct
    an accurate and fair assessment.
    Criteria for moderation of the FSA
                 instrument
   Standard of the paper (validity,
    reliability and fairness)
   Questions of various types e.g. Multiple
    Choice Questions, Case studies,
    paragraphs, data response, essay, etc
   Questions from which candidates are to
    choose – are they of equal difficulty
    level
   Correct distribution in terms of cognitive
    levels (Bloom’s Taxonomy)
            Bloom’s Taxonomy
   In 1956, Benjamin Bloom, a professor at the
    University of Chicago, shared his famous
    "Taxonomy of Educational Objectives". Bloom had
    identified six levels of cognitive complexity that
    have been used over the past 40 years to make
    sure that instruction stimulates and develops
    learners' higher-order thinking skills. The Levels
    are: Knowledge : Rote memory skills (facts, terms,
    procedures, classification systems) Comprehension
    : The ability to translate, paraphrase, interpret or
    extrapolate material. Application : The capacity to
    transfer knowledge from one setting to
    another. Analysis : The ability to discover and
    differentiate the component parts of a larger
    whole. Synthesis : The ability to weave component
    parts into a coherent whole. Evaluation : The
    ability to judge the value or use of information
    using a set of standards.
                   Bloom’s Taxonomy
   1. Knowledge: remembering of previously learned material; recall (facts
    or whole theories); bringing to mind.
   Terms: defines, describes, identifies, lists, matches, names.
   2. Comprehension: grasping the meaning of material; interpreting
    (explaining or summarizing); predicting outcome and effects (estimating
    future trends).
   Terms: convert, defend, distinguish, estimate, explain, generalize,
    rewrite.
   3. Application: ability to use learned material in a new situation; apply
    rules, laws, methods, theories.
   Terms: changes, computes, demonstrates, operates, shows, uses, solves.
   4. Analysis: breaking down into parts; understanding organization,
    clarifying, concluding.
   Identify parts: See Related Order; Relationships; Clarify.
   Terms: distinguish, diagrams, outlines, relates, breaks down,
    discriminates, subdivides.
   5. Synthesis: ability to put parts together to form a new whole; unique
    communication; set of abstract relations.
   Terms: combines, complies, composes, creates, designs, rearranges.
   6. Evaluation: ability to judge value for purpose; base on criteria;
    support judgment with reason. (No guessing).
   Terms: appraises, criticizes, compares, supports, concludes,
    discriminates, contrasts, summarizes, explains.
                   Bloom’s Taxonomy
   .Knowledge (finding out)
    a. Use - records, films, videos, models, events, media, diagrams,
    books...
    b. observed behavior - ask match, discover, locate, observe, listen.
   2. Comprehension (understanding)
    a. Use - trends, consequences, tables, cartoons....
    b. observed behavior - chart, associate, contrast, interpret, compare.
   3. Application (making use of the knowledge)
    a. use - collection, diary, photographs, sculpture, illustration.
    b. observed behavior - list, construct, teach, paint, manipulate, report.
   4. Analysis questions (taking apart the known)
    a. use - graph, survey, diagram, chart, questionnaire, report....
    b. observed behavior - classify, categorize, dissect, advertise, survey.
    5. Synthesis (putting things together in another way)
    a. use - article, radio show, video, puppet show, inventions, poetry,
    short story...
    b. observed behavior - combine, invent, compose, hypothesis, create,
    produce, write.
    6. Evaluation (judging outcomes)
    a. use - letters, group with discussion panel, court trial, survey, self-
    evaluation, value, allusions...
    b. observed behavior - judge, debate, evaluating, editorialize,
    recommend.
    Criteria for moderation of the FSA
                 instrument
   Overall: how does the standard of
    the assessment instrument compare
    in relation to other qualifications/s
    assessment instruments and
    previous assessment instruments.
Criteria for moderation of the FSA
             instrument
•Intellectually challenging and
 allowing for creative responses from
 candidates
•Suitability of examples and
 illustrations
    Relationship between mark
     allocation, level of difficulty and
     time allocation
    Criteria for moderation of the FSA
                 instrument
   FSA do not count more than
    continuous assessment of providers –
    the objective for the FSA is to check
    the standard across providers and
    across qualifications -
    STANDARDISATION
      Technical Criteria
•Cover page with all relevant details
 such as time, unit standards and
 instructions to candidates
•Clarity of instructions to candidates
•Lay out: learner friendly
•Correct numbering
      Technical Criteria
•Mark/weighting allocation clearly
 indicated
•Quality of illustrations, graphs, tables
 etc must be print ready
•Complete memorandum with model
 answers with mark/weight allocation
 and provision for alternatives
Registered Unit Standards and
          Qualifications
•Relevance to registered
 Qualification, i.e. unit standards,
 specific outcomes, assessment
 criteria
•Levels of questions
•Coverage of unit standards
•Weighting and spread of contents
             History of tool
   Full history of FSA (when it will be
    conducted, who will be assessed,
    when and where) with all drafts,
    internal moderators’ comments etc.
    must accompany the assessment
    instrument each time it is submitted
    to the external moderator (SETQAA)
                   Cognitive skills

   What conceptual constructs of the unit
    standards does the assessment
    instrument deal with?
     e.g. – reasoning ability
    •   ability   to   communicate
    •   ability   to   translate from verbal to symbolic
    •   ability   to   compare and contrast
    •   ability   to   see causal relationship
    •   ability   to   express an argument clearly
                  Cognitive skills

   Are these constructs representative of the
    best and latest developments in the
    training of this knowledge field?
   Are the questions challenging and allowing
    for creative responses from candidates?
   Suggested application of cognitive levels
    for an NQF Level 4 qualification is:
    •   10%   knowledge
    •   20%   comprehension
    •   40%   application
    •   30%   analysis, synthesis, evaluation
     Language and bias
• Correct terminology
• Appropriate language register – for the
  level of the learner
• Avoidance of gender, race, cultural,
  provincial bias
• Clear and unambiguous specification of
  instructions within questions e.g. list,
  describe
                 Time
• Relationship between difficulty level of
  questions and time
• Relationship between time and mark
  allocation and answer requirements
         Internal Moderation

   Is there evidence that the paper
    has been internally moderated?
   Quality, standard and relevance of
    input from internal moderator.
 Competency of Assessor
• Knowledge of assessor setting the FSA
  with regard to qualification.
• Knowledge, experience, expertise in
  assessment
     Marking Memorandum

   Correctness of memorandum
   Correspondence with questions.
   Alternative answers provided
   Does it facilitate assessment?
           Alignment Grid
   Did they use the grid showing the
    alignment of the instrument with the
    purpose, rational, assessment
    criteria and NQF level descriptor of
    the qualification as issued by the
    SSETA used?
Overall impression of the paper
• Fairness of the assessment instrument as
  whole
• Will the assessment instrument as a whole
  assess the achievement of the purpose of the
  qualification and the exit level outcomes
• Recommendations for improvement or
  maintenance of qualification/unit standards.
• final acceptance/rejection of individual
  assessment questions and whole assessment
  instrument needs to be substantiated
    REFERENCES OR RECORDS

   ETQA-F 005 Final Integrated
    Summative Assessment Instrument
   Only after written approval from the
    external moderator, SETQAA will
    approve and sign off the final
    assessment instrument and notify the
    Certification/Outsourced Partner to
    proceed with the distribution of the
    assessment instrument and conduct
    the assessment.
                                             CONTINUOUS ASSESSMENT VS FINAL SUMMATIVE ASSESSMENT



                                                       STAKEHOLDERS INVOLVED IN
                                                          QA OF ASSESSMENT
                                                             PROCESSES
                                                      Provider internal assessors
                                                      Provider internal moderators
                                                      Certification/Outsourced Partner
                                                      assessors
                                                      Certification/Outsourced Partner
                                                      moderators
            SETQAA DOC                                SETQAA external moderators
                                                      SETQAA external verifiers




                                  PRACTICAL COMPONENT                                      THEORETICAL
                                                                                           COMPONENT




                                                                                                    FSA (WRITTEN COMPONENT)
  TRAINING PROVIDER ASSESSMENT                                 FSA (PRACTICAL                     Conducted AT END of
Assessment should happen on a CONTINUOUS
basis during learnership/training – formative as
                                                               OBSERVATION)                       learnership/Qualification
w ell as summative                                    Should be set and conducted                 Set and moderated by registered
Evidence gathered in PoE                              CONTINUOUSLY and/or AT END of               assessors and moderators appointed
Internally moderated and reports submitted to         learnership/ qualification                  by CP
SETQAA at regular specified intervals (See                                                        Assessed and moderated by registered
                                                      Set and assessed by registered
m oderation policy)
Externally m oderated/verified by SETQAA              assessors, moderated by registered          assessors and moderators appointed
PoE subm itted for external m oderation by CP at      moderators appointed by CP                  by CP
end of learning                                       Externally verified by SETQAA               Externally moderated/verified by
                                                                                                  SETQAA appointed moderators
       FSA Moderation process
   APPLICATION FOR MODERATION OF FINAL
    INTEGRATED SUMMATIVE ASSESSMENT
    INSTRUMENT RECEIVED BY SSETQA
   EXTERNAL MODERATOR APPOINTED BY
    SSETQA
   NO COMMUNICATION PERMITTED BEWEEN
    DEVELOPER AND MODERATOR
   MODERATION OF FINAL INTEGRATED
    SUMMATIVE ASSESSMENT INSTRUMENT / RPL
    TOOL
      FSA Moderation process
   PRELIMINARY REPORT PREPARED BY
    MODERATOR AFTER THE FSA/RPL TOOL
    HAS BEEN MODERATED - REMEDIATION
    IDENTIFIED
   TOOL SENT BACK TO DEVELOPER FOR
    REMEDIATION TO BE ADDRESSED
   THE DEVELOPER MAY REMEDIATE /
    REQUEST ADDITIONAL INFORMATION
    OR APPEAL THE MODERATION DECISION
    THROUGH THE SSETA APPEALS
    PROCEDURE
      FSA Moderation process
   REMEDIATION COMPLETED
   SENT TO EXTERAL MODERATOR TO
    CONFIRM REMEDIATION & FINAL
    REPORT SUBMITTED
   ELECTRONIC VERSIONS SENT
   FSA/RPL TOOL REGISTERED ON
    DATABASE
   The Certification/Outsourced
    Partner assures the rectification is
    done and sends the final draft
    together with the Moderators report
    to SETQAA for external moderation.
   The Certification/Outsourced
    Partner assures the rectification is
    done and sends the final draft
    together with the Moderators report
    to SETQAA for external moderation.
   Only after written approval from
    the external moderator, SETQAA
    will approve and sign off the final
    assessment instrument and notify
    the Certification Partner to proceed
    with the distribution of the
    assessment instrument and
    conduct the assessment.
Thank you

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:1
posted:10/9/2012
language:English
pages:51