An Introduction to Understanding the
OTA Fieldwork Performance Evaluation
(FWPE)
Karen Atler, MS, OTR
Colorado State University, Fort Collins, CO
Roberta Wimmer, OTR/L
Pacific University, Forest Grove, OR
2003 The American Occupational Therapy Association, Inc. www.AOTA.org
Task Force Members
Carole Dennis, PhD, OTR
Ithaca College, New York
Carole Hays, MA, OTR
Springfield Hospital Center, Maryland
Becky Robler, MEd, OTR
Pueblo Community College, Colorado
Karen Atler, MS, OTR, Co-Chairperson
Colorado State University
Roberta Wimmer, OTR, Co-Chairperson
Pacific University, Oregon
2
Objectives for Today:
Describe the…
Entry-level practice competencies for OT
and OTA students.
Purpose, format, content, and scoring of
the companion evaluation forms.
New concepts and terminology used in the
FWPE from the OT Practice Framework.
Begin to score items on the FWPE.
3
Task Force’s Charge
Revise/develop evaluation tools to measure
assistant and professional Level II fieldwork
student performance.
Expectations:
Conduct review of literature across disciplines.
Synthesize feedback on current AOTA FWE/OT
forms.
Incorporate 1997 NBCOT Practice Analysis results.
Address identified desired characteristics.
4
Desired Characteristics
Companion documents for assistant and
professional level that…
Measure entry-level competence.
Focus on occupation-based practice.
Reflect current and future practice.
Can be used in a variety of settings.
Provide feedback to students.
Can be easily used in a timely manner.
5
NBCOT Practice Analysis 1997
What OTs & OTAs DO
Determining needs/priorities for interventions.
Identifying/designing interventions.
Implementing interventions.
Reporting/evaluating intervention effectiveness.
Providing OT services for populations.
Managing delivery of OT services.
Advancing effectiveness of the OT profession.
6
NBCOT Practice Analysis 1997
What OTs & OTAs NEED TO KNOW
Human development and performance.
Principles/strategies in the identification/evaluation of
strengths and needs.
Principles/strategies in intervention/treatment
planning.
Principles/strategies in intervention.
Nature of occupation and occupational performance.
Service management.
Responsibilities as a professional.
7
Standards of Practice
for OT
Identifies minimum standards.
Identifies key performance areas for the OT
and OTA:
Professional standing and responsibility
Referral
Screening
Evaluation
Intervention plan
Intervention
Transition services
Discontinuation
8
ACOTE:
Minimum Standards and Outcomes for OTA
Be a generalist.
Achieve entry-level competence.
Work under the supervision of and in
cooperation with the OT.
Articulate, apply, and justify interventions
related to occupation.
Keep current with best practice.
Uphold the ethics, values, and
attitudes of the profession.
Goal of Level II Fieldwork
Education for the OTA Student
Develop competent, entry-level
generalists.
Include an in-depth experience in
delivering OT services.
Be designed to promote reasoning,
enable ethical practice, and develop
professionalism.
10
The Process
Began with OTA evaluation.
Reviewed by experienced panel.
Submitted to COE.
Made revisions.
Completed pilot studies (2 OTA, 1 OT).
11
Design and Analysis of Pilot Studies:
The Rasch Measurement Model
less able Student Ability more able
Results of Pilot Studies
Good representation in pilot samples.
Students and educators preferred new form.
Good scale and response validity.
Inaccurate use of scale.
Rating Scale Usage in Pilot Studies II and III
Rating Scale Descriptors % Usage OT % Usage OTA
1 = Unsatisfactory 0 0
2 = Needs Improvement 10 13
3 = Meets Standards 56 53
4 = Exceeds Standards 34 29
13
FWPEs for OT and OTA Students
Companion documents
Terminology
Content layout
Purpose
Design
Rating scale
Scoring system
14
FWPEs OT and OTA
Companion Documents
Focus
The OT process
The clinical reasoning process
Roles and responsibilities of the OT and OTA
Structure
Collaborative process—student and FW
educator
Same layout
Same rating/scoring system
15
Terminology of the FWPEs
Reflects…
Standards of Practice for OT and ACOTE
Education Standards
OT Practice Framework
The glossary
16
Content Layout of FWPEs
Summary Sheet
Overview/instructions
Organization of items
Space for comments—midterm and final
Performance Rating Summary Sheet
17
Content of OT and OTA Evaluations
OTA OT
Fundamentals of practice (3) Fundamentals of practice (3)
Basic tenets (3) Basic tenets (4)
Evaluation/screening (5) Evaluation/screening (10)
Intervention (6) Intervention (9)
Service Management (5)
Communication (2) Communication (4)
Professional behavior (6) Professional behavior (7)
18
Primary Purposes of the FWPEs
Measures entry-level competence:
Designed to differentiate the competent
student from the incompetent student.
NOT designed to differentiate levels above
entry-level competence.
19
Purpose (continued)
Provides student with accurate
assessment of his/her competence for
entry-level practice over time:
Growth occurs over time.
Midterm and final scores reflect this change.
Midterm scores: Satisfactory/unsatisfactory.
Final scores: Pass/no pass.
20
Purpose (continued)
Provides feedback to student.
Provides opportunity for student self-
assessment.
21
Design of the FWPEs
The “doing” of the OT process is
evaluated, not the individual tasks in
isolation.
NOT all items are equal in level of
difficulty (i.e., simple to complex).
Evaluation is supplemented with
development of site-specific objectives.
22
Easier
2-
-
- Cultural competence; 25
- Ethics; 1
-
- Interpersonal skills;24
- Responds to Feedback;21
- Safety; 2 and 3
-
-
1-
-
-
- Work Behaviors; 22
-
- Therapeutic Use Self;16
-
- Written Communication;19
-
-
0- Verbal Communication; 18
- Self-responsibility; 20
- I Implements intervntn; 14
-
- OT/OTA Roles 5DataGather
-
-
- Selects Intervention;13
- Activity Analysis; 15
- Evidence BasedPractic 6
- 1- Plans Intervention; 12
- Reports; 10
- OT Philosophy; 4
- Administer Assessmnts;8
- Establishes Goals; 11
-
Rasch Ordering of Items OTA
- Modifies Intrven Plan;17
- Interprets Assessment;9
-
-
23
- 2-
Harder
Rating Scale of FWPEs
4 = Exceeds Performance is highly skilled and self-initiated. This
Standards rating is rarely given and would represent the top 5% of
all the students you have supervised.
3 = Meets Performance is consistent with entry-level practice.
Standards This rating is infrequently given at midterm and is a
strong rating at final.
2 = Needs Performance is progressing but still needs improvement
Improvement for entry-level practice. This is a realistic rating of
performance at midterm and some ratings of 2 may be
reasonable at the final.
1 = Unsatisfactory Performance is below standards and requires
development for entry-level practice. This rating is given
when there is concern about performance.
24
Scoring System of FWPEs
Each item must be scored.
Ethics and safety items must be passed.
Each item rating recorded on
Performance Rating Summary Sheet.
All items summed up at midterm and
final.
Score compared to scales provided.
25
Midterm and Final Scores
OTA
Overall Midterm Score
Satisfactory: 54 & above
Unsatisfactory: 53 & below
Overall Final Score
Pass: 70 & above
No Pass: 69 & below
26
Rating Performance Using the FWPEs
Case Scenarios
Sandra—OTA Evaluation/Screening
David—OTA Intervention
27
Individualizing the FWPEs
Designed for additional objectives to
be written to add clarification:
Site-specific objectives
NOT supervisor-specific
If an item is very clear and meets the
RUMBA test, then there is no need to
write another objective.
28
Objectives: An Example
From FWPE for the OT Student
16. Establishes accurate and
appropriate plan
School—Develops behavioral-based,
measurable OT goals during IEP process.
Acute Care—Overall intervention plan is
achievable within client’s length of stay.
29
Summary
Evaluations designed to measure entry-
level competence, NOT level of
performance above competency.
OT practice examined as a generalist.
Evaluations reflect the OT process.
Performance develops over time.
30