Functional Family Therapy
FUNCTIONAL FAMILY THERAPY QUALITY ASSURANCE MANUAL
Dana Phelps, M.Ed. Washington State Functional Family Therapy Quality Assurance Specialist
Introduction
The Washington State Functional Family Therapy Project is dedicated to implementing Functional Family Therapy with high model fidelity. Recent evidence suggests that evidence-based intervention programs depend on high model fidelity for successful outcomes (Barnoski, 2002; Sexton, Hollimon, Mease, & Alexander, 2002). In Functional Family Therapy, model fidelity is based on the dimensions adherence to the principles of the model as well as the competent delivery of the model. Ensuring model fidelity in a community based system of care requires an ongoing systematic system of both quality assurance and quality improvement. Quality assurance involves the ongoing and accurate monitoring and tracking of reliable measures of model implementation. Quality improvement involves the systematic implementation of activities to improve accurate implementation of the intervention. In the sections below, the principles and protocols for the Washington State Functional Family Therapy Quality Assurance and Improvement system are outlined.
Principles of Quality Assurance and Improvement
Given the sensitive nature of quality assurance information it is important to clearly identify the principles of the model prior to implementation. The Washington State FFT quality assurance and improvement system is based on the following principles: 1. The primary goal of this system of quality assurance is improvement of the delivery of FFT. As such, quality assurance information is: Intended for use primarily by FFT clinical supervisors who are most capable of determining systematic improvement plans. It is not intended as a tool for routine program administration. While program administrators need aggregate and summarized information that informs overall program implementation, specific clinical data is most useful as a tool for clinical supervision. Therapists should be provided with accurate and timely feedback directly from the FFT clinical supervisor. Therapists who perform below the national standards of model fidelity should be presented with a systematic plan for improvement. Therapists who, after all attempts at improvement, continue to demonstrate model fidelity outcomes below the minimal national standard should not practice the FFT model. Quality assurance information is intended for employment status decisions only after all possible improvement strategies have been attempted.
September 2005
Functional Family Therapy
2. Monitoring and Tracking model fidelity (quality assurance) must be based on: reliable and valid measures from multiple domains (adherence and competence) based upon multiple measures (specific case level ratings, global therapist rating) gathered from different and relevant perspectives (FFT clinical supervisor and client). Incremental measurement, that is, more specific measures of fidelity are only undertaken when global level indicators suggest that more specific and time intensive measures are necessary. 3. Quality improvement is based upon: Ongoing, specific, and timely feedback based on accurate measure of model fidelity (adherence and competence). A systematic and individualized plan of therapist improvement
Quality Assurance
Functional Family Therapy (FFT) has developed a comprehensive Fidelity and Adherence Protocol that is central to successful implementation of FFT. The Web-based computer monitoring and tracking application (Clinical Services System-FFT CSS) is the mechanism to gather, manage and feedback multiple fidelity ratings while also providing real time feedback to therapists and clinical supervisors. No single measure adequately portrays therapist adherence and competence. In the FFT system, six measure of quality assurance are used to monitor and track model fidelity. Quality assurance instruments: 1. Knowledge Review The WA-FFT Knowledge Review is a measure of therapist basic knowledge of the FFT model. The knowledge review is administered following initial three day training and at the end of the first 6 months of the initial training year. 2. Progress Notes (Therapist Report) At each treatment encounter therapists report the treatment encounter goals (what they believed was most important to accomplish in the session) and the progress they believed was made in accomplishing these goal(s). When therapist reported goals are aggregated across treatment encounters it is possible to determine whether therapist are pursuing the goals prescribed by the FFT treatment model. Goal: To obtain the therapist perspective of the process of FFT at the level of treatment encounters To provide feedback to the clinical supervisor regarding the therapist clinical decision making processes. To provide specific areas of concern to be targeted by the clinical supervisor in subsequent supervision sessions.
September 2005
Functional Family Therapy
Process: Therapists complete the progress notes following each treatment encounter and enter the progress note on the FFTCSS. Clinical supervisors review the therapist’s progress notes prior to each supervision session and determine areas to provide focused help and assistance. Adherence reports are available to help clinical supervisors monitor therapist adherence and competence in delivering FFT.
3. Counseling Process Questionnaire (Client Report). The Counseling Process Questionnaire (CPQ) is a 20-item instrument measuring the client/family experiences in FFT. All family members complete the CPQ after every other treatment session. The CPQ is designed to tap into client experiences that would be expected in each phase of FFT (six questions measure experiences that would be expect of each of the phases). Two additional questions measure degree of global change from the client perspective. Model fidelity is measured by comparing client reported experiences with model prescribed activities. Goal: To obtain the family perspective on their experience in therapy. To provide feedback to the therapist on the family experience so they determine the phase outcomes of FFT. To provide ratings of adherence and competence from the family perspective. Process: Families compete the CPQ follow every other session of FFT CPQ’s are entered into the CSS by the FFT therapist. CPQ’s are available for therapist inspection for self-monitoring The CSS produces an adherence report that can be used by either therapist or clinical supervisors. Therapists monitor the CPQ results on the CSS adherence report Clinical supervisors monitor the CPQ on the CSS adherence reports. Ratings are maintained in the FFT-CSS for use in supervision. Issues of adherence and competence addressed in subsequent supervision session. 4. Weekly Adherence & Competence ratings (Clinical supervisor report). At weekly case staffing meetings, FFT clinical supervisors rate each FFT therapist on levels of model adherence (application of necessary technical and clinical aspects of FFT) and competence (skillful application of the necessary components of FFT). These ratings represent the FFT therapist’s
September 2005
Functional Family Therapy
adherence and competence in the case discussed at each staffing. Global competence and adherence can be determined from ratings of each construct over time (across cases). Goal: To identify specific issues of adherence and competence for the individual therapist Identify specific issues of adherence and competence for the group. Provide focused supervision to the working group. Identify specific supervision issues to be addressed. Process: Ratings are maintained in the FFT-CSS for use in supervision. Issues of adherence and competence addressed in subsequent supervision session. Weekly competency and adherence ratings are entered into the CSS by the FFT clinical supervisor. The CSS produces a report of these ratings over time for use by the clinical supervisor. 5. Global Therapist Rating The Global Therapist rating (FFT-GTR) is a 35-item instrument completed by the FFT clinical supervisor three times each year. The global rating includes ratings of general model principles and behaviors, specific phase based behaviors, and service delivery items. Goal: To identify status of therapist adherence and competence in FFT. Provide specific information to therapist and site regarding the performance of therapist. Help identify any therapists that may benefit from additional training. Process: Global rating completed by the FFT implementation consultant and reviewed by FFT clinical supervisor If there are serious issues apparent the ratings are discussed with the therapist, the site administration, and the state FFT coordinator and a plan for remediation is developed (see below). Global therapist ratings are entered into the CSS by the FFT clinical supervisor. The CSS produces a report of these ratings for use by the clinical supervisor. 6. Working Group Feedback reports The Site Feedback Report (FFT-SFR) is a rating of the working group and their ability to adhere to the FFT model. The FFT-SFR is completed by the FFT clinical supervisor three times each year. Goal:
September 2005
Functional Family Therapy
To identify site barriers to successful FFT implementation. To identify general therapist adherence and competency issues that may impact successful model delivery. To identify general working group issues that impact successful model fidelity To provide specific and written feedback to the site regarding implementation issues that may impact successful model delivery. Process: Site feedback report is drafted by the FFT clinical supervisor. Site feedback report sent to the state FFT coordinator for review and potential discussion Site feedback report reviewed by the FFT implementation consultant at the follow up visit with the team and with the agency administration. Plan for remediation developed (if necessary). 7. Video Tape Rating Videotape ratings are a method to gather specific information regarding adherent and competent delivery of FFT by the therapist. Videotape rating is a time consuming task that requires a trained rater if reliable and valid measure are to be made. Because of the cost and intensity of this activity it is reserved for cases in which other methods of quality assurance indicate that further assessment is necessary. The Global Tape Rating system (FFTGTR) will be used if videotape rating is required. Goal: To specifically determine the degree of therapist model adherence and competence To provide detailed feedback to the therapist regarding potential areas of improvement. To provide detailed feedback to the FFT clinical supervisor and the state FFT coordinator for use in developing formal improvement plans. Process: FFT therapist videotapes two FFT sessions. Tapes submitted to FFT state coordinator who, in consultation with FFT LLC determines the tape rater. Tapes rated using the FFT-GRT system. Specific feedback provided to the FFT state coordinator, FFT therapist and court administrator
Quality Improvement System
Quality Improvement System is based on the principle that therapists should receive specific and timely information regarding their performance. When problems arise, the therapist should be informed early and specific plans for remediation developed. Successful quality improvement is based on specific, timely, and concrete feedback that allows for individualized plans for improvement. That feedback should come from first from the immediate clinical clinical supervisor, followed by the FFT State
September 2005
Functional Family Therapy
Coordinator. The following steps will be followed when the therapist performance on the instruments noted above follows below the national standard. Court administrators will receive regular reports regarding therapist performance (as noted below). For each new therapist summary reports will be provided to the Court Administrator by the FFT State coordinator following each follow-up visit (approximately every 4 months). For experienced FFT therapists summary reports will be provided twice each year (following the continuing education training). When informal improvement plans court administrators will be informed. When formal improvement plans are required court administrators will be involved in developing the plan Step 1: Consultation with Therapist
FFT clinical supervisor will call the therapist individually and discuss the issues of concern and develop and informal method for therapist improvement. Consultation is usually the result of a series of weekly supervision ratings falling below the standard, serious problems with therapist progress notes, and consistent CPQ ratings below that expected of the therapist at their level of experience.
Step 2:
Consultation with State Coordinator
If therapist adherence performance does not improve the FFT clinical supervisor will consult with the state FFT coordinator and discuss concerns and discuss an informal method for therapist improvement. This may involve additional reading, or individual supervision. The State FFT coordinator will, if appropriate, consult with the court administrator and inform them of the ongoing concerns. The result of this consultation will be a specific but informal plan for therapist improvement. The informal improvement plan may include: additional adherence monitoring (e. g. video tape review, knowledge review etc.), individual supervision, or additional training. If the informal improvement plan does not result in improved adherence results, a formal improvement plan will be implemented.
Step 3:
Formal Improvement Plan
If therapist adherence performance does not improve additional assessment of therapist performance will take place. These assessment will include: Tape review-therapist will provide the state coordinator with two recent FFT therapy tapes. The tapes will be reviewed using the FFT tape review system by the statewide coordinator. Specific feedback will be provided to the therapist and the court administrator. After consultation between the FFT clinical supervisor, the state FFT coordinator and the court administration will develop a formal improvement plan. This will include additional formal training (clinical training/follow-up training).
Step 4: Retraining
If therapist adherence performance does not improve the therapist will be removed from active FFT caseload and may be retrained (if deemed appropriate).
Quality Assurance and Improvement Process
September 2005
Functional Family Therapy
The quality assurance and improvement system must be ongoing. The process for new and experienced therapists will follow different timelines. The goal with newly trained FFT therapist is to identify issues/problems with model fidelity early so that additional training and supervision can be provided. For experienced therapists, the goal is to prevent model drift. In both cases the primary goal is to identify problems of model adherence and provide assistance so that the therapist can improve their practice. New Therapists Following the initial clinical training the following steps will be followed during the first year of FFT training.
September 2005
Functional Family Therapy
Training Activity/ Clinical Activity Initial Clinical Training
Quality Assurance/Improvement Activity Knowledge review
Activity:
Quality improvement Activities/Action
Administered and scored by the FFT State coordinator
Action:
a. If results below national standard informal improvement plan developed b. Therapist and Court Administrator informed and agree to the informal improvement plan c. Overall assessment provided to Court Administrator on all therapists following initial training. FFT Cases
(on going)
Progress note
Activity:
Information monitored by FFT clinical supervisor CPQ
Action:
Used by FFT clinical supervisor in next case staffing meeting Case Staffing
(on going)
Weekly Supervision Rating
(minimum 2 times each month)
Activity:
Supervision Rating entered on the CSS by FFT clinical supervisor
Action:
Information monitored by FFT clinical supervisor and FFT state coordinator Follow-up training #1 90 days after initial training Global Rating Measure Working group Feedback Report
Activity: a. GRM entered on the CSS
b. WG feedback entered on the CSS
Action:
a. Court administrators provided with global adherence levels of each therapist in their group b. If below the national standard, FFT state coordinator informed c. Informal improvement plan developed d. Court administrator informed of any informal improvement plans
Case Staffing
Weekly Supervision Rating
Activity: September 2005
Functional Family Therapy
Training Activity/ Clinical Activity
(on going)
Quality Assurance/Improvement Activity
(minimum 2 times each month)
Quality improvement Activities/Action
Supervision Rating entered on the CSS by FFT clinical supervisor
Action:
Information monitored by FFT clinical supervisor and FFT state coordinator
Follow-up training #2 6 months after initial training
Global Rating Measure Working Group feedback report Knowledge Review
Activity:
a. Supervision Rating entered on the CSS by FFT clinical supervisor b. WG Feedback report entered on the CSS c. Knowledge review entered on the CSS
Action:
a. If below the national standard, FFT state coordinator informed b. Additional quality assurance monitoring including: -Knowledge review -video tape review c. Informal improvement plan developed including: -additional supervision -additional training Case Staffing
(on going)
Weekly Supervision Rating
(minimum 2 times each month)
Activity:
Supervision Rating entered on the CSS by FFT clinical supervisor
Action:
Information monitored by FFT clinical supervisor and FFT state coordinator Follow-up training #3 9 months after initial training Global Rating Measure Site Feedback report
Activity:
a. Supervision Rating entered on the CSS by FFT clinical supervisor b. WG Feedback report entered on the CSS c. Knowledge review entered on the CSS
September 2005
Functional Family Therapy
Training Activity/ Clinical Activity
Quality Assurance/Improvement Activity
Quality improvement Activities/Action
Action:
a. If below the national standard, FFT state coordinator informed c. Additional quality assurance monitoring including: -Knowledge review -video tape review d. Formal improvement plan developed including: -additional supervision -additional training e. Court administrator informed of any improvement plans f. Improvement plan monitored by FFT state coordinator and the court administrators with progress updates provided by the FFT state coordinator. Case Staffing
(on going)
Global Rating Measure Working Group feedback report Knowledge Review
Activity:
Supervision Rating entered on the CSS by FFT clinical supervisor
Action:
Information monitored by FFT clinical supervisor and FFT state coordinator Timeline for ongoing FFT Therapists Training Activity/ Clinical Activity FFT Cases
(ongoing)
Quality Assurance/Improvement Activity Progress note
Activity:
Quality improvement Activities/Action
Information monitored by FFT clinical supervisor CPQ
Action:
Used by FFT clinical supervisor in next case staffing meeting Case Staffing
(on going)
Weekly Supervision Rating
(minimum 2 times each month)
Activity:
Supervision Rating entered on the CSS by FFT clinical supervisor
September 2005
Functional Family Therapy
Training Activity/ Clinical Activity
Quality Assurance/Improvement Activity
Action:
Quality improvement Activities/Action
a. Information monitored by FFT clinical supervisor and FFT state coordinator b. Additional quality assurance monitoring including: -Knowledge review -video tape review c. Informal improvement plan developed including: -additional supervision -additional training Continuing Education Training #1 Global Rating Measure (GRM) Working group Feedback Report (WGFR)
Activity: a. GRM entered on the CSS
b. WGFR feedback entered on the CSS
Action:
a. Court administrators provided with global adherence levels of each therapist in their group b. If below the national standard, FFT state coordinator informed c. Additional quality assurance monitoring including: -Knowledge review -video tape review d. Formal improvement plan developed including: -additional supervision -additional training e. Court administrator informed of any improvement plans f. Improvement plan monitored by FFT state coordinator and the court administrators with progress updates provided by the FFT state coordinator.
Continuing Education
Global Rating Measure
Activity: a. GRM entered on the CSS September 2005
Functional Family Therapy
Training Activity/ Clinical Activity Training #2
Quality Assurance/Improvement Activity Working group Feedback Report
Quality improvement Activities/Action
b. WG feedback entered on the CSS
Action:
a. Court administrators provided with global adherence levels of each therapist in their group b. If below the national standard, FFT state coordinator informed d. Additional quality assurance monitoring including: -Knowledge review -video tape review e. Formal improvement plan developed including: -additional supervision -additional training f. Court administrator informed of any improvement plans g. Improvement plan monitored by FFT state coordinator and the court administrators with progress updates provided by the FFT state coordinator. h. Court administrator informed of any informal improvement plans i. The CJAA Advisory Committee will be informed of formal improvement plans and will refer any problems implementing the improvement plan to the WAJCA Executive Board.
September 2005
Functional Family Therapy
FUNCTIONAL FAMILY THERAPY Weekly Supervision Checklist
Instructions: The purpose of this checklist is to document the weekly consultation phone call between the FFT consultant and the site/working group. Please complete one weekly supervisor checklist for EACH phone visit. Please complete this during the phone call. Use the scale below to make the adherence and competency ratings.
Site: Date:
FFT Supervisor:
Team Member
Case Presented
(name & number – note if none)
Phase of Case
(E/M, BC, G)
Major Issue
(Use codes below to identify specific issue and include any other comments)
Adherence Rating
(Use scale below)
Competency Rating
(Use scale below)
1. 2. 3. 4. 5. 6. 7. 8.
Major Issue Codes: 1 = following phases 2 = basic theoretical understanding of the model 3 = thinking relationally/family focus 4 = understanding relational functions 5 = clinical decision through the lens 6 = balanced alliance 7 = reframing as primary skill in E/M 8 = reducing negativity and blame 9 = developing and using organizing themes 10 = specific behavior change targets 11 = behavior change that matches relational functions 12 = generalizing change 13 = relapse prevention 14 = incorporating community resources
September 2005
Functional Family Therapy
Group Clinical Issues: What are the primary issues the group is struggling with at this point regarding the Clinical Model?
Use the codes below and include any other information.
Major Issue Codes:
1 = following phases 8 = reducing negativity and blame 2 = basic theoretical understanding of the model 9 = developing and using organizing themes 3 = thinking relationally/family focus 10 = specific behavior change targets 4 = understanding relational functions 11 = behavior change that matches relational functions 5 = clinical decision through the lens 12 = generalizing change 6 = balanced alliance 13 = relapse prevention 7 = reframing as primary skill in E/M 14 = incorporating community resources
Site/Working Group Challenges: Use the codes below and include any other information.
Site Challenge Codes
1 = case loads 2 = administrative support 3 = staffing attendance 4 = system responsiveness to clients
September 2005
Functional Family Therapy
5 = team responsiveness to clients
Adherence
Adherence is the degree to which the therapist is doing the FFT program (clinical model, assessment protocol, staffing participation, CSS). Low ratings of adherence (0-1) would indicate that the therapist is not or very rarely using the technical elements of the program (assessment protocol, CSS), is not participating (attending staffing infrequently), or using the clinical model in work with clients (following phases of the model and attempting to achieve the goals of the model in clinical work). An average rating (2-4) indicates that the therapist is doing the technical aspects (see above), but not or infrequently doing the clinical aspects of the model. High adherence (5-6) indicates that a therapist is doing all parts of the model consistently.
|
|
|
|
|
|
|
0
Low Adherence
1
2
3
Average Adherence
4
5
6
High Adherence
(using the model very little)
(attending, using CSS, not using clinical model)
(using all aspects of model regularly)
Competence
Competence reflects the skill of the therapist in doing the clinical model of FFT. Competence includes the ability to be clinically responsive to individual families (translate the model to the individual family) while remaining model-focused (goals & skills), consistently practicing the model, and thinking complexly about clients and the FFT therapy process. Low competence ratings (0-1) would indicate a therapist who is attempting to achieve the goals of each phase and using the skills of each phase but doing clinical work in ways that is rigid, not matching to the family, in a way that reflects simple thinking about the process, that involves simple application of the skills (e.g. reframing) that is applied inconsistently. An average competence rating (2-4) indicates that the therapist is thinking somewhat complexly about the family and process, using skills (e.g. reframing) with moderate complexity and doing these things most of the time. High competence ratings (5-6) indicate that the therapist has the ability to think complexly about families and the process, and to do the clinical skills of FFT with a high degree of skill in ways that match to many different kinds of families in a consistent manner.
|
|
|
|
|
|
|
0
Low Adherence
1
2
3
Average Adherence
4
5
6
High Adherence
(using the model very little)
(attending, using CSS, not using clinical model)
(using all aspects of model regularly)
September 2005
Interagency Coordination
INTERAGENCY COORDINATION
September 2005
Multidimensional Treatment Foster Care
MULTIDIMENSIONAL TREATMENT FOSTER CARE
September 2005
Dialectic Behavior Therapy
DIALECTIC BEHAVIOR THERAPY
September 2005
Family Integrated Treatment
FAMILY INTEGRATED TREATMENT
September 2005
Mentoring
MENTORING
September 2005
FFT Therapist Name:
FFT Consultant and Workgroup:
Elements needing improvement Identify specific reading, training, and consultation that will occur to impact the identified element.
Specific problems related to the elements Action Plan for achieving improvement
Timeframes for action plan
Directions: List the FFT components and principles that will be addressed.
Discuss how the identified element creates problems in the therapist’s practice of FFT. Whether the problem is related to core principles, knowledge, or performance should be reflected.
Specify dates by which steps in the action plan will occur and dates by which re-assessment will occur. Identify the end date of the improvement plan.
FFT Therapist Signature:
Date: Date: Signature:
FFT Consultant Signature:
Managing/Overseeing Delivery of Services: Therapist Improvement Plan
Completion Date:
Functional Family Therapy
September 2005