Docstoc

Performance Evaluation Tools - Excel

Document Sample
Performance Evaluation Tools - Excel Powered By Docstoc
					                                                              CORE Program Internal Review
                                                                       Template

Programmatic evidence provides assurance that reasonable opportunities to master the knowledge, skills, and dispositions
of a competent DO have been offered.
Testing evidence, if the testing is a valid and reliable measure of the knowledge, skills, and dispositions, provides assurance
that those who have attained passing scores possess the knowledge necessary to be a competent DO.
Performance evidence provides assurance that those who are judged to have met the performance criteria can perform effectively.
PIR Discussion Questions                 Reference Item   Suggested Evidence (Other Types of Does the      Notes    Recommended Action
                                         from AOA PIR     Evidence Not Listed Here May       Evidence
                                         Questionnaire    Suffice)                           Meet the
                                                                                             Criteria of
                                                                                             Compliance?
                                                                                             (Y/N)
Curriculum/Evaluation
What are the written goals and           A1               Programmatic evidence: Residency
objectives that focus on the                              Manual with goals and objectives
competencies for each major rotation?                     focusing on the competencies for each
                                                          major rotation.
How and when is the list of objectives   A1               Programmatic evidence: Residency
(identifying learning expectations)                       Manual with goals and objectives for
provided to and reviewed with                             each major rotation. Interview faculty
interns/residents?                                        and trainees to discuss learning
                                                          expectations.
What is the in-service exam utilized and A2               Programmatic evidence: Name of in-
how is it utilized?                                       service exam. Testing evidence:
                                                          Program averages, program subject
                                                          area strengths/weaknesses. Program
                                                          comparison to national average.
                                                          Resident comparison to
                                                          program/national scores. Evidence in
                                                          individual resident files that this has
                                                          been discussed and a plan completed
                                                          for remediation. Evidence from
                                                          department and MEC discussions that
                                                          curricular changes have been instituted
                                                          based on performance reports from the
                                                          ISE.




Compliments of CORE Faculty Development                                                                                                  1
                                                               CORE Program Internal Review
                                                                        Template

PIR Discussion Questions                  Reference Item   Suggested Evidence (Other Types of Does the      Notes   Recommended Action
                                          from AOA PIR     Evidence Not Listed Here May       Evidence
                                          Questionnaire    Suffice)                           Meet the
                                                                                              Criteria of
                                                                                              Compliance?
                                                                                              (Y/N)
If there is no in-service exam utilized, A2                Performance evidence: Copy of OSCE
what is another objective measure of the                   results, copy of the ICCP plan for the
effectiveness of the program with regard                   institution, clear delineation of the tools
to established goals and objectives?                       which will be used to evaluate each of
                                                           the goals, competencies, etc.


How are ethical issues integrated into    A3               Programmatic evidence: Copy of
the program? Please describe                               hospital didactic program schedule and
objectives.                                                OPTI Ed Day schedule (with
                                                           objectives).
How are socioeconomic issues              A3               Programmatic evidence: Copy of
integrated into the program? Please                        hospital didactic program schedule and
describe objectives.                                       OPTI Ed Day schedule (with
                                                           objectives).
How are medical/legal issues integrated A3                 Programmatic evidence: Copy of
into the program? Please describe                          hospital didactic program schedule and
objectives.                                                OPTI Ed Day schedule (with
                                                           objectives).
How are health policy issues integrated   A3               Programmatic evidence: Copy of
into the program? Please describe                          hospital didactic program schedule and
objectives.                                                OPTI Ed Day schedule (with
                                                           objectives).
How are health administration issues      A3               Programmatic evidence: Copy of
integrated into the program? Please                        hospital didactic program schedule and
describe objectives.                                       OPTI Ed Day schedule (with
                                                           objectives).
How are managed care issues integrated A3                  Programmatic evidence: Copy of
into the program? Please describe                          hospital didactic program schedule and
objectives.                                                OPTI Ed Day schedule (with
                                                           objectives).




Compliments of CORE Faculty Development                                                                                                  2
                                                                CORE Program Internal Review
                                                                         Template

PIR Discussion Questions                   Reference Item   Suggested Evidence (Other Types of Does the      Notes   Recommended Action
                                           from AOA PIR     Evidence Not Listed Here May       Evidence
                                           Questionnaire    Suffice)                           Meet the
                                                                                               Criteria of
                                                                                               Compliance?
                                                                                               (Y/N)
How are cost containment issues            A3               Programmatic evidence: Copy of
integrated into the program? Please                         hospital didactic program schedule and
describe objectives.                                        OPTI Ed Day schedule (with
                                                            objectives).
How is instruction on communication        A4               Programmatic evidence: Copy of
skills and biopsychosocial medicine                         hospital didactic program schedule
integrated into the program?                                (with objectives).
How is medical research methodology         A5              Programmatic evidence: Copy of
and design, statistics, and critical review                 hospital didactic program schedule
of the literature incorporated in the                       (journal club, introduction to research),
program.                                                    Research Ed Day schedule,
                                                            Documentation about Hospital and
                                                            AOA COM IRB Procedures.
                                                            Performance evidence: Copies of
                                                            approved research proposals. Copies
                                                            of completed research.


How are basic science courses              A6               Programmatic evidence: Copy of OPTI
integrated into the program?                                Ed Day schedule where basic science
                                                            sessions were offered. Copy of
                                                            specialty colleges' basic science
                                                            courses.
How is OPP integrated into the program? A7                  Programmatic evidence: Copy of
                                                            hospital didactic program schedule and
                                                            OPTI Ed Day schedule.
How often do interns/residents get         A8               Programmatic evidence: Copy of OPP
evaluated on their proficiency in the                       competency evaluation tools, OSCE
utilization of OPP techniques?                              (OMM case). Performance evidence:
                                                            OMM technique checklist.




Compliments of CORE Faculty Development                                                                                                   3
                                                              CORE Program Internal Review
                                                                       Template

PIR Discussion Questions                 Reference Item   Suggested Evidence (Other Types of Does the      Notes   Recommended Action
                                         from AOA PIR     Evidence Not Listed Here May       Evidence
                                         Questionnaire    Suffice)                           Meet the
                                                                                             Criteria of
                                                                                             Compliance?
                                                                                             (Y/N)
What are the ways in which residents     A9               Programmatic evidence: Interview
assume responsibility for teaching and                    trainees about their teaching and
supervising other residents, interns,                     supervising responsibilities.
and/or students?                                          Performance evidence: Copy of
                                                          rotation evaluation form (residents in
                                                          the service are evaluated by students).


What are the ways in which               A10              Programmatic evidence: Attendance at
interns/residents participate in                          educational programs focusing on
educational programs regarding                            physician impairment and substance
physician impairment, including                           abuse. Interview trainees about
substance abuse?                                          participation experience in educational
                                                          programs focusing on physician
                                                          impairment and substance abuse.



How do interns/residents provide a      A11               Programmatic evidence: Interview
confidential written assessment and                       trainees about the process of providing
evaluation of the curriculum? How often                   assessment and evaluation of the
is this done?                                             curriculum.
How are board review activities          A12              Programmatic evidence: Copy of
integrated into the program?                              specialty colleges' board review
                                                          courses/activities.
Evaluation of Intern/Resident
Performance
What evaluation of intern/resident       B1               Programmatic evidence: Copy of
performance does the teaching faculty                     monthly rotation evaluation completed
complete at the end of each rotation?                     in a timely manner. Performance
                                                          evidence: Logs signed by trainer(s).




Compliments of CORE Faculty Development                                                                                                 4
                                                                CORE Program Internal Review
                                                                         Template

PIR Discussion Questions                   Reference Item   Suggested Evidence (Other Types of Does the      Notes   Recommended Action
                                           from AOA PIR     Evidence Not Listed Here May       Evidence
                                           Questionnaire    Suffice)                           Meet the
                                                                                               Criteria of
                                                                                               Compliance?
                                                                                               (Y/N)
How often and with what instrument is      B2               Programmatic evidence: Copy of
the intern/resident review conducted by                     quarterly evaluation. Interview trainees
the program director?                                       to discuss quarterly evaluation process.

What instrument is used to provide an     B3                Programmatic evidence: Copy of
annual written summary of each                              annual evaluation of each trainee, MEC
intern/resident's overall performance and                   meeting minutes that reflect trainee
progression through the program that is                     performance as agenda item.
reviewed by MEC on a quarterly and
annual basis?
What instrument(s) is(are) used to       B4                 Programmatic evidence: Copy of
provide a written assessment of                             competency evaluation tools.
intern/resident performance in achieving                    Performance evidence: OSCE, Mock
competency in the seven required                            Oral Boards, Checklists, Mini-CEX.
areas?
What are the final evaluation tools used B4                 Programmatic evidence: Copy of
to evaluate the competencies at the end                     competency evaluation end-of-training
of the training?                                            report. Individual resident final
                                                            portfolios of best performance on each
                                                            of the seven competencies.


What type of remedial program is           B5               Programmatic evidence: Copy of
provided for interns/residents (if there                    remediation policy/program in
are interns/residents within the past                       residency manual.
three years that entered a remedial
program)?




Compliments of CORE Faculty Development                                                                                                   5
                                                                 CORE Program Internal Review
                                                                          Template

PIR Discussion Questions                    Reference Item   Suggested Evidence (Other Types of Does the      Notes   Recommended Action
                                            from AOA PIR     Evidence Not Listed Here May       Evidence
                                            Questionnaire    Suffice)                           Meet the
                                                                                                Criteria of
                                                                                                Compliance?
                                                                                                (Y/N)
What methods are used to monitor the        B6               Programmatic evidence: Copy of
stress and emotional health of                               hospital didactic program schedule
interns/residents?                                           highlighting health and wellness
                                                             sessions. Interview with program
                                                             director and trainees. Copy of the
                                                             Employee Assistance Program and
                                                             availability of mental health counseling.


What, if any rotations, have been           B7               Programmatic evidence: Copy of
modified due to undesirable stress?                          modified rotation (highlighting change).

Organization of Teaching Staff
What are the categories of attending        C1               Programmatic evidence: Faculty roster
staff in relationship to the program?                        with curriculum vitae. Interview faculty
                                                             to discuss teaching load. Appointment
                                                             letters from the AOA COM and OPTI
                                                             assigning faculty rank and status.


What is the written criteria defining the   C2               Programmatic evidence: Faculty roster
requirements of the teaching faculty?                        with curriculum vitae. Interview faculty
                                                             to discuss teaching requirements. Copy
                                                             of CORE Clinical Teaching Faculty
                                                             Appointment and Reappointment
                                                             Process.


What is the process for interns/residents C4                 Programmatic evidence: Copy of
to provide confidential written evaluations                  rotation evaluation form (with section
of the teaching staff?                                       on providing teaching feedback). Copy
                                                             of evaluation policy (assuring
                                                             confidentiality).
How do these evaluations include all        C5               Programmatic evidence: Copy of
physicians with whom interns/residents                       rotation evaluation form.
have contact with?




Compliments of CORE Faculty Development                                                                                                    6
                                                             CORE Program Internal Review
                                                                      Template

PIR Discussion Questions                Reference Item   Suggested Evidence (Other Types of Does the      Notes   Recommended Action
                                        from AOA PIR     Evidence Not Listed Here May       Evidence
                                        Questionnaire    Suffice)                           Meet the
                                                                                            Criteria of
                                                                                            Compliance?
                                                                                            (Y/N)
How is evaluation information shared    C6               Programmatic evidence: Copy of policy
with the teaching faculty?                               on managing and distributing teaching
                                                         evaluation data. Interview faculty to
                                                         discuss teaching feedback mechanism.

How (and how often) are the teaching    C7               Programmatic evidence: Copy of policy
staff reviewed and evaluated by the                      on managing and distributing teaching
department chairman and program                          evaluation data. Copy of ICCP (section
director?                                                on annual review of teaching
                                                         evaluations). Interview program director
                                                         to discuss teaching feedback
                                                         mechanism.
Where are the evaluations of teaching   C8               Programmatic evidence: Copy of policy
staff by department chairman/program                     on managing and distributing teaching
director available for review?                           evaluation data. Interview program
                                                         director to discuss teaching feedback
                                                         mechanism and to discuss changes in
                                                         teaching staff assignment as a result of
                                                         the evaluation process.


How does your program ensure that all C10                Programmatic evidence: Copy of FD
members of the teaching staff                            program schedule, OPTI Program
demonstrate a strong interest in the                     Directors Meetings, Individual faculty's
education of interns/residents, sound                    CME. Performance evidence:
clinical and teaching abilities, support of              Teaching/Service excellence award,
the goals and objectives of the program,                 aggregated teaching evaluation
and a commitment to their own                            summary indicating excellent, good,
continuing medical education?                            fair, or poor teaching.




Compliments of CORE Faculty Development                                                                                                7
                                                                 CORE Program Internal Review
                                                                          Template

PIR Discussion Questions                    Reference Item   Suggested Evidence (Other Types of Does the      Notes   Recommended Action
                                            from AOA PIR     Evidence Not Listed Here May       Evidence
                                            Questionnaire    Suffice)                           Meet the
                                                                                                Criteria of
                                                                                                Compliance?
                                                                                                (Y/N)
How does teaching faculty participate in    C11              Programmatic evidence: Faculty
scholarly activities?                                        curriculum vitae highlighting scholarly
                                                             activities (research poster, conference
                                                             presentations, publications).

What regular meetings are conducted to C12                   Programmatic evidence: MEC meeting
review program goals, objectives, and                        minutes, documentation of 1-1
overall effectiveness of the program?                        Consultation with FD.

Institutional Requirements
What are the goals and objectives for out D1                 Programmatic evidence: List of out
rotations?                                                   rotations (with objectives) and affiliation
                                                             agreements.
What guidelines determine who is            D2               Programmatic evidence: Copy of "out
responsible for the day-to-day activities                    rotation" guidelines that include
of the program completed at an outside                       orientation to the rotation, program
institution?                                                 director interaction with the supervising
                                                             physician, outline of oversight
                                                             expectations, and examples of
                                                             completed evaluations, which indicate
                                                             reporting structure back to the program
                                                             director.

How are interns/residents integrated into D3                 Programmatic evidence: Attendance at
the clinical review of quality of care,                      utilization review sessions. Interview
utilization, and other staff review                          trainees on participation's impact on
activities?                                                  their training.




Compliments of CORE Faculty Development                                                                                                    8
                                                                 CORE Program Internal Review
                                                                          Template

PIR Discussion Questions                    Reference Item   Suggested Evidence (Other Types of Does the      Notes   Recommended Action
                                            from AOA PIR     Evidence Not Listed Here May       Evidence
                                            Questionnaire    Suffice)                           Meet the
                                                                                                Criteria of
                                                                                                Compliance?
                                                                                                (Y/N)
What instructional materials are provided D4                 Programmatic evidence: Attendance at
to all interns/residents regarding quality                   quality assurance performance
assurance performance improvement?                           improvement sessions. Copy of
                                                             session handouts. Performance
                                                             evidence: Quality improvement
                                                             activities/projects that trainees
                                                             participated in.


What is the process provided to ensure      D5               Programmatic evidence: Copy of 80-
that interns/residents have alternate 48-                    hour policy in Residency Manual.
hour weekends off duty, or at least one                      Interview trainees to discuss duty
24-hour period off each weekend?                             hours.

What is the process provided to prevent D6                   Programmatic evidence: Copy of "Call
interns/residents from being assigned to                     Schedule". Interview trainees to
call more often than every third night?                      discuss call issues.

What circumstances, if any, have            D7               Programmatic evidence: Copy of 80-
caused interns/residents to be assigned                      hour policy in Residency Manual.
to work physically on duty in excess of                      Interview trainees to discuss duty
80 hours per week?                                           hours. List of situations/circumstances.

What situations/circumstances cause        D7                Programmatic evidence: Copy of 80-
interns/residents to be physically on duty                   hour policy in Residency Manual.
in excess of 24 hours with 6 additional                      Interview trainees to discuss duty
hours allowed for patient continuity and                     hours. List of situations/circumstances.
educational sessions?


What is the process provided to ensure D8                    Programmatic evidence: Copy of 80-
that interns/residents have (at the                          hour policy in Residency Manual.
conclusion of a 24-hour shift) a minimum                     Interview trainees to discuss duty
of 12 hours off duty before returning to                     hours.
work?




Compliments of CORE Faculty Development                                                                                                    9
                                                             CORE Program Internal Review
                                                                      Template

PIR Discussion Questions                Reference Item   Suggested Evidence (Other Types of Does the      Notes   Recommended Action
                                        from AOA PIR     Evidence Not Listed Here May       Evidence
                                        Questionnaire    Suffice)                           Meet the
                                                                                            Criteria of
                                                                                            Compliance?
                                                                                            (Y/N)
What services are provided to minimize D9                Programmatic evidence: List of
intern/resident work that is extraneous to               services provided by hospital or
the educational program?                                 program which relieves residents of
                                                         duty during educational activities.
                                                         Example of coverage policies and post-
                                                         call provisions. Copy of policy(ies) that
                                                         reflect educational value to activities (in
                                                         preference to predominantly service
                                                         requirements).


What is the procedure/policy provided to D10             Programmatic evidence: Copy of OPTI
ensure that interns/residents are freed                  Attendance Policy. Attendance at
from duties to attend conferences,                       hospital didactic programs, OPTI Ed
including OPTI Conferences?                              Days, and CME conferences.

In the past three years, how many       D11              Programmatic evidence: Copy of OPTI
residents have failed to meet OPTI or                    and Program Attendance Policy for Ed
program conference attendance                            Days. Attendance at OPTI Ed Days.
requirements?
What is the policy that requires        D12              Programmatic evidence: Copy of policy
interns/residents to be certified to                     which outlines credentialing for each
perform procedures independent of                        level of training.
direct supervision?
How are complications and deaths        D13              Programmatic evidence: Attendance at
reviewed as part of the educational                      M&M Sessions. Interview trainees to
program?                                                 discuss didactic sessions they
                                                         participated in that addressed issues
                                                         related to death (for example, book
                                                         club, communication or notification of
                                                         death training seminars, etc).




Compliments of CORE Faculty Development                                                                                                10
                                                                CORE Program Internal Review
                                                                         Template

PIR Discussion Questions                   Reference Item   Suggested Evidence (Other Types of Does the      Notes   Recommended Action
                                           from AOA PIR     Evidence Not Listed Here May       Evidence
                                           Questionnaire    Suffice)                           Meet the
                                                                                               Criteria of
                                                                                               Compliance?
                                                                                               (Y/N)
What is the number of autopsies done by D14                 Programmatic evidence: Copy of
your institution that represent diverse                     hospital global autopsy records
spectrum of disease?                                        indicating the number of autopsies
                                                            performed and types of disease (e.g.
                                                            Hospital Mortality Reporting log for
                                                            local, state regulators).
How are interns/residents notified when    D14              Programmatic evidence: Copy of policy
an autopsy of their patient is being                        that outlines how interns and residents
performed?                                                  are notified about autopsy findings.
                                                            Examples from resident logs and
                                                            resident files.
How and when are interns/residents         D14              Programmatic evidence: Copy of a
provided a copy of the autopsy report?                      sample memo/form used to notify
                                                            interns/residents.
What are the specific criteria for         D15              Programmatic evidence: Copy of
intern/resident selection? Is a personal                    interview questions. Copy of
interview required?                                         intern/resident selection criteria. Copy
                                                            of nondiscriminatory clause/policy. List
                                                            of trainees (highlighting
                                                            underrepresented groups).


What is the process used to ensure that D16                 Programmatic evidence: Copy of
interns/residents meet AOA eligibility                      trainee roster with medical license
requirements?                                               documentation and other official
                                                            documentation, including COM
                                                            graduation certificate, program transfer
                                                            documents indicating good standing,
                                                            etc.
OPTI Expectations




Compliments of CORE Faculty Development                                                                                                   11
                                                                 CORE Program Internal Review
                                                                          Template

PIR Discussion Questions                    Reference Item   Suggested Evidence (Other Types of Does the      Notes   Recommended Action
                                            from AOA PIR     Evidence Not Listed Here May       Evidence
                                            Questionnaire    Suffice)                           Meet the
                                                                                                Criteria of
                                                                                                Compliance?
                                                                                                (Y/N)
What is the process and review                               Programmatic evidence: Attendance of
procedure used to examine program                            program director at OPTI-sponsored
directors' participation in the OPTI?                        meetings. OPTI OME meeting agenda
                                                             showing that program directors and
                                                             DMEs attendance was reviewed.
                                                             Interview program director to discuss
                                                             participation in OPTI and its impact on
                                                             his/her program.


What are the program documents that                          Programmatic evidence: Graduation
show proof of an AOA COM program                             certificates, affiliation agreements, and
sponsorship and OPTI membership?                             recruitment materials.

What is the program's progress in                            Programmatic evidence: Copy of
response to previous citations? Please                       Corrective Action Plan and timeline.
indicate the corrective action plan and
specify if progress is in accordance with
an appropriate timeline.


Was the corrective action plan reviewed                      Programmatic evidence: Copy of
by the OPTI before submission to AOA                         Corrective Action Plan and timeline.
(in accordance with OPTI protocol)?




Compliments of CORE Faculty Development                                                                                                    12

				
DOCUMENT INFO
Description: Performance Evaluation Tools document sample