"360 Rating Form 360 Evaluation Instrument 360 Assessment Tool"
360 Rating Form 360 Evaluation Instrument 360 Assessment Tool Assessing professionalism: A Word of Caution in the First Steps! 1/24/2007 Malakan Rad 2 The elements are reliability and responsibility, honesty and integrity, maturity, respect for others, critique, altruism, interpersonal skills, and [absence of ] impairment (i.e., psychological/ chemical). 1/24/2007 Malakan Rad 3 additional humanistic qualities; honor, integrity, ethical and moral standards; accountability; excellence; and duty/advocacy. 1/24/2007 Malakan Rad 4 altruism demands that the best interests of patients, not self-interest, guide physicians. Honor and integrity entail the highest standards of behavior and the refusal to violate one’s personal and professional codes. Duty is the free acceptance of commitment to service. 1/24/2007 Malakan Rad 5 Society expects physicians to act professionally. In response to recent criticism regarding unprofessional behavior in medicine, some argue that improving medical professionalism can only occur through changes in teaching and .assessing it 1/24/2007 Malakan Rad 6 There are at least three types of studies, however, that may point the way for future evaluation thrusts. 1/24/2007 Malakan Rad 7 Some work evaluates professional behavior as part of clinical performance. Other studies evaluate only professional behavior, as a comprehensive entity in and of itself. Still other research evaluates single elements of professional behavior such as humanism, self-assessment, dutifulness, altruism, empathy and compassion, honesty, integrity, and ethical behavior, as well as communication. 1/24/2007 Malakan Rad 8 Although assessing professionalism poses many challenges,gauging and detecting changes in professionalism is impossible without measurement. There was a review of techniques used to assess professionalism during the past 20 years. 1/24/2007 Malakan Rad 9 The authors searched five electronic databases and reference lists from 1982 to 2002. Eighty-eight assessments were retained and organized into : 1/24/2007 Malakan Rad 10 1/24/2007 Malakan Rad 11 content area: 1-ethics 2-personal Characteristics 3- comprehensive professionalism 4- diversity 1/24/2007 Malakan Rad 12 Diversity pertained to assessments that addressed cultural issues, socioeconomic status, gender, age or disability. Validity and reliability were examined for five of 10 instruments 1/24/2007 Malakan Rad 13 Type of outcome examined: 1-affective 2- cognitive 3- behavioral 4-environmental 1/24/2007 Malakan Rad 14 personal comprehensive ethics diversity Characteristics professionalism cognitive affective behavioral environmental 1/24/2007 Malakan Rad 15 Ethics Forty-nine assessments addressed various aspects of ethics such as morality, ethical principles, honor codes, social norms, deception, abuse or mistreatment, cheating, disclosure, and sexual misconduct. 1/24/2007 Malakan Rad 16 Personal characteristics The category labeled ‘personal characteristics’ included assessments that addressed attributes such as emotional intelligence, personal values, empathy, trustworthiness, cynicism and dogmatism. All of the personal characteristics instruments assessed affective outcomes. Validity and reliability were examined for 13 of 15 (Table 1a). 1/24/2007 Malakan Rad 17 Humility Honesty Balance between commitment at hospital and commitment at home Altruism Respectfulness Compassion Sensitivity Tactfulness …. 1/24/2007 Malakan Rad 18 1/24/2007 Malakan Rad 19 Tip 1 Build a positive view of the importance of training in professional attitudes with all those involved in the curriculum 1/24/2007 Malakan Rad 20 Tip 2 Use examples of good practice from elsewhere 1/24/2007 Malakan Rad 21 Tip 3 Ensure that professional development objectives are set throughout the training curriculum 1/24/2007 Malakan Rad 22 Tip 4 Use learning methods that match the objectives 1/24/2007 Malakan Rad 23 Tip 5 Tutors must be good role models 1/24/2007 Malakan Rad 24 Tip 6 Create a learning culture that examines shortcomings openly 1/24/2007 Malakan Rad 25 Tip 7 Use peer influence systematically 1/24/2007 Malakan Rad 26 Tip 8 Assess the learning objectives formally 1/24/2007 Malakan Rad 27 Tip 9 Support the learner in vulnerability and strengthen coping 1/24/2007 Malakan Rad 28 Tip 10 Reward good performance by students and staff 1/24/2007 Malakan Rad 29 Tip 11 Secure necessary resources and skills 1/24/2007 Malakan Rad 30 Tip 12 Use levers for change within the curriculum 1/24/2007 Malakan Rad 31 As the saying goes, ‘‘It is easier to kill a man than to change his ways’’. The author hopes these tips may be useful to those who wish to preserve life in all its fullness. 1/24/2007 Malakan Rad 32 General recommendations for assessing professionalism 1/24/2007 Malakan Rad 33 •Encourage formative and Start early Summative assessment, often undertaken to classify learners, dominates medical education. Using assessment to improve learning is gaining credibility, however, with reports of formative assessments appearing more often in the literature. Some evidence suggests that focused teaching can improve moral reasoning (Self et al., 1992; Goldie et al., 2002); more specifically, that feedback derived from assessment may improve professional behaviors (Phelan et al., 1993; Papadakis et al., 2001). These findings suggest that professionalism should be formatively assessed. This means that assessment should begin early (Lowe et al., 2001) be conducted frequently, be implemented long term, and provide learners with opportunities to change (Van Luijk et al., 2000). 1/24/2007 Malakan Rad 34 The educational environment, whether through formal or informal curricula, appears to influence learner attitudes and behavior (Stern, 1996). One study discerned relationships between the ethical environment and medical students’ ethical behavior (Feudtner et al., 1994). In another study, residents reported learning most about professionalism from observing role models (Brownell & Cote, 2001). 1/24/2007 Malakan Rad 35 Research suggests that the business (Freeman et al., 1999) and cultural environment (Hoffmaster et al., 1991) influence professionalism among practicing physicians. Consequently, assessments that gauge professionalism in the environment may provide insight into the professionalism of individuals. 1/24/2007 Malakan Rad 36 Systematic assessment of professionalism should also include many different assessors, more than one assessment method and assessment in different settings (Littlefield et al., 1996; Van Luijk et al., 2000; Ginsburg et al., 2002). Research indicates that different assessors offer different perspectives, thus enhancing the breadth of assessment (Wooliscroft et al., 1994) and multiple assessors enhance reliability (Swanson, 1987). Each assessment method has strengths and weaknesses. 1/24/2007 Malakan Rad 37 Rating forms are considered relatively easy to use, but are plagued with the ‘halo’ or ‘horns’ effect (Gray, 1996). 1/24/2007 Malakan Rad 38 THE HALO EFFECT The halo effect results when a supervisor overrates employee by: • Failing to see deteriorating performance because of a good past record or personal friendship. • Rewarding those with similar beliefs or background as his or her own. • Believing that, because performance is outstanding in one area, there are no problems in other areas. 1/24/2007 Malakan Rad 39 THE HORNS EFFECT The horns effect results when a supervisor underrates employees who: • Does not meet the impossible high standards the supervisor sets. • Achieves results by different methods that the supervisor uses. • Fails at one thing – and find that this one instance is influencing the supervisor’s future evaluations. 1/24/2007 Malakan Rad 40 Using more than one assessment method may help to compensate for the weaknesses associated with any single approach. 1/24/2007 Malakan Rad 41 Using more thanBecause professionalism is a complex construct, it is unlikely that a single assessment will adequately measure it. Using a combination of assessments, however, such as a moral reasoning assessment together with a behavioral assessment, may be adequate. 1/24/2007 Malakan Rad 42 Different Settings Assessment in different settings can help to determine the generalizability of learners’ professionalism and identify context specific issues relevant to learners. Suggestions for assessing medical students, residents, and practicing physicians Medical students. 1/24/2007 Malakan Rad 43 Assessment of medical student professionalism is often delayed until clerkship rotations. The Physicianship Evaluation Form (Papadakis et al., 2001), however, indicates that it is both desirable and possible to begin assessing student professionalism during the first year of medical school. 1/24/2007 Malakan Rad 44 Initial use of this assessment is formative; students are invited to present their perspective and data are used to provide feedback and guide remediation. 1/24/2007 Malakan Rad 45 Persistent patterns of unprofessional behavior, despite remediation, may provide grounds for dismissal. This performance-based, longitudinal approach helps to set professionalism expectations early, both for learners and for faculty who must commit to addressing and attempting to improve students’ professional behavior. 1/24/2007 Malakan Rad 46 Arnold suggested that assessments could be improved by focusing on professionalism as a separate entity, including rigorous qualitative approaches, exploring the extent to which the environment supports assessment, and determining the usefulness of developmental expectations in assessing professionalism. 1/24/2007 Malakan Rad 47 Ginsburg and colleagues (2000) examined existing assessments in terms of types of raters, such as faculty supervisors and peers. They reasoned that difficulties involved in assessing professionalism are due to the frequent use of abstract idealized definitions, the context specific nature of professionalism, and evaluator reluctance to address relatively minor lapses. The same authors proposed that professionalism assessments should address cognitive and behavioral outcomes. 1/24/2007 Malakan Rad 48 For Students Students should also be prepared to address ethical issues in clinical settings. Behavioral simulations, such as the Moral Behavior Analysis (Sheehan et al., 1987) and the Ethics OSCE (Singer et al., 1996), in which ethical dilemmas are role-played, are ideal for this purpose, especially if they include post-encounter oral or written assessments that probe reasoning used during encounters. To obtain reliable data, many cases would be required, thus, these simulations should be used for formative assessment only. 1/24/2007 Malakan Rad 49 For Residents Residents typically rotate through many different settings and interact with a broad range of personnel. These characteristics of residency training may present challenges to coordinating comprehensive assessment and providing opportunities for the development of long-term relationships helpful to assessing professionalism. 1/24/2007 Malakan Rad 50 On the other hand, changing rotations provide the option of obtaining input from various observers (e.g. nurses, patients, supervising physicians and peers), and assessing the extent to which resident professionalism varies across settings. 1/24/2007 Malakan Rad 51 To assess residents’ knowledge of professionalism and to stimulate instructional discussion, the Barry’s Challenges to Professionalism Questionnaire may be used (Barry et al., 2000). This self-administered questionnaire consists of vignettes that address conflict of interest, gifts and physician impairment. Research results indicated that the questionnaire was able to detect a broad range of knowledge about professionalism and discriminate among respondents depending on years of experience. 1/24/2007 Malakan Rad 52 For Practicing physicians Assessments of ethics knowledge by surveys and of comprehensive professionalism by patient questionnaires were the most frequently used, and are probably the most feasible, approaches to assessing professionalism in practicing physicians. For instance, the Defining Issues Test, a written test in which ethical dilemmas are presented in vignettes, is capable of discriminating different levels of moral reasoning among practicing physicians (Baldwin & Bunch, 2000). The Barry’s Challenges to Professionalism Questionnaire would also be suitable for use with practicing physicians. 1/24/2007 Malakan Rad 53 Examples of patient questionnaires designed to assess practicing physician professionalism include the Humanism Scale (Hauck et al., 1990) and the Wake Forest Physician Trust Scale (Hall et al., 2002). Both have yielded valid and reliable data. 1/24/2007 Malakan Rad 54 Patient questionnaires have advantages: they obtain information about physicians’ behaviors in actual practice, they seem to be acceptable to physicians, and they may prompt changes in behaviors assessed. 1/24/2007 Malakan Rad 55 Summary At least 88 professionalism assessments have been used in medical education since 1982. 1/24/2007 Malakan Rad 56 For Student Useful approaches to assessing medical student professionalism include a longitudinal, performance- based method and behavioral simulations with post-encounter probes. 1/24/2007 Malakan Rad 57 For Resident Useful approaches to assessing resident physician professionalism include a 360-degree assessment and a cognitive assessment of professionalism. 1/24/2007 Malakan Rad 58 For Practicing Physician Useful approaches to assessing practicing physician professionalism include patient questionnaires and a cognitive assessment of professionalism. 1/24/2007 Malakan Rad 59 No single method exists for the reliable and valid evaluation of professional behavior. 1/24/2007 Malakan Rad 60 more studies on the predictive validity of assessments and their use as part of formative evaluation systems are recommended 1/24/2007 Malakan Rad 61 Instead of creating new professionalism assessments,existing assessments should be improved. 1/24/2007 Malakan Rad 62 A Word of Caution! The new professionalism needs to be tackled at a number of levels. 1/24/2007 Malakan Rad 63 Teaching :unprofessionalism(by role model) + Assessment:assessing professionalism = Absolute unprofessionalistic act 1/24/2007 Malakan Rad 64 1/24/2007 Malakan Rad 65 1/24/2007 Malakan Rad 66 Shell (surface) Core (depth) 1/24/2007 Malakan Rad 67 Shell (surface) Core (depth) 1/24/2007 Malakan Rad 68 Shell (surface) 1/24/2007 Malakan Rad 69 0 degree 360 degree 1/24/2007 Malakan Rad 70 باسپاس 1/24/2007 Malakan Rad 71