Get documents about "Psychology
Document Sample
Annual Assessment Report to the Deans
Due September 30 each Academic Year
Please use this format in submitting your assessment information. Only report data on outcomes assessed during the report
academic year.
Department/Program: Psychology
A. Published
Identify where the department’s/program’s outcomes are published.
___________on-line catalogue, http://www.keene.edu/catalog/
___________department’s web page
___________department handbook
___________VSA (Voluntary System of Accountability)
_____X______other (identify) Q Drive
If outcomes are published online, please indicate the URL:
AAASC Doc. 2-17-2010
B. Assessment Matrix (Fall 2010 – Spring 2011)
Assessment Matrix
Method
Program Learning Outcome Course/ Performance
(sample, measures, data Results Action/Improvement Plan
Experience Criterion
collection & analysis)
Sampling of incoming freshmen N/A N/A. This Administration of a N/A
in their first psychology course. sample was department generated
Fall 2010
used to 75 item multiple choice
establish a test representing core
baseline for concepts in psychology
comparison. (see attached narrative)
Sampling of 2nd sem. Juniors. Lecture A group mean Administration of a A sampled The results for personality
Mastery of content in core courses; of 50% correct department generated group of Jrs., was not due to absence of
courses (Exper. Design, Brain 200 and 300 in all five core 75 item multiple choice achieved a Mn course (92% took the
and Behavior, Personality, levels areas. test representing core of 50 % on 4 of course). We will distribute
Spring 2011
Social, and Abnormal psych.) concepts in psychology the 5 core a list of core personality
(see attached narrative) areas (all but concepts to all faculty
personality). teaching this course in
2011 – 2012.
Sampling of 2nd sem. Juniors. Lecture A group mean Administration of a A different Item analysis will be done
Mastery of content in core courses; of 50% correct commercial multiple group of jrs. on commercial test to
courses (Exper. Design, Brain 200 and 300 in all five core choice test (ACAT) of achieved a Mn assess content validity. In
and Behavior, Personality, level areas. core concepts in of 50% on 2 of SP 2011 we will switch to a
Social, and Abnormal psych.) psychology (see the 5 core fully within group
attached narrative) areas. comparison.
AAASC Doc. 2-17-2010
C. Fall 2010-Spring 2011Program Outcomes
Please attach all program outcomes for Fall 2010-Spring 2011.
Have these program outcomes changed since the last report?
Yes No X
D. Program Outcomes Summary – KSC Department of Psychology 2010 – 2011
History and Overview
AY 2008 – 2009
During the AY 2008 – 2009, the KSC Psychology department began to formulate an assessment plan. The initial focus was on
developing a quantitative exit exam, primarily for seniors.
The development of this test was rational or theory-based. We began with a seminal article published in the American Psychologist
by Boneau (1990) whose survey of psychology departments yielded lists of core concepts by sub-discipline (e.g., Physiological,
Experimental Design, Social Psychology, Personality Theory). These lists were circulated among members of the KSC psychology
department for review and comment. After a consensus was reached on a set of core concepts, department members were asked
to submit sample questions. A final pool of 136 items was generated reflecting the following 12 sub-disciplines: History and
AAASC Doc. 2-17-2010
Systems, Statistics and Research Methods, Brain and Behavior, Sensation and Perception, Learning, Cognitive, Developmental,
Motivation, Emotion, Social, Personality, and Abnormal/Clinical.
In the spring of 2009, we gave the 136 item test to 119 juniors and seniors. In the Fall of 2009 we submitted our first report. To
focus our assessment on those who had already taken courses linked to the material on the test, we narrowed our analysis to four of
the twelve content areas: Statistics and Research Methods, Brain and Behavior, Social Psychology, and Personality. Courses in these
four areas are taken by most, if not of our majors, as part of our “gateway” into the major, or as an elective to complete “group”
requirements.
Quantitative results from spring 2009:
Statistics and Research Methods
Strengths: Experimental Design (control groups, random selection, IV, DV), Likert scale, IQ scores
Weaknesses: Psychometrics (reliability, validity), T distribution, p values associated with correlations
Brain and Behavior
Strengths: Neuron structure, action potential, basic neuroanatomy and physiology (except emotion)
Weaknesses: phrenology, corpus callosum, L-R brain, ANS (branches), emotion, naturalistic studies
Social Psychology
Strengths: Obedience and Conformity, Attitudes, TV and aggression
Weaknesses: Rosenthal study (self-fulfilling prophecies)
Personality Psychology
Strengths: Humanistic tradition, Freud (stages, dreams)
Weaknesses: NEO Five-factor, Eysenck, Allport (religion, prejudice), Freud (transference)
AAASC Doc. 2-17-2010
Recommendations submitted fall 2009:
The psychology faculty were alerted to student’s weaknesses. After a discussion of the results on an item by item basis, we decided
to reduce the 136 items by approximately half the total. The consensus was also to focus our analyses on the same four content
areas. In addition, a qualitative assessment of critical thinking was planned for spring 2010.
AY 2009 –2010:
During AY 2009 – 2010 department members expressed interest in adding a qualitative assessment geared towards critical thinking
skills. We consulted the APA Board of Educational Affairs Task Force on Psychology Major Competencies (2006). Specifically, we
focused on “Critical Thinking Skills in Psychology”. The APA task force listed 22 potential outcomes that could be assessed (e.g.,
challenge claims that arise from myth, stereotype, or untested assumptions). Some outcomes were more relevant for assessing a
student’s appraisal of someone else’s work while others were more relevant for assessing a student’s own work. We created a 10-
point rubric from the first set of outcomes (evaluating the work of others, see Appendix A). A short excerpt from Newsweek taken
from the book, The Secret, was selected. In the spring of 2010, 14 seniors participated in this qualitative assessment. Their essays
were read by a member of the assessment committee who scored them for the presence of any of the 10 items in the critical
thinking rubric. In addition, we employed our 136 item test for a second time. Seventy senior psychology majors participated in this
quantitative assessment.
Results from spring 2010: (Bolded concepts are new strengths or weakness or were in the opposite category last year, i.e. a strength
this year but a weakness last year, or vice versa)
Quantitative Results from spring 2010:
Statistics and Research Methods
Strengths: Experimental Design (control groups, random selection, IV, DV), Likert scale
Weaknesses: Psychometrics (reliability, validity), T scores, correlations, IQ scores, interaction effects
AAASC Doc. 2-17-2010
Brain and Behavior
Strengths: Neuron structure, action potential, basic neuroanatomy and physiology (except emotion)
Weaknesses: phrenology, corpus callosum, L-R brain, ANS (branches), emotion, sleep and brain waves
Social Psychology: 9 Questions
Strengths: Conformity, Attitudes, cognitive dissonance
Weaknesses: Rosenthal study (self-fulfilling prophecies), TV and aggression, Milgram study
Personality Psychology
Strengths: Humanistic tradition, Freud (psychosexual stages, dreams)
Weaknesses: NEO Five-factor, Eysenck, Allport (religion, prejudice), Freud (transference)
Qualitative Results
Most of the students “avoided being swayed by appeals to emotion or authority”, and were able to “evaluate quality of information,
differentiating empirical evidence from speculation” or “challenge claims that arise from myth, stereotype, or untested assumptions.
About a half of the students “deconstructed arguments (e.g., conclusions, premises/assumptions, gaps, counter-arguments” or
showed the ability to “distinguish among assumptions, emotional appeals, speculations, and defensible evidence”. Only two
“weighed support for conclusions to determine how well reasons support conclusions” and “identified weak, contradictory, and
inappropriate assertions”. No one “used scientific principles and evidence to resolve conflicting claims”.
Our assessment may have been limited by the nature of the selected work sample (A “pop” psychology book with little “data” to
evaluate or challenge).
Comparison of spring 2009 versus spring 2010 (quantitative results on 136 item exam)
A comparison across years shows remarkable consistency in strengths and weaknesses. In the area of statistics and research
methods, in both years, our students appear well versed in sampling procedures, identifying independent and dependent variables,
AAASC Doc. 2-17-2010
and understand the concept of a Likert scale. However, they struggle with notions of reliability and validity (test construction), and
are not sure about the mean and standard deviation of a standard IQ Test. In spring 2010, our students showed a little more
difficulty working with two-way Anovas (main effects vs. interactions). In the area of brain and behavior, our students were strong
both years on the topic of neuron structure and function but demonstrated more difficulty with the autonomic nervous system
(stress response) as well as the anatomy and physiology of emotion. In the spring of 2010, another weak area was brain wave
activity associated with sleep. In Social Psychology, conformity and attitudes were strong areas in both years. In 2010, our students
did better with cognitive dissonance, but worse with the effects of TV on aggression and the Milgram experiment. In Personality
theory, the pattern was identical across the two years. The strengths included Maslow and Rogers, and aspects of Freud; the
weaknesses included the NEO five-factor model, Allport’s significance, Eysenck’ research on traits, and the clinical concept of
“transference”.
Recommendations submitted in fall 2010
Faculty will be alerted to student strengths and weaknesses. In the spring of 2011 the reduced (75 item) test will be used instead of
the 136 item version. In future years we may consider a nationally-normed test provided by an outside source. We will continue to
focus on the four areas listed above (Statistics and Research Methods, Brain and Behavior, Social Psychology, and Personality
Theory). A better work sample will be used to assess critical thinking skills. (Ideally, an empirical study or at least a report with some
data will be obtained for this purpose.)
AY 2010 –2011:
In the fall of 2010 and the spring of 2011 we instituted our reduced (75 item) test. In the fall we assessed Freshmen taking their first
course in psychology to establish a baseline. In the spring we assessed 2nd semester juniors. Our rationale was that all of them
should have completed our core courses by this point in their program of studies.
In the spring of 2011 we also incorporated a nationally-normed test provided by an outside source, ACAT. We continued to focus on
the four areas listed above (Statistics and Research Methods, Brain and Behavior, Social Psychology, and Personality Theory). We
also assessed abnormal psychology. Our rationale for adding this fifth content area was that many of our majors take this course
because they are interesting in pursuing careers in clinical or counseling psychology, or they find it relevant for their future work in
AAASC Doc. 2-17-2010
social work, education, or health. We administered our internal measure to one group of juniors while another group received the
national test develop by ACAT. Below we first summarize our results involving freshman vs. juniors, as they performed on our
internal measure. Secondly, we summarize our result involving just the two groups of juniors as they performed on our internal
measure as compared to the national test.
Fall freshmen versus spring 2nd semester juniors (Department Psychology Test)
There was a clear increase in mastery of core concepts. On average we found an increase of approximately 16 percent across the
content areas, ranging from 10 percent for personality to 19 percent for research design and abnormal psychology. Among our
juniors, four of the five group means were 50 percent or higher. The one exception was personality psychology (44%).
Research Design Brain/Behavior Social Psych. Personality Abnormal Composite
Freshmen 42% 39% 38% 34% 31% 37%
Juniors 61% 56% 54% 44% 50% 53%
Spring 2nd semester juniors (Department vs. Commercial Test): Note: Bolded areas targeted for AY 2010 – 2011
Research Design Brain/Behavior Social Psych. Personality Abnormal Composite
Department Test 61% 56% 54% 44% 50% 53%
Commercial Test 42% 39% 42% 48% 57% 46%
Overall, our students appeared to perform a slightly better on our department psychology test. However, they did perform better on the
commercial test in personality and abnormal psychology. The greatest discrepancies were noted in research design and brain and
behavior. At this point, we are in the pilot testing and refinement phase of our quantitative assessment module for psychology. We
AAASC Doc. 2-17-2010
plan on doing an item analysis of the contents of both tests and review content validity as well as refining the departmental consensus
regarding the contents for each core area. In SP 2012 we will administer both tests to one group of 40 students.
AAASC Doc. 2-17-2010
