Docstoc

Psychology

Document Sample
Psychology Powered By Docstoc
					                                            Annual Assessment Report to the Deans
                                            Due September 30 each Academic Year

Please use this format in submitting your assessment information. Only report data on outcomes assessed during the report
academic year.

Department/Program:    Psychology

A. Published

   Identify where the department’s/program’s outcomes are published.


              ___________on-line catalogue, http://www.keene.edu/catalog/

              ___________department’s web page

              ___________department handbook

              ___________VSA (Voluntary System of Accountability)

              _____X______other (identify) Q Drive

              If outcomes are published online, please indicate the URL:




AAASC Doc. 2-17-2010
              B. Assessment Matrix (Fall 2010 – Spring 2011)
                                                                        Assessment Matrix
                                                                                          Method
                Program Learning Outcome           Course/       Performance
                                                                                 (sample, measures, data         Results        Action/Improvement Plan
                                                  Experience       Criterion
                                                                                   collection & analysis)
              Sampling of incoming freshmen       N/A           N/A. This        Administration of a                            N/A
              in their first psychology course.                 sample was       department generated
Fall 2010




                                                                used to          75 item multiple choice
                                                                establish a      test representing core
                                                                baseline for     concepts in psychology
                                                                comparison.      (see attached narrative)




              Sampling of 2nd sem. Juniors.       Lecture       A group mean       Administration of a        A sampled         The results for personality
              Mastery of content in core          courses;      of 50% correct     department generated       group of Jrs.,    was not due to absence of
              courses (Exper. Design, Brain       200 and 300   in all five core   75 item multiple choice    achieved a Mn     course (92% took the
              and Behavior, Personality,          levels        areas.             test representing core     of 50 % on 4 of   course). We will distribute
Spring 2011




              Social, and Abnormal psych.)                                         concepts in psychology     the 5 core        a list of core personality
                                                                                   (see attached narrative)   areas (all but    concepts to all faculty
                                                                                                              personality).     teaching this course in
                                                                                                                                2011 – 2012.
              Sampling of 2nd sem. Juniors.       Lecture       A group mean       Administration of a        A different       Item analysis will be done
              Mastery of content in core          courses;      of 50% correct     commercial multiple        group of jrs.     on commercial test to
              courses (Exper. Design, Brain       200 and 300   in all five core   choice test (ACAT) of      achieved a Mn     assess content validity. In
              and Behavior, Personality,          level         areas.             core concepts in           of 50% on 2 of    SP 2011 we will switch to a
              Social, and Abnormal psych.)                                         psychology (see            the 5 core        fully within group
                                                                                   attached narrative)        areas.            comparison.


              AAASC Doc. 2-17-2010
C. Fall 2010-Spring 2011Program Outcomes

Please attach all program outcomes for Fall 2010-Spring 2011.

Have these program outcomes changed since the last report?

Yes           No X




D. Program Outcomes Summary – KSC Department of Psychology 2010 – 2011
History and Overview

AY 2008 – 2009

During the AY 2008 – 2009, the KSC Psychology department began to formulate an assessment plan. The initial focus was on
developing a quantitative exit exam, primarily for seniors.

The development of this test was rational or theory-based. We began with a seminal article published in the American Psychologist
by Boneau (1990) whose survey of psychology departments yielded lists of core concepts by sub-discipline (e.g., Physiological,
Experimental Design, Social Psychology, Personality Theory). These lists were circulated among members of the KSC psychology
department for review and comment. After a consensus was reached on a set of core concepts, department members were asked
to submit sample questions. A final pool of 136 items was generated reflecting the following 12 sub-disciplines: History and




AAASC Doc. 2-17-2010
Systems, Statistics and Research Methods, Brain and Behavior, Sensation and Perception, Learning, Cognitive, Developmental,
Motivation, Emotion, Social, Personality, and Abnormal/Clinical.

In the spring of 2009, we gave the 136 item test to 119 juniors and seniors. In the Fall of 2009 we submitted our first report. To
focus our assessment on those who had already taken courses linked to the material on the test, we narrowed our analysis to four of
the twelve content areas: Statistics and Research Methods, Brain and Behavior, Social Psychology, and Personality. Courses in these
four areas are taken by most, if not of our majors, as part of our “gateway” into the major, or as an elective to complete “group”
requirements.

Quantitative results from spring 2009:

Statistics and Research Methods
     Strengths: Experimental Design (control groups, random selection, IV, DV), Likert scale, IQ scores
  Weaknesses: Psychometrics (reliability, validity), T distribution, p values associated with correlations

Brain and Behavior
    Strengths: Neuron structure, action potential, basic neuroanatomy and physiology (except emotion)
 Weaknesses: phrenology, corpus callosum, L-R brain, ANS (branches), emotion, naturalistic studies

Social Psychology
     Strengths: Obedience and Conformity, Attitudes, TV and aggression
  Weaknesses: Rosenthal study (self-fulfilling prophecies)

Personality Psychology
     Strengths: Humanistic tradition, Freud (stages, dreams)
  Weaknesses: NEO Five-factor, Eysenck, Allport (religion, prejudice), Freud (transference)




AAASC Doc. 2-17-2010
Recommendations submitted fall 2009:

The psychology faculty were alerted to student’s weaknesses. After a discussion of the results on an item by item basis, we decided
to reduce the 136 items by approximately half the total. The consensus was also to focus our analyses on the same four content
areas. In addition, a qualitative assessment of critical thinking was planned for spring 2010.

AY 2009 –2010:

During AY 2009 – 2010 department members expressed interest in adding a qualitative assessment geared towards critical thinking
skills. We consulted the APA Board of Educational Affairs Task Force on Psychology Major Competencies (2006). Specifically, we
focused on “Critical Thinking Skills in Psychology”. The APA task force listed 22 potential outcomes that could be assessed (e.g.,
challenge claims that arise from myth, stereotype, or untested assumptions). Some outcomes were more relevant for assessing a
student’s appraisal of someone else’s work while others were more relevant for assessing a student’s own work. We created a 10-
point rubric from the first set of outcomes (evaluating the work of others, see Appendix A). A short excerpt from Newsweek taken
from the book, The Secret, was selected. In the spring of 2010, 14 seniors participated in this qualitative assessment. Their essays
were read by a member of the assessment committee who scored them for the presence of any of the 10 items in the critical
thinking rubric. In addition, we employed our 136 item test for a second time. Seventy senior psychology majors participated in this
quantitative assessment.

Results from spring 2010: (Bolded concepts are new strengths or weakness or were in the opposite category last year, i.e. a strength
this year but a weakness last year, or vice versa)

Quantitative Results from spring 2010:

Statistics and Research Methods
     Strengths: Experimental Design (control groups, random selection, IV, DV), Likert scale
  Weaknesses: Psychometrics (reliability, validity), T scores, correlations, IQ scores, interaction effects




AAASC Doc. 2-17-2010
Brain and Behavior
    Strengths: Neuron structure, action potential, basic neuroanatomy and physiology (except emotion)
 Weaknesses: phrenology, corpus callosum, L-R brain, ANS (branches), emotion, sleep and brain waves

Social Psychology: 9 Questions
     Strengths: Conformity, Attitudes, cognitive dissonance
  Weaknesses: Rosenthal study (self-fulfilling prophecies), TV and aggression, Milgram study

Personality Psychology

   Strengths: Humanistic tradition, Freud (psychosexual stages, dreams)
  Weaknesses: NEO Five-factor, Eysenck, Allport (religion, prejudice), Freud (transference)

Qualitative Results

Most of the students “avoided being swayed by appeals to emotion or authority”, and were able to “evaluate quality of information,
differentiating empirical evidence from speculation” or “challenge claims that arise from myth, stereotype, or untested assumptions.
About a half of the students “deconstructed arguments (e.g., conclusions, premises/assumptions, gaps, counter-arguments” or
showed the ability to “distinguish among assumptions, emotional appeals, speculations, and defensible evidence”. Only two
“weighed support for conclusions to determine how well reasons support conclusions” and “identified weak, contradictory, and
inappropriate assertions”. No one “used scientific principles and evidence to resolve conflicting claims”.

Our assessment may have been limited by the nature of the selected work sample (A “pop” psychology book with little “data” to
evaluate or challenge).

Comparison of spring 2009 versus spring 2010 (quantitative results on 136 item exam)

A comparison across years shows remarkable consistency in strengths and weaknesses. In the area of statistics and research
methods, in both years, our students appear well versed in sampling procedures, identifying independent and dependent variables,



AAASC Doc. 2-17-2010
and understand the concept of a Likert scale. However, they struggle with notions of reliability and validity (test construction), and
are not sure about the mean and standard deviation of a standard IQ Test. In spring 2010, our students showed a little more
difficulty working with two-way Anovas (main effects vs. interactions). In the area of brain and behavior, our students were strong
both years on the topic of neuron structure and function but demonstrated more difficulty with the autonomic nervous system
(stress response) as well as the anatomy and physiology of emotion. In the spring of 2010, another weak area was brain wave
activity associated with sleep. In Social Psychology, conformity and attitudes were strong areas in both years. In 2010, our students
did better with cognitive dissonance, but worse with the effects of TV on aggression and the Milgram experiment. In Personality
theory, the pattern was identical across the two years. The strengths included Maslow and Rogers, and aspects of Freud; the
weaknesses included the NEO five-factor model, Allport’s significance, Eysenck’ research on traits, and the clinical concept of
“transference”.

Recommendations submitted in fall 2010

Faculty will be alerted to student strengths and weaknesses. In the spring of 2011 the reduced (75 item) test will be used instead of
the 136 item version. In future years we may consider a nationally-normed test provided by an outside source. We will continue to
focus on the four areas listed above (Statistics and Research Methods, Brain and Behavior, Social Psychology, and Personality
Theory). A better work sample will be used to assess critical thinking skills. (Ideally, an empirical study or at least a report with some
data will be obtained for this purpose.)

AY 2010 –2011:

In the fall of 2010 and the spring of 2011 we instituted our reduced (75 item) test. In the fall we assessed Freshmen taking their first
course in psychology to establish a baseline. In the spring we assessed 2nd semester juniors. Our rationale was that all of them
should have completed our core courses by this point in their program of studies.

In the spring of 2011 we also incorporated a nationally-normed test provided by an outside source, ACAT. We continued to focus on
the four areas listed above (Statistics and Research Methods, Brain and Behavior, Social Psychology, and Personality Theory). We
also assessed abnormal psychology. Our rationale for adding this fifth content area was that many of our majors take this course
because they are interesting in pursuing careers in clinical or counseling psychology, or they find it relevant for their future work in



AAASC Doc. 2-17-2010
social work, education, or health. We administered our internal measure to one group of juniors while another group received the
national test develop by ACAT. Below we first summarize our results involving freshman vs. juniors, as they performed on our
internal measure. Secondly, we summarize our result involving just the two groups of juniors as they performed on our internal
measure as compared to the national test.

Fall freshmen versus spring 2nd semester juniors (Department Psychology Test)

There was a clear increase in mastery of core concepts. On average we found an increase of approximately 16 percent across the
content areas, ranging from 10 percent for personality to 19 percent for research design and abnormal psychology. Among our
juniors, four of the five group means were 50 percent or higher. The one exception was personality psychology (44%).

                   Research Design    Brain/Behavior      Social Psych.      Personality         Abnormal           Composite
Freshmen           42%                39%                 38%                34%                 31%                37%

Juniors            61%                 56%                54%                44%                 50%                53%


Spring 2nd semester juniors (Department vs. Commercial Test): Note: Bolded areas targeted for AY 2010 – 2011

                Research Design       Brain/Behavior      Social Psych.      Personality         Abnormal           Composite
Department Test 61%                   56%                 54%                44%                 50%                53%

Commercial Test    42%                39%                 42%                48%                 57%                46%


Overall, our students appeared to perform a slightly better on our department psychology test. However, they did perform better on the
commercial test in personality and abnormal psychology. The greatest discrepancies were noted in research design and brain and
behavior. At this point, we are in the pilot testing and refinement phase of our quantitative assessment module for psychology. We




AAASC Doc. 2-17-2010
plan on doing an item analysis of the contents of both tests and review content validity as well as refining the departmental consensus
regarding the contents for each core area. In SP 2012 we will administer both tests to one group of 40 students.




AAASC Doc. 2-17-2010

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:12
posted:10/18/2012
language:English
pages:9