Allen, James D. and Razvi, Summar. 2006. Student’s perspectives, levels of epistemological
understanding, and critical thinking dispositions related to the use of case studies in an
educational psychology course. Paper presented at the annual meeting of the American
Educational Research Association. Available online:
df. Accessed on May 2, 2007.

         While case studies often have been implemented in different educational courses
(teacher preparation, educational psychology), their effectiveness in improving cognitive
development has not been effectively studied. This study focused on the level of
epistemological understanding of students when case studies are discussed in an educational
psychology course and the relationship between the nature of the students’ critical thinking and
level of understanding. Educational psychology classes are meant to help future teachers
assess, understand, and handle difficult situations that might arise in the classroom, as well as
enhance their understanding of the educational environment and help develop their teaching
strategies. Case studies can expose students to possible scenarios that they may encounter and
develop tactics on how to resolve them and to appreciate the situations from multiple
perspectives. Epistemological levels of understanding in which one can evolve are Realist,
Absolutist, Mutliplist, and Evaluativist, which move from copies to facts to opinions to judgment.
The last category requires good critical thinking skills in order to evaluate and compare.
         For this study, 19 undergraduate pre-service education students in two educational
psychology classes were assessed. All students were White and most were female, thereby
limiting the usefulness of any generalizations derived from this study. Case studies were read
and written analyses were prepared prior to in-class discussion. Data were collected via surveys,
observation, interview, evaluation of video-taped in-class discussions, and the California Critical
Thinking Disposition Inventory assessment tool. The critical thinking assessment measured critical
thinking in seven areas: truth-seeking, open-mindedness, analyticity, systematicity, self-
confidence, inquisitiveness, and maturity.
         Based on analyses of the recorded in-class discussions, there was an increase in level of
understanding from the first case study to the later studies, with the largest gains occurring
between the Multiplist and Evaluativist levels. Over time, students became more comfortable
with presenting assertions than they had earlier in the term. They spoke their opinions and
supported them with logical statements and evidence. The authors did find a slight positive
association between maturity scores on the CCTDI assessment and the Evaluativist level
demonstrated in the recorded discussions. However, there was also a slight negative
association between analytical scores and number of Evaluativist responses, which essentially
negates the gain seen between Evaluativist level and maturity. Also, student performance on
the CCTDI had no real relationship with the level of participation in the discussions. In this study,
there wa no definitive relationship between critical thinking scores and level of epistemological
understanding. However, the authors believed that the limitations of this particular study may
have significantly influenced the findings.
         The authors suggest that in order to get good, productive and deep discussions of case
studies, one needs to urge the group to move beyond Realist discussions of the case study and
begin with Absolutist responses. Instructors need to design questions that will help direct students
to these higher cognitive level thought processes. To have more effective case study
discussions, professors must develop their skill at leading discussions.
Bucy, Mary C. 2006. Encouraging critical thinking through expert panel discussions. College
Teaching 54(2):222-224

         A method for developing critical thinking skills in classroom discussions when students
have similar backgrounds and professional experience is to introduce controversy to stimulate
debate. The author discusses her experience in her emerging information technologies class, a
requirement for education master’s students. She assigned each student a “personality” to
research, an outspoken commentator in the field of emerging technologies. Once familiarizing
themselves with the views and opinions of their assigned personalities, students would
participate in a panel discussion, adopting the opinions of the person whom they researched.
         While the author originally expected that students would only bring in these
“personalities” on the assigned day of panel discussion, she soon realized that students quickly
began discussing their assigned person and began bringing the associated opinions into class
discussion far in advance of the scheduled panel discussions. Students seemed intrigued by the
interesting, controversial views presented by these personalities and quickly began doing
additional research. Students on different panels had elected to learn mannerisms and dressed
in a fashion similar to their personality. They devised different ways of doing panel discussions
and getting their fellow classmates involved. This use of panel discussions, leaving students free
to develop the exact format that they would take, truly seemed to stimulate the students’
engagement in class and with the material and exposed them to different perspectives on
controversial topics that they might not have before considered.
         The author found this form of panel discussion to be an excellent way to introduce
diversity into the classroom and engage the students with the material, a connection that often
lasted beyond the end of the term. Getting students to move outside their comfort zone and
consider different perspectives that challenge the foundation of their own beliefs contributes to
cognitive development.
Burbach, Mark E., Matkin, Gina S. Fritz, and Susan M. 2006. Teaching critical thinking in an
introductory leadership course utilizing active learning strategies: a confirmatory study. College
Student Journal 38(3):482-493

         To determine if active learning in an introductory leadership course increased critical
thinking skills, a pre- and post-test using the Watson-Glaser Critical Thinking Appraisal was
administered. The course was designed to develop interpersonal skills for leadership, requiring
students to use critical thinking in order to evaluate leadership skills, to understand themselves
and others, and to utilize the leadership skills (and people skills) to become an effective leader.
As part of this course, students had to participate in a service learning project, and they were
advised to choose a project that was a new area of experience for them (e.g., new perspective
or cultural experience). In-class scenarios, role playing, case study assessments, reflective journal
writing, and student presentations were also used in this course.
         The sample consisted of 80 students in six sections of an introductory leadership course
taught by three instructors at a Midwestern university. The majority of participants were male (57
to 23) and lower classmen. While most sections had 17-18 students, some has less than 10
students. Results of the Watson-Glaser Critical Thinking Appraisal revealed gains in deduction
and interpretation and overall critical thinking, supporting the hypothesis that active learning in
this leadership course did contribute critical thinking development. However, it is not known
which active learning strategy had the most impact on increasing critical thinking.
Elder, Linda and Paul, Richard. 2001. Critical thinking: Thinking to some purpose. Journal of
Developmental Education 25(1):40-41

       In this reflective essay on critical thinking, the authors list questions that students should
consider as they think, which should help them develop stronger critical thinking skills. These
questions include (Elder and Paul, 2001:41):
       1. Is your purpose clear? Can you distinguish it from related purposes? (Take time to
            state your purpose clearly.)
       2. Is your purpose significant? (Choose significant purposes.)
       3. Is your purpose realistic (achievable)? (Focus on what you are capable of achieving.
            Keep your purpose realistic.)
       4. Is your purpose consistent with your other purposes (goals, ends)? (Periodically review
            your fundamental purposes. Look for inconsistencies.)
       5. Is your purpose just and fair? (Make sure your purpose is fair in context).
       6. Do you stick to your purpose throughout? (Check periodically to be sure you are still
            focused on your purpose and haven’t wandered from your target.)

An instructor can encourage students to think about “purpose” every day in the classroom or in
assignments by having the student write what they think is the purpose of the activity, discussion,
reading, and lecture.
Elder, Linda and Paul, Richard. 2002. Critical thinking: Teaching students how to study and learn
(Part II). Journal of Developmental Education 26(2):34-5

         This is a continuation of a guide to critical thinking, whose first part was presented by Paul
and Elder (2002). In this particular segment, the focus is on determining the importance of ideas
or lines of thinking, how to think within the realm of particular field of study, and how to
effectively learn ideas presented in textbooks. Students must learn to evaluate their own ideas
and challenge themselves to view things from alternative perspectives, assess their validity
without allowing personal experience or feelings to cloud judgment. Trying to think within the
subject (historically, anthropologically, sociologically), students can be more objective in
evaluating personal ideas as well as those encountered in class. Understanding the nuance of
language, and appreciating how these nuances can change depending on who is speaking,
can also affect the way in which students evaluate ideas and their foundations. The authors
stress that students must learn where our ideas originate: personal experience, native language,
socialization, and academic fields of study. If one can assess the use of ideas understand their
origination, a student can become more effective in controlling ideas instead of vice versa.
         The authors advise students to learn how to think within the subject. Students need to
think like a biologist, chemist, historian. Students should use textbooks and encyclopedias to
help them understand the “essence” of the subject. As one reads, writes, listens, and discusses,
the student should unite every new idea with the essential foundation that s/he discovered.
Students should look for ways to connect these ideas together and with this foundation.
Students need to learn the markers that identify the subject and use them to help guide their
way of thinking within that field.
         Textbooks are a tool for students to use, helping them to gain mastery in a course.
Students should determine the system of ideas used in the text, creating a list of key terms that
each student personally finds to be important for the class. Students can test their knowledge
by quizzing themselves on key ideas and topics, as well as defining any ideas or theories that are
contrary to them. Trying to relate and integrate ideas can help students identify deeper
connections and develop a deeper understanding of the materials, instead of leaving each
idea as an isolated entity. Students can also generate lists of key ideas for the subject and each
chapter or article, defining them in their own words and explaining them to others, as well as
interconnecting ideas, with constant revision and assessment of progress.
Garside, Colleen. 1996. Look who’s talking: A comparison of lecture and group discussion
teaching strategies and critical thinking skills. Communication Education 45:212-227

         It has been argued that oral communication activities, such as presentations, group
discussions, and debates, can increase communication competence and enhance learning;
however, this has not been often empirically assessed. In this particular study, the ability of
group discussion to increase critical thinking skills is measured and compared to critical thinking
increases seen in lecture-only settings. Since students need to verbalize their ideas, defend their
positions or beliefs, and evaluate the statements of others, it is believed that group discussion is
an ideal climate for improving critical thinking skills.
         The sample consisted of 118 undergraduate students in six sections of an introductory
interpersonal communication course. Three sections had new information presented in a lecture
format and the remaining three used a group discussion approach. The sections with group
discussion were broken into small groups and students were given a list of questions that
addressed the same topics covered in the lectures. Each group would then collectively write
responses to those questions after discussion. Both lectures and group discussions were
videotaped and assessed for concepts covered and modes of communication used.
         To measure the learning that took place, all sections were given an objective test that
covered the materials that should have been addressed in both lecture and group discussion.
As controls, one lecture and one discussion class each had completed a pre-test. Whether or
not the student had completed the assigned readings prior to class was recognized as a
conditional factor. However, other factors, such as age, sex, major, course load, etc., were not
         Their results did not find any significant differences in the impact on critical thinking skills
for either lecture of group discussion sections, which does not necessarily follow previous
research. One possible explanation is that students were more accustomed to objective tests
based on lecture material and not so much with discussion-based courses. Also, most students
in the course were lower classmen and their variable experience with college lecture or
discussion courses may have been a limitation of this study. Also, the project was administered
in the early portion of the term, when students were still unfamiliar with course material (and
perhaps even the course formats). Also, a loosely structured discussion approach was used in
this study, and it is possible that a more structured format is needed to improve critical thinking
gains from the experience. Given the parameters of the group discussion sections, the
researchers were pleased that the students actually demonstrated similar amounts of learning.
A larger sample, involving multiple courses, more institutions, and more structured discussion
sessions, could improve the results of this study.
Gellin, A. 2004. The effect of undergraduate student involvement on critical thinking: A meta-
analysis of the literature 1991-2000. Journal of College Student Development 44(6):746-762

          Studies on the effects of co-curricular student involvement on critical thinking skills have
had mixed results. To help find some clarification, an analysis of the literature from 1991-2000
was conducted. Articles from this time period looked at participation in athletics,
fraternities/sororities, clubs/organizations, faculty interaction, peer interaction, on-campus living,
and/or employment. Also, these studies have assessed the impact of any of these participations
on critical thinking, using tools such as the Watson-Glaser Critical Thinking Appraisal, Cornell
Critical Thinking Test, California Critical Thinking Skills Test, and the Collegiate Assessment of
Academic Proficiency. Also, sample sizes had to be sufficiently large for statistical analysis of
significance and effect size, which allows for comparison of different studies using different
assessment tools and variables. After a review of journals, dissertations, etc., eighteen studies
met the criteria and were included for the study.
          Overall, involvement in activities outside of class had a .14 effect gain on critical thinking,
likely because these activities included exposure to diversity in ideas, peoples, experiences,
cultures, etc. It is also hypothesized that students may have become aware of the benefits of
co-curricular activities that spearheaded their interest to find more benefits with in-class
participation. Also, involvement outside of class might give students opportunities to apply what
they have learned in class to the real world.
          The authors also looked at the impact of involvement in individual activities to determine
if critical thinking gains were more likely to occur in conjunction with one activity over another.
Participation in a club or organization had an effect score of .11 and may indicate a
commitment to involvement by actively seeking out a group in which one is interested and
develop a sense of belonging. Membership in these groups may also expose students to a more
diversified array of ideas and experiences, helping to develop their critical thinking.
          Belonging to a club or organization had the lowest effect score, though the authors think
that this lowered score may be due to the varied nature of clubs/organizations. Peer interaction
had a positive effect, which would have contributed to the positive impact of participation in a
club or organization. Living on campus had a positive effect gain, which likely correlated with
participation on clubs/organizations and peer interaction, since students who lived on campus
were more likely to have both experiences. Among students who were employed while taking
classes, there was a positive effect gain. However, one must consider where the job was
located, the nature of the job and how many hours the student spent working. But only one
previous study had differentiated between students who worked on or off campus and none
noted students who were employed in jobs within their intended field and the impact on their
cognitive growth. Studies are inconsistent regarding the number of hours working for pay that
serve as a benefit to critical thinking skills or as a detriment. It is possible that peer interaction on
the job may contribute to cognitive growth, especially for on-campus employment. No specific
effect of fraternity/sorority involvement could be calculated based on the literature review.
Also, no specific effect for interaction with faculty was identified, which surprised the authors. In
the literature, the reported effect sizes from three studies ranged from highly negative to positive.
This may indicate different skills of professors, with some having better interpersonal
communication skills than others.
          A limitation of the literature review is that there were few studies that studied all of the
same factors. So while 18 studies had been identified for inclusion in this review, some only
addressed one or two of the co-curricular activities. Also, the relationship between independent
and dependent variables could not be assessed. Student and institutional characteristics could
have had a major influence on the results.
          Despite these limitations, administration and faculty may find some important things to
consider from this study. Since there was a critical thinking gain due to involvement in co-
curricular activities overall, schools should continue to fund these activities and perhaps develop
more from which students can choose. Institutions may want to consider encouraging students
to become involved in more than one activity. Living on campus, not truly an option for most
community colleges, had the largest impact on critical thinking grains. However, if schools have
the money, they may want to invest in creating more on-campus jobs for students. Studies have
shown that working while in college can help improve critical thinking skills, though how many
hours should be worked is debated. Having more on-campus jobs will give students the
opportunity for more peer interaction, involvement on campus, and earning ability. The authors
also recommend that academic and student affairs can work more closely on this common
goal of student success by working together to create more opportunities for critical thinking
development on campus, recognizing that activities outside of the classroom can be as
important as those within the classroom.
Lauer, Thomas. 2005. Teaching critical-thinking skills using course content material: A reversal
of roles. Journal of College Science Teaching 34(6): 34-37

         In this article, the author implements the theory that using laboratory sessions, active-
learning approaches, and inquiry-based learning is more conducive to teaching students how
to improve their critical thinking skills, rather than purely lecturing to them on the subject. The
author tests the ability to use course content material to teach higher-order critical thinking skills
using critical thinking terminology that was not expressly designated as such in the class. He
tested this approach on 94 freshman biology majors in an introductory biology class with a
laboratory component. The author integrated critical thinking in three phases: introduction,
mastery and evaluation. Using questions of increasing critical complexity, the author escorted
students through the critical thinking process. Simply, he began the first day of class by asking
them to identify the color of a notecard and discussed how this was simple factual recall. He
followed up by asking why the card was white, which demonstrated comprehension and ability
to apply ideas learned elsewhere (e.g., light reflection and wave lengths learned in a high
school science course). He then asked students how he would change the color to blue,
allowing students to formulate ideas. The author then instructed his students that he would be
asking his students to think at the level of the second and third questions throughout the term
(though he never expressly stated that these were part of a higher critical thinking thought
         To develop mastery, he would begin class with a “question of the day,” which involved
analysis, synthesis or evaluation, followed by presentation of course material and class
discussion. Typically, the answer required information that was not presented in class or in the
textbook but called upon the student to postulate a response based on other information that
s/he knew. Multiple choice questions that were categorized as knowledge-based, application
of analysis, and synthesis/evaluation were administered and allowed the author to determine
how well in each section students were performing. Personal feedback with students was used
to help them address questions they had about their exams. Students who missed many
knowledge-based questions needed to study more. Those who missed questions in the
application section needed to develop their skills at associating facts. Students who missed
synthesis/evaluation questions had difficulty doing just that—synthesizing and evaluating. The
meetings with the students allowed them to see the areas where they needed to improve and
to discuss strategies to do so. The author concluded class by asking students what was the most
important thing that they had learned in class that term. In general, learning how to improve
their thinking skills was as important as the course material.
Mahaffy, Mardi. 2006. Encouraging critical thinking in student library research: An application of
national standards. College Teaching 54(4):324-327

          With the rise in reliance of students on internet-based resources, instructors are often
disappointed that students are not using library reference materials appropriately for research
projects, and that this dependence on the internet is causing a decrease in the ability of
students’ research and critical thinking skills. Some feel that students are essentially doing a
cursory review of the material and not developing any deeper understanding of the ideas
presented. The Association of College and Research libraries have published a report that lists
the core competencies that the group feels that students should know to be truly literate.
          The standards presented in this report include that the student be able to judge what
type and how much information is needed to complete the project. This is essentially the
formation of the research question and thesis statement. Instructors can help students by
presenting them with clearer instructions for the research subject, helping them to narrow their
focus and concentrate their efforts. This direction can be given by telling what reference source
to read, which may help them develop their thesis statement. Also, requiring that students
submit their thesis statements in advance will allow the instructor to approve or help students
revise their statements early in the process.
          The second standard involves the development of an effective research strategy and
skills to utilize library resources efficiently. Instructors should not assume that their students have
already mastered the skills needed to do this. Assignments should be designed to include
pointer on how to do an effective search, such as using references cited in one resource as a
launching pad for finding additional resources.
          The third standard entails evaluation of the research material to determine which is most
applicable and what aspects of that material warrant inclusion in the final paper. To help
students in this area, instructors may limit what resources students may use (e.g., no internet
resources, only peer-reviewed journals). Such injunctions may be necessary to ensure that
students do not develop an over-reliance on web-based sources. However, it is more beneficial
to the student to help them learn how to critically evaluate resources and allow them to explore
the different merits of the material that are available. One way to help students do this is by
design a project that requires them to use a popular magazine, newspaper, scholarly journal
and internet site and have the students compare and contrast.
          The fourth standard centers on using the researched materials to support the topic or
purpose of the assignment. This can range from well-written research paper, skit, outline, class
presentation, debate or annotated bibliography accompanied with a paragraph on why the
resource was or was not helpful and which was most beneficial to the student in understanding
course materials.
          The fifth standard entails raising student awareness regarding plagiarism, copyright
infringement and ethics in research. The internet has made it extremely easy for students to
borrow the work of others and claim it as their own, either intentionally or unintentionally. To help
raise awareness in these areas, students may be required to research piracy, censorship, and
other such ethical issues as part of their project.
          While designing assignments with these standards in mind can help students to become
better critical thinkers, researchers, and writers, these projects also give the professors an
opportunity to work more with their librarians, appreciating the role they play in the
development of students’ cognitive skills.
Pascarella, Ernest T., Bohr, Louise, Nora, Amaury, and Terenzini, Patrick T. 1996. Is differential
exposure to college linked to the development of critical thinking? Research in Higher education

          The authors utilized a longitudinal, multi-institutional dataset to address two questions: 1.
to what extent do the number of credit hours taken during the first academic year relate to the
first-year critical thinking skills of students in two-year and four-year institutions?; and 2. how do
the first-year effects on critical thinking differ between groups of students based on the number
of credit hours taken? The sample consisted of students from 18 four-year and five two-year
institutions, who collectively were representative of the general undergraduate population
nationwide. The first-year student sample essentially was drawn from the National Study of
Student Learning project in the early 1990s limited to those who were enrolled at institutions using
the semester system (rather than trimesters), which resulted in a final sample of 1,860 students.
          The researchers controlled for individual pre-college critical thinking scores, sex, race,
pre-college academic motivation, age, work responsibilities, number of classes taken during first
year of college in five different areas, and the average level of critical thinking in the first-year
class at each institution. Controlling for all of these factors should allow for a more accurate
estimate of the net impact of college on first-year critical thinking. The authors did find a
significant positive relationship between number of credit hours taken and first-year critical
thinking gains, and this result was more prominent among students at four-year institutions than
two-year colleges (roughly 1.56 times more). Among students at four-year schools, with each
increase in credit hour taken, there was a 0.458 of a point increase in critical thinking by the end
of the first year.
          It was concluded that students taking 24 credit hours or more would have a 2.29 point
net advantage in critical thinking compared to students taking six credit hours or less. Among
students at two-year colleges, those who were full-time had a 1.47 point advantage in critical
thinking over students taking six credit hours or fewer. The effects on critical thinking based on
credit hours taken were of similar magnitude when compared against the controlled factors
such as age, sex, and race. Another interesting aspect of the research also indicated that the
lowest levels of critical thinking were found among students at two-year colleges who took
between 12 and 15 credit hours during their first year.
          In general, first-year students taking more credit hours have higher critical thinking gains,
independent of conditional factors such as age, sex, and race, but this positive increase is
modest. Students at four-year institutions demonstrated this more clearly and more strongly than
students at two-year institutions. Surprisingly, students at two-year colleges who took between
12-15 credit hours had the lowest critical gains, even lower than students who only took six
Paul, Richard and Elder, Linda. 2002. Critical thinking: Teaching students how to study and learn
(part 3). Journal of Developmental Education 26(1):36-7

        The authors write that in order for students to study well and truly learn any subject, they
must learn how to be disciplined thinkers and immerse themselves within that subject and
become critical thinkers. Students must evaluate and communicate ideas within that discipline,
using the mindset, language and logic of that discipline. The authors have prepared a guide on
how students can become better thinkers, and they present the first 18 ideas in this article.
Paraphrased, these ideas are:
    1. Know the expectations of the class. If they are not clear, ask. This also pertains to
        grading policy and class preparation.
    2. Actively listen, write, read, and discuss. Participate in your own education.
    3. Consider the subject of the course as a way of thinking (e.g., chemistry=chemically
        thinking; history= historically thinking; anthropology=anthropologically thinking)
    4. Ask questions in lecture, discussion, or anytime.
    5. Always look for ways to connect what you learn in one class with another! Relate new
        things to old things. Makes deeper connections for learning than simply memorizing.
    6. Do not consider the professor the enemy. S/he is a coach, helping you to prepare for
        the “big game.”
    7. Reading a textbook is like roleplaying. You take on the thinking of the author. Presenting
        ideas from the book to a classmate using the thinking of the author can help you
        internalize them.
    8. Never be passive! Class time is time to practice thinking within the subject. Think about
        the concepts, principles and logic of the subject as materials are presented in class.
    9. Look for ways to relate course materials to your life. Make the material personal. Find
        the relevance!
    10. Experiment with study skills, identify which methods are weak, and improve upon them
        with practice.
    11. Always ask yourself if you know that material well enough that you would be able to
        explain it to someone else. If not, you need to learn it better!
    12. Develop your own definitions of the fundamental ideas of the class. Pick a key concept
        early on, define it, and then relate it to every lecture, assignment, and reading
        throughout the term.
    13. Always ask yourself if you can elaborate ideas or give examples. If you cannot, you
        need to learn the subject better.
    14. Do not wait until the class meets to test yourself. Constantly ask yourself to summarize the
        main points of the previous lecture or section of material.
    15. Always ask yourself if your thinking is clear, logical, relevant? Are you able to identify the
        significance of things? Hold yourself to a higher intellectual standard.
    16. Use writing to help you learn the material. Write summaries in your own words. Define
        key terms in your own words. Write practice tests and take them!
    17. Assess yourself to determine if you are really listening in class. Can you summarize the
        professor’s main points in your own words? Can you expand upon definition and
        discussion of key terms?
    18. Assess your reading. Are you asking questions while you are reading and can you tell
        when you do not really understand the material?
Paul, Richard and Elder, Linda. 2003. Critical thinking: Teaching students how to study and learn
(Part III). Journal of Developmental Education 26(3):36-7

        In this edition of the critical thinking series by Paul and Elder, the authors provide useful
tips on how students can become effective, critical readers, deriving essential information from
written material. To determine what an author’s reasoning is, students should ask themselves to
identify the following:
     1. main purpose of the article, book, etc.
     2. key question addressed
     3. most important information used to support the author’s arguments
     4. key conclusions or inferences made by the author
     5. key ideas needed to understand the article and what do they mean (to the author)
     6. main assumptions underlying the author’s thinking
     7. if the author’s reasoning is taken seriously, what are the implications
     8. if the author’s reasoning is not taken seriously, what are the implications
     9. what is the main point of view used by the author
Being able to answer these questions will help the student to think like the author, which can be
an effective tool for internalizing the information. This approach is useful for books, articles,
textbooks and the like.
        After determining the essential components of the author’s reasoning, the students can
evaluate them to judge the quality of the reasoning.
     1. Is the purpose clear and justifiable?
     2. Is the key question unbiased, clearly stated or implied, reflective of the issue’s complexity,
        relate to the purpose?
     3. Is relevant evidence and information used? Is it accurate and relevant to the question?
        Are the complexities of the issue discussed?
     4. Are key ideas defined when necessary and used in a valid way?
     5. Is the author aware of the possible issues with the any assumptions used in the argument?
        Are these possible issues addressed?
     6. Do the inferences/conclusions made by the author logically follow the information
        presented? Are the conclusions unfounded? Are alternative conclusions presented by
        the author?
     7. Does the author give due consideration to alternative viewpoints on the subject?
     8. Does the author address any consequences of the position taken in the textbook? Does
        the author appear open to consideration of these implications?

{These suggested guidelines will not only make students more effective readers. If they make
sure that their own writings stand up to this test, they will become better critical writers as well.}
Renaud, Robert D. and Murray, Harry G. 2007. The validity of higher-order questions as a
process indicator of educational quality. Research in Higher Education 48(3):319-351

         As defined by the authors (Renaud and Murray, 2007: 320), process indicators, also
referred to as the college environment, are “those aspects of the institution capable of affecting
the student (e.g., teaching quality, physical characteristics).” One method of measuring the
validity of a process indicator is by comparing it with an outcome variable to determine if the
indicator truly does influence the outcome. Such assessments are highly important, especially
considering the import that many publications place on ranking postsecondary institutions
based on process indicators without doing any evaluation of the relationship with actual
         Higher-order questions will reflect synthesis and evaluation as described by Bloom’s
taxonomy, which entail the creation of new answers generated by the student and not just
repetition of material stated in lecture or text. Higher-order processes indicators are something
that can be measured easily and be included into the design of course materials and exams.
          In this study, the authors wanted to test the hypothesis that the frequency of higher-
order questions was a valid indicator of educational quality and was correlated to gains in
critical thinking skills. Experiments included giving students lower order questions for one chapter
and higher order questions for another and later testing their comprehension of the material,
with the hypothesis being that the students would perform better on questions pertaining to the
higher order chapters. Another experiment included administering a pre-test and a post-test to
measure the change in critical thinking over the term. The final experiment included a
comparison of mean course critical thinking gain with the proportion of higher-order questions
used on assignments and exams over several courses, disciplines, and years. Pre-tests and post-
tests were administered, and it was expected that greater frequency of higher order questions
would correlate with greater critical thinking gains. Students were from a large university in
central Canada and sample sizes ranged from 131 for the first experiment, 190 in the second,
and 781 in the final experiment.
         The hypothesis for the first experiment, that students would perform better on parts of the
exam relating to chapters for which higher-order questions had been prepared, was slightly
supported. For the second experiment using the general test for critical thinking, there was no
significant different in critical thinking gain between pre- and post-test. But the course-specific
test on critical thinking did show a larger and significant gain among students who answered
higher-order questions on the pre- and post-test than for students who answered lower-order
questions. For the third experiment, a significant, positive correlation was found between the use
of higher-order questions and gain in critical thinking skills across courses, disciplines and time.
However, this study was not able to control for any confounding factors such as other courses
and the effect they may had on the results. So, it is difficult to determine if the gains seen were
due to that course alone or a combination of other courses. While each of these experiments
have their limitations, in general, they share similar conclusions that the use of higher-order
questions does appear to have a positive impact on gains in critical thinking skills.
Sedlak, Carol A., Doheny, Margaret O., Panthofer, Nancy, and Anaya, Ella. 2004. Critical
thinking in students’ service-learning experiences. College Teaching 51(3):99-103

         This study assesses the impact of service-learning experiences on critical thinking skills in
nursing students. Ninety-four sophomore nursing students in their first clinical class volunteered at
various community agencies, such as day cares, nursing homes, Meals on Wheels and the
American Red Cross. Students were asked to conduct research on service learning and clinical
agencies. The students were advised to select a community agency that suited their learning
needs and they had to design a written contract with the agency for 14 hours of service, duties
to be performed, the agencies own objectives and the students’ learning objectives. Students
had to keep a reflective journal of their service learning experience, focusing on decision-
making processes, consideration of the associated emotions, and evaluation of outcomes and
possible alternatives. They also had to do a poster presentation that focused on the manner in
which the services they performed matched their learning goals, could impact nursing
practices, and considered the experience within the context of the general academic
community. Evaluation of the student materials and experiences revealed the development of
“professional self-perspective” that entailed being a care-provider and a good communicator,
as well as formation of a community perspective of health issues and diversity.
         For service learning to be truly beneficial to a course and assist in the development of
critical thinking skills, the service-learning project needs to be related to the course objectives
and is it a good fit for the course, as well as the needs of the agencies involved.

To top