THE RESEARCH CRITIQUE

Document Sample
THE RESEARCH CRITIQUE Powered By Docstoc
					                                          THE RESEARCH CRITIQUE

What is a critique? It is a professional analysis of the weaknesses and strengths of a particular piece of research.
A critique may be done for a variety of purposes: acting as an expert reviewer to assess whether this research
paper should be published; providing helpful comments on a work before it is submitted for publication (e.g.,
panel discussant or reading a friend’s paper); or, as in the case here, as a learning experience for emerging
scholars to practice their developing research skills.

A critique emphasizes the same elements as a research design (and then some) and evaluates how well the author
has carried out these elements. The “and then some” part is that, for completed research, a critique also covers
the data analysis and whether that has been properly carried out and interpreted. A critique is not necessarily 100
percent, or even any percent, critical. It may be predominantly positive. Below are some items to consider in
preparing a critique. Not all will be applicable for every research paper nor are you required to comment on all
of them for each paper. Instead focus on items you identified as the most serious potential problems or most
praiseworthy. Each critique should be about 2-3 pages single spaced or 4-5 pages double spaced in length.

The Research Question.
         a). What is the research question? Does the author actually have one? Is there only one? If more than
one, is it clear that this particular research needs multiple questions (e.g., we must first answer X before we can
answer Y) or do the multiple questions reflect an inability of the author to clearly identify his own purpose? If,
instead of a question, the author states a research purpose or task, identify this purpose but also attempt to restate
it as a question.
         b) How does the author justify the importance of this question to the scholarly community? If important,
how important? Is it something likely to be publishable in a major journal, a minor one, or only as a conference
paper?
         c) How does the author demonstrate that the question remains unanswered? What prior research on this
question is mentioned and what is the author’s explanation of why answers provided by the prior research are
inadequate?

Literature Review
Has the author examined the relevant literature on the topic and persuasively explained why the research question
posed has not yet been satisfactorily answered? An author does not need to include all literature, no matter how
marginally relevant, but should have discussed what the field holds to be the latest and best investigations of the
topic. (NOTE: Journal reviewers and conference discussants are usually (not always successfully, but usually)
selected because they are experts in a field.) Graduate students reading the paper drafts of colleagues may not be
in that position. If so, focus on other elements of the critique.

Research Approach
How does the author explain and justify the contribution s/he will make with the research? What research
approach / data generation method will be used -- cross-sectional survey, time series, participant observation,
etc.? Are any sampling issues involved / discussed? What is the population about which generalizations are
drawn and do the cases examined justify such a generalization?) If an experimental method is used, what is the
generalizability / external validity of the results? Is the unit of analysis clear?

Dependent Variable(s), Independent Variable(s), and Hypotheses.
Treat each variable separately, do not simply make general comments about all of them. Begin with the
dependent variable, then consider each independent variable in turn. Consider does not necessarily mean
comment upon, but rather has the author produced a novel conceptualization or operationalization that deserves
applause, used some measures you think are doubtful, or just presented what they are going to do without any
logical explanation of why?

How is each variable conceptualized and operationalized? Did the author develop the conceptual / operational
definition herself, borrow it from a specific piece of prior research, or simply refer to it as a variable commonly
used in the discipline (e.g., party identification)? Is the operational definition a valid one -- isomorphic with the
conceptual definition? You do not need to comment on every variable, but should comment on those that are
new or for which the author uses a different operationalization than is common with past literature.

Are the hypothesis to be tested clear? Is it global (X is related to Y) or conditional (under condition C, X is
related to Y)? What is the author’s rationale for expecting the independent variables to be related to the
dependent variable? Is this logic persuasive to you? Is it well thought-out and fully developed or overly
simplistic? Are the operationalizations of the variables consistent with this logic? For example, if the author
talks about a college / non-college distinction as being related to political participation, is the variable
operationalized as this dichotomy or as a continuous variable reflecting years of education?

Is the direction of causality discussed and / or clear? Is it likely that causality might flow in ways not discussed
by the author, from the dependent variable to the independent variable or between independent variables?

Confounding / Control Variables.
Are any variables introduced primarily to control for their confounding effects? Which ones? What is the
explanation given for why they might have a confounding effect? Are there other confounding variables you can
think of that the author ignores? Alternatively, if the authors contend no controls are necessary is their reasoning
persuasive to you?

Analytic Approach
What statistical technique does the author use? Based on the questions being asked and the data available, is this
an appropriate technique? What does the author have to say about meeting the assumptions of the technique?

Expectations
Is there a clear statement of expectations, either a separate statement or embedded in the hypotheses?

Findings.
What did the author find as the result of her research? Which expectations were borne out and which ones not?
Are you persuaded that the inferences drawn from the research about the hypotheses are valid? Are any post hoc
explanations given for some of the patterns discovered in the data?

Time and Space
What temporal (e.g., 1996 election) and space (e.g., United States), limitations are there to the data used in this
study? Is the study generalizable beyond these data limits, or are there reasons to believe that different
conditions across time and space may limit the findings? If so, what do you believe those conditions to be?

Conclusions and limitations
What does the author suggest, if anything, as possibilities for future research? If the analysis is clearly limited in
some way – e.g., only crude data or operationalizations are currently available – is the author honest about those
limitations? How severe do you think they might be?

Writing Style
Is the author’s argument understandable? Are the arguments presented logically and coherently? Is the paper
well organized so you can easily follow what the author is doing and why? Is the narrative wordy and redundant?
Are there irrelevant sections that can be deleted? Are there errors in spelling or grammar? If so identify them
(e.g. p. 12 george washington should be capitalized).

WHAT NOT TO INCLUDE IN A CRITIQUE

a) It is not legitimate to criticize the research paper for something beyond its purpose. Don’t complain that an
author did not include something unless that something is a necessary part of a research article. For example, if a
study examines presidential use of news conferences from Kennedy to Reagan, don’t complain that it did not
include later presidents unless there is a clear reason why doing so undermines the research. Focus on what the
author sought to accomplish, not what you wished they had done.
b) Do not require the unobtainable. We all would like perfect data and ideal measures for variables, but neither
usually exists in reality. It is legitimate to criticize data or measures if better ones are readily available.
Otherwise don’t complain if the author has done the best she can with imperfect data.

c) Do not make an abundance of broad and general statements of the type: “This research was well done with
an interesting question and good data.” This means nothing. You must be concrete, describing specific strengths
and weaknesses. Clearly state your reasons for concluding that the author has either done a good or less-than-
good job on one or more parts of the elements of research.

d) Friends don’t let friends turn in sloppy research. Making nothing but positive comments on a fellow
student’s paper is of no value whatsoever. We are here to learn, both as a graduate student and throughout your
scholarly life. Definitely praise items where the author was particularly creative, industrious, or ingenious. But
also comment (politely, but firmly) on weaknesses so the author can address them before turning a paper in for a
grade or as a journal submission.

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:2/8/2013
language:English
pages:3