|
The questions below are designed to guide your reading of
an entire research article.
1. What is the basic research question, or problem? Try to
state it in just one sentence. (Chapters 1, 2) top
2. Is the purpose of the study explanatory, evaluative, exploratory,
or descriptive? Did the study have more than one purpose?
(Chapter 1) top
3. What prior literature was reviewed? Was it relevant to
the research problem? To the theoretical framework? Does the
literature review appear to be adequate? Are you aware of
(or can you locate) any important studies that have been omitted?
(Chapter 2) top
4. Was a theoretical framework presented? What was it? Did
it seem appropriate for the research question addressed? Can
you think of a different theoretical perspective that might
have been used? (Chapter 2) top
5. How well did the study live up to the guidelines for science?
Do you need additional information in any areas to evaluate
the study? To replicate it? (Chapters 1, 2) top
6. Did the study seem consistent with current ethical standards?
Were any trade-offs made between different ethical guidelines?
Was an appropriate balance struck between adherence to ethical
standards and use of the most rigorous scientific practices?
(Chapter 2 and all Methods Chapters) top
7. Were any hypotheses stated? Were these hypotheses justified
adequately in terms of the theoretical framework? In terms
of prior research? (Chapter 2) top
8. What were the independent and dependent variables in the
hypothesis(es)? Did these variables reflect the theoretical
concepts as intended? What direction of association was hypothesized?
Were any other variables identified as potentially important?
(Chapters 2, 3) top
9. What were the major concepts in the research? How, and
how clearly, were they defined? Were some concepts treated
as unidimensional that you think might best be thought of
as multidimensional? (Chapter 3) top
10. Did the instruments used, the measures of the variables,
seem valid and reliable? How did the authors attempt to establish
this? Could any more have been done in the study to establish
measurement validity? (Chapter 3) top
11. Was a sample or the entire population of elements used
in the study? What type of sample was selected? Was a probability
sampling method used? Did the authors think the sample was
generally representative of the population from which it was
drawn? Do you? How would you evaluate the likely generalizability
of the findings to other populations? (Chapter 4) top
12. Was the response rate or participation rate reported?
Does it appear likely that those who did not respond or participate
were markedly different from those who did participate? Why
or why not? Did the author(s) adequately discuss this issue?
(Chapter 4) top
13. What were the units of analysis? Were they appropriate
for the research question? If some groups were the units of
analysis, were any statements made at any point that are open
to the ecological fallacy? If individuals were the units of
analysis, were any statements made at any point that suggest
reductionist reasoning? (Chapter 4) top
14. Was the study design cross-sectional or longitudinal,
or did it use both types of data? If the design was longitudinal,
what type of longitudinal design was it? Could the longitudinal
design have been improved in any way, as by collecting panel
data rather than trend data, or by decreasing the dropout
rate in a panel design? If cross-sectional data were used,
could the research question have been addressed more effectively
with longitudinal data? (Chapter 5) top
15. Were any causal assertions made or implied in the hypotheses
or in subsequent discussion? What approach was used to demonstrate
the existence of causal effects? Were all five issues in establishing
causal relationships addressed? What, if any, variables were
controlled in the analysis to reduce the risk of spurious
relationships? Should any other variables have been measured
and controlled? How satisfied are you with the internal validity
of the conclusions? (Chapter 5) top
16. Was an experimental, survey, participant observation,
or some other research design used? How well was this design
suited to the research question posed and the specific hypotheses
tested, if any? Why do you suppose the author(s) chose this
particular design? How was the design modified in response
to research constraints? How was it modified in order to take
advantage of research opportunities? (Chapters 6-10). top
17. Was this an evaluation research project? If so, which
type of evaluation was it? Which design alternatives did it
use? (Chapter 11) top
18. Was a historical comparative design used? Which type
was it? Were problems due to using historical and/or cross-national
data addressed? (Chapter 10) top
19. Did the analysis rely on secondary data? Did the author(s)
take into account the limitations due to use of secondary
data? Did they provide an adequate description of the strengths
and weaknesses of the original study? (Chapter 10) top
20. Summarize the findings. How clearly were statistical
and/or qualitative data presented and discussed? Were the
results substantively important? (Chapters 9, 12) top
21. Did the author(s) adequately represent the findings in
the discussion and/or conclusions sections? Were conclusions
well grounded in the findings? Are any other interpretations
possible? (Chapter 13) top
22. Compare the study to others addressing the same research
question. Did the study yield additional insights? In what
ways was the study design more or less adequate than the design
of previous research? (Chapter 13) top
23. What additional research questions and hypotheses are
suggested by the study's results? What light did the study
shed on the theoretical framework used? On social policy questions?
(Chapters 2, 13) top
|
