|
The questions below are designed to guide your reading of an
entire research article.
1.
What is the social condition under study? What is the basic research question, or problem? Try to state it in just one sentence. (Chapter 2) top
2.
Is the purpose of the study explanatory, evaluative, exploratory, or descriptive? Did the study have more than one purpose? (Chapter 1) top
3.
How did the author(s) explain the importance of the research question? Is the research question relevant to social work practice and/or social welfare policy? (Chapter 2) top
4.
What prior literature was reviewed? Was it relevant to the research problem? To the theoretical framework? Does the literature review appear to be adequate? Are you aware of (or can you locate) any important omitted studies? Is the literature review up to date? (Chapter 2) top
5.
Was a theoretical framework presented? What was it? Did it seem appropriate for the research question addressed? Can you think of a different theoretical perspective that might have been used? (Chapter 2) top
6.
Were any hypotheses stated? Were these hypotheses justified adequately in terms of the theoretical framework? In terms of prior research? (Chapter 2) top
7.
What were the independent and dependent variables in the hypothesis or hypotheses? Did these variables reflect the theoretical concepts as intended? What direction of association was hypothesized? Were any other variables identified as potentially important? (Chapter 2) top
8.
What were the major concepts in the research? Did the author(s) provide clear and complete nominal definitions for each concept? What are the nominal definitions? Were some concepts treated as unidimensional that you think might best conceptualized as multidimensional? (Chapter 3) top
9.
How are variables operationally defined by the author(s)? Are the operational definitions adequate? Did the instruments used, the measures of the variables, seem valid and reliable? How did the author(s) attempt to establish measurement reliability and measurement validity? Could anything else have been done in the study to establish measurement validity? Have the measures used in the study been evaluated in terms of reliability and validity with populations similar to the study sample? (Chapter 3) top
10.
Was a sample of the entire population of elements used in the study? Was a probability or nonprobability sampling method used? What specific type of sampling method was used? How was the sample recruited and selected? How large is the sample? Did the authors think the sample was generally representative of the population from which it was drawn? Do you think the sample was generally representative of the population from which it was drawn? How could you evaluate the likely generalizability of the findings to other populations? Are women and people of color adequately represented in the sample? (Chapter 4) top
11.
Was the response rate or participation rate reported? Does it appear likely that those who did not respond or participate were markedly different from those who did participate? Why or why not? Did the author(s) adequately discuss this issue? (Chapters 4, 8) top
12.
What were the units of analysis? Were they appropriate for the research question? If some groups were the units of analysis, were any statements made at any point that are open to the ecological fallacy? If individuals were the units of analysis, were any statements made at any point that suggest reductionist reasoning? (Chapter 5) top
13.
Was the study design cross-sectional or longitudinal, or did it use both types of data? If the design was longitudinal, what type of longitudinal design was it? Could the longitudinal design have been improved in any way (for instance, by collecting panel data rather than trend data or by decreasing the dropout rate in a panel design)? If cross-sectional data were used, could the research question have been addressed more effectively with longitudinal data? (Chapter 5) top
14.
Were any causal assertions made or implied in the hypotheses or in the subsequent discussion? What approach was used to demonstrate the existence of causal effects? Were all five issues related to establishing causal relationships addressed? What, if any, variables were controlled in the analysis to reduce the risk of spurious relationships? Should any other variables have been measured and controlled? (Chapter 5) top
15.
Was an experimental, single-subject, survey, participant observation, or some other research design used? How does the author describe the design? How well was this design suited to the research question posed and/or the specific hypotheses tested? Why do you suppose the author(s) chose this particular design? (Chapters 6-9) top
16.
Did the design eliminate potential alternative explanations and how did the design do this? How satisfied are you with the internal validity of the conclusions? Why? (Chapter 6). top
17. What is the setting for the study? Does the setting limit the generalizability of the results to other similar settings or to the broader population? Is reactivity a problem? Are there other threats to external validity? (Chapters 5, 6) top
18.
Was this an evaluation research project? If so, which type of evaluation was it? Which design alternatives did it use? (Chapter 10) top
19.
How were data collected? What were the advantages and disadvantages of the particular data collection method(s) used? (Chapter 8) top
20.
What did the author(s) find? Are the statistical techniques used appropriate for the level of measurement of the variables? How clearly were statistical and/or qualitative data presented and discussed? Were the results substantively important? (Chapters 11, 12) top
21.
Did the author(s) adequately represent the findings in the discussion and/or conclusion sections? Were conclusions well grounded in the findings? Can you think of any other interpretations of the findings? (Chapter 13) top
22.
Compare the study to others addressing the same research question. Did the study yield additional insights? In what ways was the study design more or less adequate than the design of previous research? (Chapter 13) top
23.
What additional research questions and hypotheses are suggested by the study's results? What light did the study shed on the theoretical framework used? On social work practice questions? On social policy questions? (Chapters 2, 13) top
24.
How well did the study live up to the guidelines for science? Do you need any additional information to evaluate the study? To replicate it? (Chapter 2) top
25.
Did the study seem consistent with current ethical standards? Were any trade-offs made between different ethical guidelines? Was an appropriate balance struck between adherence to ethical standards and use of the most rigorous scientific practices? (Chapter 2) top |