You are here

Evaluation Methodology Basics
Share
Share

Click 'Additional Materials' for downloadable samples

"I welcome this student-friendly text to complement my favorite checklists. This text develops the core checklist framework to provide evaluation methodology basics for, presumably mainly, introductory courses on program evaluation."
Colin Sharp, Evaluation Journal of Australasia

"What is evaluation-specific logic and methodology? This book answers that question in a way that is persuasive, accessible, and understandable. It presents a set of principles and procedures to guide the task of blending descriptive data with relevant values to draw explicitly evaluative conclusions. The book makes a significant contribution to positioning evaluation as a unique and special field of inquiry and judgment."

—Michael Quinn Patton, Union Institute and University

"Amidst the wash of methods books available to evaluators, Davidson's book provides powerful techniques for asking and answering the important foundational questions in any program evaluation."
—Doug Leigh, Pepperdine University

"This is a very well written book that offers a unique perspective on long-practiced evaluation techniques and presents several new, potentially very useful, techniques that return "valuation" to the evaluation process."
—Greg Roberts, University of Texas, Austin

Evaluation theorists for years have advised evaluators to "take into account" all relevant values as part of an evaluation. But especially for the relatively new evaluator (even one who is knowledgeable and experienced in research methodology), there is not a lot of guidance about how this is done. Evaluation Methodology Basics: The Nuts and Bolts of Sound Evaluation provides a step-by-step guide for doing a real evaluation. It focuses on the main kinds of "big picture" questions that evaluators usually need to answer, and how the nature of such questions is linked to evaluation methodology choices. Jane Davidson explains how to combine a mix of qualitative and quantitative data with "relevant values" (such as needs) to draw explicitly evaluative conclusions.

Many students and evaluators find it difficult to visualize what evaluation logic and methodology "look like" in practice. To address this, Davidson presents readers with useful rubrics and flowcharts that may be used during each stage of the evaluation. Many of the concepts presented in the chapters are illustrated with specific examples from a range of disciplines. Exercises and "pop quiz" questions help reinforce the key points covered in each chapter, provide homework assignments for those teaching an evaluation course, and allow learners to develop slices of an evaluation plan as they work their way through the text.

Evaluation Methodology Basics is an ideal text for students of evaluation and students in programs that have evaluation course requirements, such as education, health, sociology, psychology, and many others throughout the social sciences. It will also be essential reading for practitioners who find themselves thrown into evaluation roles without the benefit of specialized evaluation training.


 
What is Evaluation?
Basic Definitions

 
A Note About Fitting Evaluation Approach to Purpose

 
The Steps Involved

 
The Ingredients of a Good Evaluation

 
Identifying the Evaluand

 
Additional Readings

 
Exercises

 
 
Defining the Purpose of the Evaluation
Evaluations for Determining Overall Quality or Value

 
Evaluations for Finding Areas for Improvement

 
Questions About Absolute vs. Relative Quality/Value

 
Summary and Final Comments

 
Additional Readings

 
Exercises

 
 
Identifying Evaluative Criteria
Why Not Just Use Goals?

 
Identifying Criteria: Basic Concepts and Tools

 
Conducting a Needs Assessment

 
Identifying Other Relevant Criteria

 
Additional Readings

 
Exercises

 
 
Organizing the Criteria & Identifying Potential Sources of Evidence
The Importance of Choosing Multiple Sources of Quality Evidence

 
The Process Evaluation Checkpoint

 
The Outcome Evaluation Checkpoint

 
The Comparative Cost-Effectiveness Checkpoint

 
The Exportability Checkpoint

 
Additional Readings

 
Exercises

 
 
Dealing With the Causation Issue
Certainty about causation

 
Inferring causation: Basic principles

 
Inferring causation: Seven strategies

 
Choosing a blend of methods to address the causation issue

 
Additional Readings

 
Exercises

 
 
"Values" in Evaluation
The Controversy

 
The Three Types of Subjectivity

 
The Tensions Between Subjectivism and Common Sense

 
Where do the "Values" in an Evaluation Come From?

 
Additional Readings

 
Exercises

 
 
Determining Importance
Determining Importance: What and Why

 
Determining Importance: Six Strategies

 
Additional Readings

 
Exercises

 
 
The Merit Determination Step
Determining Merit: What and Why

 
Using Rubrics for Determining "Absolute" Merit

 
Using Rubrics for Determining "Relative" Merit

 
Making practical use of merit determination rubrics

 
Additional Readings

 
Exercises

 
 
Synthesis Methodology
Synthesis: What and Why

 
The Role of Costs and Comparisons in Synthesis

 
Synthesizing for "Grading"

 
Synthesizing for "Ranking"

 
Additional Readings

 
Exercises

 
 
Putting it All Together
The KEC Preliminary Checkpoints

 
The KEC Foundations Checkpoints

 
The KEC Subevaluations Checkpoints

 
The KEC Conclusions Checkpoints

 
Additional Readings

 
 
Meta-Evaluation
The five criteria for evaluating evaluations

 
Key points to remember in meta-evaluation

 
Other options for conducting a meta-evaluation

 
Formative meta-evaluation

 
Should I use meta-evaluation myself?

 
Additional Readings

 
Exercises

 
 
Answers to Selected Exercises
 
Sample Exam Questions
Tips for students answering these exam questions

 
The questions

 
A rubric for grading assignment and exam answers

 
Suggested answers to practice exam questions

 
 
Glossary
 
References

“Amidst the wash of methods books available to evaluators, Davidson’s book provides powerful techniques for asking and answering the important foundational questions in any program evaluation.”

Doug Leigh
Pepperdine University

“This is a very well written book that offers a unique perspective on long-practiced evaluation techniques and presents several new, potentially very useful, techniques that return “valuation” to the evaluation process.”

Greg Roberts
University of Texas, Austin

"What is evaluation-specific logic and methodology?  This book answers that question in a way that is persuasive, accessible, and understandable.  It presents a set of principles and procedures to guide the task of blending descriptive data with relevant values to draw explicitly evaluative conclusions. The book makes a significant contribution to positioning evaluation as a unique and special field of inquiry and judgment."

Michael Quinn Patton
Union Institute and University

"Evaluation Methodology Basics is yet another excellent book from Sage. This would be a good book to use for a graduate-level course in evaluation techniques or as a template for anyone who is new to the techniques of program evaluation."

Annette M. Matthews
Psychiatric Services, Volume 57, Issues 10

"The text is not only a great introduction for novices, but a toolbox with new perspectives for conducting sound evaluation that integrates values with facts into evaluative conclusions. As such Evaluation Methodology Basics should be on every evaluator's shelf."

Daniela Schroeter
Canadian Journal of Evaluation

"I welcome this student-friendly text to complement my favorite checklists.  This text develops the core checklist framework to provide evaluation methodology basics for, presumably mainly, introductory courses on program evaluation."

Colin Sharp
Evaluation Journal of Australasia

Well written. Easy to follow and understand. Very informative.

Dr Kimberly Harris
Human Development and Services, North Carolina Agricultural and Technical State University
March 9, 2014

The text provides practical steps for applying the theory of performance and program evalution.

Mr Cordell Riley
Van Loan School of Graduate & Professional Studies , Endicott College (Bermuda Program)
October 9, 2011
Key features
  • This book gives a very brief introduction to evaluation, focusing on the main kinds of "big picture" questions that evaluations usually need to answer, and how the nature of such questions are linked to evaluation methodology choices.
  • The text covers how to identify the right criteria for your evaluation (i.e.: the aspects that distinguish a good, valuable, effective program from a weak or poor one).  Evaluation-specific methods covered include multi-level needs assessment and ethical analysis.
  • The work discusses how to objectively figure out which criteria are more important than the others (in a way that goes beyond simply asking other people what is important).
  • The book delves into how to combine a mix of qualitative and quantitative data with "relevant values" (such as needs) to draw explicitly evaluative conclusions.
  • Davidson discusses how to systematically and transparently pull together all of the strengths and weaknesses that one uncovers to answer the fundamental evaluation question(s) for the client and other right-to-know audiences.
  • The book includes tips for packaging and presenting evaluative information for different audiences.

 

 

 

Sage College Publishing

You can purchase or sample this product on our Sage College Publishing site:

Go To College Site

This title is also available on SAGE Research Methods, the ultimate digital methods library. If your library doesn’t have access, ask your librarian to start a trial.