Standard Setting

A Guide to Establishing and Evaluating Performance Standards on Tests
First Edition
Gregory J. Cizek
Michael B. Bunch
Standard Setting
November 2006 | 368 pages | Sage US
Create Flyer

If you’re in North America, please visit our Sage College Publishing website to purchase or sample this book:

Go to College Publishing Website

Description

"Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests is a well-timed text, given the legislated requirement of performance standards in our K-12 educational system, as well as the continued need in the process of credentialing professionals. This book would be useful to individuals retained to produce or coordinate standard-setting activites." —PSYCCRITIQUES


In Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests, authors Gregory J. Cizek and Michael B. Bunch provide the only "go-to" reference on how to set standards on tests in education, licensure, and certification. This book is comprehensive in scope, practical in nature, and definitive in terms of cataloguing the essential conceptual and procedural fundamentals of setting performance standards.

Key Features:  
  • Offers a detailed description of procedures: In a practical, straightforward, and virtually jargon-free style, this book begins with important foundational and conceptual information about standard setting followed by step-by-step procedures for all currently used methods, including Angoff, Item Mapping, Analystic Judgment, etc. In addition, a full chapter is devoted to future issues and research in standard setting.
  • Enables readers to design, implement, and evaluate a standard setting study: Using this book, readers will be able to: select an appropriate standard setting method; schedule and plan implementation; identify and train qualified standard-setting participants; conduct the standard setting meeting; calculate one or more cut scores; and evaluate the success of the procedure they have implemented.
  • Allows readers to adapt written materials to their situation: Examples are provided in widely available formats such as Excel, SPSS, .pdf and Word files so readers do not have to purchase specialized software or learn to use unfamiliar packages. The book includes downloadable forms, sample correspondence, checklists, data sets, and analytical software for performing calculations required for various standard setting procedures.

Intended Audience:  
This is an excellent supplementary textbook for graduate courses such as Applied Measurement in Education; Psychometric Theory; Advanced Educational Measurement; Special Topics in Measurement and Evaluation; Seminar in Educational and Psychological Measurement in the departments of Educational Psychology, Measurement and Evaluation, Psychology, and others.  

Contents

SECTION I: FUNDAMENTALS OF STANDARD SETTING

  • 1. Contemporary Standard Setting: An Enduring Need
  • The Need to Make Decisions
  • The Benefits of Standard Setting
  • General Approaches to Standard Setting
  • Current Contexts for Standard Setting
  • 2. What is Standard Setting?
  • Kinds of Standards
  • Definitions of Standard Setting
  • Policy Issues and Standard Setting
  • Item Scoring Criteria and Total-Test Performance Standards
  • Conclusions
  • 3. Common Elements in Setting Preformance Standards
  • Purpose
  • Choosing a Standard Setting Method
  • Performance Level Labels
  • Performance Level Descriptions
  • Key Conceptualizations
  • Selecting and Training Participants
  • Evaluating Standard Setting
  • Providing Feedback to Participants
  • Professional Guidelines for Standard Setting
  • Evaluating Standard Setting
  • Conclusions

SECTION II: STANDARD SETTING METHODS

  • 4. The Nedelsky Method
  • Procedures for the Nedelsky Method
  • Alternative Procedures and Limitations
  • 5. The Ebel Method
  • Procedures for the Ebel Method
  • Alternative Procedures and Limitations
  • 6. The Angoff Method and Angoff Variations
  • Procedures for the Angoff Method
  • Procedures for Angoff Variations
  • Alternative Procedures and Limitations
  • 7. The Direct Consensus Method
  • Procedures for the Direct Consensus Method
  • Alternative Procedures and Limitations
  • 8. The Contrasting Groups and Borderline Group Methods
  • Procedures for the Contrasting Groups Method
  • An Example Using the Contrasting Groups Method
  • The Borderline Group Method
  • Alternative Procedures and Limitations
  • 9. The Body of Work and Other Holistic Methods
  • The Judgmental Policy Capturing Method
  • The Dominant Profile Method
  • The Analytic Judgment Method
  • Summary Analysis of Three Holistic Methods
  • Overview of the Body of Work Method
  • Procedures for the Body of Work Method
  • An Application the Body of Work Method
  • Alternative Procedures and Limitations
  • 10. The Bookmark Method
  • Overview of the Bookmark Method
  • An Implementation of the Bookmark Method
  • Alternative Procedures and Limitations
  • 11. The Item-Descriptor Matching Method
  • Procedures for the IDM Method
  • Alternative Procedures and Limitations
  • 12. The Hofstee and Beuk Methods
  • The Hofstee Method
  • Procedures for Implementing the Hofstee Method
  • The Beuk Method
  • Procedures for Implementing the Beuk Method
  • Alternative Procedures and Limitations

SECTION III: CHALLENGES AND FUTURE DIRECTIONS IN STANDARD SETTING

  • 13. Scheduling Standard Setting Activities
  • Scheduling Standard Setting for Educational Assessments
  • Scheduling Standard Setting for Credentialing Programs
  • Conclusions and Recommendations
  • 14. Vertically-Moderated Standard Setting
  • The Interrelated Challenges
  • A Brief History of Vertically-Moderated Standard Setting
  • What Is Vertically-Moderated Standard Setting?
  • Approaches to VMSS
  • Applications of VMSS
  • An Illustration of VMSS Procedures
  • Alternative Procedures and Limitations
  • 15. Standard Setting on Alternate Assessments
  • The Unique Challenges of Alternate Assessment
  • Necessary Conditions for Alternate Assessment Systems
  • A Generalized Holistic (GH) Method
  • Limitations and Alternative Procedures
  • 16. Special Topics and Next Steps
  • Rounding
  • Methods of Adjusting Cut Scores
  • Deciding How to Incorporate Uncertainty
  • Generalizability of Standards
  • Using Multiple Methods of Standard Setting
  • Improving Participant Training

Description

"Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests is a well-timed text, given the legislated requirement of performance standards in our K-12 educational system, as well as the continued need in the process of credentialing professionals. This book would be useful to individuals retained to produce or coordinate standard-setting activites." —PSYCCRITIQUES


In Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests, authors Gregory J. Cizek and Michael B. Bunch provide the only "go-to" reference on how to set standards on tests in education, licensure, and certification. This book is comprehensive in scope, practical in nature, and definitive in terms of cataloguing the essential conceptual and procedural fundamentals of setting performance standards.

Key Features:  
  • Offers a detailed description of procedures: In a practical, straightforward, and virtually jargon-free style, this book begins with important foundational and conceptual information about standard setting followed by step-by-step procedures for all currently used methods, including Angoff, Item Mapping, Analystic Judgment, etc. In addition, a full chapter is devoted to future issues and research in standard setting.
  • Enables readers to design, implement, and evaluate a standard setting study: Using this book, readers will be able to: select an appropriate standard setting method; schedule and plan implementation; identify and train qualified standard-setting participants; conduct the standard setting meeting; calculate one or more cut scores; and evaluate the success of the procedure they have implemented.
  • Allows readers to adapt written materials to their situation: Examples are provided in widely available formats such as Excel, SPSS, .pdf and Word files so readers do not have to purchase specialized software or learn to use unfamiliar packages. The book includes downloadable forms, sample correspondence, checklists, data sets, and analytical software for performing calculations required for various standard setting procedures.

Intended Audience:  
This is an excellent supplementary textbook for graduate courses such as Applied Measurement in Education; Psychometric Theory; Advanced Educational Measurement; Special Topics in Measurement and Evaluation; Seminar in Educational and Psychological Measurement in the departments of Educational Psychology, Measurement and Evaluation, Psychology, and others.  

Contents

SECTION I: FUNDAMENTALS OF STANDARD SETTING

  • 1. Contemporary Standard Setting: An Enduring Need
  • The Need to Make Decisions
  • The Benefits of Standard Setting
  • General Approaches to Standard Setting
  • Current Contexts for Standard Setting
  • 2. What is Standard Setting?
  • Kinds of Standards
  • Definitions of Standard Setting
  • Policy Issues and Standard Setting
  • Item Scoring Criteria and Total-Test Performance Standards
  • Conclusions
  • 3. Common Elements in Setting Preformance Standards
  • Purpose
  • Choosing a Standard Setting Method
  • Performance Level Labels
  • Performance Level Descriptions
  • Key Conceptualizations
  • Selecting and Training Participants
  • Evaluating Standard Setting
  • Providing Feedback to Participants
  • Professional Guidelines for Standard Setting
  • Evaluating Standard Setting
  • Conclusions

SECTION II: STANDARD SETTING METHODS

  • 4. The Nedelsky Method
  • Procedures for the Nedelsky Method
  • Alternative Procedures and Limitations
  • 5. The Ebel Method
  • Procedures for the Ebel Method
  • Alternative Procedures and Limitations
  • 6. The Angoff Method and Angoff Variations
  • Procedures for the Angoff Method
  • Procedures for Angoff Variations
  • Alternative Procedures and Limitations
  • 7. The Direct Consensus Method
  • Procedures for the Direct Consensus Method
  • Alternative Procedures and Limitations
  • 8. The Contrasting Groups and Borderline Group Methods
  • Procedures for the Contrasting Groups Method
  • An Example Using the Contrasting Groups Method
  • The Borderline Group Method
  • Alternative Procedures and Limitations
  • 9. The Body of Work and Other Holistic Methods
  • The Judgmental Policy Capturing Method
  • The Dominant Profile Method
  • The Analytic Judgment Method
  • Summary Analysis of Three Holistic Methods
  • Overview of the Body of Work Method
  • Procedures for the Body of Work Method
  • An Application the Body of Work Method
  • Alternative Procedures and Limitations
  • 10. The Bookmark Method
  • Overview of the Bookmark Method
  • An Implementation of the Bookmark Method
  • Alternative Procedures and Limitations
  • 11. The Item-Descriptor Matching Method
  • Procedures for the IDM Method
  • Alternative Procedures and Limitations
  • 12. The Hofstee and Beuk Methods
  • The Hofstee Method
  • Procedures for Implementing the Hofstee Method
  • The Beuk Method
  • Procedures for Implementing the Beuk Method
  • Alternative Procedures and Limitations

SECTION III: CHALLENGES AND FUTURE DIRECTIONS IN STANDARD SETTING

  • 13. Scheduling Standard Setting Activities
  • Scheduling Standard Setting for Educational Assessments
  • Scheduling Standard Setting for Credentialing Programs
  • Conclusions and Recommendations
  • 14. Vertically-Moderated Standard Setting
  • The Interrelated Challenges
  • A Brief History of Vertically-Moderated Standard Setting
  • What Is Vertically-Moderated Standard Setting?
  • Approaches to VMSS
  • Applications of VMSS
  • An Illustration of VMSS Procedures
  • Alternative Procedures and Limitations
  • 15. Standard Setting on Alternate Assessments
  • The Unique Challenges of Alternate Assessment
  • Necessary Conditions for Alternate Assessment Systems
  • A Generalized Holistic (GH) Method
  • Limitations and Alternative Procedures
  • 16. Special Topics and Next Steps
  • Rounding
  • Methods of Adjusting Cut Scores
  • Deciding How to Incorporate Uncertainty
  • Generalizability of Standards
  • Using Multiple Methods of Standard Setting
  • Improving Participant Training
SAGE Publishing Logo

Standard Setting

A Guide to Establishing and Evaluating Performance Standards on Tests

Gregory J. Cizek , Michael B. Bunch
Cancel Print

November 2006 | 368 pages | Sage US

Format Published Date ISBN Price

"Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests is a well-timed text, given the legislated requirement of performance standards in our K-12 educational system, as well as the continued need in the process of credentialing professionals. This book would be useful to individuals retained to produce or coordinate standard-setting activites." —PSYCCRITIQUES


In Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests, authors Gregory J. Cizek and Michael B. Bunch provide the only "go-to" reference on how to set standards on tests in education, licensure, and certification. This book is comprehensive in scope, practical in nature, and definitive in terms of cataloguing the essential conceptual and procedural fundamentals of setting performance standards.

Key Features:  
  • Offers a detailed description of procedures: In a practical, straightforward, and virtually jargon-free style, this book begins with important foundational and conceptual information about standard setting followed by step-by-step procedures for all currently used methods, including Angoff, Item Mapping, Analystic Judgment, etc. In addition, a full chapter is devoted to future issues and research in standard setting.
  • Enables readers to design, implement, and evaluate a standard setting study: Using this book, readers will be able to: select an appropriate standard setting method; schedule and plan implementation; identify and train qualified standard-setting participants; conduct the standard setting meeting; calculate one or more cut scores; and evaluate the success of the procedure they have implemented.
  • Allows readers to adapt written materials to their situation: Examples are provided in widely available formats such as Excel, SPSS, .pdf and Word files so readers do not have to purchase specialized software or learn to use unfamiliar packages. The book includes downloadable forms, sample correspondence, checklists, data sets, and analytical software for performing calculations required for various standard setting procedures.

Intended Audience:  
This is an excellent supplementary textbook for graduate courses such as Applied Measurement in Education; Psychometric Theory; Advanced Educational Measurement; Special Topics in Measurement and Evaluation; Seminar in Educational and Psychological Measurement in the departments of Educational Psychology, Measurement and Evaluation, Psychology, and others.  

Table Of Contents:

  • SECTION I: FUNDAMENTALS OF STANDARD SETTING
  • 1. Contemporary Standard Setting: An Enduring Need
  • The Need to Make Decisions
  • The Benefits of Standard Setting
  • General Approaches to Standard Setting
  • Current Contexts for Standard Setting
  • 2. What is Standard Setting?
  • Kinds of Standards
  • Definitions of Standard Setting
  • Policy Issues and Standard Setting
  • Item Scoring Criteria and Total-Test Performance Standards
  • Conclusions
  • 3. Common Elements in Setting Preformance Standards
  • Purpose
  • Choosing a Standard Setting Method
  • Performance Level Labels
  • Performance Level Descriptions
  • Key Conceptualizations
  • Selecting and Training Participants
  • Evaluating Standard Setting
  • Providing Feedback to Participants
  • Professional Guidelines for Standard Setting
  • Evaluating Standard Setting
  • Conclusions
  • SECTION II: STANDARD SETTING METHODS
  • 4. The Nedelsky Method
  • Procedures for the Nedelsky Method
  • Alternative Procedures and Limitations
  • 5. The Ebel Method
  • Procedures for the Ebel Method
  • Alternative Procedures and Limitations
  • 6. The Angoff Method and Angoff Variations
  • Procedures for the Angoff Method
  • Procedures for Angoff Variations
  • Alternative Procedures and Limitations
  • 7. The Direct Consensus Method
  • Procedures for the Direct Consensus Method
  • Alternative Procedures and Limitations
  • 8. The Contrasting Groups and Borderline Group Methods
  • Procedures for the Contrasting Groups Method
  • An Example Using the Contrasting Groups Method
  • The Borderline Group Method
  • Alternative Procedures and Limitations
  • 9. The Body of Work and Other Holistic Methods
  • The Judgmental Policy Capturing Method
  • The Dominant Profile Method
  • The Analytic Judgment Method
  • Summary Analysis of Three Holistic Methods
  • Overview of the Body of Work Method
  • Procedures for the Body of Work Method
  • An Application the Body of Work Method
  • Alternative Procedures and Limitations
  • 10. The Bookmark Method
  • Overview of the Bookmark Method
  • An Implementation of the Bookmark Method
  • Alternative Procedures and Limitations
  • 11. The Item-Descriptor Matching Method
  • Procedures for the IDM Method
  • Alternative Procedures and Limitations
  • 12. The Hofstee and Beuk Methods
  • The Hofstee Method
  • Procedures for Implementing the Hofstee Method
  • The Beuk Method
  • Procedures for Implementing the Beuk Method
  • Alternative Procedures and Limitations
  • SECTION III: CHALLENGES AND FUTURE DIRECTIONS IN STANDARD SETTING
  • 13. Scheduling Standard Setting Activities
  • Scheduling Standard Setting for Educational Assessments
  • Scheduling Standard Setting for Credentialing Programs
  • Conclusions and Recommendations
  • 14. Vertically-Moderated Standard Setting
  • The Interrelated Challenges
  • A Brief History of Vertically-Moderated Standard Setting
  • What Is Vertically-Moderated Standard Setting?
  • Approaches to VMSS
  • Applications of VMSS
  • An Illustration of VMSS Procedures
  • Alternative Procedures and Limitations
  • 15. Standard Setting on Alternate Assessments
  • The Unique Challenges of Alternate Assessment
  • Necessary Conditions for Alternate Assessment Systems
  • A Generalized Holistic (GH) Method
  • Limitations and Alternative Procedures
  • 16. Special Topics and Next Steps
  • Rounding
  • Methods of Adjusting Cut Scores
  • Deciding How to Incorporate Uncertainty
  • Generalizability of Standards
  • Using Multiple Methods of Standard Setting
  • Improving Participant Training

Recent Product Reviews:

It is abundantly apparent that the authors know standard setting and themselves have implemented numerous standard-setting procedures using different methods for different types of assessment programs.... Although the book is very practical, someone who has a fair amount of knowledge in standard setting will not feel shortchanged. There is a richness in the discussions of the methods that engages the reader, such as the different modifications of a method and the situational challenge that warranted each modification. Also, when a specific implementation of a method is discussed in detail, the authors are very conscientious in discussing the impetus for the adaptations. The historical perspective presented for some of the methods is quite intriguing.
Journal of Educational Measurement
A well-timed text, given the legislated requirement of performance standards in our K-12 educational system, as well as the continued need in the process of credentialing professionals. This book would be useful to individuals retained to produce or coordinate standard-setting activites.
PsycCritiques

Recommendations