Assessing Teacher Dispositions

Five Standards-Based Steps to Valid Measurement Using the DAATS Model
Judy R. Wilkerson - Florida Gulf Coast University, USA
William S. Lang - University of South Florida, St. Petersburg
Assessing Teacher Dispositions
May 2007 | 296 pages | Corwin
Create Flyer

Purchase

Paperback
ISBN: 9781412953689
Available from January 0001

Description

"There is a vitally important link between teacher preparation and the performance of those teachers and their students. Assessing Teacher Competency and Assessing Teacher Dispositions provide a strong underpinning to improve teacher competencies in both the cognitive and affective domains in ways that we can hope will endure post-licensure."
—From the Foreword by Richard C. Kunkel

"Well researched and standards based, with activities, worksheets, definitions, and rubrics. Addresses a topic that has been a mystery to assessment gurus."
—Marilyn K. Troupe, Director
Division of Educator Preparation, Kentucky Education Professional Standards Board

At last, a step-by-step guide for assessing teacher dispositions that addresses national accreditation standards.

While school leaders have long sought a definitive tool for assessing teacher affect and dispositions, a practical method for measurement has proven elusive—until now.

Assessing Teacher Dispositions presents a conceptual framework that helps educators understand what "appropriate dispositions" are, why it is important to measure them, and how to implement an assessment process in their schools and districts.

This indispensable companion to Assessing Teacher Competency introduces the authors' research-based five-step DAATS model, combining user-friendly definitions and guiding questions with an examination of assessment design, planning, instrument development, decision making, and data management. Linked to national standards for best practice set by NCATE, INTASC, and NBPTS, the DAATS approach offers:

  • A step-by-step implementation sequence with worksheets and training activities
  • Examples from preservice and inservice settings
  • A comprehensive assessment system when used with the CAATS model for assessing teacher competency (knowledge and skills)

This groundbreaking text offers a field-tested, valid, and reliable process for dispositions assessment that is ideal for schools of education, teacher induction programs, and preservice and inservice training.

Contents

List of Tables

List of Tables

List of Figures

List of Figures

List of Boxes

List of Boxes

List of Abbreviations and Acronyms

List of Abbreviations and Acronyms

Foreword by Richard C. Kunkel

Foreword by Richard C. Kunkel

Preface

Preface

Acknowledgments

Acknowledgments

About the Authors

  • 1. What Are Dispositions and Why Should We Measure Them?
  • What This Chapter Is About
  • The Importance of Measuring Dispositions
  • The Challenge
  • What Are Standards-Based Dispositions?
  • Hierarchical Relationships Among Knowledge, Skills, and Dispositions
  • Remembering Bloom
  • Dispositions and Accreditation Requirements: Requirements and Definitions
  • Measuring Dispositions: Sources of Confusion
  • Measuring Dispositions: Morals, Ethics, or Standards Based?
  • Different Construct, Different Assessments, Similar Assessment Design Process
  • Wrap Up
  • Activity 1.1: Questions for Exploration
  • Activity 1.2: What Have You Noticed?
  • Activity 1.3: Assessment Belief Scale
  • Activity 1.4: Cognitive, Affective, and Psychomotor Objectives and Assessments
  • 2. Methods for Assessing Dispositions
  • What This Chapter Is About
  • A Conceptual Framework for Measuring Dispositions
  • Measuring Teacher Dispositions: The State of the Art
  • Back to Basics: Bloom and Krathwhol
  • Available Methods for Measuring Dispositions or Affect
  • The Importance of Inference in Measuring Dispositions
  • Wrap Up
  • Activity 2.1: Questions for Exploration
  • Activity 2.2: Bloom and the INTASC Principles
  • Activity 2.3: Field Work
  • Activity 2.4: Review Your Feelings
  • 3. DAATS Step 1: Assessment Design Inputs
  • Where We Have Been So Far
  • What This Chapter Is About
  • Why Are Purpose, Use, Propositions, and Content So Important?
  • DAATS Step 1A: Define the Purpose(s) and Use(s) of the System
  • DAATS Step 1B: Define the Propositions or Principles That Guide the System
  • DAATS Step 1C: Define the Conceptual Framework or Content of the System
  • DAATS Step 1D: Review Local Factors That Impact the System
  • Wrap Up
  • Worksheet 3.1: Purpose, Use, Propositions, Content, and Context Checksheet
  • Worksheet 3.2: Purpose, Use, Content, Draft
  • Worksheet 3.3: Propositions
  • Worksheet 3.4: Contextual Analysis
  • 4. DAATS Step 2: Planning With a Continuing Eye on Valid Assessment Decisions
  • Where We Have Been So Far
  • What This Chapter Is About
  • DAATS Step 2A: Analyze Standards and Indicators
  • All Those Indicators
  • Why Bother?
  • DAATS Step 2B: Visualize the Teacher Demonstrating the Affective Targets
  • DAATS Step 2C: Select Assessment Methods at Different Levels of Inference
  • DAATS Step 2D: Build an Assessment Framework Correlating Standards and Methods
  • Wrap Up
  • DAATS
  • Worksheet 4.1: Organizing for Alignment (Version 1)
  • Worksheet 4.2: Organizing for Alignment (Version 2)
  • Worksheet 4.3: Visualizing the Dispositional Statements
  • Worksheet 4.4: Selecting Assessment Methods for INTASC Indicators
  • Worksheet 4.5: Assessment Methods for INTASC Indicators: Blueprint
  • Worksheet 4.6: Cost/Benefit and Coverage Analysis of Assessment Methods
  • 5. DAATS Step 3: Instrument Development
  • Where We Have Been So Far
  • What This Chapter Is About
  • DAATS Step 3A: Draft items for Each Instrument
  • Thurstone Agreement Scales
  • Questionnaires, Interviews, and Focus Groups
  • Observed Performance
  • Thematic Apperception Tests or Situation Reflection Assessment
  • DAATS Step 3B: Review Items for Applicability to Values, Domain Coverage, Job Relevance
  • Wrap Up
  • Worksheet 5.1: Creating Scales
  • Worksheet 5.2: Creating Questionnaires, Interviews, or K-12 Focus Group Protocols
  • Worksheet 5.3: Creating an Affective Behaviour Checklist
  • Worksheet 5.4: Creating an Affective Behaviour Rating Scale
  • Worksheet 5.5: Creating a Tally Sheet for Affective Observation
  • Worksheet 5.6: Checklist for Reviewing Scale Drafts
  • Worksheet 5.7: Review Sheets for Questionnaires and Interviews
  • Worksheet 5.8: Review Sheets for K-12 Focus Group Protocols
  • Worksheet 5.9: Checklist for Reviewing Observations and Behavioral Checklists
  • Worksheet 5.10: Coverage Check
  • Worksheet 5.11: Rating Form for Stakeholder Review
  • 6. DAATS Step 4: Decision Making and Data Management
  • Where We Have Been So Far
  • What This Chapter Is About
  • DAATS Step 4A: Develop Scoring Rubrics
  • Dichotomous Response Scoring Keys
  • Rating Scale Rubrics
  • DAATS Step 4B: Determine How Data Will Be Combined and Used
  • Need for Shared Data
  • Data Storage
  • Data Aggregation
  • Maximizing the Utility of the Data for Decision Making
  • DAATS Step 4C: Develop Implementation Procedures and Materials
  • Preponderance of the Evidence vs. Cut Scores
  • Advising and Due Process
  • Scoring Procedures
  • Implementation
  • Wrap Up
  • Worksheet #6.1: Explanation of Dichotomous Scoring Decisions
  • Worksheet #6.2: Rubric Design
  • Worksheet #6.3: Sample Format for Candidate/Teacher Tracking Form
  • Worksheet #6.4: Format for Data Aggregation
  • Worksheet #6.5: Sample Disposition Event Report
  • Worksheet #6.6: Management Plan
  • 7. DAATS Step 5: Credible Data
  • Where We Have Been So Far
  • What This Chapter Is About
  • What Is Psychometric Integrity and Why Do We Have to Worry About It?
  • DAATS Step 5A: Create a Plan to Provide Evidence of Validity, Reliability, Fairness, and Utility
  • Elements of a Plan
  • Element 7.1: Purpose and Use
  • Element 7.2: Construct Measured
  • Element 7.3: Interpretation and Reporting of Scores
  • Element 7.4: Assessment Specifications and Content Map
  • Element 7.5: Assessor/Rater Selection and Training Procedures
  • Element 7.6: Analysis Methodology
  • Element 7.7: External Review Personnel and Methodology
  • Element 7.8: Evidence of Validity, Reliability, and Fairness (VRF)
  • Psychometric Evidence Collected Already
  • Next Steps in Collecting Evidence of Validity, Reliability, and Fairness
  • Future Studies
  • DAATS Step 5B: Implement the Plan Conscientiously
  • Wrap Up
  • Worksheets and Examples
  • Worksheet 7.1: Assessment Specifications
  • Worksheet 7.2: Analysis of Appropriateness of Decisions for Teacher Failures
  • Worksheet 7.3: Analysis of Rehire Data
  • Worksheet 7.4: Program Improvement Record
  • Worksheet 7.5: Expert Rescoring
  • Worksheet 7.6: Fairness Review
  • Worksheet 7.7: Analysis of Remediation Efforts and EO Impact
  • Worksheet 7.8: Psychometric Plan Format
  • Example 1: Logistic Ruler for Content Validity
  • Example 2: Computation of the Lawshe (1975) Content Validity Ratio
  • Example 3: Disparate Impact Analysis
  • Example 4: Computation of Cohen's Kappa (1960) for Inter-rater Reliability
  • Example 5: Two Pearson Correlation Coefficients and Scatterplots: Disposition Scores Correlated with PRAXIS and Portfolio Scores
  • Example 6: Spearman Correlation Coefficient and Scatterplot: Disposition Scores Correlated With Principal Ratings
  • Example 7: Correlation Matrix and Scatterplots Knowledge, Impact, Dispositions, Skills (KIDS)
  • Example 8: T-Test Comparing Dispositions of Mathematics and Science Teachers
  • Example 9: DIF Analysis for Programs
  • 8. Using Teacher Scores for Continuous Improvement
  • What This Chapter Is About
  • Reasons Why We Use the Rasch Model
  • The Classical Approach
  • A Quick Overview of Where Rasch Fits Into the Grand Scheme of IRT Models
  • Rasch: The Basics
  • Getting Started
  • Differences That Item Writers Make
  • Guttman Scaling
  • A Sample Rasch Ruler
  • From Pictures to Numbers
  • The Fit Statistic
  • Gain Scores " Real or Imagined'
  • Ratings and Raters
  • Learning More About Rasch
  • Wrap Up
  • Activity #1: Decision-Making Tool for Measurement
  • 9. Legal Integrity
  • What This Chapter Is About
  • Why Not Portfolios?
  • Why the Pied Piper?
  • What IF?? A Legal Scenario: Mary Beth JoAnne Sues XYZ University
  • MBJ Helps Us to Understand the Convergency of Psychometrics and Legal Requirements
  • Background Facts
  • Scenario #1
  • Scenarios #2, 3, and 4
  • Psychometric Issues and Legal Challenges in the Real World
  • Legal Issues and Precedents
  • Three Landmark Dispositions Cases in Two Years
  • Tide Changing in NCATE
  • Standards Are the Vanguard!
  • MBJ Revisited
  • End Note

Resource I. DAATS Steps and Worksheets

Resource I. DAATS Steps and Worksheets

Resource II. INTASC Disposition Indicators

Resource II. INTASC Disposition Indicators

Glossary

Glossary

Index

Index

Description

"There is a vitally important link between teacher preparation and the performance of those teachers and their students. Assessing Teacher Competency and Assessing Teacher Dispositions provide a strong underpinning to improve teacher competencies in both the cognitive and affective domains in ways that we can hope will endure post-licensure."
—From the Foreword by Richard C. Kunkel

"Well researched and standards based, with activities, worksheets, definitions, and rubrics. Addresses a topic that has been a mystery to assessment gurus."
—Marilyn K. Troupe, Director
Division of Educator Preparation, Kentucky Education Professional Standards Board

At last, a step-by-step guide for assessing teacher dispositions that addresses national accreditation standards.

While school leaders have long sought a definitive tool for assessing teacher affect and dispositions, a practical method for measurement has proven elusive—until now.

Assessing Teacher Dispositions presents a conceptual framework that helps educators understand what "appropriate dispositions" are, why it is important to measure them, and how to implement an assessment process in their schools and districts.

This indispensable companion to Assessing Teacher Competency introduces the authors' research-based five-step DAATS model, combining user-friendly definitions and guiding questions with an examination of assessment design, planning, instrument development, decision making, and data management. Linked to national standards for best practice set by NCATE, INTASC, and NBPTS, the DAATS approach offers:

  • A step-by-step implementation sequence with worksheets and training activities
  • Examples from preservice and inservice settings
  • A comprehensive assessment system when used with the CAATS model for assessing teacher competency (knowledge and skills)

This groundbreaking text offers a field-tested, valid, and reliable process for dispositions assessment that is ideal for schools of education, teacher induction programs, and preservice and inservice training.

Contents

List of Tables

List of Tables

List of Figures

List of Figures

List of Boxes

List of Boxes

List of Abbreviations and Acronyms

List of Abbreviations and Acronyms

Foreword by Richard C. Kunkel

Foreword by Richard C. Kunkel

Preface

Preface

Acknowledgments

Acknowledgments

About the Authors

  • 1. What Are Dispositions and Why Should We Measure Them?
  • What This Chapter Is About
  • The Importance of Measuring Dispositions
  • The Challenge
  • What Are Standards-Based Dispositions?
  • Hierarchical Relationships Among Knowledge, Skills, and Dispositions
  • Remembering Bloom
  • Dispositions and Accreditation Requirements: Requirements and Definitions
  • Measuring Dispositions: Sources of Confusion
  • Measuring Dispositions: Morals, Ethics, or Standards Based?
  • Different Construct, Different Assessments, Similar Assessment Design Process
  • Wrap Up
  • Activity 1.1: Questions for Exploration
  • Activity 1.2: What Have You Noticed?
  • Activity 1.3: Assessment Belief Scale
  • Activity 1.4: Cognitive, Affective, and Psychomotor Objectives and Assessments
  • 2. Methods for Assessing Dispositions
  • What This Chapter Is About
  • A Conceptual Framework for Measuring Dispositions
  • Measuring Teacher Dispositions: The State of the Art
  • Back to Basics: Bloom and Krathwhol
  • Available Methods for Measuring Dispositions or Affect
  • The Importance of Inference in Measuring Dispositions
  • Wrap Up
  • Activity 2.1: Questions for Exploration
  • Activity 2.2: Bloom and the INTASC Principles
  • Activity 2.3: Field Work
  • Activity 2.4: Review Your Feelings
  • 3. DAATS Step 1: Assessment Design Inputs
  • Where We Have Been So Far
  • What This Chapter Is About
  • Why Are Purpose, Use, Propositions, and Content So Important?
  • DAATS Step 1A: Define the Purpose(s) and Use(s) of the System
  • DAATS Step 1B: Define the Propositions or Principles That Guide the System
  • DAATS Step 1C: Define the Conceptual Framework or Content of the System
  • DAATS Step 1D: Review Local Factors That Impact the System
  • Wrap Up
  • Worksheet 3.1: Purpose, Use, Propositions, Content, and Context Checksheet
  • Worksheet 3.2: Purpose, Use, Content, Draft
  • Worksheet 3.3: Propositions
  • Worksheet 3.4: Contextual Analysis
  • 4. DAATS Step 2: Planning With a Continuing Eye on Valid Assessment Decisions
  • Where We Have Been So Far
  • What This Chapter Is About
  • DAATS Step 2A: Analyze Standards and Indicators
  • All Those Indicators
  • Why Bother?
  • DAATS Step 2B: Visualize the Teacher Demonstrating the Affective Targets
  • DAATS Step 2C: Select Assessment Methods at Different Levels of Inference
  • DAATS Step 2D: Build an Assessment Framework Correlating Standards and Methods
  • Wrap Up
  • DAATS
  • Worksheet 4.1: Organizing for Alignment (Version 1)
  • Worksheet 4.2: Organizing for Alignment (Version 2)
  • Worksheet 4.3: Visualizing the Dispositional Statements
  • Worksheet 4.4: Selecting Assessment Methods for INTASC Indicators
  • Worksheet 4.5: Assessment Methods for INTASC Indicators: Blueprint
  • Worksheet 4.6: Cost/Benefit and Coverage Analysis of Assessment Methods
  • 5. DAATS Step 3: Instrument Development
  • Where We Have Been So Far
  • What This Chapter Is About
  • DAATS Step 3A: Draft items for Each Instrument
  • Thurstone Agreement Scales
  • Questionnaires, Interviews, and Focus Groups
  • Observed Performance
  • Thematic Apperception Tests or Situation Reflection Assessment
  • DAATS Step 3B: Review Items for Applicability to Values, Domain Coverage, Job Relevance
  • Wrap Up
  • Worksheet 5.1: Creating Scales
  • Worksheet 5.2: Creating Questionnaires, Interviews, or K-12 Focus Group Protocols
  • Worksheet 5.3: Creating an Affective Behaviour Checklist
  • Worksheet 5.4: Creating an Affective Behaviour Rating Scale
  • Worksheet 5.5: Creating a Tally Sheet for Affective Observation
  • Worksheet 5.6: Checklist for Reviewing Scale Drafts
  • Worksheet 5.7: Review Sheets for Questionnaires and Interviews
  • Worksheet 5.8: Review Sheets for K-12 Focus Group Protocols
  • Worksheet 5.9: Checklist for Reviewing Observations and Behavioral Checklists
  • Worksheet 5.10: Coverage Check
  • Worksheet 5.11: Rating Form for Stakeholder Review
  • 6. DAATS Step 4: Decision Making and Data Management
  • Where We Have Been So Far
  • What This Chapter Is About
  • DAATS Step 4A: Develop Scoring Rubrics
  • Dichotomous Response Scoring Keys
  • Rating Scale Rubrics
  • DAATS Step 4B: Determine How Data Will Be Combined and Used
  • Need for Shared Data
  • Data Storage
  • Data Aggregation
  • Maximizing the Utility of the Data for Decision Making
  • DAATS Step 4C: Develop Implementation Procedures and Materials
  • Preponderance of the Evidence vs. Cut Scores
  • Advising and Due Process
  • Scoring Procedures
  • Implementation
  • Wrap Up
  • Worksheet #6.1: Explanation of Dichotomous Scoring Decisions
  • Worksheet #6.2: Rubric Design
  • Worksheet #6.3: Sample Format for Candidate/Teacher Tracking Form
  • Worksheet #6.4: Format for Data Aggregation
  • Worksheet #6.5: Sample Disposition Event Report
  • Worksheet #6.6: Management Plan
  • 7. DAATS Step 5: Credible Data
  • Where We Have Been So Far
  • What This Chapter Is About
  • What Is Psychometric Integrity and Why Do We Have to Worry About It?
  • DAATS Step 5A: Create a Plan to Provide Evidence of Validity, Reliability, Fairness, and Utility
  • Elements of a Plan
  • Element 7.1: Purpose and Use
  • Element 7.2: Construct Measured
  • Element 7.3: Interpretation and Reporting of Scores
  • Element 7.4: Assessment Specifications and Content Map
  • Element 7.5: Assessor/Rater Selection and Training Procedures
  • Element 7.6: Analysis Methodology
  • Element 7.7: External Review Personnel and Methodology
  • Element 7.8: Evidence of Validity, Reliability, and Fairness (VRF)
  • Psychometric Evidence Collected Already
  • Next Steps in Collecting Evidence of Validity, Reliability, and Fairness
  • Future Studies
  • DAATS Step 5B: Implement the Plan Conscientiously
  • Wrap Up
  • Worksheets and Examples
  • Worksheet 7.1: Assessment Specifications
  • Worksheet 7.2: Analysis of Appropriateness of Decisions for Teacher Failures
  • Worksheet 7.3: Analysis of Rehire Data
  • Worksheet 7.4: Program Improvement Record
  • Worksheet 7.5: Expert Rescoring
  • Worksheet 7.6: Fairness Review
  • Worksheet 7.7: Analysis of Remediation Efforts and EO Impact
  • Worksheet 7.8: Psychometric Plan Format
  • Example 1: Logistic Ruler for Content Validity
  • Example 2: Computation of the Lawshe (1975) Content Validity Ratio
  • Example 3: Disparate Impact Analysis
  • Example 4: Computation of Cohen's Kappa (1960) for Inter-rater Reliability
  • Example 5: Two Pearson Correlation Coefficients and Scatterplots: Disposition Scores Correlated with PRAXIS and Portfolio Scores
  • Example 6: Spearman Correlation Coefficient and Scatterplot: Disposition Scores Correlated With Principal Ratings
  • Example 7: Correlation Matrix and Scatterplots Knowledge, Impact, Dispositions, Skills (KIDS)
  • Example 8: T-Test Comparing Dispositions of Mathematics and Science Teachers
  • Example 9: DIF Analysis for Programs
  • 8. Using Teacher Scores for Continuous Improvement
  • What This Chapter Is About
  • Reasons Why We Use the Rasch Model
  • The Classical Approach
  • A Quick Overview of Where Rasch Fits Into the Grand Scheme of IRT Models
  • Rasch: The Basics
  • Getting Started
  • Differences That Item Writers Make
  • Guttman Scaling
  • A Sample Rasch Ruler
  • From Pictures to Numbers
  • The Fit Statistic
  • Gain Scores " Real or Imagined'
  • Ratings and Raters
  • Learning More About Rasch
  • Wrap Up
  • Activity #1: Decision-Making Tool for Measurement
  • 9. Legal Integrity
  • What This Chapter Is About
  • Why Not Portfolios?
  • Why the Pied Piper?
  • What IF?? A Legal Scenario: Mary Beth JoAnne Sues XYZ University
  • MBJ Helps Us to Understand the Convergency of Psychometrics and Legal Requirements
  • Background Facts
  • Scenario #1
  • Scenarios #2, 3, and 4
  • Psychometric Issues and Legal Challenges in the Real World
  • Legal Issues and Precedents
  • Three Landmark Dispositions Cases in Two Years
  • Tide Changing in NCATE
  • Standards Are the Vanguard!
  • MBJ Revisited
  • End Note

Resource I. DAATS Steps and Worksheets

Resource I. DAATS Steps and Worksheets

Resource II. INTASC Disposition Indicators

Resource II. INTASC Disposition Indicators

Glossary

Glossary

Index

Index

SAGE Publishing Logo

Assessing Teacher Dispositions

Five Standards-Based Steps to Valid Measurement Using the DAATS Model

Judy R. Wilkerson , William S. Lang
Cancel Print

May 2007 | 296 pages | Corwin

Format Published Date ISBN Price
Paperback 27/02/2023 9781412953689 $48.95
Hardcover 27/02/2023 9781412953672 $99.95

"There is a vitally important link between teacher preparation and the performance of those teachers and their students. Assessing Teacher Competency and Assessing Teacher Dispositions provide a strong underpinning to improve teacher competencies in both the cognitive and affective domains in ways that we can hope will endure post-licensure."
—From the Foreword by Richard C. Kunkel

"Well researched and standards based, with activities, worksheets, definitions, and rubrics. Addresses a topic that has been a mystery to assessment gurus."
—Marilyn K. Troupe, Director
Division of Educator Preparation, Kentucky Education Professional Standards Board

At last, a step-by-step guide for assessing teacher dispositions that addresses national accreditation standards.

While school leaders have long sought a definitive tool for assessing teacher affect and dispositions, a practical method for measurement has proven elusive—until now.

Assessing Teacher Dispositions presents a conceptual framework that helps educators understand what "appropriate dispositions" are, why it is important to measure them, and how to implement an assessment process in their schools and districts.

This indispensable companion to Assessing Teacher Competency introduces the authors' research-based five-step DAATS model, combining user-friendly definitions and guiding questions with an examination of assessment design, planning, instrument development, decision making, and data management. Linked to national standards for best practice set by NCATE, INTASC, and NBPTS, the DAATS approach offers:

  • A step-by-step implementation sequence with worksheets and training activities
  • Examples from preservice and inservice settings
  • A comprehensive assessment system when used with the CAATS model for assessing teacher competency (knowledge and skills)

This groundbreaking text offers a field-tested, valid, and reliable process for dispositions assessment that is ideal for schools of education, teacher induction programs, and preservice and inservice training.


Table Of Contents:

  • List of Tables
  • List of Figures
  • List of Boxes
  • List of Abbreviations and Acronyms
  • Foreword by Richard C. Kunkel
  • Preface
  • Acknowledgments
  • About the Authors
  • 1. What Are Dispositions and Why Should We Measure Them?
  • What This Chapter Is About
  • The Importance of Measuring Dispositions
  • The Challenge
  • What Are Standards-Based Dispositions?
  • Hierarchical Relationships Among Knowledge, Skills, and Dispositions
  • Remembering Bloom
  • Dispositions and Accreditation Requirements: Requirements and Definitions
  • Measuring Dispositions: Sources of Confusion
  • Measuring Dispositions: Morals, Ethics, or Standards Based?
  • Different Construct, Different Assessments, Similar Assessment Design Process
  • Wrap Up
  • Activity 1.1: Questions for Exploration
  • Activity 1.2: What Have You Noticed?
  • Activity 1.3: Assessment Belief Scale
  • Activity 1.4: Cognitive, Affective, and Psychomotor Objectives and Assessments
  • 2. Methods for Assessing Dispositions
  • What This Chapter Is About
  • A Conceptual Framework for Measuring Dispositions
  • Measuring Teacher Dispositions: The State of the Art
  • Back to Basics: Bloom and Krathwhol
  • Available Methods for Measuring Dispositions or Affect
  • The Importance of Inference in Measuring Dispositions
  • Wrap Up
  • Activity 2.1: Questions for Exploration
  • Activity 2.2: Bloom and the INTASC Principles
  • Activity 2.3: Field Work
  • Activity 2.4: Review Your Feelings
  • 3. DAATS Step 1: Assessment Design Inputs
  • Where We Have Been So Far
  • What This Chapter Is About
  • Why Are Purpose, Use, Propositions, and Content So Important?
  • DAATS Step 1A: Define the Purpose(s) and Use(s) of the System
  • DAATS Step 1B: Define the Propositions or Principles That Guide the System
  • DAATS Step 1C: Define the Conceptual Framework or Content of the System
  • DAATS Step 1D: Review Local Factors That Impact the System
  • Wrap Up
  • Worksheet 3.1: Purpose, Use, Propositions, Content, and Context Checksheet
  • Worksheet 3.2: Purpose, Use, Content, Draft
  • Worksheet 3.3: Propositions
  • Worksheet 3.4: Contextual Analysis
  • 4. DAATS Step 2: Planning With a Continuing Eye on Valid Assessment Decisions
  • Where We Have Been So Far
  • What This Chapter Is About
  • DAATS Step 2A: Analyze Standards and Indicators
  • All Those Indicators
  • Why Bother?
  • DAATS Step 2B: Visualize the Teacher Demonstrating the Affective Targets
  • DAATS Step 2C: Select Assessment Methods at Different Levels of Inference
  • DAATS Step 2D: Build an Assessment Framework Correlating Standards and Methods
  • Wrap Up
  • DAATS
  • Worksheet 4.1: Organizing for Alignment (Version 1)
  • Worksheet 4.2: Organizing for Alignment (Version 2)
  • Worksheet 4.3: Visualizing the Dispositional Statements
  • Worksheet 4.4: Selecting Assessment Methods for INTASC Indicators
  • Worksheet 4.5: Assessment Methods for INTASC Indicators: Blueprint
  • Worksheet 4.6: Cost/Benefit and Coverage Analysis of Assessment Methods
  • 5. DAATS Step 3: Instrument Development
  • Where We Have Been So Far
  • What This Chapter Is About
  • DAATS Step 3A: Draft items for Each Instrument
  • Thurstone Agreement Scales
  • Questionnaires, Interviews, and Focus Groups
  • Observed Performance
  • Thematic Apperception Tests or Situation Reflection Assessment
  • DAATS Step 3B: Review Items for Applicability to Values, Domain Coverage, Job Relevance
  • Wrap Up
  • Worksheet 5.1: Creating Scales
  • Worksheet 5.2: Creating Questionnaires, Interviews, or K-12 Focus Group Protocols
  • Worksheet 5.3: Creating an Affective Behaviour Checklist
  • Worksheet 5.4: Creating an Affective Behaviour Rating Scale
  • Worksheet 5.5: Creating a Tally Sheet for Affective Observation
  • Worksheet 5.6: Checklist for Reviewing Scale Drafts
  • Worksheet 5.7: Review Sheets for Questionnaires and Interviews
  • Worksheet 5.8: Review Sheets for K-12 Focus Group Protocols
  • Worksheet 5.9: Checklist for Reviewing Observations and Behavioral Checklists
  • Worksheet 5.10: Coverage Check
  • Worksheet 5.11: Rating Form for Stakeholder Review
  • 6. DAATS Step 4: Decision Making and Data Management
  • Where We Have Been So Far
  • What This Chapter Is About
  • DAATS Step 4A: Develop Scoring Rubrics
  • Dichotomous Response Scoring Keys
  • Rating Scale Rubrics
  • DAATS Step 4B: Determine How Data Will Be Combined and Used
  • Need for Shared Data
  • Data Storage
  • Data Aggregation
  • Maximizing the Utility of the Data for Decision Making
  • DAATS Step 4C: Develop Implementation Procedures and Materials
  • Preponderance of the Evidence vs. Cut Scores
  • Advising and Due Process
  • Scoring Procedures
  • Implementation
  • Wrap Up
  • Worksheet #6.1: Explanation of Dichotomous Scoring Decisions
  • Worksheet #6.2: Rubric Design
  • Worksheet #6.3: Sample Format for Candidate/Teacher Tracking Form
  • Worksheet #6.4: Format for Data Aggregation
  • Worksheet #6.5: Sample Disposition Event Report
  • Worksheet #6.6: Management Plan
  • 7. DAATS Step 5: Credible Data
  • Where We Have Been So Far
  • What This Chapter Is About
  • What Is Psychometric Integrity and Why Do We Have to Worry About It?
  • DAATS Step 5A: Create a Plan to Provide Evidence of Validity, Reliability, Fairness, and Utility
  • Elements of a Plan
  • Element 7.1: Purpose and Use
  • Element 7.2: Construct Measured
  • Element 7.3: Interpretation and Reporting of Scores
  • Element 7.4: Assessment Specifications and Content Map
  • Element 7.5: Assessor/Rater Selection and Training Procedures
  • Element 7.6: Analysis Methodology
  • Element 7.7: External Review Personnel and Methodology
  • Element 7.8: Evidence of Validity, Reliability, and Fairness (VRF)
  • Psychometric Evidence Collected Already
  • Next Steps in Collecting Evidence of Validity, Reliability, and Fairness
  • Future Studies
  • DAATS Step 5B: Implement the Plan Conscientiously
  • Wrap Up
  • Worksheets and Examples
  • Worksheet 7.1: Assessment Specifications
  • Worksheet 7.2: Analysis of Appropriateness of Decisions for Teacher Failures
  • Worksheet 7.3: Analysis of Rehire Data
  • Worksheet 7.4: Program Improvement Record
  • Worksheet 7.5: Expert Rescoring
  • Worksheet 7.6: Fairness Review
  • Worksheet 7.7: Analysis of Remediation Efforts and EO Impact
  • Worksheet 7.8: Psychometric Plan Format
  • Example 1: Logistic Ruler for Content Validity
  • Example 2: Computation of the Lawshe (1975) Content Validity Ratio
  • Example 3: Disparate Impact Analysis
  • Example 4: Computation of Cohen's Kappa (1960) for Inter-rater Reliability
  • Example 5: Two Pearson Correlation Coefficients and Scatterplots: Disposition Scores Correlated with PRAXIS and Portfolio Scores
  • Example 6: Spearman Correlation Coefficient and Scatterplot: Disposition Scores Correlated With Principal Ratings
  • Example 7: Correlation Matrix and Scatterplots Knowledge, Impact, Dispositions, Skills (KIDS)
  • Example 8: T-Test Comparing Dispositions of Mathematics and Science Teachers
  • Example 9: DIF Analysis for Programs
  • 8. Using Teacher Scores for Continuous Improvement
  • What This Chapter Is About
  • Reasons Why We Use the Rasch Model
  • The Classical Approach
  • A Quick Overview of Where Rasch Fits Into the Grand Scheme of IRT Models
  • Rasch: The Basics
  • Getting Started
  • Differences That Item Writers Make
  • Guttman Scaling
  • A Sample Rasch Ruler
  • From Pictures to Numbers
  • The Fit Statistic
  • Gain Scores " Real or Imagined'
  • Ratings and Raters
  • Learning More About Rasch
  • Wrap Up
  • Activity #1: Decision-Making Tool for Measurement
  • 9. Legal Integrity
  • What This Chapter Is About
  • Why Not Portfolios?
  • Why the Pied Piper?
  • What IF?? A Legal Scenario: Mary Beth JoAnne Sues XYZ University
  • MBJ Helps Us to Understand the Convergency of Psychometrics and Legal Requirements
  • Background Facts
  • Scenario #1
  • Scenarios #2, 3, and 4
  • Psychometric Issues and Legal Challenges in the Real World
  • Legal Issues and Precedents
  • Three Landmark Dispositions Cases in Two Years
  • Tide Changing in NCATE
  • Standards Are the Vanguard!
  • MBJ Revisited
  • End Note
  • Resource I. DAATS Steps and Worksheets
  • Resource II. INTASC Disposition Indicators
  • Glossary
  • Index

Recent Product Reviews:

"This book is chock full of worksheets, assessment design techniques, staff discussion and training activities, and a variety of examples as to how the above can be used for pre-and in-service settings, with or without a statistical focus. This is not a book written for the average consumer, but rather for teacher educators, assessment designers, and K-12 school administrators and staff developers. From an instructor’s perspective, sections of the text would be excellent as an introduction to quantitative research methods for the social sciences."
Teachers College Record, August 2008
"School leaders will find an important, key tool to assessing teacher dispositions. It's been field-tested, so it isn't just theory. An essential guide."
California Bookwatch, August 2007
"The text has been well researched, is standards-based, and includes activities, worksheets, definitions, and rubrics. It addresses a topic that has been a mystery to assessment gurus."
Marilyn K. Troupe, Director, Division of Educator Preparation, Kentucky Education Professional Standards Board
“We have all been through statistics classes and measurement classes, but this text actually helps us apply measurement ideas to the concept of dispositions. It should be extremely helpful to education faculty as well as to their liberal arts and science counterparts who often do not understand the concept of dispositions or why they are important and need to be measured or how one even begins to develop an assessment system that includes such measurements.”
Martha Gage, Director of Teacher Education and Licensure, Kansas State Department of Education

Recommendations