Appendix: Reliability and Validity of Assessment Measures

Introduction

A great deal of effort was expended to ensure both the reliability and validity of these assessments. Internal consistency was used as the approach in measuring reliability. Each assessment was meant to be administered multiple times to skilled practitioners to rate organizations with which they had been associated. During each assessment's development, the internal consistency of each construct was measured using Cronbach's coefficient alpha, which is the most common statistic used to quantify the degree to which all items within a given construct measure the same attribute. Based on a review of scholarly literature, a minimum alpha level of 0.7 was applied. Only items that were included in the final alpha calculation are included in the instrument. The range across all constructs of the performance improvement maturity assessment was between 0.8147 and 0.9085. The range across all constructs for the change-readiness assessment was between 0.8212 and 0.9085

To assess face validity, each instrument was reviewed by several subject matter experts (SMEs) to ensure the understandability of all items. For construct validity, a separate set of trained performance-improvement practitioners reviewed the instruments and felt that each item really measured what it was intended to measure. Concurrent validity was difficult to determine because most other maturity models are simply descriptive in nature and do not include an assessment instrument. Concurrent validity also was challenging to determine for the change-readiness assessment because its results could not be compared with those of other instruments within the same organization. During the test for internal consistency, the overall findings were compared with the practitioner estimate of the respective organization's maturity and change-readiness levels. The alignment of these expert estimates with the results of the instrument was used to confirm concurrent validity, as prescribed by Litwin. Assessing the predictive validity of each assessment is an ongoing activity that we hope this book's readers will contribute to by providing the results of any use of these instruments along with comments on efficacy relating to predicting the current maturity state and change-readiness level of the organization. Please provide your results (feel free to mask any organizational information as you deem appropriate) to bookfeedback@novaces.com.

Performance-Improvement Maturity Assessment

All items in this survey will be assessed using a five-item scale ranging from "Strongly Disagree" to "Strongly Agree."

1. My organization has a structured approach for solving complex quality-related problems.

   • Strongly Disagree

   • Disagree

   • Neutral

   • Agree

   • Strongly Agree

2. Within my organization, a plan has been developed that outlines the path forward for deploying continuous process improvement.

   • Strongly Disagree

   • Disagree

   • Neutral

   • Agree

   • Strongly Agree

3. Leaders in my organization follow up to ensure that improvements are implemented and applied over time.

   • Strongly Disagree

   • Disagree

   • Neutral

   • Agree

   • Strongly Agree

4. My organization actively pursues opportunities to quickly implement obvious and low-risk improvements.

   • Strongly Disagree

   • Disagree

   • Neutral

   • Agree

   • Strongly Agree

5. My organization regularly communicates improvement activity successes.

   • Strongly Disagree

   • Disagree

   • Neutral

   • Agree

   • Strongly Agree

6. Action has ...