Item reduction based on rigorous methodological guidelines is necessary to maintain validity when shortening composite measurement scales

Abstract

Objective

To review current practice and update guidelines for the methodology of shortening composite measurement scales (CMSs).

Study Design and Setting

A literature review gathered data on 91 shortening processes from 1995 to 2009. The validity of the initial CMS, the shortening methods, and the validity of the derived short-form scales were examined. The results were compared with those from a previous literature review (articles from 1985 to 1995) to develop updated guidelines for CMS shortening.

Results

The literature review revealed a persisting lack of use of rigorous methodology for CMS shortening. Of the 91 cases of CMS shortening, 36 combined a content approach and a statistical approach; 45 used only a statistical approach and 10 (11%) only a content approach. The updated guidelines deal with the validity and conceptual model of the initial CMS, the preservation of content and psychometric properties during shortening, the selection of items, and the validation of the short form.

Conclusion

Item reduction based on a rigorous methodology is necessary if the short-form instrument aims to maintain the validity and other measurement properties of the parent instrument, which in turn supports application in research and clinical practice.

Introduction

Many health constructs are too complex to be captured by direct measurement. When these constructs need to be examined, one of the most popular methods requires the use of a composite measurement scale (CMS). The CMS generally consists of items or questions that assess one or several attributes scored by a scale.

Over the years, the measurement of health constructs has led to the production of a large number of scales, with often a high number of items [1].

The burden of long scales and the increasing need for multiple instruments in the same study have logically created a strong need to shorten CMSs. To shorten a CMS consists in reducing its number of items while trying to preserve or improve its psychometric properties. The methodology to develop new CMSs is well documented [2] and includes consideration of different psychometric properties but has limited applicability as a means of informing the shortening of instruments. Guidelines for shortening existing CMSs are scarce. In 1997, Coste et al. [3] noted that most articles reporting on scale shortening lacked rigorous methodology: shortening processes were often inadequately conceptualized, and excessive credit was given to statistical techniques. The authors recommended carefully choosing the original scale according to its content, its possibility for shortening, and its psychometric properties; focusing on criterion validity if the original scale can be considered the gold standard and if not, preferring an expert-based approach to content validity, possibly helped by statistical considerations; and finally, performing a validation study in an independent sample. In 2000, Smith et al. [4] also noted methodological pitfalls concerning CMS shortening in psychology and recommended using a validated original scale, clarifying the intended use of the short form, estimating a priori the properties of the short form to balance resource or time savings against the loss of validity, preserving content, and using an independent sample to validate the short form. In 2002, Stanton et al. [5] also observed a lack of methodological recommendations for scale reduction. The authors argued that focusing on internal consistency should be avoided and proposed a set of item “quality indices” to help conceptualize the competing issues that influence item retention decisions. However, new approaches and new statistical methods such as item response theory (IRT) are now used for scale shortening [6], but we lack published recommendations integrating these methods. Another shortfall in current guidelines concerns content validity; retaining content during the shortening process is important, but only methodological guidelines for developing new CMSs focus on content analysis [7], [8].

This article aims to describe the methodology currently used to shorten CMSs through a literature review and to compare with a previous review for proposing updated and structured guidelines for CMS shortening.