On the identification of clinically significant client changes: Reinterpretation of Jacobson's cut scores

Abstract

“Cut points” or “cut scores” play a central role in Jacobson's popular method of identifying clinically significant changes in psychotherapy. When pre- and posttherapy scores of a client are on different sides of one of these cut scores, the change is considered clinically significant, provided that it is also “reliable” (i.e., not due to measurement errors). This article critically examines the meanings and implications of these cut scores. Contrary to popular beliefs, they are generallynot the test scores for which the probability of “belonging” to the Functional population is equal to the probability of “belonging” to the Dysfunctional population. When the Functional population distribution is above that of the Dysfunctional population, persons scoring above these cut scores can, in fact, have much greater probabilities of belonging to the Dysfunctional than to the Functional population. Goals of Jacobson's method can be attained only with Bayesian methods. Bayesian modifications of Jacobson's cut scores are proposed, although their use is limited by the availability of relevant base rates. Bayesian methods (a) can provide information about the probability that an individual belongs to each population, given his (her) score, and (b) are expected to yield total misdiagnosis rates that are many times lower than those of Jacobson's method. Users of Jacobson's method are cautioned against interpreting ratios of likelihoods as if they were ratios of posterior probabilities.