nach oben

Prevention Science

Erschienen in:

01.10.2015

Increasing the Sensitivity of Measures to Change

verfasst von: Carlotta Ching Ting Fok, David Henry

Erschienen in: Prevention Science | Ausgabe 7/2015

Einloggen, um Zugang zu erhalten

Abstract

Little attention is paid in prevention research to the ability of measures to accurately assess change, termed “responsiveness” or “sensitivity to change.” This paper reviews definitions and measures of responsiveness, and suggests five strategies for increasing sensitivity to change, with central focus on prevention research with small samples: (a) improving understandability and cultural validity, (b) assuring that the measure covers the full range of the latent construct being measured, (c) eliminating redundant items, (d) maximizing sensitivity of the device used to collect responses; and (e) asking directly about change. Examples of the application of each strategy are provided. The discussion focuses on using the issues as a checklist for improving measures and the implications of sensitivity to change for prevention research with small samples.

Allen, J., Mohatt, G., Fok, C. C. T., Henry, D., & People Awakening Team. (2009). Suicide prevention as a community development process: Understanding circumpolar youth suicide prevention through community level outcomes. International Journal of Circumpolar Health, 68, 274–291. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2875412/.PubMedCentralCrossRefPubMed

Allen, J., Mohatt, G. V., Fok, C. C. T., Henry, D., Burket, R., & People Awakening Team. (2014). A protective factor model for alcohol abuse and suicide prevention among Alaska Native youth. American Journal of Community Psychology, 54, 125–139. doi:10.1007/s10464-014-9661-3.PubMedCentralCrossRefPubMed

Bartunek, J. M., & Moch, M. K. (1987). First-order, second-order, and third-order change and organization development interventions: A cognitive approach. The Journal of Applied Behavioral Science, 23, 483–500. doi:10.1177/002188638702300404.CrossRef

Beneke, M., & Rasmus, W. (1992). ‘Clinical global impressions’ (ECDEU): Some critical comments. Pharmacopsychiatrie, 25, 171–176.CrossRef

Campbell, D. T., Stanley, J. C., & Gage, N. L. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton, Mifflin and Company.

Cox, E. P. (1980). The optimal number of response alternatives for a scale: A review. Journal of Marketing Research, 17, 407–422. doi:10.2307/3150495.CrossRef

Dahlke, F., Lohaus, A., & Gutzmann, H. (1992). Reliability and clinical concepts underlying global judgments in dementia: Implications for clinical research. Psychopharmacology Bulletin, 28, 425–432.PubMed

Ebesutani, C., Drescher, C. F., Reise, S. P., Heiden, L., Hight, T. L., Damond, J. D., & Young, J. (2012). The Loneliness Questionnaire–Short version: An evaluation of reverse-worded and non-reverse-worded items via item response theory. Journal of Personality Assessment, 94, 427–437. doi:10.1080/00223891.2012.662188.CrossRefPubMed

Fok, C. C. T., Allen, J., Henry, D., Mohatt, G., & People Awakening Team. (2011a). Multicultural Mastery Scale for Youth: Multidimensional assessment of culturally mediated coping strategies. Psychological Assessment. doi:10.1037/a0025505.PubMedCentralPubMed

Fok, C.C.T., Allen, J.A., & Henry, D., The People Awakening Team. (2011b). The Brief Family Relationship Scale: A brief measure of the relationship dimension in family functioning. doi:10.1177/1073191111425856.

Fok, C.C.T., Henry, D., & Allen, J.A. (this issue). Maybe small is too small a term: Introduction to advancing small sample prevention science.

Frank, E., Prien, R. F., Jarrett, R. B., Keller, M. B., Kupfer, D. J., Lavori, P. W., Rush, A. J., & Weissman, M. M. (1991). Conceptualization and rationale for consensus definitions of terms in major depressive disorder. Archives of General Psychiatry, 48, 851–855.CrossRefPubMed

Grice, H. P. (1975). Logic and conversation. In P. Cole & J. L. Morgan (Eds.), Syntax and semantics (Vol. 3, pp. 41–58). New York: Academic.

Guastello, S. J., Aruka, Y., Doyle, M., & Smerz, K. E. (2008). Cross-cultural generalizability of a cusp catastrophe model for binge drinking among college students. Nonlinear Dynamics, Psychology, and Life Sciences, 12, 397–407.PubMed

Guy, W. (1976). Clinical global impressions. In W. Guy (Ed.), ECDEU assessment manual for psychopharmacology, revised (pp. 218–222). Rockville: National Institute of Mental Health.

Guyatt, G., Walter, S., & Norman, G. (1987). Measuring change over time: Assessing the usefulness of evaluative instruments. Journal of Chronic Diseases, 40, 171–178.CrossRefPubMed

Guyatt, G. H., Deyo, R. A., Charlson, M., Levine, M. N., & Mitchell, A. (1989). Responsiveness and validity in health status measurement: A clarification. Journal of Clinical Epidemiology, 42, 403–408.CrossRefPubMed

Hanley, J. A., & McNeil, B. J. (1983). A method of comparing the areas under receiver operating characteristics curves derived from the same cases. Radiology, 148, 839–843.CrossRefPubMed

Hays, R. D., & Hadorn, D. (1992). Responsiveness to change: An aspect of validity, not a separate dimension. Quality of Life Research, 1, 73–75.CrossRefPubMed

Henry, D., Pavuluri, M. N., Youngstrom, E., & Birmaher, B. (2008). Accuracy of brief and full forms of the child mania rating scale. Journal of Clinical Psychology, 64, 1–14. doi:10.1002/jclp.20464.CrossRef

Himadi, W. G., Boice, R., & Barlow, D. H. (1986). Assessment of agoraphobia—II: Measurement of clinical change. Behaviour Research and Therapy, 24, 321–332.CrossRefPubMed

Jackson, T., McKenzie, J., & Hobfoll, S. E. (2000). Communal aspects of self-regulation. In M. Boekaerts, P. R. Pintrich, & M. Zeider (Eds.), Handbook of self-regulation. San Diego: Academic.

Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12.CrossRefPubMed

Kristal, A. R., Beresford, S. A., & Lazovich, D. (1994). Assessing change in diet-intervention research. The American Journal of Clinical Nutrition, 59, 185S–189S.PubMed

Lewin, K. (1947). Frontiers in group dynamics. Human Relations, 1, 143–153.CrossRef

Liang, M. H., Fossel, A. H., & Larson, M. G. (1990). Comparisons of five health status instruments for orthopaedic evaluation. Medical Care, 28, 632–642.CrossRefPubMed

Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3, 85–106.PubMed

Metz, C. E. (1978). Basic principles of ROC analysis. Seminars in Nuclear Medicine, 8, 283–298.CrossRefPubMed

Mohatt, G. V., Fok, C. C. T., Henry, D., Allen, J., & People Awakening Team. (2014). Feasibility of a community intervention for the prevention of suicide and alcohol abuse with Yup’ik Alaska Native youth: The Elluam Tungiinun and Yupiucimta Asvairtuumallerkaa studies. American Journal of Community Psychology, 54, 153–169. doi:10.1007/s10464-014-9646-2.PubMedCentralCrossRefPubMed

Norman, G. R., Stratford, P., & Regehr, G. (1997). Methodological problems in the retrospective computation of responsiveness to change: The lesson of Cronbach alpha. Journal of Clinical Epidemiology, 50, 869–879.CrossRefPubMed

Pavuluri, M. N., Henry, D., Naylor, M., Carbray, J., Sampson, G., & Janicak, P. G. (2004). A pharmacotherapy algorithm for stabilization and maintenance of pediatric bipolar disorder. Journal of American Academy of Child and Adolescent Psychiatry, 43, 859–867.CrossRef

Pavuluri, M. N., Henry, D. B., Devineni, B., Carbray, J. A., & Birmaher, B. (2006). Child Mania Rating Scale: Development, reliability, and validity. Journal of the American Academy of Child and Adolescent Psychiatry, 45, 550–560. doi:10.1097/01.chi.0000205700.40700.50.CrossRefPubMed

Pearlin, L. I., Menaghan, E. G., Lieberman, M. A., & Mullan, J. T. (1981). The stress process. Journal of Health & Social Behavior, 22, 337–356.CrossRef

Prochaska, J. O., & DiClemente, C. C. (1983). Stages and processes of self-change of smoking: Toward an integrative model of change. Journal of Consulting and Clinical Psychology, 51, 390–395.CrossRefPubMed

Puhan, M. A., Bryant, D., Guyatt, G. H., Heels-Ansdell, D., & Schünemann, H. J. (2005). Internal consistency reliability is a poor predictor of responsiveness. Health and Quality of Life Outcomes, 3, 1–8. doi:10.1186/1477-7525-3-33.CrossRef

Raudenbush, S. W., & Liu, X. (2001). Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change. Psychological Methods, 6, 387–401. doi:10.1037/1082-989X.6.4.387.CrossRefPubMed

Rizopoulos, D. (2006). ltm: An R package for latent variable modelling and item response theory analyses. Journal of Statistical Software, 17, 1–25.CrossRef

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 34, 100–114.

Sato, T. L., Turnbull, C. D., Davidson, J. R. T., & Madakasira, S. (1984). Depressive illness and placebo response. International Journal of Psychiatry Medicine, 14, 171–179.CrossRef

Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93–105.CrossRef

Schwarz, N., Knäuper, B., Hippler, H. J., Noelle-Neumann, E., & Clar, F. (1991). Rating scales: Numeric values may change the meaning of scale labels. Public Opinion Quarterly, 55, 570–582.CrossRef

Sharma, E., Beck, K. H., & Clar, P. I. (2013). Social context of smoking hookah among college students: Scale development and validation. Journal of American College Health, 61, 204–211. doi:10.1080/07448481.2013.787621.CrossRefPubMed

Spearing, M. K., Post, R. M., Levericha, G. S., Brandt, D., & Nolen, W. (1997). Modification of the Clinical Global Impressions (CGI) scale for use in bipolar illness (BP): The CGI-BP. Psychiatric Research, 73, 159–171.CrossRef

Spybrook, J., Raudenbush, S. W., Congdon, R., & Martínez, A. (2009). Optimal design for longitudinal and multilevel research: Documentation for the “Optimal Design” software. New York: William T. Grant Foundation.

Streiner, D. L., & Norman, G. R. (1995). Health measurement scales: A practical guide to their development and use (3rd ed.). Oxford: Oxford University Press.

Wakschlag, L.S., Briggs-Gowan, M.J., Choi, S.W., Nichols, S.R., Kestler, J., Burns, J.L., … & Henry, D. (2014). Advancing a multidimensional, developmental spectrum approach to preschool disruptive behavior. Journal of the American Academy of Child & Adolescent Psychiatry, 53(1), 82–96.

Wästberg, B. A., Haglund, L., & Eklund, M. (2012). The work environment impact scale–self-rating (WEIS-SR) evaluated in primary health care in Sweden. Work: A Journal of Prevention, Assessment and Rehabilitation, 42, 447–457.

West, A. E., Celio, C. I., Henry, D. B., & Pavuluri, M. N. (2011). Child mania rating scale-parent version: A valid measure of symptom change due to pharmacotherapy. Journal of Affective Disorders, 128, 112–119.PubMedCentralCrossRefPubMed

Zimmerman, M., McGlinchey, J. B., Posternak, M. A., Friedman, M., Attiullah, N., & Boerescu, D. (2006). How should remission from depression be defined? The depressed patient’s perspective. American Journal of Psychiatry, 163, 148–150.CrossRefPubMed

Titel: Increasing the Sensitivity of Measures to Change
verfasst von: Carlotta Ching Ting Fok
David Henry
Publikationsdatum: 01.10.2015
Verlag: Springer US
Erschienen in: Prevention Science / Ausgabe 7/2015
Print ISSN: 1389-4986
Elektronische ISSN: 1573-6695
DOI: https://doi.org/10.1007/s11121-015-0545-z

Springer Medizin

Abstract

Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten

Weitere Artikel der Ausgabe 7/2015

Standards of Evidence for Efficacy, Effectiveness, and Scale-up Research in Prevention Science: Next Generation

Sample Size Considerations in Prevention Research Applications of Multilevel Modeling and Structural Equation Modeling

Bayesian Methods for Prevention Research

Methods for Multilevel Ordinal Data in Prevention Research

The Intersection Between Prevention Science and Evidence-Based Policy: How the SPR Evidence Standards Support Human Services Prevention Programs

Research Designs for Intervention Research with Small Samples II: Stepped Wedge and Interrupted Time-Series Designs