Skip to main content
Erschienen in: Quality of Life Research 12/2020

11.07.2020

The importance of development standards for anchoring vignettes: an illustrative example from pediatric localized scleroderma quality of life

verfasst von: Christina K. Zigler, Heidi Jacobe, Kaveh Ardalan, Theresa M. Coles, Suzane Lane, Kaila L. Schollaert, Kathryn S. Torok

Erschienen in: Quality of Life Research | Ausgabe 12/2020

Einloggen, um Zugang zu erhalten

Abstract

Purpose

Anchoring vignettes (AVs) are a promising measurement technique to reduce bias in patient-reported outcome (PRO) measures by helping researchers understand differences in how individuals and groups interpret response options. However, little attention has been paid to ensure quality development of AVs, and their performance has not been well assessed in pediatric populations. In this study, we explore the application of a rigorous development process for AVs based upon current standards for PROs, as well as feasibility of AVs when administered to children and adolescents.

Methods

We developed AVs using a rigorous, patient-centered mixed methods process including three phases: (1) development, (2) a pilot study, and (3) a field test. Our proposed process included the generation of a conceptual framework based on the PRO, the Localized Scleroderma Quality of Life Instrument, and numerous vignette-specific considerations. We qualitatively explored readability and comprehension of the AVs (pilot study) and then analyzed ranking patterns within vignette sets (field test).

Results

Four sets of four vignettes were developed. Revisions were suggested at each phase of development. The pilot study demonstrated that children ≥ 10 years had no trouble indicating understanding of the AVs. In the field test, although appropriate rankings of vignettes were generally demonstrated by participants, the percentage of tied rankings was higher than expected in this pediatric group.

Conclusions

This work supports the need for rigorous developmental standards for AVs, as each stage of development suggested revisions. Additionally, AVs showed initial promise for use with pediatric populations; general feasibility and understanding were supported.
Literatur
1.
Zurück zum Zitat U. S. Department of Health and Human Services, F. a. D. A., Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, and Center for Devices and Radiological Health. (2006). Guidance for industry: Patient-reported outcome measures: use in medical product development to support labeling claims: Draft guidance. Health Qual Life Outcomes, 4, 79. https://doi.org/10.1186/1477-7525-4-79.CrossRef U. S. Department of Health and Human Services, F. a. D. A., Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, and Center for Devices and Radiological Health. (2006). Guidance for industry: Patient-reported outcome measures: use in medical product development to support labeling claims: Draft guidance. Health Qual Life Outcomes, 4, 79. https://​doi.​org/​10.​1186/​1477-7525-4-79.CrossRef
2.
Zurück zum Zitat Reeve, B. B., Wyrwich, K. W., Wu, A. W., Velikova, G., Terwee, C. B., Snyder, C. F., et al. (2013). ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Quality of Life Research, 22(8), 1889–1905. https://doi.org/10.1007/s11136-012-0344-y.CrossRefPubMed Reeve, B. B., Wyrwich, K. W., Wu, A. W., Velikova, G., Terwee, C. B., Snyder, C. F., et al. (2013). ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Quality of Life Research, 22(8), 1889–1905. https://​doi.​org/​10.​1007/​s11136-012-0344-y.CrossRefPubMed
3.
Zurück zum Zitat Eignor D.R. (2013). The standards for educational and psychological testing: American psychological association. Eignor D.R. (2013). The standards for educational and psychological testing: American psychological association.
4.
Zurück zum Zitat King, G., Murray, C. J., Salomon, J. A., & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 98(1), 191–207.CrossRef King, G., Murray, C. J., Salomon, J. A., & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 98(1), 191–207.CrossRef
6.
Zurück zum Zitat King, G., & Wand, J. (2007). Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes. Political Analysis, 15(1), 46–66.CrossRef King, G., & Wand, J. (2007). Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes. Political Analysis, 15(1), 46–66.CrossRef
7.
Zurück zum Zitat Schwartz, C. E. (2010). Applications of response shift theory and methods to participation measurement: A brief history of a young field. Archives of Physical Medicine Rehabilitation, 91(9), S38–S43.CrossRef Schwartz, C. E. (2010). Applications of response shift theory and methods to participation measurement: A brief history of a young field. Archives of Physical Medicine Rehabilitation, 91(9), S38–S43.CrossRef
8.
Zurück zum Zitat Bolt, D. M., Lu, Y., & Kim, J.-S. (2014). Measurement and control of response styles using anchoring vignettes: A model-based approach. Journal Psychological Methods, 19(4), 528.CrossRef Bolt, D. M., Lu, Y., & Kim, J.-S. (2014). Measurement and control of response styles using anchoring vignettes: A model-based approach. Journal Psychological Methods, 19(4), 528.CrossRef
9.
Zurück zum Zitat Primi R., Zanon C., Santos D., De Fruyt F., John O.P. (2016). Anchoring vignettes. European Journal of Psychological Assessment. Primi R., Zanon C., Santos D., De Fruyt F., John O.P. (2016). Anchoring vignettes. European Journal of Psychological Assessment.
10.
Zurück zum Zitat Vonkova, H., Bendl, S., & Papajoanu, O. (2017). How students report dishonest behavior in school: Self-assessment and anchoring vignettes. The Journal of Experimental Education, 85(1), 36–53.CrossRef Vonkova, H., Bendl, S., & Papajoanu, O. (2017). How students report dishonest behavior in school: Self-assessment and anchoring vignettes. The Journal of Experimental Education, 85(1), 36–53.CrossRef
11.
Zurück zum Zitat Vonkova H., Zamarro G., Deberg V., Hitt C. (2015). Comparisons of student perceptions of teacher’s performance in the classroom: Using parametric anchoring vignette methods for improving comparability. EDRE Working Paper No. 2015–01 Vonkova H., Zamarro G., Deberg V., Hitt C. (2015). Comparisons of student perceptions of teacher’s performance in the classroom: Using parametric anchoring vignette methods for improving comparability. EDRE Working Paper No. 2015–01
12.
Zurück zum Zitat Marksteiner T., Kuger S., Klieme E. (2018). The potential of anchoring vignettes to increase intercultural comparability of non-cognitive factors. Assessment in Education: Principles, Policy & Practice, 1–21. Marksteiner T., Kuger S., Klieme E. (2018). The potential of anchoring vignettes to increase intercultural comparability of non-cognitive factors. Assessment in Education: Principles, Policy & Practice, 1–21.
13.
Zurück zum Zitat Vonkova, H., & Hrabak, J. (2015). The (in) comparability of ICT knowledge and skill self-assessments among upper secondary school students: The use of the anchoring vignette method. Computers Education, 85, 191–202.CrossRef Vonkova, H., & Hrabak, J. (2015). The (in) comparability of ICT knowledge and skill self-assessments among upper secondary school students: The use of the anchoring vignette method. Computers Education, 85, 191–202.CrossRef
15.
Zurück zum Zitat Primi R., Santos D., John O.P., De Fruyt F., Hauck-Filho N. (2017). Dealing with person differential item functioning in social-emotional skill assessment using anchoring vignettes. In The Annual Meeting of the Psychometric Society, (pp. 275–286): Springer Primi R., Santos D., John O.P., De Fruyt F., Hauck-Filho N. (2017). Dealing with person differential item functioning in social-emotional skill assessment using anchoring vignettes. In The Annual Meeting of the Psychometric Society, (pp. 275–286): Springer
16.
Zurück zum Zitat Grol-Prokopczyk, H., Verdes-Tennant, E., McEniry, M., & Ispány, M. (2015). Promises and pitfalls of anchoring vignettes in health survey research. Demography, 52(5), 1703–1728.CrossRef Grol-Prokopczyk, H., Verdes-Tennant, E., McEniry, M., & Ispány, M. (2015). Promises and pitfalls of anchoring vignettes in health survey research. Demography, 52(5), 1703–1728.CrossRef
20.
21.
Zurück zum Zitat Zigler, C. K., Ardalan, K., Hernandez, A., Calliendo, A. E., Magee, K., Mann, C. M., et al. (2020). Exploring the impact of paediatric localized scleroderma on health-related quality of life: Focus groups with youth and caregivers. British Journal of Dermatology. https://doi.org/10.1111/bjd.18879.CrossRefPubMed Zigler, C. K., Ardalan, K., Hernandez, A., Calliendo, A. E., Magee, K., Mann, C. M., et al. (2020). Exploring the impact of paediatric localized scleroderma on health-related quality of life: Focus groups with youth and caregivers. British Journal of Dermatology. https://​doi.​org/​10.​1111/​bjd.​18879.CrossRefPubMed
22.
Zurück zum Zitat Hopkins, D. J., & King, G. (2010). Improving anchoring vignettes: Designing surveys to correct interpersonal incomparability. Public Opinion Quarterly, 74(2), 201–222.CrossRef Hopkins, D. J., & King, G. (2010). Improving anchoring vignettes: Designing surveys to correct interpersonal incomparability. Public Opinion Quarterly, 74(2), 201–222.CrossRef
23.
Zurück zum Zitat Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12(3), 229–238.CrossRef Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12(3), 229–238.CrossRef
24.
Zurück zum Zitat Wand, J. (2013). Credible comparisons using interpersonally incomparable data: Nonparametric scales with anchoring vignettes. American Journal of Political Science, 57(1), 249–262.CrossRef Wand, J. (2013). Credible comparisons using interpersonally incomparable data: Nonparametric scales with anchoring vignettes. American Journal of Political Science, 57(1), 249–262.CrossRef
25.
Zurück zum Zitat Wand J., King G., Lau O. (2011). Anchors: Software for anchoring vignette data. Journal of Statistical Software. Wand J., King G., Lau O. (2011). Anchors: Software for anchoring vignette data. Journal of Statistical Software.
27.
Zurück zum Zitat Murray, K. J., & Laxer, R. M. (2002). Scleroderma in children and adolescents. Rheumatic Diseases Clinics of North America, 28(3), 603–624.CrossRef Murray, K. J., & Laxer, R. M. (2002). Scleroderma in children and adolescents. Rheumatic Diseases Clinics of North America, 28(3), 603–624.CrossRef
Metadaten
Titel
The importance of development standards for anchoring vignettes: an illustrative example from pediatric localized scleroderma quality of life
verfasst von
Christina K. Zigler
Heidi Jacobe
Kaveh Ardalan
Theresa M. Coles
Suzane Lane
Kaila L. Schollaert
Kathryn S. Torok
Publikationsdatum
11.07.2020
Verlag
Springer International Publishing
Erschienen in
Quality of Life Research / Ausgabe 12/2020
Print ISSN: 0962-9343
Elektronische ISSN: 1573-2649
DOI
https://doi.org/10.1007/s11136-020-02575-6

Weitere Artikel der Ausgabe 12/2020

Quality of Life Research 12/2020 Zur Ausgabe