Skip to main content
Log in

Assessing the Straightforwardly-Worded Brief Fear of Negative Evaluation Scale for Differential Item Functioning Across Gender and Ethnicity

  • Published:
Journal of Psychopathology and Behavioral Assessment Aims and scope Submit manuscript

Abstract

The Brief Fear of Negative Evaluation Scale (BFNE; Leary Personality and Social Psychology Bulletin, 9, 371–375, 1983) assesses fear and worry about receiving negative evaluation from others. Rodebaugh et al. Psychological Assessment, 16, 169–181, (2004) found that the BFNE is composed of a reverse-worded factor (BFNE-R) and straightforwardly-worded factor (BFNE-S). Further, they found the BFNE-S to have better psychometric properties and provide more information than the BFNE-R. Currently there is a lack of research regarding the measurement invariance of the BFNE-S across gender and ethnicity with respect to item thresholds. The present study uses item response theory (IRT) to test the BFNE-S for differential item functioning (DIF) related to gender and ethnicity (White, Asian, and Black). Six data sets consisting of clinical, community, and undergraduate participants were utilized (N = 2,109). The factor structure of the BFNE-S was confirmed using categorical confirmatory factor analysis, IRT model assumptions were tested, and the BFNE-S was evaluated for DIF. Item nine demonstrated significant non-uniform DIF between White and Black participants. No other items showed significant uniform or non-uniform DIF across gender or ethnicity. Results suggest the BFNE-S can be used reliably with men and women and Asian and White participants. More research is needed to understand the implications of using the BFNE-S with Black participants.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. To test whether the results of the DIF analyses were dependent on anchor items were-ran the analyses for both gender and ethnicity using items three and eight which did not exhibit LD. The results from the DIF analyses for using items three and eight versus the items described above were identical.

References

  • Adams, C. E., Myers, V. H., Barbera, B. L., & Brantley, P. J. (2011). The role of fear of negative evaluation in predicting depression and quality of life 4 years after bariatric surgery in women. Psychology, 2, 150–154. doi:10.4236/psych.2011.23024.

    Article  Google Scholar 

  • Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57, 289–300. doi:10.2307/2346101.

    Google Scholar 

  • Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246. doi:10.1037/0033-2909.107.2.238.

    Article  PubMed  Google Scholar 

  • Blanchard, J. J., Mueser, K. T., & Bellack, A. S. (1998). Anhedonia, positive and negative affect, and social functioning in schizophrenia. Schizophrenia Bulletin, 24, 413–424. http://schizophreniabulletin.oxfordjournals.org/.

  • Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika, 46, 443–459. doi:10.1007/BF02293801.

    Article  Google Scholar 

  • Breslau, J., Aguilar-Gaxiola, S., Kendler, K. S., Su, M., Williams, D., & Kessler, R. C. (2006). Specifying race-ethnic differences in risk for psychiatric disorder in a USA national sample. Psychological Medicine, 36, 57–68. doi:10.1017/S0033291705006161.

    Article  PubMed Central  PubMed  Google Scholar 

  • Brown, P. J., & Roose, S. P. (2011). Age and anxiety and depressive symptoms: the effect on domains of quality of life. International Journal of Geriatric Psychiatry, 26, 1260–1266.

    Article  PubMed Central  PubMed  Google Scholar 

  • Cai, L. (2008). SEM of another flavor: Two new applications of the supplemented EM algorithm. British Journal of Mathematical and Statistical Psychology, 61, 309–329. doi:10.1348/000711007X249603.

    Article  PubMed  Google Scholar 

  • Cai, L. (2012). flexMIRT™ version 1.88: A numerical engine for multilevel item factor analysis and test scoring. [Computer Software]. Seattle: Vector Psychometric Group.

    Google Scholar 

  • Cai, L., Thissen, D., & du Toit, S. H. C. (2013). IRTPRO:Flexible, multidimensional, multiple categorical IRT modeling [Computer software]. Chicago: Scientific Software International.

    Google Scholar 

  • Carleton, N. R., Collimore, K. C., McCabe, R. E., & Antony, M. M. (2011). Addressing revisions to the brief fear of negative evaluation scale: measuring fear of negative evaluation across anxiety and mood disorders. Journal of Anxiety Disorders, 25, 822–828. doi:10.1016/j.janxdis.2011.04.002.

    Article  PubMed  Google Scholar 

  • Carleton, N. R., McCreary, D. R., Norton, P. J., & Asmundson, G. G. (2006). Brief fear of negative evaluation scale-revised. Depression and Anxiety, 23, 297–303. doi:10.1002/da.20142.

    Article  PubMed  Google Scholar 

  • Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265–289. doi:10.2307/1165285.

    Article  Google Scholar 

  • Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255. doi:10.1207/S15328007SEM0902_5.

    Article  Google Scholar 

  • Collins, K. A., Westra, H. A., Dozois, D. A., & Stewart, S. H. (2005). The validity of the brief version of the fear of negative evaluation scale. Journal of Anxiety Disorders, 19, 345–359. doi:10.1016/j.janxdis.2004.02.003.

    Article  PubMed  Google Scholar 

  • De Ayala, R. J. (2009). The theory and practice of item response theory. New York, NY: The Guilford Press. http://www.guilford.com/.

  • DiNardo, P. A., Brown, T. A., & Barlow, D. H. (1994). Anxiety disorders interview schedule for DSM-IV: Lifetime version (ADIS-IV-L). San Antonio: The Psychological Corporation.

    Google Scholar 

  • Elosua, P. (2011). Assessing measurement equivalence in ordered-categorical data. Psicológica, 32, 403–421. http://www.uv.es/revispsi/paraARCHIVES/2011.html.

  • First, M.B., Spitzer, R.L., Gibbon, M., et al. Structured Clinical Interview for DSM-IV Axis I Disorders (SCID), Clinician Version: Administration Booklet. Washington, DC: American Psychiatric Press; 1996. http://www.appi.org/Home

  • Gilbert, N., & Meyer, C. (2005). Fear of negative evaluation and the development of eating psychopathology: a longitudinal study among nonclinical women. International Journal of Eating Disorders, 37, 307–312. doi:10.1002/eat.20105.

    Article  PubMed  Google Scholar 

  • Haikal, M., & Hong, R. Y. (2010). The effects of social evaluation and looming threat on self-attentional biases and social anxiety. Journal of Anxiety Disorders, 24, 345–352. doi:10.1016/j.janxdis.2010.01.007.

    Article  PubMed  Google Scholar 

  • Hambrick, J. P., Rodebaugh, T. L., Balsis, S., Woods, C. M., Mendez, J. L., & Heimberg, R. G. (2010). Cross-ethnic measurement equivalence of measures of depression, social anxiety, and worry. Assessment, 17, 155–171. doi:10.1177/1073191109350158.

    Article  PubMed  Google Scholar 

  • Harpole, J. K., & Woods, C. M. (2013). The impact of local dependence on differential item functioning. Paper presented at the annual meeting of the Psychometric Society, Arnhem, Netherlands.

  • Harpole, J. K., & Woods, C. M. (2014). The impact of local dependence on differential item functioning. Manuscript submitted for publication.

  • Heimberg, R. G., Brozovich, F. A., & Rapee, R. M. (2010). A cognitive-behavioral model of social anxiety disorder: update and extension. In S. G. Hofmann & P. M. DiBartolo (Eds.), Social anxiety: Clinical, developmental, and social perspectives. New York: Elsevier. doi:10.1016/B978-0-12-375096-9.00015-8.

    Google Scholar 

  • Honaker, J., King, G., & Blackwell, M. (2011). Amelia II: A program for missing data. Journal of Statistical Software, 45, 1–47. http://www.jstatsoft.org

  • Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424–453. doi:10.1037/1082-989X.3.4.424.

    Article  Google Scholar 

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118.

    Article  Google Scholar 

  • Langer, M. (2008). A reexamination of Lord’s Wald test for differential item functioning using item response theory and modern error estimation, Unpublished doctoral dissertation. University of North Carolina at Chapel Hill. http://dc.lib.unc.edu/cdm/singleitem/collection/etd/id/2084.

  • Leary, M. R. (1983). A brief version of the fear of negative evaluation scale. Personality and Social Psychology Bulletin, 9, 371–375. doi:10.1177/0146167283093007.

    Article  Google Scholar 

  • Levinson, C. A., & Rodebaugh, T. L. (2012). Social anxiety and eating disorders: the role of negative social evaluation fears. Eating Behaviors, 13, 27–35. doi:10.1016/j.eatbeh.2011.11.006.

    Article  PubMed Central  PubMed  Google Scholar 

  • Levinson, C. A., & Rodebaugh, T. L. (2011). Validation of the social appearance anxiety scale: factor, convergent, and divergent validity. Assessment, 18, 350–356. doi:10.1177/1073191111404808.

    Article  PubMed  Google Scholar 

  • Levinson, C. A., Rodebaugh, T. L., Menatti, A., & Weeks, J. W. (2013). Development and validation of the Social Exercise and Anxiety Measure (SEAM): assessing fears, avoidance, and importance of social exercise. Journal of Psychopathology and Behavioral Assessment, 35, 244–253. doi:10.1007/s10862-012-9326-1.

    Article  Google Scholar 

  • Lewis, B. A., & O’Neill, H. (2000). Alcohol expectancies and social deficits relating to problem drinking among college students. Addictive Behaviors, 25, 295–299. doi:10.1016/S0306-4603(99)00063-5.

    Article  PubMed  Google Scholar 

  • Liebowitz, M.R. (1987). Social phobia. Modern Problems of Pharmacopsychiatry, 22, 141–173. http://www.karger.com/BookSeries/Home/223929.

  • Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale: Lawrence Erlbaum.

    Google Scholar 

  • Lubke, G. H., & Muthén, B. O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11, 514–534. doi:10.1207/s15328007sem1104_2.

    Article  Google Scholar 

  • MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149. doi:10.1037/1082-989X.1.2.130.

    Article  Google Scholar 

  • Mattick, R. P., & Clarke, J. C. (1998). Development and validation of measures of social phobia, scrutiny, fear, and social interaction anxiety. Behaviour Research and Therapy, 36, 455–470. doi:10.1016/S0005-7967(97)10031-6.

    Article  PubMed  Google Scholar 

  • Maydeu-Olivares, A., & Joe, H. (2005). Limited-and full-information estimation and goodness-of-fit testing in 2n contingency tables. Journal of the American Statistical Association, 100, 1009–1020. doi:10.1198/016214504000002069.

    Article  Google Scholar 

  • Maydeu-Olivares, A., & Joe, H. (2006). Limited information goodness-of-fit testing in multidimensional contingency tables. Psychometrika, 71, 713–732. doi:10.1007/s11336-005-1295-9.

    Article  Google Scholar 

  • Meyer, T. J., Miller, M. L., Metzger, R. L., & Borkovec, T. D. (1990). Development and validation of the Penn State Worry Questionnaire. Behaviour Research and Therapy, 28, 487–495. doi:10.1016/0005-7967(90)90135-6.

    Article  PubMed  Google Scholar 

  • Millsap, R. E. (2011). Statistical approaches to measurement invariance. New York: Routledge. http://www.routledge.com.

  • Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide. Seventh Edition. Los Angeles, CA: Muthén & Muthén. http://www.statmodel.com.

  • Norton, P. J., & Weeks, J. W. (2009). A multi-ethnic examination of socioevaluative fears. Journal of Anxiety Disorders, 23, 904–908. doi:10.1016/j.janxdis.2009.05.008.

    Article  PubMed Central  PubMed  Google Scholar 

  • O’Connor, L. E., Berry, J. W., Weiss, J., & Gilbert, P. (2002). Guilt, fear, submission, and empathy in depression. Journal of Affective Disorders, 71, 19–27. doi:10.1016/S0165-0327(01)00408-6.

    Article  PubMed  Google Scholar 

  • Okazaki, S. (1997). Sources of ethnic differences between Asian American and White American college students on measures of depression and social anxiety. Journal of Abnormal Psychology, 106, 52–60. doi:10.1037/0021-843X.106.1.52.

    Article  PubMed  Google Scholar 

  • Orlando, M., & Thissen, D. (2003). Further investigation of the performance of S-χ 2: an item-fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27, 289–298. doi:10.1177/0146621603027004004.

    Article  Google Scholar 

  • Orlando, M., & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50–64. doi:10.1177/01466216000241003.

    Article  Google Scholar 

  • Core Team, R. (2013). R: A Language and Environment for Statistical Computing [Computer Software]. Vienna: R Foundation for Statistical Computing.

    Google Scholar 

  • Rodebaugh, T. L., Shumaker, E. A., Levinson, C. A., Fernandez, K. C., Langer, J. K., Lim, M. H., & Yarkoni, T. (2013). Interpersonal constraint conferred by generalized social anxiety disorder is evident on a behavioral economics task. Journal of Abnormal Psychology, 22, 39–44. doi:10.1037/a0030975.

    Article  Google Scholar 

  • Rodebaugh, T. L., Woods, C. M., Thissen, D. M., Heimberg, R. G., Chambless, D. L., & Rapee, R. M. (2004). More information from fewer questions: the factor structure and item properties of the original and brief fear of negative evaluation scale. Psychological Assessment, 16, 169–181. doi:10.1037/1040-3590.16.2.169.

    Article  PubMed  Google Scholar 

  • Rodebaugh, T. L., Heimberg, R. G., Brown, P. J., Fernandez, K. C., Blanco, C., Schneier, F. R., & Liebowitz, M. R. (2011). More reasons to be straightforward: Findings and norms for two scales relevant to social anxiety. Journal of Anxiety Disorders, 25, 623–630. doi:10.1016/j.janxdis.2011.02.002.

    Article  PubMed Central  PubMed  Google Scholar 

  • Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York: Wiley. doi:10.1002/9780470316696.indsub.

    Book  Google Scholar 

  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17). Richmond, VA: Psychometric Society. http://www.psychometrika.org.

  • Schaefer, J. L., & Olsen, M. K. (1998). Multiple imputation for multivariate missing-data problems: a data analysts perspective. Multivariate Behavioral Research, 33, 545–571. doi:10.1207/s15327906mbr3304_5.

    Article  Google Scholar 

  • Smith, S. M., Stinson, F. S., Dawson, D. A., Goldstein, R., Huang, B., & Grant, B. F. (2006). Race/ethnic differences in the prevalence and co-occurrence of substance use disorders and independent mood and anxiety disorders: results from the national epidemiologic survey on alcohol and related conditions. Psychological Medicine, 36, 987–998. doi:10.1017/S0033291706007690.

    Article  PubMed  Google Scholar 

  • Stark, S. (2008). MODFIT: Plot theoretical item response functions and examine the fit of dichotomous and polytomous IRT models to response data [Computer program]. Department of Psychology, University of South Florida at Tampa.

  • Steiger, J. H., & Lind, J. (1980). Statistically-based tests for the number of common factors. Paper presented at the Annual Spring Meeting of the Psychometric Society, Iowa City, IA. http://www.statpower.net.

  • Steinberg, L., & Thissen, D. (2006). Using effect sizes for research reporting: examples using item response theory to analyze differential item functioning. Psychological Methods, 11, 402–415. doi:10.1037/1082-989X.11.4.402.

    Article  PubMed  Google Scholar 

  • Thissen, D., Steinberg, L., & Gerrard, M. (1986). Beyond group-mean differences: the concept of item bias. Psychological Bulletin, 99, 118–128. doi:10.1177/014662169501900105.

    Article  Google Scholar 

  • Thissen, D., Steinberg, L., & Kuang, D. (2002). Quick and easy implementation of the Benjamini-Hochberg procedure for controlling the false positive rate in multiple comparisons. Journal of Educational and Behavioral Statistics, 27, 77–83. doi:10.3102/10769986027001077.

    Article  Google Scholar 

  • Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67–111). Hillsdale: Lawrence Erlbaum.

    Google Scholar 

  • Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10. doi:10.1007/BF02291170.

    Article  Google Scholar 

  • Wang, W.-C., Shih, C.-L., & Yang, C.-C. (2009). The MIMIC method with scale purification for detecting differential item functioning. Educational and Psychological Measurement, 69, 713–731. doi:10.1177/0013164409332228.

    Article  Google Scholar 

  • Watson, D., & Friend, R. (1969). Measurement of social-evaluative anxiety. Journal of Consulting and Clinical Psychology, 33, 448–457. doi:10.1037/h0027806.

    Article  PubMed  Google Scholar 

  • Weeks, J. W., Heimberg, R. G., Fresco, D. M., Hart, T. A., Turk, C. L., Schneier, F. R., et al. (2005). Empirical validation and psychometric evaluation of the brief fear of negative evaluation scale in patients with social anxiety disorder. Psychological Assessment, 17, 179–190. doi:10.1037/1040-3590.17.2.179.

    Article  PubMed  Google Scholar 

  • Weeks, J. W., Heimberg, R. G., & Rodebaugh, T. L. (2008). The fear of positive evaluation scale: assessing a proposed cognitive component of social anxiety. Journal of Anxiety Disorders, 22, 44–55. doi:10.1016/j.janxdis.2007.08.002.

    Article  PubMed  Google Scholar 

  • Weeks, J. W., Heimberg, R. G., Rodebaugh, T. L., Goldin, P. R., & Gross, J. J. (2012). Psychometric evaluation of the fear of positive evaluation scale in patients with social anxiety disorder. Psychological Assessment, 24, 301–312. doi:10.1037/a0025723.

    Article  PubMed Central  PubMed  Google Scholar 

  • Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: current approaches and future directions. Psychological Methods, 12, 58–79. doi:10.1037/1082-989X.12.1.58.

    Article  PubMed Central  PubMed  Google Scholar 

  • Woods, C. M. (2009a). Empirical selection of anchors for tests of differential item functioning. Applied Psychological Measurement, 33, 42–57. doi:10.1177/0146621607314044.

    Article  Google Scholar 

  • Woods, C. M. (2009b). Evaluation of MIMIC-model methods for DIF testing with comparison to two-group analysis. Multivariate Behavioral Research, 44, 1–27. doi:10.1080/00273170802620121.

    Article  Google Scholar 

  • Woods, C. M., Cai, L., & Wang, M. (2013). The Langer-improved Wald test for DIF testing with multiple groups: evaluation and comparison to two-group IRT. Educational and Psychological Measurement, 73, 532–547. doi:10.1177/0013164412464875.

    Article  Google Scholar 

  • Zimmerman, M., & Mattia, J. I. (1998). Body dysmorphic disorder in psychiatric outpatients: recognition, prevalence, comorbidity, demographic, and clinical correlates. Comprehensive Psychiatry, 39, 265–270. doi:10.1016/S0010-440X(98)90034-7.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This research was supported in part by the National Institute of Mental Health (NIMH) grant F31-MH096433-01 to Cheri A. Levinson; National Institute of Health (NIH)/NIMH grant MH090308 to Thomas L. Rodebaugh; NIH grant UL1 RR024992 to Washington University St. Louis; National Institute of Aging (NIA) grant T32 AG0030 and NIMH grant T32 MH20004 to Patrick J. Brown; NIMH grant R01 MH064481-01A1 and GlaxoSmithKline Pharmaceuticals grant 101618 to Richard G. Heimberg; NIH grant K02 DA023200 to Carlos Blanco; NIH grant R01 MH064726 to Michael Liebowitz. Carlos Blanco, Franklin Schneier, and Michael Liebowitz were also supported by the New York Psychiatric Institute. The authors would like to thank the Center for Research Methods and Data Analysis at the University of Kansas for assistance in preparing this manuscript.

Conflict of Interest

Jared K. Harpole, Cheri A. Levinson, Carol M. Woods, Thomas L. Rodebaugh, Justin W. Weeks, Patrick J. Brown, Richard G. Heimberg, Andrew R. Menatti, Carlos Blanco, Franklin Schneier, and Michael Liebowitz all declare that there were no conflicts of interest.

Experiment Participants

All data sets that were collected in this study had Institutional Review Board approval and all participants gave informed consent before entering the studies.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jared K. Harpole.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Harpole, J.K., Levinson, C.A., Woods, C.M. et al. Assessing the Straightforwardly-Worded Brief Fear of Negative Evaluation Scale for Differential Item Functioning Across Gender and Ethnicity. J Psychopathol Behav Assess 37, 306–317 (2015). https://doi.org/10.1007/s10862-014-9455-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10862-014-9455-9

Keywords

Navigation