Internet administration of self-report measures commonly used in research on social anxiety disorder: A psychometric evaluation

Abstract

The Internet has become increasingly popular as a way to administer self-report questionnaires, especially in the field of Internet delivered psychological treatments. Collecting questionnaire data over the Internet has advantages, such as ease of administration, and automated scoring. However, psychometric properties cannot be assumed to be identical to the paper-and-pencil versions. The aim of this study was to test the equivalence of paper-and-pencil and Internet administered versions of self-report questionnaires used in social phobia research. We analyzed data from two trials in which samples were recruited in a similar manner. One sample (N = 64) completed the paper-and-pencil version of questionnaires and the second sample (N = 57) completed the same measures online. We included the Liebowitz Social Anxiety Scale-self-assessment (LSAS-SR), the Social Interaction and Anxiety Scale (SIAS), and the Social Phobia Scale (SPS) as measures of social anxiety. Also included were the Montgomery Åsberg Depression Rating Scale-self-assessment (MADRS-S), the Beck Anxiety Inventory (BAI), and the Quality of Life Inventory (QOLI). Results showed equivalent psychometric properties across administration formats. Cronbach’s α ranged between 0.77 and 0.94. There was an indication of a somewhat higher construct validity when participants filled out questionnaires using paper-and-pencil. We conclude that the LSAS-SR, SIAS, and SPS can be administered via the Internet with maintained psychometric properties.

Introduction

With the advent of the Internet and the development of Internet-administered psychological treatments (Andersson et al., 2005, Spek et al., 2007), researchers and clinicians are increasingly using the Internet to administer self-report questionnaires. While the number of controlled trials showing promising effects of guided Internet-delivered Cognitive Behavior Therapy (CBT) is increasing rapidly (Andersson et al., 2008), fewer studies in the field of clinical psychology have focused on the use of the Internet for information gathering purposes. Internet administration of questionnaires can have some advantages. For example, they can be completed in the patients own home with instant data storing, which may reduce risks that are associated with paper questionnaires, e.g., lost in the mail. In addition, the internal drop-out rate in terms of uncompleted items can be reduced by making it impossible to skip items. Furthermore, scripts can be used to calculate scores, which can be easily exported to statistics software. However, when self-report questionnaires are transferred to the Internet, psychometric properties cannot be assumed to be identical to their paper version equivalents (Andersson et al., 2008, Buchanan, 2003). Several studies have investigated the effect of administration mode of questionnaires in various contexts. Internet-administered questionnaires are strongly correlated with scores obtained using paper-and-pencil versions and internal consistencies tend to be equal (Andersson et al., 2003, Austin et al., 2006, Carlbring, Brunt, et al., 2007, Richter et al., 2008). Still, it has been suggested that each Internet-administered questionnaire should be evaluated separately (Buchanan, 2003), in particular when establishing norms. One large clinical population for which Internet administration of questionnaires could seem attractive are people suffering from social phobia, or social anxiety disorder (Erwin et al., 2004, Shepherd and Edelmann, 2005). However, psychometric research on this topic is scarce and separate norms might be needed. One reason for doing so is to investigate whether respondents consistently give higher severity ratings when giving their answers via the Internet, which was one of the first reported systematic differences between Internet-administered questionnaires and the original versions (Joinson, 1998). To our knowledge, very little research has been conducted to test the psychometric properties of Internet-administered questionnaires commonly used in the assessment of social anxiety disorder. Given the increasing use of Internet-administered social anxiety disorder questionnaires, due to the development of Internet-delivered CBT and the high prevalence of the disorder, we find it very important to evaluate their psychometric properties.

The aim of this study was to test the equivalence of paper-and-pencil and Internet administered versions of self-report questionnaires that measure social anxiety disorder, general anxiety, depression, and quality of life. More specifically, we compared mean scores, internal consistencies and construct validity across administration formats. To estimate construct validity we used the criteria for intercorrelations proposed by Campbell and Fiske (1959).