Evaluating audio computer assisted self-interviews in urban south African communities: evidence for good suitability and reduced social desirability bias of a cross-sectional survey on sexual behaviour

Sexual behavioural research largely depends on self-reported data in the absence of more objective biomarkers. Ensuring the validity of these data is a common challenge faced by traditional survey methods. Specifically, self-reports of sensitive issues such as engagement in multiple, concurrent sexual relationships may be subject to social desirability bias (SDB)— or the inclination to report behaviours that are viewed in a favourable way by society. For sexual behaviours the bias is often gendered because of sexual double standards in society, which frequently lead men to over-report and women to under-report certain behaviours [1‐4].

For several countries in Africa, young unmarried women tend to report a later age for their first sexual experience compared to young unmarried men [5]. In regard to sexual concurrency —overlapping sexual relationships where a new sexual relationship starts before an existing relationship ends— several studies from Africa indicate that men are far more likely to report having multiple sexual partners at once, or extra-marital partners, compared to women [6‐9]. Women have also been found to report fewer non-spousal partners in the past year compared to men [10]. Gendered discrepancies also exist in reporting of condom use in Africa. Men consistently are more likely to report having ever used condoms [6] and always using condoms with all of their partners [9, 11] than women. A study conducted specifically to test SDB among married couples, found that a higher percentage of husbands reported that they used condoms with their spouse, than their wives did [12].

When participants consciously modify their answers to project a socially ‘appropriate’ response, this is called impression management [13], and the effects of it can theoretically be ameliorated by changing the design of the study. To combat the problem of SDB, several studies have investigated how mode of inquiry might reduce this phenomenon. Audio Computer-Assisted Self-Interviewing (ACASI) techniques have emerged as a way to curtail SDB because they allow participants to answer questions in privacy by using a computer to read the question, while simultaneously listening to an audio recording of the questions being asked. Several studies have attempted to evaluate the accuracy of ACASI-generated survey results and have produced disparate results. For example, studies done in India have shown that women reported forcibly being touched [14] and domestic violence less often in ACASI compared to traditional Face-to-face-interviewing (FTFI) methods [15]. In the U.S. a study was conducted to see if reported condom use in an ACASI could be substantiated by a Y-chromosome polymerase chain reaction (Yc-PCR) test to detect sperm in vaginal fluid. It demonstrated that even with ACASI methods, there were still inconsistencies between the Yc-PCR and reported results [16]. However, in the African context ACASI methods seemed to precipitate the reporting of more sexual behaviours [17]. Increases in the reporting of number of sexual partners [18], two or more sex partners at once [19], and for women, having had sex with a relative, stranger, or older man [20] in ACASI compared to FTFI seem to indicate that participants in Africa may be more comfortable answering questions about sexual behaviour when they have access to greater privacy.

These discrepant findings about ACASI compared to FTFI are indicative of the need for more context-specific research on ACASI methods. Particularly, since little is known about the validity of ACASI and its suitability for the context of urban, disadvantaged communities in South Africa, where the burden of HIV/AIDS is among the highest in the world. We conducted a cross-sectional survey in three urban communities of Cape Town, South Africa that are known to have a high prevalence of HIV. As a part of the survey, we asked participants questions about their interview experience. In this paper, we report the results of these questions and we compare key summary statistics known to be highly sensitive to SDB from our survey with those from the Demographic and Health Surveys (DHS), conducted in Lesotho, Swaziland, and Zimbabwe [21‐23].

After exclusions, we were able to evaluate the ACASI user experience for 798 participants. Our study population was comprised mostly of women (68%) and participants identifying as black (75%). The median age of participants was 35.5 years old (IQR 26–46). Most participants chose to take the survey in isiXhosa language (66.5%), followed by Afrikaans (21.9%) and English (11.4%). Table 1 displays the results from our questions about interview modality. A majority of participants found that using ACASI was easy (‘very easy’: 67.2% and ‘somewhat easy’: 18.7%). Most participants also found that the ACASI method offered a certain level of privacy (‘very private’: 88.7% and ‘somewhat private’: 7.6%). Eighty-six percent of the participants claimed to have answered all questions truthfully. When questioned about their preferred mode of inquiry 82.5% said they would like to do similar surveys about sexual behaviour using a touch screen computer with ACASI.

Table 2 shows the results of the univariate models, developed to identify the groups of participants most likely to have a positive/negative user experience. Female participants (OR 1.68; 95% CI: 1.23–2.29) and those with a secondary (OR 2.00; 95% CI: 1.48–2.71) or tertiary education (OR 7.06; 95% CI: 1.62–30.82) were more likely to find the ACASI questionnaire easy to use, while those who were in the oldest age group (OR 0.37; 95% CI: 0.23–0.58 and the unemployed (OR 0.69; 95% CI: 0.47–1.01) were less likely to find it easy. Participants aged 25–39 (OR 2.10; 95% CI: 1.16–3.81) were more likely to view their experience as being confidential. Coloured participants were less likely to think our methods granted privacy (OR 0.46; 95% CI: 0.29–0.74). The oldest age group (OR 1.81; 95% CI: 1.06–3.09) and those with a secondary education (OR 1.53; 95% CI: 1.02–2.30) were the groups most likely to answer all questions truthfully. Participants with a secondary education (OR 2.40; 95% CI: 1.64–3.50) were the most likely to choose touch screen ACASI survey as their preferred mode of inquiry, while those who belonged to the oldest age group (OR 0.44; 95% CI: 0.24–0.79) were less likely to choose this modality.

Table 3 provides a comparison of different reported sexual behaviours, by men and women, in our survey with three different FTFI-based surveys in southern Africa. In this comparison, we used only 503 of our original participants, creating a subset of participants with demographics most closely resembling the DHS sample population. For all sexual behaviours examined, our survey produces smaller male–female prevalence ratios than the DHSs in Lesotho, Swaziland, and Zimbabwe. The ratio of men to women reporting greater than 2 partners in the past year in our survey was 3.5 (95% CI: 2.3–5.2) compared to 10.6 (95% CI: 7.3–15.2) in Lesotho, 20.0 (95% CI: 7.1–72.9) in Swaziland, and 22.0 (95% CI: 9.4–49.0) in Zimbabwe. For the prevalence of greater than 2 lifetime partners, our survey produced a ratio of 1.2 (95% CI: 1.0–1.3) compared to 2.1 (95% CI: 2.0–2.2) in Swaziland and 4.9 (95% CI: 4.6–5.2) in Zimbabwe. Our survey also produced a ratio of 2.6 (95% CI: 1.9–3.6) for respondents reporting a concurrent relationship in the past year (Swaziland 15.4, 95% CI: 10.7–21.7; Zimbabwe 11.3, 95% CI: 8.8–16.0). Finally, Table 3 also shows that the prevalence’s of all reported behaviours are larger for men and women in our survey compared to the others.

We assessed the user-friendliness, privacy and truthfulness of an ACASI-based sexual behaviour survey that was administered in disadvantaged, urban communities in Cape Town, South Africa. We found that an overwhelming majority of our participants preferred ACASI on touch screen computers to other modes of inquiry and that they answered all questions truthfully, probably owing in large part to participants’ perceived ease of use and privacy.

Despite the predominately positive experience, our analysis indicates that more effort may be required to improve the user experience for certain subgroups of the South African population. For instance, respondents over the age of 40 were less likely to think ACASI was easy to use, which might explain why they were also less likely to prefer ACASI as a mode of inquiry. Indeed, others have found that older generations are often intimidated by computers [28]. These results are not surprising given that computer literacy skills are only beginning to be taught in disadvantaged communities in South Africa. Moreover, unemployed participants found ACASI methods difficult to use probably due to the lack of computer exposure from a work environment. Since these groups of people found the survey challenging, it may have affected their recall of information due to the additional cognitive strain of navigating ACASI [13].

We also found that coloured participants were less likely to find our methods private. We suspect the reason for this is that a plurality of the coloured participants in our survey belonged to the oldest age group. McCallum and Peterson postulate that younger generations may find it more socially acceptable to share sensitive information due to changing views of privacy in the new technology age [13]. In fact, among Cape Town youth, there is a high use of mobile internet for instant messaging, digital media, and social networking [29], making their standards for privacy different from older generations. It should also be noted that in the multivariate analysis, adjusting for all of the potential predictors, the predictors did not change, only the size of the effects.

Our results also indicate that in this context, ACASI may produce more accurate results by partially removing some of the gendered SDB. In our survey the gender gap between reported sexual behaviours for men and women was narrowed, compared to the DHS that used FTFI techniques. Additionally, all of the sexual behaviours were reported more frequently in our survey than in the DHSs. This is particularly noticeable when looking at female reports of > 2 partners in the past year and had a concurrent relationship: less than 1% of women reported these behaviors in all DHS studies. This could be due to many different confounders or cultural differences between the two populations. However, we believe such a large difference in reported behaviours is most likely due to increased perception of privacy, which made our participants feel more at ease when answering intimate questions [30].

Our study had some limitations, which may affect the interpretation of our results. Firstly, we recognise that the questions we asked with respect to user experience, may also be subject to forms of bias. However, we believe that the bias would be minimal because typical forms of survey bias (e.g. inaccurate recall, social desirability, etc.) were probably not factors in answering these specific questions. Secondly, the positive results from the first two parts of our analysis may have been confounded by additional mechanisms we built into the survey to reduce bias, other than the interview mode. For one, we made use of a visual timeline to help respondents recall beginning and end dates of distinct episodes within each of their relationships. These visual cues have been demonstrated to encourage internal consistency in reporting relationship histories [31, 32]. Additionally, our survey was relatively short, asking only a few questions for each reported relationship episode and partner, thus minimizing fatigue bias compared to other long surveys, such as DHSs. While these are an overall strength of the survey, they may have contributed to the positive user experience, as much or more than the ACASI itself.

The greatest strength of this study is that it is one of few sexual behaviour surveys that measured the user experience of the instrument [33]. This allowed us not only to find a proxy for how accurate our results were, but it enabled us to quantitatively assess how appropriate our methods were for the communities where the survey was administered. By asking questions about user experience we were able to determine that old age groups and unemployed people in similar contexts may be at risk for having an adverse experience with ACASI and consequently inaccurately reporting their results. Going forward, this knowledge will allow researchers to target these at-risk groups and provide more assistance and training for them before conducting a survey with similar methods.

The results of our study indicate that ACASI methods are not only suitable for disadvantaged, urban African contexts, but they may considerably reduce SDB in sexual behaviour surveys. By targeting groups at-risk for finding ACASI methods challenging, and providing them with additional support and training prior to taking the survey, the accuracy of data may be improved. However, future work needs to be done on how to best train these people to make them feel comfortable with computerized survey methods. Correspondingly, more work needs to be done on how computer literacy and numeracy training affect the validity of ACASI responses.