Incorporating a quiz into informed consent processes: Qualitative study of participants' reactions

Individual informed consent is a key ethical obligation for clinical studies. However, empirical studies show that the key requirements are often not met [1‐5]. Suggestions for improving informed consent abound [2‐12], but few have been evaluated in the field.

There is some evidence that a formal assessment, administered during the informed consent process, improves understanding[4]. Checks of study comprehension, often in the form of a simple quiz with a check-list of facts, are, therefore, increasingly recommended for clinical studies[10, 13, 14]. In a recently published Nuffield discussion document, the monitoring of participants' understanding by a separate team was described as a 'valuable addition to many trials conducted in developed countries, where participants may have an incomplete understanding of the implications of their participation' [14]. The need to assess participants' general understanding of the implications of the trial, rather than to simply test their retention of information with a check-list, was emphasized.

In this paper, community reactions to a semi-structured questionnaire aimed at exploring participant perceptions and understanding of a malaria vaccine trial are described. The trial was conducted at the KEMRI-Wellcome Trust Collaborative Research Programme on the Kenyan Coast. The trial, information giving processes, and community perceptions and understanding of the trial itself are described in detail elsewhere[15, 16]. Briefly, information-giving included: creating awareness of the study among community representatives and the general community at public meetings, and then with parents in their own homes; holding detailed group and individual discussions with parents who volunteered to attend the local dispensary to enrol their children in the trial; and a further explanation of the study at least a week after signing consent, just prior to administration of the vaccine (Figure 1). The findings of the questionnaire (or quiz) are briefly reported here. The unexpected reactions to the quiz are then described in more detail, in order to inform appropriate consenting processes in similar communities. This research was part of a protocol reviewed and approved by the Kenyan National Ethical Review Committee (No. 925).

The Social and Behavioural Research (SBR) group coordinate community liaison activities at the KEMRI-Wellcome Trust Research Programme. SBR members (SM, CG and VM) adapted a quiz passed by the local and national ethics review committees into a semi-structured questionnaire (hereafter 'quiz') administered to 189 caretakers, primarily mothers (see Additional file 1). The quiz was administered by four experienced SBR fieldworkers, all of whom were from the same locality as the participants and had been carefully trained in communication skills. To avoid disrupting consenting and clinical screening processes, caretakers were interviewed just prior to their departure from the dispensary on the screening day (Step 3; Figure 1). Community reactions to the quiz itself were initially raised by the community to the trial field staff and PI (PB) and were then explored by CG through focus group discussions with community members (n = 9) and fieldworkers (n = 2) once the trial was underway. Community members were mothers of children in the trial, selected on the basis of village of residence. Focus Group Discussions (FGDs) continued to be held until a point of redundancy was reached (no new information being presented in the groups).

The proportion of correct responses regarding different key study details ranged from 29% to 85% (Table 1), and the proportion responding 'don't know' from 14% to 45%. 35% of participants responded to the question 'what is this work for' with a correct response. Most participants providing an 'incorrect' response described vaccine or malaria prevention work in general terms, without specifying testing for safety or potential efficacy of a malaria vaccine. In subsequent FGDs, and in responses to other questions, it appeared that many more parents did in fact understand that the study was a trial, and that the problem was primarily with the question and it's administration rather than with participant knowledge. For the remaining questions, 'incorrect' responses included 6% (n = 11) reporting that all children would be given a malaria vaccine, 18% (n = 35) that their child would receive one or two rather than three injections, and 20% (n = 39) that children with the vaccine would have assured protection against malaria infection.

No concerns were openly vocalized about the quiz itself while it was being administered. However, over the course of quiz one mother became increasingly agitated and finally withdrew from the study, demanding that her child's blood sample be returned and destroyed to prevent 'devil-worshipping'. Many other potential participants observed this dramatic and public event. Discussions around it revealed that rumours were widespread in the community: rumours that the information given by the research team is a lie; that unnecessarily large samples of blood are taken to be sold or given to devils (often associated with 'wazungus', or foreigners); that children will fit and die immediately after the vaccine is administered; that children will slowly die off after the end of the vaccine trial; and that biscuits and juice are given to children to increase their blood for further blood taking for devils.

Recognition of the huge sensitivity around blood sampling led to public meetings at which the PI showed and answered questions about how blood is processed and used in the laboratory; including demonstrations of microscope slides and assay wells. These meetings were an opportunity to focus on an area that had not been covered in depth in previous information giving sessions. More importantly, they were an additional opportunity for community members to test the research teams' openness and consistency in messages. These are behaviours supportive of trust-building, and appeared essential to the successful conduct of the trial (see also Gikonyo et al [16]).

Over the course of public meetings and in other informal interactions, concerns about the quiz itself began to emerge. Questions about the quiz were, therefore, included in FGDs with fieldworkers and community members. In almost all FGDs, participants mentioned at least one disadvantage of the quiz. An important view expressed in community groups was that the questions asked, and the way they were asked, heightened concerns about rumours circulating in the community; and that raising concerns in this way was unnecessary (see above event, and Quotes 1–3; Figure 2). Further concerns included: people were not given a specific explanation of the quiz in previous information giving steps and so were not prepared for it; a person was being taken 'very far and interviewed alone'; and possible poor verbal and non-verbal communication by interviewers (Figure 2, quote 4). Reports of questions asked suggest that either the questions were being confused with a larger census that is on-going in the area, or that some of the interviewers embellished the questions and contributed to the concerns raised (Figure 2; quote 5). Participants' concerns about the questions may have led to a relatively high level of 'don't know' responses to the quiz, either in protest at the questions, or in an attempt to gather additional information from SBR fieldworkers to cross-check vaccine trial team information.

In six FGDs participants mentioned at least one positive aspect of the quiz, including that it helped people understand, gave them a chance to seek clarification on specific issues (from the study team or friends and relatives), and assisted in decision-making about study participation (Figure 3; Quotes 1 and 2). Those who reacted badly were sometimes reported not to have understood, and any concerns raised were reportedly resolved once the trial began.

Nevertheless, the quiz was clearly an unnerving experience for some participants (Figure 2; Quote 6), particularly given the relatively low education level of many mothers compared with the field assistants interviewing them. One person wondered what good the quiz did for those who decided not to join, because many of the rumours circulating in the community were proven untrue (for example children did not die immediately after receiving the first vaccine), and because those not in the trial did not benefit from careful monitoring and free treatment: P: (Interrupted a discussion on the positive aspects of the quiz)... it helped some; those who decided to continue coming. But then those that misunderstood, it didn't help them because they ran away! So, did they finish the journey?

Formal approaches to checking study understanding are increasingly popular for clinical research. The findings presented in this paper suggest that any such checks should be employed with sensitivity and caution. In the absence of other studies that have formally explored this issue, the experiences in Kenya suggest the following key issues influencing community reactions and the validity of the quiz itself:

The interviewer. The value of quizzes is dependent on the interpersonal skills of the people administering it, and their ability to ask questions in an open and non-threatening way. Difficulties were faced in Kenya even with fieldworkers fluent in the local language who were carefully trained in qualitative methodologies and in communication skills. Interviewers from within the team may have been more appropriate in this case.

Qualitative versus quantitative data. The findings of quantitative data acquired can be misleading without complementary qualitative data. Using quiz data alone to make definitive comments on individual levels of understanding or eligibility to access a trial is not therefore recommended. Imbalances in resources and research knowledge between researchers and community members can also make individual interviews intimidating for participants. In Kenya, a qualitative, group based, or less formal quiz may well have achieved the same positive impacts for all with fewer concerns raised for community members.

The follow up provided after a quiz. A formal check of understanding can lead to difficulties for individuals, for the community and for researchers in the short term. Nevertheless, this is an opportunity for the issues raised to be dealt with, leading to better understanding on both sides, and a stronger relationship between participants and researchers. This is particularly the case if the assessment is incorporated into and informs a broader strategy for involving communities in aspects of the research.

Expertise is required. Developing, administering and responding to assessments of understanding for clinical trials requires significant social science capacity. Strengthening the role of good quality social science in clinical trials is an important strategy for improving ethical practice.

The findings may be specific to the study population and to the socio-cultural and institutional context of the study. Use of a similar quiz in phase 1 trials, on a much smaller sample size of well-educated parents, provoked less controversy. However this was not formally explored. Further studies on quizzes would indicate the generalizability of the findings and identify important contextual influences. A greater understanding of community perspectives on quizzes, and of other ethical requirements, will help give research participants and communities a voice in national and international debates on how to strengthen ethical practice.