Treatment fidelity in the Tinnitus Retraining Therapy Trial

Adherence or fidelity is a critical component of randomized trials. Importantly, treatment fidelity impacts the ability of investigators to estimate accurately the treatment effect; poor adherence to the treatment protocol may result in enhancement of either type I or type II errors. When the intervention is not a drug, but a behavioral intervention, treatment fidelity allows for the identification of the critical elements of the experimental and comparator interventions for replication in future studies or in practice. Ultimately, poor adherence can lead to inaccurate translation of trial findings to clinical practice.

In medication trials, adherence monitoring may involve counting unused tablets or pills or measurement of a biochemical marker. However, when the intervention involves a counseling or behavioral component, measures to assess adherence are less well-developed. Treatment fidelity for behavioral interventions is defined as ensuring that the recipient receives the intended intervention. Measures to enhance treatment fidelity in trials aim to confirm that individuals take the medication or receive the type of counseling or other intervention assigned at randomization. These measures also aim to prevent errors due to contamination of the treatment. Typically, treatment fidelity involves a number of steps, as recommended by the National Institutes of Health (NIH) Behavior Change Consortium and described by Bellg et al. [1]. These steps include:

Thoughtful pre-planning is required to implement these adherence steps to achieve consistency and accuracy for most behavioral interventions. The details of these steps often varies across trials. For example, some recent reports described fidelity steps for a variety of interventions, including music therapy for people with dementia [2], best practices for implementing mindfulness-based interventions [3], or motivational interviewing [4]. Trials that report on measures taken to enhance treatment fidelity tend to emphasize delivery, rather than participant engagement in treatment [5]. Although researchers recognize the importance of implementing measures for treatment fidelity, few have received adequate training in fidelity measures or include descriptions of these measures in trial reports [6]. Excellent resources for planning include the model developed by the NIH Behavior Change Consortium, as described by Bellg et al. [1] and Borrelli [6] and implemented by Kechter et al. [3] and Robb et al. [7]. There are also various articles describing methods used for fidelity monitoring either generally [8] or across many areas of behavioral interventions as described above.

Tinnitus, the perception of sound in the absence of a corresponding external stimulus for which there is no apparent cause is a debilitating condition for some individuals. Various options for management exist, including tinnitus retraining therapy (TRT). TRT involves non-psychiatric tinnitus-specific educational counseling (TC) and low-level sound therapy (ST), typically achieved using sound generators (SGs) and/or enriched environmental sound, to habituate the individual’s associated negative emotional reactions (e.g., annoyance or anxiety) and perception (awareness) of the tinnitus, and ultimately its impact on the patient’s life [9]. The overall goal of TC is to educate the patient about his or her condition, which is achieved through didactic education. We compared the efficacy of TRT with the standard of care (SOC) in the Tinnitus Retraining Therapy Trial (TRTT). The TRTT, a multi-center trial conducted at US military medical sites, enrolled individuals with severe tinnitus to the trial. The TRTT also evaluated the necessity for the use of SGs by including a third study arm, the Partial TRT group, which used TC and short-acting placebo SGs as a control for ST. The SOC group received counseling that used an interactive patient-centered care approach focusing on assisting the study patient to become actively engaged in his or her symptom relief [10]. All three study groups were encouraged to use environmental sound at all times. No difference was found between treatment groups on numerous measures of tinnitus-specific health-related quality of life [11], so it is critical to emphasize that these were distinct interventions. Table 1 summarizes the main differences between the two types of counseling, TC and SOC. All three treatment groups involved one-on-one counseling, with the TRT and Partial TRT groups receiving TC counseling. Study audiologists administered both types of counseling in the TRTT primarily to avoid a counselor effect, minimize the required number of audiologists, and allow all audiologists to receive training in TRT and the patient-centered approach for SOC. Blinding was maintained by having one audiologist provide the intervention (counseling) and the second measure all audiologic outcomes for an individual participant. To avoid cross-contamination, it was necessary to develop a protocol to enhance adherence to each management strategy. In this report, we describe how we followed the steps recommended by the NIH Behavior Change Consortium and described by Bellg et al. [1] to achieve treatment fidelity in the TRTT.

Twenty-five audiologists were certified for TC and/or SOC counseling and 24 completed at least one counseling session.

Twenty-five audiologists submitted audiotapes for TC. Of these, 22 (88%) demonstrated competency for TC. An additional 2 (8%) audiologists demonstrated partial competency requiring a telephone call with the protocol monitor regarding some deficiencies in the recordings. One of these two audiologists re-submitted an audiotape demonstrating full competency. The other audiologist did not submit an acceptable audiotape and chose not to re-submit a second one.

Twenty-five audiologists submitted audiotapes for SOC. Of these, 15 (60%) demonstrated competency for SOC, with 2 audiologists demonstrating adequate competency but requiring a telephone call with the protocol monitor. The remaining 8 (32%) audiologists did not submit an acceptable audiotape, and 2 of these chose not to submit a second audiotape. Of 6 audiologists who submitted a second audiotape, 2 were classified as acceptable and 3 as partially acceptable. The remaining audiologist re-submitted another two audiotapes with the fourth classified as acceptable.

Audiologists submitted 101 checklists comprised of 33 items for TC (Table 3). Per the study protocol, although all sessions were recorded, only the first two treatment sessions of each type of counseling for each audiologist were reviewed by the protocol monitor for adherence to protocol. One randomly selected audiotape from among the next five counseling sessions also was reviewed. Of the 101 audiotapes and checklists submitted, protocol monitors reviewed 37 TC counseling sessions. Adherence to each of 33 critical items on the TC checklist as assessed by the protocol monitor ranged from 70 to 100% across 37 counseling sessions (median 97%). There was no difference between adherence for TRT (median, 97%) and Partial TRT (median, 100%) (data not shown). Only one checklist item, “Summary reviewed” (70% agreement), failed to achieve adherence of at least 80%.

Audiologists submitted 45 checklists for SOC counseling sessions comprised of 44 items (Table 4). Of these, 30 audiotapes were reviewed by the protocol monitor. Adherence to each of 44 critical items on the SOC checklist across 30 SOC counseling sessions ranged from 42 to 100% (median, 87.5%). Some items failed to achieve adherence of at least 80%, including “Communicated empathy and understanding of Participant’s thought and feelings” (63%), “Emphasized rationale or/relevance of this particular recommendation” for stress (65%) or sleep issues (63%), “Changes Participant thinks would be helpful” for sleep (42%), “Importance of ability to concentrate” (58%), and various items under the topic of concentration (54–57%). Most of these topics are related to the patient-centered approach, suggesting that this approach was somewhat difficult for counselors to deliver, notwithstanding the previous training.

In the TRTT, adherence with the main components of both the TC and SOC counseling interventions was assessed by comparison of the checklists completed by the audiologist with the corresponding review by the protocol monitors. Rigorous review by the protocol monitors generally showed agreement with the audiologists’ perception. Audiologists’ perception of adherence and their self-report of adherence tended to be somewhat more favorably rated than that by the protocol monitor. Items that were less in agreement in the SOC counseling checklist tended to be those most related to the patient-oriented approach, which focuses on shared-decision making and goal setting [10]. Although the study audiologists routinely provided counseling of tinnitus patients in their clinical practices prior to the TRTT, most were not experienced with the SOC patient-oriented approach to managing tinnitus in the trial. This inexperience is reflected in the poorer performance during the initial competency assessment compared with their performance on TC counseling. Thus, the less favorable agreement of the patient-oriented SOC checklist items is not a surprising outcome. In retrospect, additional training seems to have been warranted for the SOC arm of the trial.

One challenge we faced while the trial was ongoing was the constant turnover in audiologists. This high turnover, primarily due to re-posting of the military study personnel participating in the trial, required constant training to ensure consistency and faithfulness to the interventions from the beginning to the end of the trial. In addition, the quality assurance procedures, including checklists and audiotape review, continued throughout the trial. Because this “extended time period” could impact treatment fidelity [14], monitoring procedures put into place continued to be maintained until trial end.

Treatment efficacy may indirectly reflect treatment fidelity. If so, then on average, all study groups in the TRTT may be judged to have been faithful to the intervention protocol. The majority of participants in all groups showed improvement on average by at least 30% in the Tinnitus Questionnaire at 18 months and improvement on all other tinnitus-specific health-related quality of life instruments used as secondary measures [11]. It is possible that a participant may have received the intervention, but chose not to or was unable to act upon it. Investigators who have examined this concept have found mixed results, i.e., some found a relation while others have not. However, it appears clear that there is a difference between treatment effectiveness and enactment and that enactment is not necessarily an element of treatment fidelity [8].

Although we are unable to explore fidelity further because the trial has ended, we would suggest that other investigators query both interventionists and participants following the intervention to understand their perspective on the treatment. For example, questions we might have asked the audiologists would be items such as preferences for one versus the other treatment arm or how likely they are to continue with either treatment arm in their practice. We would query study participants on how much they believe the treatment helped them and if they plan to continue using the skills they might have acquired during the trial.