Background
Chronic infection with Hepatitis C Virus (HCV) affects 130–200 million people worldwide [
1]. It is the leading cause of cirrhosis, liver cancer and liver transplants which result in 350,000 deaths worldwide [
2]. HCV is divided into 6 genotypes, with genotype 1 being the hardest to treat and the most prevalent in Western countries. The goal of treatment is to achieve a sustained virologic response (SVR), marker of viral eradication, assessed by a viral load HCV RNA (VL) below the limit of detection (LOD) six months after cessation of therapy. Until 2011, the only available treatment was based on weekly injections of pegylated interferon (peg-IFN) and daily oral ribavirin (RBV) during 48 weeks, with SVR rate lower than 50% in treatment-naïve HCV genotype 1 patients [
3].
In 2011, the approval of two protease inhibitors (PI), telaprevir and boceprevir, in combination with peg-IFN/RBV (triple therapy), marked a milestone for anti-HCV therapy with SVR rates larger than 70% in treatment-naïve HCV genotype 1 patients [
4,
5]. Dozens of compounds targeting different viral proteins are currently in different stages of clinical trials, raising the expectation that several IFN-free regimens might be available in the coming years.
Viral kinetic modeling aims at characterizing the main mechanisms that govern the virologic response to treatment using mathematical models. Following the recommendations of the Food and Drug Administration [
6], this approach has been increasingly used in phase 1/2 of clinical development to estimate viral kinetic parameters and to evaluate drug antiviral effectiveness
in vivo[
7,
8]. Parameter estimation is often achieved using non-linear mixed effect models (NLMEM) [
9]. The popularity of this approach is due to the fact that it optimizes the information available by borrowing strength from the whole sample to provide precise estimation of the parameters, including covariate effects [
10‐
12]. Moreover it naturally accounts for the information brought by VL data below the limit of detection (BLD) and reduces the bias in parameter estimation as compared to empirical approaches where BLD data are ignored or assigned to half the LOD [
10,
13,
14].
So far, viral kinetic models and NLMEM have mostly been used in phase 1/2 clinical trials with large number of patients and/or frequent assessment of VL data within each patient. However in most clinical trials, in particular when they are not sponsored by the industry, it is not possible to hospitalize patients and to get frequent viral load samples. In this challenging context, the capacity of NLMEM to precisely estimate viral kinetic parameters is not known. In particular the performance of tests used to assess the effect of a covariate which have good asymptotic properties (Wald test, likelihood ratio test or score test) is not warranted when one is far from the asymptotic conditions. For instance an inflation of the type I error has been reported in another clinical context where data were sparse [
15]. With the new potent triple therapies against HCV the amount of information available may also be limited by the fact that a large proportion of VL data are below LOD.
Here our goal was to evaluate the capacity of NLMEM to precisely estimate the parameters of viral kinetic models when there is a large proportion of BLD data and a limited number of data per patient. In particular we aimed to evaluate by simulation the type I errors and the power of the Wald test to compare the antiviral effectiveness of two groups receiving different triple therapies (noted PI-A and PI-B in the following). Parameter estimation and Wald test property were evaluated according to the number of patients, the number of samples per patient and the expected difference in antiviral effectiveness between the two treatment groups.
Discussion
The goal of this study was to evaluate the capacity of NLMEM to provide precise and accurate estimates of viral kinetic parameters when only sparse data with a large proportion of BLD data are available. In particular we aimed to evaluate the ability of this approach to correctly reject or not the null hypothesis of equal treatment effectiveness when two groups with different antiviral strategies are compared.
Our results showed that NLMEM provide very precise and accurate estimates for both the fixed effects and the inter-individual variance parameters, even when only 5 data points (at days 0, 2, 3, 7 and 14) were available within each patient. This allowed circumventing the need for intensive VL sampling measurements at treatment initiation, which are difficult to obtain in current clinical practice. Of note the viral clearance rate,
c and its associated variability ω
c, were poorly estimated in this sparse initial sampling. However this parameter is mostly involved in the initial rate of viral decline and thus a poor estimation of
c did not substantially deteriorate the estimation of the other parameters (Table
2).
By comparing the results obtained with and without a LOD for VL, we demonstrated that maximum likelihood appropriately handle BLD, consistent with results found previously [
10]. The conclusion was somewhat different when considering the outcome of Wald test for comparing antiviral effectiveness. In this case the lack of information due to BLD contributed to an inflation of the type I error as compared to the results obtained with no LOD of VL, suggesting that the development of real-time PCR assays with lower LOD may improve the estimation of viral kinetic parameters. Interestingly, even when there was no LOD of VL, we still found that the type I error was inflated when the number of observations n
tot was lower than 140. This suggests that the outcome of Wald test should be taken with caution when the number of patients is low and in that case we suggested to use a threshold correction for the Wald test to limit the impact of this inflation. Here we used an empirical threshold correction but other corrections exist such as the Galland correction or the permutation test [
15]. On the other hand the power of the Wald test (corrected or not) was found to be very high, especially when compared with that obtained using a Wilcoxon test on the mean viral decline at day 14. This result clearly shows the benefit of viral kinetic analyzed with NLMEM over empirical approaches done in most clinical studies. Although better results may be obtained by comparing the viral decline at earlier time points (such as day 2 or 7) the power of the Wilcoxon test remained lower than those achieved by modeling approach (not shown). Consistent with results found elsewhere, the power increases when the number of observations per patient increases and was much less sensitive to the number of measurements within each patient [
26]. From a clinical standpoint this finding indicates that the enrollment of a large population of study is to be preferred to small population sample with frequent assessments of VL.
We focused here on the properties of the Wald test and further studies would be needed to study how these results apply to other tests that require more computation time, such as likelihood ratio tests (LRT) or score test. Interestingly previous simulation studies using the SAEM algorithm in MONOLIX showed that the outcomes of these tests were largely comparable [
15]. Of note this result may not hold when other estimation methods are used and for instance the outcomes of Wald test and LRT were found to be different when using the FOCE-I algorithm in NONMEM version 7 [
15]. Indeed the Wald test had a lower power than LRT with FOCE-I, which was probably due to the poor estimation of the standard error of the covariate effect [
25]. The advantage of the Wald test is that results are immediately obtained and do not require to compute the likelihood or its derivatives, as done for the LRT and the score test. Computation time needed by simulations could be largely reduced by using information theory and approximations to derive Fisher information matrix. For instance the software PFIM uses a first order approximation of the likelihood and, under this approximation, an analytical form of the Fisher matrix can be obtained [
27]. Thus the expected variance of viral kinetic parameters could be obtained without the intensive simulations done here. Although such approximations worked well even with limited number of patients [
11], it does not take into account BLD data and hence could underestimate the standard error when a large proportion of data are BLD. It should be noted that optimal design theory predicts that an increase of variances in random effect may deteriorate the precision of parameter estimates and the power of the Wald test. However this possibility was not investigated in this study where the inter-individual variance parameters were fixed.
Here we focused on the comparison of treatment antiviral effectiveness in the first two weeks of treatment. On this short time scale the standard biphasic model of viral kinetics has been shown to provide a good fit to the data [
7,
16]. However more complex models may be needed to fit long-term VL data, such as models that relax the assumption of constant target cells and/or account for the emergence of treatment resistant viruses [
28,
29]. Moreover viral decline during PI therapy is faster than what is observed with IFN-based therapy [
30]. This feature is captured in the standard biphasic model by assuming that PIs lead to an enhancement of the treatment effectiveness,
ϵ, and of the clearance rate of infected cells,
δ[
7,
29,
31]. Consistent with this observation we set here large mean values for both ϵ and
δ, equal to 0.999 and 0.58 day
-1 as compared to 0.92 and 0.14 day
-1 with IFN-based therapy, respectively [
30]. However this dual mode of action of PIs may be integrated in a more physiological way by using new multiscale viral kinetic models that explicitly integrate the effect of PIs on the intra-cellular viral dynamics [
9].
Although the use of NLMEM has been shown to provide very precise and accurate estimates of the parameters even in presence of sparse designs, it should be acknowledged that these estimates are done on the population parameters, i.e., the mean and the variance of parameters in the population. How NLMEM also allow precise and accurate estimation of the individual parameters for individualized treatment duration remains to be evaluated.
Competing interests
All authors declare that they have no competing interests.
Authors’ contributions
CL, JG and FM designed the simulation study and the MODCUPIC trial. CL carried out the simulations and drafted the manuscript. JG participated to the work of estimation (with CL). CL, JG and FM participated in the statistical analysis and helped to draft the manuscript. All authors read and approved the final manuscript.