Scenarios.

Our objective was to compare the performance of sampling with and without ROSE in common FNAB situations. We classified FNAB procedures into two generic procedure types: simple and complex. The simple procedure type was intended to represent FNAB procedures that would be performed in a physician’s office (e.g., palpation-guided FNAB or US-guided FNAB of thyroid) and require limited personnel and a short set-up time. The complex procedure represents procedures such as EUS-FNAB and CT-guided FNA that require more personnel and a longer setup time. The procedure type is designed to capture differences in the cost structure of different types of FNAB procedures. The scenarios are described in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Design of Computer Experiments.

We investigated 32 scenarios which are described in the table below. Each scenario is a combination of a procedure type (simple or complex), a per-pass adequacy rate (low or high), and a set of sampling protocols (one for ROSE and one for fixed sampling). Procedure types are categorized as simple (palpation guided) or as complex (image guided). Simple procedures have shorter setup times and require fewer resources than complex procedures. Variable definitions are provided in Appendix (S1 Text). Sampling protocols are defined by stopping rules (number of needle passes for fixed sampling, and number of adequate samples for ROSE). Please see the text for additional explanation of procedure types, adequacy rates and sampling protocols. In the simple procedure: t_setup = 4 min, c_var,o = $30/hr, c_var,c = $100/hr, c_np = $0/pass and c_pat = $20/hr. In the complex procedure: t_setup = 30 min, c_var,o = $260/hr, c_var,c = $100/hr, c_np = $30/pass and c_pat = $20/hr.

https://doi.org/10.1371/journal.pone.0135466.t002

The adequacy rate represents the per-pass probability, p, of obtaining an adequate sample. The per-pass adequacy rate would vary on a wide range of factors such the lesion characteristics (size, solid vs cystic, anatomic location) and operator characteristics. The overall adequacy rate depends on the per-pass adequacy rate as well as the number of needle passes. Our model explicitly separates these two factors (per-pass adequacy, number of passes) to study the impact of different sampling protocols. We used two different per-pass adequacy rates (low vs high) to capture the impact of lesion and operator characteristics on sampling performance.

Sampling protocols are defined by a stopping rule. For sampling without ROSE, sampling stops after a fixed number of needle passes, n_F. With ROSE, sampling stops after the cytologist observes a required number of adequate samples, n_R. The relative performance of ROSE depends on the stopping rules. For that reason, we surveyed a wide combination of stopping rules. We compared sampling policies in which n_R varied from one to two adequate samples and n_F varied from two to five needle passes. This provided a total of eight sampling protocols which provided a comparison of particular combination of ROSE vs fixed sampling.

We defined a scenario as a comparison of ROSE vs fixed sampling with a specific combination of stopping rules (stopping rule for ROSE, stopping rule for fixed), per-pass adequacy rate (low vs high), and procedure type (simple vs complex). For example, a scenario might involve a comparison of ROSE (stopping after first adequate sample) and fixed sampling (three passes) in a simple procedure with a high per-pass adequacy rate. This created a total of 32 scenarios (4 fixed sampling stopping rules x 2 ROSE stopping rules x 2 per-pass adequacy rates x 2 procedure types). The scenarios are described in Table 2.

Sensitivity analysis.

We conducted one-way and probabilistic sensitivity analysis (see Glossary, S2 Text) One way sensitivity analysis captures the impact of changing a single variable at a time. Probabilistic sensitivity analysis varies all variables and is able to capture the impact of interactions. For one-way analysis, we selected a base-case sampling protocol (n_F = 3, n_R = 1) and determined the impact of variation of input parameters (see Glossary, S2 Text) on the total cost. We performed one-way analysis on each of the four case-types. The input parameters for each case-type and the range over which each parameter was varied are presented in Table 3. One-way analysis was performed using @RISK (Palisade Corp, Ithaca, NY).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Assumptions for Procedure Types.

Procedure types were categorized as simple (palpation guided) or as complex (image guided). Simple procedures have shorter setup times and require fewer resources than complex procedures. Variable definitions are provided in Appendix (S1 Text).Each cell shows the base value and the range used for sensitivity analysis. Cells that span both columns indicate values that are the same for both types of procedures. Please refer to the Methods and Glossary (S2 Text) for a more detailed description of procedure types.

https://doi.org/10.1371/journal.pone.0135466.t003

We used probabilistic sensitivity analysis to study the overall variability in outcomes due to variation in all the inputs. We used Monte Carlo simulation to generate 100,000 samples for each of the 32 case scenarios (see Glossary, S2 Text). Each sample was obtained by randomly drawing input values from the ranges shown in Table 3. Using this method, we were able to determine the variation in outcome that would be obtained using a particular sampling protocol on a particular case-type. Probabilistic sensitivity analysis was performed using Stata (Stata Corp, College Station, TX).

Overview.

Our model has three components: a sampling model, an operational model, and a cost model (Fig 2). The sampling model estimates the number of needle passes and the number of repeat procedures based on a set of technical inputs. The operational model estimates the total time per procedure. Finally, the cost model takes the outputs from the operational model and estimates the overall cost to obtain a diagnosis. Each component of the model is described in separate sections below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Explanation of cost analysis.

The mathematical model estimates sampling performance (number of needle passes, number of procedures) based on a combination of controlled inputs (decision variables) and uncontrolled inputs (technical inputs). The operational model estimates operational performance (sampling time, setup time) based on the sampling performance and operational inputs. Finally, the cost model estimates the overall cost based on the operational performance and cost inputs.

https://doi.org/10.1371/journal.pone.0135466.g002

Notation.

Input variables (parameters) are indicated with lower case letters (e.g., a) and outcomes (random variables) are indicated with upper case letters (e.g., A). Expected values of outcome variables are indicated by a bar over the symbol (e.g., ). Values for inputs and outcomes may be associated with a particular sampling protocol. We use a superscript to designate the sampling protocol: F for fixed sampling (i.e., non-ROSE) and R for ROSE (e.g., or ). We use the symbol, π, to indicate a generic sampling protocol (e.g., ) Thus, designates the expected value of outcome A with fixed sampling, designates the expected value of outcome A with ROSE sampling, and designates the expected value of outcome A for any sampling protocol. All variables are defined in the Appendix (S1 Text).

Sampling model: Number of Needle Passes and Adequacy Rate.

The sampling model estimates the number of needle passes and the number of repeat procedures as a function of parameters that specify the sampling protocol. Overall, the sampling performance depends on five inputs: sn, sp, p, n_F and n_R (defined below). Collectively, we refer to these inputs as the technical inputs to the model. The technical inputs determine the sampling performance. Sampling performance is described in terms of two outcomes: the number of needle passes and the adequacy rate (via Eqs 2–6). These technical outcomes are used to estimate the operational performance of FNAB (Fig 2).

We assume a sampling process in which there is a constant per-pass probability, p, of success. A success is defined as an adequate sample (i.e., one with sufficient material for diagnostic assessment). The parameter, p, depends on many factors such as the experience and skill of the aspirator, the anatomic site, and the characteristic of the lesion (size, solid vs. cystic, etc). The overall sampling process consists of multiple needle passes and is considered successful if at least one needle pass produces an adequate sample. We use the term “per-procedure probability of success,” P(S), to designate the overall probability of success of multiple (as distinct from the per-pass probability, p). The per-pass probability, p, is a parameter of the model (i.e., an input) whereas the per-procedure probability of success, P(S), is a random outcome that depends on p. Importantly, the per-procedure success rate is equivalent to the adequacy rate.

In a fixed sampling protocol, sampling stops after a fixed number of needle passes, n_F. Each needle pass can be viewed as a Bernoulli trial that has a probability, p, of producing an adequate sample. Let X be the total number of adequate samples obtained in n_F needle passes. In any procedure, the outcome, X can range from 0 to n_F. The probability that X takes on some particular value, k, is given by the binomial distribution: (1)

A procedure is considered successful if at least if at least one sample is adequate. The adequacy rate is the probability, P(S), that a procedure is successful. Thus, the expected adequacy rate is given by (2) where is the expected adequacy rate of a fixed sampling protocol. The adequacy rate, is the success rate of the overall procedure (result of multiple needle passes) whereas p represents the success rate of an individual needle pass.

We can verify the result in Eq 2. The procedure fails only if all n_F needle passes fail to produce an adequate sample. The probability that a single needle pass fails is (1 − p). Because each needle pass is independent, the probability that all n_F samples are inadequate is the product of the probability of failure of each individual needle pass, The overall procedure has two possible outcomes: success or failure. Therefore, the probability of success is one minus the probability of failure: .

In a fixed sampling protocol, there is no variation in the number of needle passes. The expected number of needle passes is: (3)

The binomial distribution is based on the assumption that the per-pass probability of success, p, is constant and is unaffected by the number of passes. This assumption is probably not strictly true in FNAB sampling because each needle pass disrupts the tissue and affects the probability of success of subsequent samples; however, the assumption that the per-pass probability of success is constant is a useful approximation and has been shown to be robust to deviations in which the probability of success declines with each needle pass.[18]

With ROSE, the number of needle passes varies. Each sample is assessed by a cytologist, and sampling stops when the cytologist observes a stopping point, n_R. Most commonly, sampling stops after observing one adequate sample (i.e. n_R = 1) but the stopping point, n_R, could vary depending on the type of case or the risk tolerance of the cytologist. We assume that n_R is set prior to sampling and could depend on physical characteristics of the lesion, clinical information, imaging studies, etc. In practice the stopping point, n_R, might be adjusted during the procedure in light of the information obtained in prior samples. We regard n_R as the average number of adequate samples that a particular cytologist would require to cease sampling. For example, one cytologist may usually be able to render a decision after the first adequate sample (n_R = 1) whereas another cytologist may only feel comfortable after the second adequate sample (n_R = 1). Our model is flexible because it enables us to investigate the cost implications of these choices.

The accuracy of the cytologist is determined by the sensitivity (sn) and specificity (sp) of the adequacy assessment (sn and sp of the cytologist are judged relative to the final cytological assessment after processing). We assume that the sensitivity and specificity are known or estimated. In general, the sensitivity and specificity of the cytologist, sn and sp, could depend on a number of factors such as experience level, the anatomic site, etc. Under these circumstances, the overall probability of success depends on the per-pass probability of success as well as the accuracy (sn and sp) of the cytologist. For example, a false positive assessment would terminate sampling and could lead to sampling failure. False negative assessments would result in unnecessary needle passes. The expected adequacy rate and expected number of needle passes for variable sampling (ROSE) with assessment by an imperfect cytologist are given by:[16] (4) (5)

We assume that the FNAB procedure is repeated if it fails. The expected number of required procedures is: (6) where π indicates the type of sampling protocol (fixed or ROSE). We provide evidence that our model can predict clinical outcomes in the Supporting Material (S4 Text).

Operational Model: Time per procedure.

The operational model estimates the total time required for each procedure using estimates of the number of needle passes obtained from the sampling model.

Our general strategy is to divide the procedure into different time periods and to estimate the resource consumption in each time period. Resource use may vary during the procedure and, in principle, one could divide the procedure into many different time periods to account for pattern of resource use.[19] Thus, the cost for each period would be the length of the period times the sum of the hourly cost of the resources used in each period, t_i ∑_j cij. The total variable (i.e. time-based) cost would be the sum of the costs in all periods: ∑_i(t_i ∑_j cij).

For simplicity, we divided the procedure time into two periods: setup and sampling. Let t_setup be the time required for set up. Let be the time required per needle pass under sampling protocol, π. (The symbol, π, is used to designate any sampling protocol. We use the symbols F and R to designate a specific sampling protocol). The time per needle pass depends on the sampling protocol. In general, the time per needle pass is longer for ROSE sampling. For example, the time per pass for fixed sampling, might be 3 minutes and the time per pass for ROSE sampling, 4, might be 8 minutes. We refer to the time per needle pass and setup time as operational inputs (Fig 2). The total sampling time is given by the time per needle pass times the expected number of needle passes: (7)

Eq 7 shows that the sampling protocol has two effects on the total sampling time. First, it affects the expected number of needle passes, Second, the time per needle pass, also depends on the sampling protocol. ROSE tends to decrease the number of needle passes but increases the time per needle pass. The total procedure time would be the sum of the setup time and the sampling time: (8)

We assume that the setup time is independent of the sampling protocol ( For example, the setup time for an EUS-FNAB does not depend on whether or not ROSE is used. In our model, the variation in total procedure time is driven by the variation in sampling time which, in turn, is determined by the expected number of needle passes, and the time per pass, Both of these quantities depend on the sampling protocol as indicated by the superscript, π.

Cost Model.

We use a societal perspective as recommended by current guidelines.[20, 21] The societal view includes all medical costs, costs to the patient, and costs to society such as lost productivity. The societal perspective is preferred because it incorporates more limited perspectives such as the government, medical, or payer perspectives. It is easy to modify a model based on the societal perspective to create a model based on more limited perspective (e.g. government, medical or payer perspectives).

We categorized costs as time-dependent (variable) and procedure-dependent (fixed). Time-dependent costs include personnel as well as items such as equipment and space that incur an opportunity cost as a function of time. (The opportunity cost is the cost of the forgone alternative. For example, the opportunity cost of endoscopy suite time is the profit that would be obtained if that time could be used to schedule an additional procedure). For convenience, we distinguish variable costs that are incurred by the cytologist from all other variable costs (the cost of the cytologist is the major difference between the hourly cost of ROSE and fixed sampling). The overall hourly variable cost for each policy is: (9A) (9B) where

c_var,c = Variable cost of the cytologist, $/hr.

c_var,o = All other variable costs aside those associated with cytologists, $/hr.

c_pat = Hourly wage of the patient (only included in societal perspective)

The cost of the cytologist would vary depending on whether the procedure is performed by an on-site cytopathologist, on-site cytotechnologist, or a combination of telcytology cytopathologist and an anciallary staff person. Our model can account for all these contingencies by adjusting the variable cost parameters, c_var,c and c_var,o.

Variable costs are incurred in two different time periods: setup and sampling. For simplicity, we will assume that the variable costs are approximately the same during setup and sampling periods. We also assume that is the same for both fixed and ROSE sampling. Thus, the total variable costs of fixed and ROSE policies differ by the cost of the cytologist. The expected total variable cost is the product of the variable cost and the time required for each period.

(10)

Fixed costs are defined as costs that are independent of time and are incurred as a function of a procedure. Most commonly, the fixed costs would be supplies. For simplicity, we assume that the fixed costs, c_fixed, are independent of the sampling protocol ().

Our model uses the number of needle passes to estimate the time of the sampling period. Each needle pass not only consumes time but is also associated with morbidity. Let be the expected cost of a needle pass. This quantity includes the weighted average of the probability of each type of adverse event and its associated cost.

The expected total cost per-procedure associated with a particular sampling protocol is given by the sum of the variable (time dependent), fixed (procedure dependent) and the costs associated with needle passes.

(11)

We assume that a procedure is repeated if it fails to obtain at least one adequate sample. The total overall expected cost per case is given by the cost per procedure times the expected number of procedures: (12)

After substituting equations, we obtain the following estimates for the expected cost of FNAB sampling with and without ROSE. The expected cost of fixed sampling (i.e. without ROSE) is given by: (13A)

The expected cost of sampling with ROSE is given by: (13B) where = .

The overall difference in the expected cost per case is: (14)

Eqs 13A and 13B show how the cost per diagnosis can be estimated from decision variables (type of sampling: fixed or ROSE, the number of required samples or stopping point, n_F or n_R), technical parameters (p, sn, sp), operational parameters (), and cost parameters as shown in Fig 2. Our model (Eq 13) is based on several simplifying assumptions. We provide a general model in the Supporting Material (S3 Text).

Input Assumptions.

As noted in the overview, our model depends on three types of inputs: technical inputs, operational inputs, and cost inputs. We assumed values for each of these inputs. The assumptions are listed in Table 2 and are described below.

Technical inputs.

The technical inputs include the sampling protocol (n_F, n_R), the per-pass success rate (p), and the accuracy of the cytologist (sn, sp). We assumed that n_F varied between two and five passes and n_R varied between one and two adequate samples. For the base-case sampling protocol (used in one-way analysis), we assumed that fixed sampling uses three passes (n_F = 3) and that ROSE sampling stops after observing one adequate sample (n_R = 1). We assumed that the cytologist has a sensitivity of 0.95 and a sensitivity of 0.99 for sample accuracy.[22–26] Previous studies have shown that the benefits of ROSE depend on the per-pass probability of success. We compared two different levels of sampling success: low per-pass success rate (p = 0.3) and high per-pass success rate (p = 0.6). With a fixed sampling policy using three passes, the per-case probability of success would be 66% and 94%, respectively for low and high per-pass probability of success. These per-case probabilities of success represent the low and high values typically reported in adequacy studies.[11]

Operational inputs.

We assumed that the setup time was three minutes for simple procedures and 20 minutes for complex procedures (please refer to the section on Scenarios for definition of simple and complex procedures). The time per needle pass depends on the sampling protocol. [9, 27] We assumed that the time per needle pass was 2 minutes for fixed sampling and 9 minutes for ROSE sampling.

Cost inputs.

We used a micro-costing approach to estimate costs. We assumed that all procedures incurred a fixed cost of $150 for final cytopathological diagnosis. [10] We assumed that complex procedures require additional supplies (e.g., anesthesia, special needles) that add $150 to the fixed costs. So, the total fixed cost, c_fixed, was $150 for simple procedures and $300 for complex procedures. We assumed that simple procedures required a technician and either a physician or cytologist. We assumed technician time cost $30 per hour and physician/cytologist time cost $100 per hour. Thus, the variable cost for a simple procedure was $130 per hour. We assumed that complex procedures required a specialist (radiologist or endoscopist) and two assistants (technicians, nurses) at $30 per hour. We assumed specialist time cost $200 per hour. Thus, the baseline variable cost (c_var,o) for complex procedures was $260 per hour. With ROSE sampling, the cost of the cytologist added an additional $100 per hour so the total variable cost for a complex procedure with ROSE was $310 per hour. We assumed the cost per needle pass, c_np, was zero for simple procedures and $30 per pass for complex procedures (accounting for morbidity, special needles, etc).