A probabilistic model for predicting the probability of no-show in hospital appointments

Abstract

The number of no-shows has a significant impact on the revenue, cost and resource utilization for almost all healthcare systems. In this study we develop a hybrid probabilistic model based on logistic regression and empirical Bayesian inference to predict the probability of no-shows in real time using both general patient social and demographic information and individual clinical appointments attendance records. The model also considers the effect of appointment date and clinic type. The effectiveness of the proposed approach is validated based on a patient dataset from a VA medical center. Such an accurate prediction model can be used to enable a precise selective overbooking strategy to reduce the negative effect of no-shows and to fill appointment slots while maintaining short wait times.

Appendix - Gaussian Mixture Models (GMM) and Expectation Maximization (EM) Algorithm

Gaussian Mixture Models (GMM) assume data points are drawn from a distribution that can be approximated by a mixture of Gaussian distributions. In this regard, assuming Q, the no-show rate of each clinic, is the feature vector, and k is the number of components (clinic clusters), the mixture model can be rewritten as:

$$ p\left( {Q|\Theta } \right) = \sum\nolimits_{{i = 1}}^k {{a_i}prob\left( {Q|{\theta_i}} \right)} $$

(11)

Where $\left\{ {{a_1},...,{a_k},{\theta_1},...,{\theta_k}} \right\}$ is the collection of parameters with $0 \leqslant {a_i} \leqslant 1,\forall i = 1,2,...,k$ and $\sum\nolimits_{{i = 1}}^k {{a_i} = 1}$ and $p\left( {Q|{\theta_i}} \right) = \frac{1}{{\sigma \sqrt {{2\pi }} }}\exp \left( { - \frac{{Q - {\mu_i}}}{{2\sigma_i^2}}} \right)$. Having as a set of n, i.i.d samples $Q = \left\{ {{q^{{(1)}}},{q^{{(2)}}},...,{q^{{(n)}}}} \right\}$ from the above model the log-likelihood function can be rewritten as:

$$ \begin{array}{*{20}{c}} {\log p\left( {Q|{\theta_i}} \right) = } \hfill \\{\log \prod\nolimits_{{j = 1}}^n {p\left( {{q^{{(j)}}}|\Theta } \right)} = \sum\nolimits_{{j = 1}}^n {\log } \sum\nolimits_{{i = 1}}^k {{\alpha_i}p} \left( {{q^{{(j)}}}|{\theta_j}} \right)} \hfill \\\end{array} $$

(12)

Here, the goal is to find Θ that maximizes the log-likelihood function:

$$ {\hat{\Theta }_{{MLE}}} = \arg \;\max \left\{ {\log \,p\left( {Q|\Theta } \right)} \right\} $$

(13)

The surface of the above likelihood function is highly nonlinear, and no closed form solution exists for the above likelihood function. One way to deal with this problem is by introducing a hidden variable Z:

$$ \begin{array}{*{20}{c}} {\log p\left( {Q,Z|{\theta_i}} \right) = } \hfill \\{\sum\nolimits_{{j = 1}}^n {\sum\nolimits_{{i = 1}}^k {z_i^{{(j)}}\log \left[ {{\alpha_i}p\left( {{q^{{(j)}}}|z_i^{{(j)}}{\theta_j}} \right)} \right]} } } \hfill \\\end{array} $$

(14)

and using Expectation Maximization (EM) algorithm as follows [33]:

• i. Initializing parameters Θ

• ii. Iterating the following until convergence:

$$ E - Step:\,Q\left( {\Theta |{\Theta^{{(t)}}}} \right) = {E_z}\log \left[ {p\left( {Q,Z|\Theta } \right)|{\Theta^{{(t)}}}} \right] $$

(15)

$$ M - Step:\,{\Theta^{{\left( {t + 1} \right)}}} = \arg \,\max Q\left( {\Theta |{\Theta^{{(t)}}}} \right) $$

(16)