Prostaglandin E (PGE₁) is necessary to maintain ductus arteriosus patency in many newborns with congenital heart disease. Because PGE₁ therapy commonly leads to fever, and given this population’s fragile state, a complete sepsis workup is often performed in febrile, but otherwise asymptomatic, patients. This practice of liberal evaluation with bacterial cultures, empiric antibiotic treatment, and delays in essential surgical intervention may result in poor resource utilization and lead to increased iatrogenic morbidity. This study sought to determine the incidence of fever and culture-positive infection in patients receiving PGE₁, and identify diagnostic variables that predict culture-positive infection. The study included a single-center retrospective review of all neonates receiving PGE₁ between 2011 and 2014. Logistic regression and receiver operator characteristic analysis were used to identify significant predictors of positive bacterial cultures. Among 435 neonates, 175 (40%) had fevers (≥ 38.3 °C) while concurrently receiving PGE₁, but only 9 (2%) had culture-positive infection and 1 (< 1%) had culture-positive bacteremia. Among 558 cultures collected, only 16 (3%) had bacterial growth. Multivariable analysis revealed age (p = 0.049, AUC 0.604), hospital length of stay (p = 0.002, AUC 0.764) and hypoxemia (p = 0.044, AUC 0.727) as the only significant predictors of positive cultures. Fever (p = 0.998, AUC 0.424) was not a significant predictor. In conclusion, given that fever occurs frequently in neonates receiving PGE₁ and it is a very non-specific marker and not a predictor of positive cultures, the common practice of complete sepsis workup should be re-examined in febrile patients at low risk of bacterial illness.