Handgrip strength (HGS) is a useful measure of health-related quality of life and general muscle strength. Serum total bilirubin (T-B) may present potential beneficial effects in preventing oxidative changes which are associated with a risk of metabolic syndrome and the development of cardiovascular disease. Limited information is available regarding whether HGS is an independent confounding factor for serum T-B. The study participants were 214 men aged 71 ± 8 (mean ± standard deviation) years and 302 women aged 71 ± 7 years that were enrolled consecutively from among paticipants aged ≥ 50 years through an annual check-up process. We evaluated the relationship between serum T-B and confounding factors within each sex. HGS related significantly with serum T-B in both men (r = 0.156, p = 0.023) and women (r = 0.173, p = 0.003). Multiple linear regression analysis showed that in men, HGS (β = 0.173) as well as smoking status (β = −0.147), exercise habit (β = 0.138), low-density lipoprotein cholesterol (β = 0.146), and hemoglobin A1c (HbA1c) (β = −0.198) were significantly and independently associated with serum T-B. In women, HGS (β = 0.159) as well as smoking status (β = −0.116), high-density lipoprotein cholesterol (β = 0.159), and HbA1c (β = −0.161) were significantly and independently associated with serum T-B. Multivariate-adjusted serum T-B levels were significantly lower in subjects with the lowest HGS level in both sexes. Increased HGS is strongly associated with increased serum T-B, independent of confounding factors in both sexes.