Figures
Abstract
Background
Breastfeeding self-efficacy can be measured with the Breastfeeding Self-Efficacy Scale-Short Form (BSES-SF). Mothers with low BSES-SF scores stop exclusive breastfeeding prematurely, but specific interventions can prevent that undesirable outcome. Because those interventions can be expensive, often one must decide which mothers will receive them. For that purpose, a cut-off BSES-SF score would be useful, but none is available. Therefore, we aimed to assess the overall accuracy of BSES-SF scores as predictors of not practicing post-discharge exclusive breastfeeding, and to choose an appropriate cut-off score for making that prediction.
Methods
This is a secondary data analysis of an intervention study. Data from 378 women in two non-Baby-Friendly Hospitals were analyzed. Participants were women in their third trimester who were 16 years of age or older, were able to read and write Japanese, were expected to have a singleton birth, and had completed the BSES-SF before discharge. BSES-SF scores were measured before discharge. Breastfeeding status was assessed 4 weeks and 12 weeks postpartum. Receiver Operating Characteristic (ROC) curves were used to assess the predictive ability of the BSES-SF and to inform the choice of a cut-off point.
Results
For both of the ROC curves (4 and 12 weeks postpartum) the area under the curve was 0.74. To obtain a high sensitivity, a cut-off score of 50 was chosen. With that cut-off score the sensitivity was 79% and the specificity was 52% 4 weeks postpartum, and they were 77% and 52%, respectively, 12 weeks postpartum.
Conclusion
In conclusion, the BSES-SF has moderate overall accuracy to distinguish women who will not practice exclusive breastfeeding after discharge from those who will. At non-Baby-Friendly hospitals in Japan, interventions to support exclusive breastfeeding might be considered for new mothers who have BSES-SF scores that are less than or equal to 50.
Citation: Nanishi K, Green J, Taguri M, Jimba M (2015) Determining a Cut-Off Point for Scores of the Breastfeeding Self-Efficacy Scale–Short Form: Secondary Data Analysis of an Intervention Study in Japan. PLoS ONE 10(6): e0129698. https://doi.org/10.1371/journal.pone.0129698
Editor: Umberto Simeoni, Centre Hospitalier Universitaire Vaudois, FRANCE
Received: January 30, 2015; Accepted: April 19, 2015; Published: June 24, 2015
Copyright: © 2015 Nanishi et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper.
Funding: This study was funded by St. Luke's Life Science Institute in Tokyo. The grant we received is named as "Heisei 21 nenndo rinshouekigakukennkyuu nadoni kannsuru kennkyuujyosei" but there is no specific grant number (https://sllsi.or.jp/en/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Despite the proven benefits of exclusive breastfeeding [1, 2], many mothers cease it prematurely. In Japan, 96% of mothers intend to breastfeed during pregnancy [3] and almost all women initiate breastfeeding [4]; however, only 52% of mothers were found to predominantly breastfeed their infants 4 weeks postpartum [5].
Breastfeeding self-efficacy can affect breastfeeding initiation, duration, and exclusivity, and it is modifiable [6–10]. Breastfeeding self-efficacy is a mother’s perception of her ability to breastfeed [11], and it is said to be affected by four factors: (1) performance accomplishments (e.g., past breastfeeding experience), (2) vicarious experiences (e.g., watching other women breastfeeding, peer counseling), (3) verbal persuasion (e.g., encouragement from influential others), and (4) physiologic or affective states (e.g., pain, fatigue, anxiety, stress, etc.) [11, 12]. Interventions targeting those four factors can enhance breastfeeding self-efficacy, and also breastfeeding duration and exclusivity [9, 10, 13, 14].
To measure breastfeeding self-efficacy, the Breastfeeding Self-Efficacy Scale-Short Form (BSES-SF) was developed in Canada in 2003 [8]. BSES-SF data are collected via a self-administered questionnaire, which has been translated from English into various languages, including Japanese [15–19]. Results of psychometric tests of the BSES-SF indicate that it can be used in various cultures and with women of various ages.
Mothers with lower BSES-SF scores are more likely to wean their babies from exclusive breastfeeding prematurely than are mothers with higher scores [8, 15, 16, 18, 19]. A longitudinal study with 262 Japanese pregnant women, which included reliability testing and validation testing, indicated that the Japanese version of the BSES-SF could be used to measure breastfeeding self-efficacy. Women were significantly more likely to discontinue exclusive breastfeeding by 4 weeks postpartum if they had a BSES-SF score lower than the sample mean [16]. Therefore, those scores can be used to predict which mothers will stop breastfeeding prematurely, and additional interventions can be offered to those mothers.
For clinical use, a cut-off point among BSES-SF scores would be useful. By using a cut-off point, health professionals can identify mothers who need additional breastfeeding support for exclusive breastfeeding. To the best of our knowledge, no such cut-off point has yet been proposed.
The percentage of mothers who practice exclusive breastfeeding is low, especially in non-Baby-Friendly Hospitals (nBFH) [10]. Thus, we aimed to determine a cut-off point for scores obtained before discharge to identify mothers in nBFHs who were at risk of discontinuing exclusive breastfeeding by 4 weeks and 12 weeks postpartum. We also sought to assess the overall performance of BSES-SF scores as predictors of not practicing exclusive breastfeeding after discharge.
Materials and Methods
Study design
This was a secondary analysis of data collected in a self-efficacy intervention study. Those data were collected between August 2010 and January 2011 in Japan [10]. A total of 781 women participated in that intervention study, of whom 574 completed the BSES-SF before discharge. Of those 574 women, 196 were in BFHs and 378 were in nBFHs. For the present study we used data collected from the 378 women at nBFHs. Data from the 196 women at BFHs was not used to avoid influence from the self-efficacy intervention. The intervention improved breastfeeding self-efficacy through 4 weeks postpartum in BFH (p = 0.037), however, the positive effect was not observed in nBFHs (p = 0.982) [10]. Another reason for the use of data from nBFH was that in Japan the vast majority women deliver in nBFH. While 2,576 facilities provide medical care for delivery in Japan [20], only 75 have been certified as BFH [21]. A cut-off determined using data from nBFH would therefore be generalizable to the majority of women in Japan.
All of the eligible participants were pregnant women in their third trimester who were 16 years of age or older, were able to read and write Japanese, were expected to have a singleton birth, and had completed the BSES-SF before discharge. Exclusion criteria were as follows: (a) an intention to formula-feed or a contraindication to breastfeeding, (b) a pregnancy that ended in either miscarriage or stillbirth, and (c) a medical condition of the women and/or the infant that could significantly interfere with breastfeeding. Further details of the larger study in which these data were collected have been published elsewhere [10].
The current study was approved by the Research Ethics Committee of the Graduate School of Medicine at The University of Tokyo. The data was collected using self-administered questionnaire. Women who read a booklet describing the detail of the study and agreed to participate in the study returned the completed questionnaire. Written consent was not collected in accordance with the Japan’s ethics guideline of epidemiological study. The Research Ethics Committee approved the inclusion of minors without parental consent because: 1) the study was explained in plain language such that it could be easily understood by teenage women, and 2) women often attend the antenatal appointment alone.
Instrument
We used scores on the Japanese version of the BSES-SF completed before discharge. The BSES-SF comprises 14 items. Each item has 5 response choices on a Likert-type scale, from “not at all confident” (1 point) to “always confident” (5 points). All the items are presented positively and their scores are summed to produce a total score ranging from 14 to 70 [8]. Higher total scores indicate higher levels of breastfeeding self-efficacy.
Post-discharge exclusive breastfeeding 4 weeks and 12 weeks postpartum
Breastfeeding status was assessed 4 weeks postpartum (by self-administered questionnaire, at the baby’s regular checkup) and again 12 weeks postpartum (by self-administered questionnaire, sent by postal mail with a stamped return envelope). In this study, we were not concerned with breastfeeding practices during hospitalization, so we measured post-discharge exclusive breastfeeding, which we defined as not giving infants any foods or liquids other than breast milk after discharge from the hospital.
Characteristics of the mothers
To understand the characteristics of the participants, factors known to be associated with breastfeeding were measured during pregnancy or before discharge through self-administered questionnaires. These included age, marital status, economic status, parity, mistimed or unwanted pregnancy, previous experience of breastfeeding, perceived support for breastfeeding from a partner, plan to return to work within 6 months after birth, mode of delivery, infant’s birth weight [16, 22–26]. We also collected data on intentions regarding infant feeding for the first five-to-six months [6, 24], attitude to infant feeding [6, 27] (Iowa Infant Feeding Attitude Scale score [27]), symptoms of depression [23] (Edinburgh Postnatal Depression Scale score [28, 29]), and general family support [22, 24] (Family Apgar score [30]).
Analysis
Receiver operating characteristic (ROC) curves were used to determine the utility of BSES-SF scores for differentiating between mothers who were at risk to not breastfeed exclusively after discharge from those who were likely to breastfeed exclusively, and to aid in the decision on a cut-off point [31]. An ROC curve is constructed by plotting the true-positive ratio (sensitivity), against the false-positive ratio (1 –specificity). One ROC curve was constructed with the data on post-discharge exclusive breastfeeding 4 weeks postpartum, and one was constructed with data from 12 weeks postpartum. Both were constructed with BSES-SF scores at discharge.
The area under the ROC curve (AUC) was computed because it is an index of the discriminative utility of a test across the full range of possible cut-off points [31]. Higher values of the AUC indicate better discrimination. An AUC of 0.5 indicates a chance result, and the highest possible value of an AUC is 1.0 [32]. Fisher et al. proposed three categories: a test with an AUC greater than 0.9 can be said to be highly accurate, while 0.7–0.9 indicates moderate accuracy and 0.5–0.7 indicates low accuracy [32].
When false-positive and false-negative results are equally undesirable, sensitivity and specificity may be equally weighted, and in such cases Youden’s J (J = sensitivity + specificity– 1) can be used to identify the optimal cut-off point [33]. J is the maximum vertical distance from the ROC curve to the line between (0, 0) and (1, 1). When sensitivity and specificity are equally weighted, the optimal cut-off point is the point with the highest value of J. However, when predicting which mothers will not practice post-discharge exclusive breastfeeding, there is no reason to assume that false-negative and false-positive results are equally undesirable, and so there is no reason for sensitivity and specificity to be equally weighted. In such cases one needs to account for “the relative loss (cost) of a false negative as compared with a false positive” [33], that is, the “cost ratio of false-positive and false-negative results” [34].
With the goal of identifying mothers who are not likely to breastfeed exclusively after discharge, we considered a high sensitivity to be more desirable than a high specificity. That is because we considered false-positive results to be more tolerable than false-negative results (i.e., we considered false-positive results to be less costly, less serious, than false-negative results). We arrived at that conclusion by considering the likely consequences of false-negative and false-positive results, as follows. As a consequence of a false-positive result, a health worker would offer an unnecessary intervention, which might result in some unnecessary healthcare expenditure. If resources were very limited, it also might result in the loss of a chance to provide a needed service to a different mother. Nonetheless, a mother who actually does not need the intervention might realize that fact and refuse the unnecessary intervention, or the health worker may eventually notice that it is unnecessary and withdraw it. In contrast, we judged the consequences of a false-negative result to be quite serious. A false-negative result would cause an intervention not to be provided to a mother in need. As a consequence, both the mother and the infant would lose all of the benefits of exclusive breastfeeding.
We could not assign exact numerical values to the consequences of false-positive and false-negative results. However, “the exact cost ratio is not needed in a decision theoretic framework; all that is needed is a qualitative indication of which of the two classification errors is more important to avoid” [35], and we judged that in the present context false negatives are more important to avoid. Therefore, for the cut-off point we decided to choose a BSES-SF score corresponding to a sensitivity higher than the sensitivity at the point with the greatest J.
Results
Characteristics of participants
Table 1 shows the characteristics of the participants. The majority of them did not have family budget concerns and their education levels were beyond the high-school level. While about half of them had no previous experience of exclusive breastfeeding, the majority of the participants intended to exclusively breastfeed for the first six months after their child’s birth.
Reliability, BSES-SF scores, and post-discharge exclusive breastfeeding
Cronbach’s α of the BSES-SF in this study was 0.94. The mean (SD) of the BSES-SF scores was 42.39 (10.57). The percentages of mothers who reported practicing post-discharge exclusive breastfeeding 4 weeks and 12 weeks postpartum were 15% (48/330) and 9% (29/307), respectively.
Cut-off score for post-discharge exclusive breastfeeding
The ROC curves in Figs 1 and 2 show the accuracy of the BSES-SF scores before discharge as predictors of not practicing post-discharge exclusive breastfeeding at each possible cut-off point. The AUCs were 0.74 (95% CI: 0.70 to 0.80) 4 weeks postpartum and 0.74 (95% CI: 0.66 to 0.82) 12 weeks postpartum. For the two ROC curves, Table 2 shows the coordinates of each point on each curve (that is, sensitivity and 1 –specificity), and the value of J at each point. J was greatest at BSES-SF scores of 44 (4 weeks postpartum) and 49 (12 weeks postpartum). At a BSES-SF score of 50, the sensitivity and specificity were 79% and 52% (4 weeks), and 77% and 52% (12 weeks). With regard to post-discharge exclusive breastfeeding 4 weeks postpartum in this sample, if the score with the highest value of J were used as the cut-off point, the result would be 92 false-negative results. In contrast, if a score of 50 were used as the cut-off point, there would be only 58 false-negative results (Table 3). Given the goal of a high sensitivity (even if that entails low specificity), the cut-off of 50 would be preferred in this context.
The area under the ROC curve is 0.74. The arrow indicates the point corresponding to a BSES-SF score of 50. At that point the sensitivity is 0.79 and the specificity is 0.52.
The area under the ROC curve is 0.74. The arrow indicates the point corresponding to a BSES-SF score of 50. At that point the sensitivity is 0.77 and the specificity is 0.52.
Discussion
This was the first attempt to establish a cut-off score on the BSES-SF while assessing the overall performance of BSES-SF scores as predictors of not practicing exclusive breastfeeding after discharge. The results show again that the Japanese version of the BSES-SF can be used to identify women who are likely to stop exclusive breastfeeding prematurely. The AUCs were above 0.7, both 4 weeks and 12 weeks postpartum. Therefore, according to the categories proposed by Fischer et al. [32], the BSES-SF has “moderate” overall accuracy in this context.
Based on the results of this study, we recommend that a BSES-SF score of 50 be used as a cut-off when screening to predict which mothers will not practice post-discharge exclusive breastfeeding, whether 4 weeks or 12 weeks postpartum. Scores less than or equal to 50 can be taken as indicators of a need for interventions to support exclusive breastfeeding. When used as a cut-off, that score correctly identified 79% of the mothers who were not practicing post-discharge exclusive breastfeeding 4 week postpartum, and for the 12-week results the comparable value was 77%.
Any decision regarding a cut-off point involves a trade-off between sensitivity and specificity [36]. For a given test, if one changes the cut-off point to increase sensitivity, then specificity will decrease. As noted above, if the net expected costs of false-negative and false-positive results can be quantified precisely, then the ideal balance between sensitivity and specificity can be computed and the ideal cut-off point can be identified [37]. However, as is the case in almost all such analyses, in this study we could not precisely quantify the total expected costs of false-negative and false-positive BSES-SF results. Nonetheless, “the exact cost ratio is not needed in a decision theoretic framework” [35]. It seems reasonable to consider that for the purposes of this study the cost of a false-negative result would be greater than that of a false-positive result. That led us to favor sensitivity over specificity, and to recommend a score of 50 as the cut-off point. In other settings a different cut-off might be more appropriate, particularly if the word “settings” is interpreted broadly. It should include economic circumstances and practical limitations. For example, if breastfeeding support can be offered at a very low cost, then a cut-off with a higher sensitivity can be used, but if breastfeeding support is very costly, then one might need to use a cut-off that gives less sensitivity and greater specificity.
In this study we used only the BSES-SF total scores. Clinicians can also use the responses to the various BSES-SF questions to design individualized interventions. By checking the responses to each item and understanding what made a particular mother’s score low, a clinician can identify areas in which additional breastfeeding support for that mother should be focused [38].
One limitation of this study is that the longest follow-up was only 12 weeks, while international [39] and national [2, 40] organizations recommend exclusive breastfeeding for the first 6 months of life. We cannot be sure how accurate the BSES-SF scores would be as predictors of events beyond 12 weeks postpartum. Another limitation stems from the fact that the data we used came from a self-efficacy intervention study. That intervention did not have a statistically significant effect on breastfeeding self-efficacy or exclusivity [10], but replication studies in other groups would be helpful. Finally, we note again that the participants were recruited through convenience sampling at two nBFHs in Japan, and that the majority were married, educated, and not in financial distress. Further study is needed of the relevance of the recommended cut-off point to other groups such as socioeconomically deprived women.
Conclusions
This is, to the best of our knowledge, the first study to establish a cut-off point for BSES-SF scores in Japan. That cut-off point can be used to identify mothers who are likely to stop exclusive breastfeeding prematurely. We can recommend that, at nBFH in Japan, interventions to support exclusive breastfeeding should be considered for new mothers whose scores on the Japanese version of the BSES-SF are less than or equal to 50.
Additional study is warranted to find out whether BSES-SF scores can be used to predict exclusive breastfeeding 6 months postpartum, and also to answer questions about generalizability to other groups.
Acknowledgments
We appreciate the cooperation of the women who participated in the study and of the staff members of the hospitals. We are also grateful to Xiu-Fang Wang, Sumiyo Okawa, Yuko Matsuoka, and Kayoko Yoshikawa for their support in data collection.
Author Contributions
Conceived and designed the experiments: KN JG MT MJ. Performed the experiments: KN JG. Analyzed the data: KN JG MT. Wrote the paper: KN JG MT MJ.
References
- 1. Ip S, Chung M, Raman G, Chew P, Magula N, DeVine D, et al. Breastfeeding and maternal and infant health outcomes in developed countries. Evid Rep Technol Assess (Full Rep). 2007;(153):1–186. pmid:17764214.
- 2. Section on breastfeeding American Acadamy of Pediatics. Breastfeeding and the use of human milk. Pediatrics. 2012;129(3):e827–41. peds.2011-3552 [pii] pmid:22371471.
- 3.
Equal Employment Children and Family Section, Japanese Ministry of Health, Labour and Welfare. Reports on infant and young child feeding 2005. Tokyo,Japan: Japanese Ministry of Health, Labour and Welfare, 2006.
- 4.
Department of Statistics and Information. First Longtudinal Survey of Babies in the 21st Century Tokyo, Japan: Ministry of Health, Labour, and Welfare; 2001 [cited 2013 19 December].
- 5.
Ministry of Health Labour and Welfare. Survey on the growth of infants and preschool children in Japan, 2010. Tokyo: Equal employment, Children and Families Bureau, Ministry of Health Labour and Welfare, 2011.
- 6. Meedya S, Fahy K, Kable A. Factors that positively influence breastfeeding duration to 6 months: a literature review. Women Birth. 2010;23(4):135–45. pmid:20299299.
- 7. Dennis CL, Faux S. Development and psychometric testing of the Breastfeeding Self-Efficacy Scale. Res Nurs Health. 1999;22(5):399–409. pmid:10520192.
- 8. Dennis CL. The breastfeeding self-efficacy scale: psychometric assessment of the short form. J Obstet Gynecol Neonatal Nurs. 2003;32(6):734–44. pmid:14649593.
- 9. Nichols J, Schutte NS, Brown RF, Dennis CL, Price I. The impact of a self-efficacy intervention on short-term breast-feeding outcomes. Health Educ Behav. 2009;36(2):250–8. 1090198107303362 [pii] pmid:17893124.
- 10.
Otsuka K, Taguri M, Dennis CL, Wakutani K, Awano M, Yamaguchi T, et al. Effectiveness of a Breastfeeding Self-efficacy Intervention: Do Hospital Practices Make a Difference? Matern Child Health J. 2013. https://doi.org/10.1007/s10995-013-1265-2 pmid:23592322.
- 11. Dennis CL. Theoretical underpinnings of breastfeeding confidence: a self-efficacy framework. J Hum Lact. 1999;15(3):195–201. pmid:10578797.
- 12. Bandura Albert. Self-efficacy:Toward a unifying theory of behavioral change. Psychological Review. 1977;84(2):191–215. pmid:847061
- 13. McQueen KA, Dennis CL, Stremler R, Norman CD. A pilot randomized controlled trial of a breastfeeding self-efficacy intervention with primiparous mothers. J Obstet Gynecol Neonatal Nurs. 2011;40(1):35–46. pmid:21244493.
- 14. Noel-Weiss J, Rupp A, Cragg B, Bassett V, Woodend AK. Randomized controlled trial to determine effects of prenatal breastfeeding workshop on maternal breastfeeding self-efficacy and breastfeeding duration. J Obstet Gynecol Neonatal Nurs. 2006;35(5):616–24. JOGN77 [pii] pmid:16958717.
- 15. Dennis CL, Heaman M, Mossman M. Psychometric testing of the breastfeeding self-efficacy scale-short form among adolescents. J Adolesc Health. 2011;49(3):265–71. S1054-139X(10)00829-3 [pii] pmid:21856518.
- 16. Otsuka K, Dennis CL, Tatsuoka H, Jimba M. The relationship between breastfeeding self-efficacy and perceived insufficient milk among Japanese mothers. J Obstet Gynecol Neonatal Nurs. 2008;37(5):546–55. JOGN277 [pii] pmid:18811774.
- 17. Wutke K, Dennis CL. The reliability and validity of the Polish version of the Breastfeeding Self-Efficacy Scale-Short Form: translation and psychometric assessment. Int J Nurs Stud. 2007;44(8):1439–46. pmid:16982056.
- 18. Aluş Tokat M, Okumuş H, Dennis CL. Translation and psychometric assessment of the Breast-feeding Self-Efficacy Scale-Short Form among pregnant and postnatal women in Turkey. Midwifery. 2010;26(1):101–8. S0266-6138(08)00043-0 [pii] pmid:18541350.
- 19. Oliver-Roig A, d'Anglade-González ML, García-García B, Silva-Tubio JR, Richart-Martínez M, Dennis CL. The Spanish version of the Breastfeeding Self-Efficacy Scale-Short Form: reliability and validity assessment. Int J Nurs Stud. 2012;49(2):169–73. S0020-7489(11)00328-2 [pii] pmid:21930270.
- 20.
Statistics Bureau, Ministry of Internal Affaires and Communications. Dynamic and static statistics of health facilities in Japan in 2011. Tokyo, Japan: 2011.
- 21.
Japan Breast Feeding Association. List of the Baby Friendly Hospitals in Japan Tokyo, Japan: Japan Breast Feeding Association; 2013 [cited 2013 September 23]. Available from: http://www.bonyu.or.jp/index.asp?patten_cd=12&page_no=11.
- 22. Dennis CL. Breastfeeding initiation and duration: a 1990–2000 literature review. J Obstet Gynecol Neonatal Nurs. 2002;31(1):12–32. pmid:11843016.
- 23. Callen J, Pinelli J. Incidence and duration of breastfeeding for term infants in Canada, United States, Europe, and Australia: a literature review. Birth. 2004;31(4):285–92. pmid:15566341.
- 24. Thulier D, Mercer J. Variables associated with breastfeeding duration. J Obstet Gynecol Neonatal Nurs. 2009;38(3):259–68. JOGN1021 [pii] pmid:19538614.
- 25. Kaneko A, Kaneita Y, Yokoyama E, Miyake T, Harano S, Suzuki K, et al. Factors associated with exclusive breast-feeding in Japan: for activities to support child-rearing with breast-feeding. J Epidemiol. 2006;16(2):57–63. pmid:16537985.
- 26. Taylor JS, Cabral HJ. Are women with an unintended pregnancy less likely to breastfeed? J Fam Pract. 2002;51(5):431–6. pmid:12019050.
- 27. de la Mora A, Russell DW, Dungy CI, Losch M, Dusdieker L. The Iowa Infant Feeding Attitude Scale: Analysis of Reliability and Validity. Journal of Applied Social Psychology. 1999;29(11):2362–80.
- 28. Cox JL, Holden JM, Sagovsky R. Detection of postnatal depression. Development of the 10-item Edinburgh Postnatal Depression Scale. Br J Psychiatry. 1987;150:782–6. pmid:3651732.
- 29. Okano T, Murata M, Masuji S.Validation and reliability of a Japanese version of the EPDS. Archives of Psychiatric Diagnosis and Clinical Evaluation. 1996;7(4):525–33. pmid:1997169221.[in Japanese]
- 30. Smilkstein G, Ashworth C, Montano D. Validity and reliability of the family APGAR as a test of family function. J Fam Pract. 1982;15(2):303–11. Epub 1982/08/01. pmid:7097168.
- 31. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. pmid:7063747.
- 32. Fischer JE, Bachmann LM, Jaeschke R. A readers' guide to the interpretation of diagnostic test properties: clinical example of sepsis. Intensive Care Med. 2003;29(7):1043–51. pmid:12734652.
- 33. Perkins NJ, Schisterman EF. The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve. Am J Epidemiol. 2006;163(7):670–5. pmid:16410346; PubMed Central PMCID: PMCPMC1444894.
- 34. Greiner M, Pfeiffer D, Smith RD. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev Vet Med. 2000;45(1–2):23–41. pmid:10802332.
- 35. Smits N. A note on Youden's J and its cost ratio. BMC Med Res Methodol. 2010;10:89. pmid:20920288; PubMed Central PMCID: PMCPMC2959030.
- 36. Rey JM, Morris-Yates A. Diagnostic accuracy in adolescents of several depression rating scales extracted from a general purpose behavior checklist. J Affect Disord. 1992;26(1):7–16. pmid:1430671.
- 37. Smits N, Smit F, Cuijpers P, De Graaf R. Using decision theory to derive optimal cut-off scores of screening instruments: an illustration explicating costs and benefits of mental health screening. Int J Methods Psychiatr Res. 2007;16(4):219–29. pmid:18188835.
- 38. Dennis CL, Faux S. Development and psychometric testing of the Breastfeeding Self-Efficacy Scale. Res Nurs Health. 1999;22(5):399–409. [pii]. pmid:10520192.
- 39.
WHO, UNICEF. Baby-Friendly Hospital Initiative-Revised, Updated, and Expanded for Integrated Care 2009 [updated 23 November 2009; cited 2011 December 28]. Available from: http://www.unicef.org/nutrition/index_24850.html.
- 40.
Ministry of Health Labour and Welfare. The guidelines for health professionals to support infant and young child feeding. Tokyo,Japan: Ministry of Health Labour, and Welfare; 2007. p. 1–80.