The INFVo perceptual rating scale for substitution voicing: development and reliability

In this paper, an experimental study of inter-judge consistency for the different dimensions of a recently proposed new scale for the rating of substitution voices is presented. The IINFVo rating scale tries to score five parameters, namely impression, intelligibility, noise, fluency and voicing. Each parameter is scored between 0 (very good substitution voicing) and 10 (very deviant substitution voicing) on a visual analogue scale. Inter-judge consistencies were measured among semi-professional as well as among professional jury members. The consistencies among semi-professionals, expressed as Pearson correlation coefficients, ranged from moderate to good (0.57–0.68), whereas those among professionals were good to excellent (0.82–0.87) and compared favourably to consistency figures published for traditional perceptual evaluation scales such as the GRBAS scale for laryngeal dysphonia. Since there is a strong correlation between the scores of impression and intelligibility, and since intelligibility is hard to score by non-native listeners, we suggest taking the mean of the two scores as the “impression” of a modified dimensional INFVo rating scale. Our experiments demonstrate that the INFVo rating scale has good potential to become a routine perceptual evaluation method in a multidimensional assessment protocol for substitution voicing.