Exploring cranial macromorphoscopic variation and classification accuracy in a South African sample

The sample consisted of 660 crania of black, white and coloured South Africans (Table 1). The South African population is diverse and consists of four major groups: South African blacks (81.0%), whites (7.7%) and coloureds (8.8%) make up the majority of the population; the remaining 2.6% of the population consists of individuals classified as Asian and Indian [24] (Statistics South Africa, 2022). Each group has a unique history within the country leading to the vast heterogeneity observed within and among the groups. Black South Africans descend from Bantu-speaking groups that migrated throughout sub-Saharan Africa from western-central Africa approximately 3000 to 5000 years ago [25]. Further divisions among the southern Bantu-speakers based on factors associated with kinship, religion and language resulted in the numerous subgroups residing in southern Africa today [26]. Colonization of the Cape during the 17th century introduced European settlers to South Africa, shaping the heritage of white South Africans. The settlers were mainly of Dutch origin, with additional contributions from French Huguenots and Germans that arrived in the 18th century. Late in the 18th century South Africa was also colonized by the English [27]. Coloured South African refers to a self-identified group unique to South Africa. The group is a result of the complex history of South Africa with genetic contributions from Khoe-San (considered indigenous South Africans), Bantu-speakers, Europeans, as well as Indians and other Asian groups that were brought to South Africa as slaves to maintain the Cape colony. The complex population structure and history of the coloured South Africans manifests as a genetically and skeletally heterogeneous group with substantial variation [8]. While the varying origins of each group resulted in a uniquely heterogeneous population with distinct structures, the group differences employed to attempt population affinity estimations persisted as a result of socio-political boundaries. Sociocultural identity in South Africa is based on the categorizations assigned to individuals during the Apartheid era, which contributed to widespread endogamy among groups [28].

The crania were sampled from the Pretoria Bone Collection (University of Pretoria) and the Kirsten Collection (Stellenbosch University) in South Africa. The remains accessioned into the collections are of documented sex, age at death, and peer-reported population affinity [29, 30]. Ethical approval (770/2018) to conduct the study was obtained from the Faculty of Health Sciences Research Ethics Committee at the University of Pretoria.

A total of 17 MMS traits were visually assessed and scored following the methodology described by Hefner [12] and Plemons and Hefner [13] as used in the Macromorphoscopic Traits collection module (MMS version 1.6.1) (Table 2). The MMS module was used to capture the scores for each individual. Where traits are bilaterally expressed, only the left side was recorded. If the left side was not available, the right side was used.

All statistical analyses were completed using the software R version 4.1.0 [31], and included assessments of observer agreement, exploratory analyses, and the creation of classification models. Ten crania were randomly selected to test observer agreement. Two observers scored the crania; both observers are experienced with skeletal analyses, but only one observer has extensive experience with the traits. The observers discussed the trait definitions and methodology prior to collecting the scores for analysis. The repeatability of the traits was assessed with Cohen’s kappa using the irr package in R; different weights were given to the scores depending on the data structure of the trait. Standard, unweighted kappa was used for the ordinal scores where the different trait states are unranked. For the ranked scores (i.e., ANS, INA, MT, NAW, and PZT), a quadratic weighted kappa was employed. Calculated kappa values can range from − 1 to 1, where values closer to 1 indicate greater agreement. No universally accepted cut-off point for satisfactory observer agreement currently exists. However, to be consistent with nomenclature when describing the strength of agreement associated with kappa statistics, the parameters proposed by Landis and Koch [32] was used.

The MMS scores were used to create frequency distributions to assess the occurrence of each trait per group. Kruskal-Wallis tests were used to identify if any traits demonstrated significant differences among the populations. Kruskal-Wallis is a non-parametric test used to compare three or more groups which operates under the assumptions of independence of scores but is not bound by assumptions of normality or homogeneity of variance [33]. Additionally, a post-hoc Dunn’s multiple comparisons test (with a Holm’s adjustment) was used to further explore differences in the trait frequencies among the populations. The Holm’s adjustment counteracts the effects of multiple comparisons and prevents increased probability of Type I errors occurring [34]. More specifically, where Kruskal-Wallis indicates the presence of significant differences, the Dunn’s test indicates which groups in a multiple comparison differ from one another to better interpret group overlap.

Random forest models (RFM) were created to classify the crania according to population affinity, as well as population affinity and sex concurrently. RFM is a non-parametric machine learning method that was introduced as an improvement upon decision trees [35]. Decision trees are a type of classification model that uses sequential splitting values (such as MMS traits) to predict the probability of an unknown belonging to a certain class (i.e., population affinity) to separate a dataset into groups [36]. Within each data split, known as “nodes” in the tree, the variable that is most strongly associated with the response variable (a specific group) is selected for the next split until a stopping condition is met. In the case of the current study, the stopping condition is an overall population estimate based on the ensemble of multivariate trees. The overall population estimate is reached by combining the most likely response from all of the nodes, or in the case of RFM, all of the trees in the ensemble. This is achieved by means of voting in classification; simply put, the population group that receives the most “votes” from the trees is returned as the overall prediction [35]. A total of 2500 classification trees were used for each model with four variables at each split. Furthermore, RFM ranks the importance of each variable included in the classification ensemble, giving an indication of which variables are most discriminatory in the model and which variables do not contribute to the classification [14]. Two measures of variable importance were employed, namely the mean decrease in the Gini index, and the mean decrease in the permutation accuracy. The Gini index measures how much each predictor variable contributes to the overall reduction in node impurity achieved by splitting the data on each variable across all trees in the forest. The mean decrease is calculated for each variable by averaging the reduction in the Gini index across all nodes where that specific variable is used for splitting. The Gini criterion has been shown to favour variables that have many categories (or trait states) and can be influenced by highly correlated variables; thus, the Gini index should not be used as the only indicator of variable importance [37]. The mean decrease in the permutation accuracy was also assessed, where the relative importance of each predictor variables is calculated by measuring the decrease in model accuracy across all trees upon removal of the variable. With both measures of variable importance, the higher the value, the more a variable contributes to the classification (i.e. the more important a variables is to the model). Finally, out-of-bag observations can be used to gauge the external prediction accuracy of the tree (comparable to leave-one-out cross-validation commonly used with discriminant analysis). The original training data is randomly sampled with replacement for each tree, which generates a smaller subset of data for each tree; essentially this is the training data. The observations excluded from the training data, or the out-of-bag observations, are a random subset of data that is essentially an internal test sample. The tree will then be used to classify the test sample to obtain a more realistic classification accuracy [38]. In the case of missing data, the mode was calculated for each trait per each sex and population group separately. The mode was used as an imputation value specifically because it appears the most in a set of values which in this case, is a population and sex group, most individuals are likely to depict that value. Data imputation was only performed when variables had less than 10% of the observations missing. For variables where more than 10% of the observations would have to be replaced, the variable was omitted from the model. After the missing data were imputed, the sample was divided so that 75% was used as the training set to create the model, and the remaining 25% was the holdout set to test the accuracy of the model on an independent set of crania. The randomForest package was used to generate the RFM classifications [39].