Latent spaces of generative models for forensic age estimation

To evaluate the effectiveness and robustness of generative models for forensic age estimation, we utilized two distinct datasets that reflect the diversity of data sources encountered in real-world scenarios.

The first dataset, used for training and validation, comprised a total of 486 postmortem computed tomography (PMCT) scans collected between 2010 and 2017 by the Office of the Medical Investigator (OMI) of the University of New Mexico [17], a centralized, university-based institution operating as a state-level medicolegal death investigation agency. These data were selected because the OMI provides a dataset characterized by standardized imaging protocols and a broad diversity of cases. The availability of a curated, anonymized dataset enabled efficient research without additional ethical and logistical hurdles. Furthermore, the diversity of cases ensures the development of a model that can be applied across various demographic and geographic contexts. To obtain access to the dataset, a formal research request was submitted, which included the specific research objectives, hypotheses and potential benefits of the study, ensuring that the data would be used for legitimate scientific purposes. The OMI research committee reviewed and approved our research request and a written confirmation of approval was given. Although all personal identifiers were fully anonymized, the dataset still included comprehensive metadata about the cases, such as age, sex and cause of death. The dataset included scans from both male and female individuals aged between 10 and 30 years who passed away under various circumstances, including natural causes, accidents and violent deaths. The individuals underwent PMCT scanning as part of routine for forensic procedures to help determine the cause of death. The age range was specifically selected due to its association with distinct skeletal development patterns. Only individuals without known traumatic bone fractures affecting MCE were included. Additionally, only patients who had undergone whole-body PMCT scans as part of the forensic investigation were considered. The scans, stored in the well-established DICOM format, were obtained using a Philips Brilliance Big Bore CT Scanner (Philips, Amsterdam, The Netherlands). The technical parameters were standardized, featuring a slice thickness of 3.0 mm, a scan speed between 0.567 s and 0.933 s, a current strength ranging from 104 mA to 254 mA, and a voltage setting of 120 kV.

The second dataset of 40 PMCT scans for testing was provided by the Institute of Legal Medicine at the University Medical Center Hamburg-Eppendorf (UKE). These scans were obtained using a Philips Incisive 128 CT Scanner (Philips, Amsterdam, The Netherlands). The technical specifications included a slice thickness of 0.67–1.0 mm, a scan speed of 0.3–0.6 s, a voltage setting of 120 kV and a current strength of 104–399 mA. As the scans in the second dataset were recorded as part of routine forensic procedures and included individuals outside the relevant age range (10–30 years), we manually filtered the UKE scans to retain only those within this age range, excluding all others and then anonymize all scans technically before investigating them. Subsequently, we harmonized the datasets by subsampling the second dataset with a higher resolution to the slice thickness of the training dataset.

We focused on the assessment of the MCE for age estimation as this region provides a reliable indicator of skeletal maturity [14]. For the validation of the approaches, a human rater (AC), who has a solid foundation in the annotation of image data, annotated corresponding masks of the MCE on the PMCT scans. To ensure the accuracy of the annotations, random samples of the results were reviewed by a clinician (AJW), who confirmed the reliability of the annotations. To increase the data volume and facilitate the segmentation process, we virtually split the PMCT and the corresponding mask data vertically, along the midline through the sternum, treating each side of the MCE as an independent sample. For the model input, only the MCE regions were specifically masked. Further preprocessing included random cropping for data augmentation to prevent overfitting by limiting the model’s options to remember samples despite the limited amount of data. Additionally, the image intensities were normalized based on Hounsfield units to highlight bone contrasts and minimize irrelevant information. Those masks defined the ROIs used as input for the generative models.

In selecting the model architecture, we considered various paradigms of generative models. For the advantages of an explicit latent space, we focused on models providing an explicit density. Given their popularity and extensive literature regarding attractive properties for their latent spaces, we chose variational autoencoders (VAEs) directly approximating this density for this study. These models from the autoencoding framework share the usage of two distinctive components: their encoder component maps data points from a high-dimensional data space into a low-dimensional space. The decoder component is trained to map these latent embeddings back to the high-dimensional space. As all data must run through, the model is forced to “compress” the spatial information within the images in a far smaller latent space. Crucially, VAEs treat the vectors from the latent space as proper random variables. Any vector within the latent space, despite being created from a sample, randomly sampled or otherwise created, could be converted back to an image. A more rigid mathematical formulation and details regarding the training by maximizing the evidence lower bound can be found in Kingma et al. [7].

While VAEs have proven to be powerful in knowledge extraction, there is no guarantee regarding the precise information stored in the latent representation and their accessibility for other tasks. Accordingly, a body of literature focuses on additional training objectives for VAEs to separate the underlying factors of variation into variables with semantic significance, a concept known as disentangled learning. To assess the impact of these learning objectives, we first consider a standard VAE as the baseline model. Additionally, we explored two more specialized VAE variants: ArVAE [12] which enforces a structured relationship between the latent space and specific continuous attributes, allowing for controlled manipulation of these attributes. Lastly, we employed AttriVAE [2] which incorporates the regression task into the training. These training strategies may enable a more interpretable representation, focusing on the influence of these attributes on the data.

The general architecture of the used deep-learning model is described in Fig. 1 and [7]. The same architecture was used for all three previously named training objectives. The encoder utilized 3D convolutional layers with increasing channel numbers (16–256), followed by batch normalization [6], an activation using Leaky ReLU [9] and dropout [16]. The weights are initialized using Kaiming initialization [5] except the final layer with a sigmoid activation layer and initialized according to the Xavier initialization [4]. The decoder mirrors this structure with transposed convolutional layers.

Only the first dataset from Mexican PMCT scans was used for training and validation of the models. Hyperparameters such as the learning rate were fine-tuned iteratively by utilizing 30% of this dataset for validation purposes. From the results of the first experiments, we chose 128 as dimensionality of the latent space. The dimensionality refers to the number of features that the model uses to represent the data in its compressed form. The latent space is a lower dimensional representation of the input data, where the model learns the most important patterns and structures. The dimensionality was determined through an iterative process based on model evaluation. The dimension was selected after testing several candidate values and assessing the corresponding performance metrics. Specifically, the quality of the generated outputs and the stability of the learning curve were closely monitored. The dimensionality of 128 was chosen as it provided the best balance between effective data compression and the preservation of important features, as evidenced by the improved model performance and the convergence of the training process.

We used the Adam optimizer with a learning rate of 1e‑4 and a batch size of 16 as an optimization technique through gradient descent. The learning rate was dynamically reduced on missing improvement on the training dataset. Additionally, we stopped the training early once overfitting became evident due to missing improvements on the validation dataset. All models were trained until convergence of their loss. The second dataset from UKE scans was exclusively reserved for the final testing purposes.

We analyzed the resulting latent spaces of the different learning objectives both qualitatively and quantitatively. Depending on the set-up, we reduced the dimension of the latent space further by projecting it into a visualizable two-dimensional space. For this dimensionality reduction, we applied a principal component analysis (PCA) as a simple baseline with a focus on the global structure of the latent space. Additionally, we employed uniform manifold approximation and projection (UMAP) in its supervised formulation [11] as a non-linear alternative with tuneable control for preserving both local and global structures.

For the qualitative analysis, we directly assessed the quality of the reconstructions and the obtained embeddings. By coloring the projected embeddings given the age associated with the corresponding sample, we assessed whether this information was implicitly encoded within the learned representations.

For the quantitative evaluation, we focused on the predictive performance and its generalizability across the datasets when the embeddings rather than the original samples were used for age prediction. Again, we conducted the calculation both for the original embedding and the representation with reduced dimensionality to quantify the trade-off for using an interpretable decision. As regression algorithms, we chose three different parametric and non-parametric methods: a simple linear regression, a k-nearest neighbor algorithm and a boosted ensemble of decision trees. For all comparisons, we used the mean absolute difference between prediction and ground truth. Unlike in the training of the models with a fixed validation dataset, we used a 10-fold cross-validation to obtain an estimate regarding the uncertainty within the prediction quality.

As an additional quantitative evaluation, we conducted a classification task exclusively on the limited data from the UKE to assess the usability of the learned representation for solving the related but not identical task of predicting whether a person is older than 21 years. Given the small number of samples, solving such a task without a pretrained feature extractor appears particularly challenging. Therefore, we applied a simple and interpretable support vector machine with a linear kernel to enforce utilization of the latent space derived from the significantly larger training dataset. In this case, we did not conduct hyperparameter optimization and used a grouped five-fold cross-validation to avoid overfitting.

The demographic distribution included 70% males (n = 341) and 30% females (n = 147) (Table 1). The age groups were categorized as follows: 10–14 years (n = 65), 15–17 years (n = 107), 18–21 years (n = 107), and 22–30 years (n = 209). The 22–30 years group represented 43% of the dataset, while the 10–14 years group was the smallest, comprising 13%. The training data consisted of 80 samples from the UKE, while the remaining 896 samples were randomly split into 627 for training (70%) and 269 for validation (30%).

In our quantitative analysis, the architecture incorporating all training objectives demonstrated effective learning of embeddings and successfully reconstructed content within the masked regions. To evaluate the visual quality of these reconstructions, Fig. 2 presents five samples from the test set from different individuals alongside their original counterparts.

On direct comparison, the models showed the expected blurry effect well-documented for VAEs. Especially when assessing the MCE, the visual reconstruction alone could not be used for an assessment without the original ground truth due to this effect. While other generative models might provide a “crisper” representation, the shape itself was well-preserved and could be overlayed to ensure proper identification and parsing of the clavicle. Noticeably, the impact of additional training objectives is clearly discernible in the ArVAE and AttriVAE models. Embedding additional information during training appeared to significantly support the reconstruction quality on the test set.

In terms of the latent space for direct and interpretable age estimations as visualized in Fig. 3, the difference between the methods where only minor. When using a PCA for a projection focusing on the global structure of the semantic space, age did not appear to represent the primary axis of variances identified by this method; however, using a method adjustable for the influence of global and local properties issued far better results. In all cases, UMAP created projections that could, depending on the task of interest, be easily used by humans to assess the rough age of a person within the visualization. The actual effect of the training objective was only of minor influence in this case.

The analysis of the results visualized in Fig. 4, indicates several noteworthy patterns concerning the generative models’ efficacy in forensic age estimation tasks phrased as regression task. The performance across the datasets, dimensionality reduction through UMAP and the utilized regression techniques reveal the interplay between those different components.

In terms of absolute performance, the difference between the algorithms is relatively low and falls well behind the standard deviation of results on different folds of the cross-validation. Given the learned structure of the latent space, the complex ensemble models with their demand for more data do not perform better than a simple linear regression. The trend remains between the training and test dataset: predictive performance using latent representations differed not significantly between both set-ups and might indicate robust findings and the linear regression as an attractive tool for the task.

Reducing the latent space dimensionality using UMAP into the interpretable 2D domain caused a performance drop across all models and regression algorithms. Nevertheless, the embeddings retained predictive power, demonstrating their potential utility in enabling fully interpretable downstream analyses.

In the classification task of determining whether a person was above 21 years old, the state simple vector machine with the linear kernel model was able to well utilize the “pretrained” semantic space. The best results were obtained on the ArVAE and are depicted in Fig. 5. Despite the smaller number of individuals below the age of 21 years in the dataset, the model achieved a precision of 76% and a recall of 73%. For individuals reaching 21 years or older, the model demonstrated higher performance, with a precision of 90% and a recall of 91%. Overall, the model achieved an accuracy of discrimination of 86%. The individuals rated as 21 years of age or older by AI but had not reached the age cut-off in reality included two individuals (18 years old and 19 years old) where both sides were predicted to be over 21 years old, as well as a 16-year-old and a 19-year-old, where only one side of the MCE was predicted to be over 21 years old.