http://orcid.org/0000-0001-5904-1112
Figures
Abstract
Background
In persons with intellectual and developmental disabilities, not only cognitive brain functions, but also socio-emotional processing networks may be impaired. This study aims to validate the Scale of Emotional Development—Short (SED-S) to provide an instrument for the assessment of socio-emotional brain functions.
Method
The SED-S was applied in 160 children aged 0–12 years. Criterion validity was investigated at item and scale level in terms of the agreement between the scale classification and the child’s chronological age. Additionally, interrater reliability and internal consistency were assessed.
Results
For the majority of items, the expected response pattern emerged, showing the highest response probabilities in the respective target age groups. Agreement between the classification of the different SED-S domains and chronological age was high (κw = 0.95; exact agreement = 80.6%). Interrater reliability at domain level ranged from κw = .98 to 1.00 and internal consistency was high (α = .99).
Citation: Sappok T, Böhm J, Birkner J, Roth G, Heinrich M (2019) How is your mind-set? Proof of concept for the measurement of the level of emotional development. PLoS ONE 14(4): e0215474. https://doi.org/10.1371/journal.pone.0215474
Editor: Marina A. Pavlova, Univdersity Hospital of TübingenUniversitatsklinikum Tubingen, GERMANY
Received: November 19, 2018; Accepted: April 2, 2019; Published: April 18, 2019
Copyright: © 2019 Sappok et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data contain potentially identifying information. The ethics committee of Charite university hospital has imposed them. The data is available upon request by contacting: tanja.sappok@charite.de or an.hartmann@keh-berlin.de.
Funding: The study is supported by funds of the von Bodelschwinghsche Stiftungen Bethel.
Competing interests: Tanja Sappok receives royalties for books about the developmental approach and the manual of the SED-S published by Hogrefe. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Background
For a comprehensive understanding of mental functioning, the traditional focus on intellectual competencies has to be widened to acknowledge socio-emotional brain functions as well [1]. These socio-emotional competencies can be conceptualized according to the emergence of the respective social processing networks alongside the trajectory of typically developing infants [2]. Thus, this comprehensive view of mental functioning comprises various aspects such as object permanency, self-other-differentiation, secure bounding, stress regulation, affect differentiation, impulse control and theory of mind [3; 4]. The maturation of emotional brain functions is the product of various internal (epi-) genetic and sensory and external factors such as bounding experience, trauma, education and learning. In a bottom-up process, the neuronal substrate, i.e. the developing brain, modifies social interaction abilities, while in a top down-process, these interactive experiences shape the developing brain networks [5]. As such, emotional functioning occurs in coordination with various processes, including cognitive, sexual, motor and moral development [2]. Each developmental level is associated with specific emotional needs, motivations, and coping skills, which affects a person’s ability to adapt to the environment [6, 7]
Neuroanatomical, the ‘emotional brain’ is located in the various centres of the limbic system [8]. The deep limbic level includes the diencephalon (hypothalamus and brain stem, including periventricular grey and vegetative nuclei) and the central nucleus of the amygdala, and processes basal functions that are necessary for survival, such as feeding, reproduction, and fight-flight reactions. Moreover, autonomic functions and the stress-regulation system are located in this part of the brain. The processes are mostly unconscious and genetically/epigenetically determined, and are only minimally influenced by environmental factors [8]. These structures develop mainly prenatally and predominate during the first months of life, when sensory stimuli from within the body, such as bowel movements, and external stimuli, such as noise and touch, are integrated and processed. The ‘way of thinking’ in this early phase of life is triggered by the sensory input and is goal- and action-oriented [9].
The mesolimbic system starts to develop prenatally and further maturates within the first months and years of life. It consists of subcortical areas and comprises the basolateral amygdala, the ventral tegmental area and the nucleus accumbens/ventral striatum, which are both parts of the basal ganglia and also functions predominantly unconscious [8, 10, 11]. As a result of the sensory stimulation and integration during the first months of live, step by step object permanence arises at the end of the first year. Now the person has an inner picture of the outside environment, the first steps towards mentalization are taken [9]. However, in this stage, persons cannot discriminate between their own thoughts and the outside world: i.e., thinking is reality.
In the mesolimbic system, basic emotional functions such as anxiety, sadness, disgust, happiness and anger are determined. The emotional conditioning derives from the bonding experiences to the central attachment figures during early life: The developing child learns to acknowledge, differentiate, understand and regulate his or her own emotions through interactive contact with his or her caregiver [12]. Moreover, the nonverbal communication network, i.e. the recognition of emotional-communicative signals evolves. Stimulated by the constant interaction with the close carers, with around 2 years, the infant is able to differentiate between the self and the other [9]. Therefore, the person is able to distinguish between his or her own thoughts and the (outside) reality. This differentiation is practised and trained in role and imaginative plays, and thereby the pretend mode of thinking arises.
The brain areas of the mesolimbic system are also decisive for reward (endogenous opioids) and reward expectation (dopamine) as a basis for motivation [8].
The upper limbic level consists of the allocortical structures of the limbic system: that is, the prefrontal, orbitofrontal and ventromedial frontal cortex, the anterior cingulate cortex and the insular cortex [8]. The upper limbic system evolves in late childhood and adolescence and derives from socio-emotional experiences with the wider social environment, including friends, peers and more distant relatives. By the end of the forth year of life, the child is able to acknowledge that other persons have different feelings, thoughts, intentions, and motivations: The Theroy of Mind network developed and the child is able to mentalize. This is the prerequisite for more complex emotional states such as empathy, friendship, loyalty and moral thinking, which are all located in the upper limbic system. Socialization and social motivation develop via conscious emotional-social learning. Moreover, socio-emotional core competencies, such as impulse control, delayed gratification, frustration tolerance and balancing of the consequences of one’s own behaviour, are determined in this brain area [8]. Advantages and disadvantages of actions can be appreciated, risks assessed properly and behaviours planned accordingly. The social brain networks processing executive functions, risk assessment and reality awareness are situated in the upper limbic system.
According to ICD-11, disorders of intellectual development are a group of etiologically diverse conditions originating during the developmental period and characterized by significantly below average intellectual functioning and adaptive behavior that are approximately two or more standard deviations below the mean [13]. As such, not only cognitive brain functions as assessed with standardized intelligence tests, but also the socio-emotional competencies and the related brain networks as described above are impaired. Additional sensory impairments may further aggravate emotional reactivity and coping strategies [14]. Generally, individuals with intellectual and developmental disability (IDD) pass through the same stages as do typically developing children, but with an increased risk of delay and incompletion [15, 16]. The developmental approach may be particularly supportive in the treatment and care of persons with developmental delays as it gives insights in the respective brain networks active at a certain developmental level [4, 7, 17]. Persons with intellectual and developmental disability (IDD) show high point prevalence rates of mental disorders and challenging behaviours (41%) [18]. The recognition of the level of development of the socio-emotional brain circuits presents an additional perspective to the mechanisms leading to particular (problem) behaviours [19, 20]. Without a thorough maturation of the mesolimbic and upper limbic system, for example, one cannot expect impulse control and empathy, but rather external support for self-regulation and physical well-being. Insight into emotional functioning is therefore crucial for appropriate treatment and support in persons with IDD [21, 22]. Subsequently, better-tailored interventions and treatment options may arise, encourage the acquisition of functional life skills, and result in increased well-being.
The level of mental functioning comprising the socio-emotional competencies can be determined with the Scale of Emotional Development–Short (SED-S) which is based on the developmental approach proposed by Anton Došen [4, 23, 24]. This developmental model describes five levels of socio-emotional development, which are aligned to the milestones of the developmental trajectories during childhood and the therewith associated maturation of the respective brain circuits. The SED-S consists of 200 binary items and is applied in the form of a semi-structured interview with a parent or caregiver. It describes the five levels of emotional functioning depicted within eight domains [23–25]: Relating to his/her Own Body, Relating to Significant Others, Dealing with Change–Object Permanence, Differentiating Emotions, Relating to Peers, Engaging with the Material World, Communicating with Others, and Regulating Affect. With five binary items for each level of functioning in each of the eight domains, the SED-S results in a profile and an overall ED score. Being aware of the continuous nature of emotional development, the SED-S is conceptualized in a stage-wise model. This stage-wise approach seems beneficial when aiming to guide work in clinical practice and behavioural therapy, as it generates clear action-advice based on the assessed level of ED [4, 22].
This study aims to assess the psychometric properties of the SED-S. Therefore, the SED-S is applied in a sample of typically developing children aged 0 to 12 years. The current research is necessary to ensure that the behaviours described in the different items really reflect the actual target behaviours which can be observed at a certain reference age to support the criterion validity of the SED-S. In detail, (1) the age specificity of a certain behaviour for the respective developmental age-group, including the profile homogeneity, is used to assess criterion validity; and (2) the agreement between two different raters is used to assess interrater reliability.
Method
Setting and design
The study sample consists of 160 typically developing children aged 0 to 12 years who were recruited from 12/2016 to 06/2017 in different institutions such as kindergartens, schools, sports clubs, and day-care centres, as well as from the families of staff at the organizing institution. Interested parents were given comprehensive information about the study. Inclusion criteria for the study were: Child aged from 0 to 12 years, no developmental delay or mental illness, and a declaration of informed consent by the parents. Typical development was ensured by taking a thorough developmental and medical history with a parent or close carer about the child’s birth, developmental milestones and life events, and by reviewing the child’s personal health record (PCHR; “gelbes Untersuchungsheft”), which is taken regularly for all children in Germany from birth up to adolescence and completed by the primary physician. This personal child health record was thus available to be checked for every participating child. The SED-S was applied in semi-structured interview form by two psychologists experienced in the concept of ED. For the estimation of ED on the domain level, the domain with the highest number of items rated as applicable for a child was chosen. The overall level of ED was marked by the domain with the fourth-lowest score (Details c.f. [23, 24]) The assigned level of ED, as derived from the assessment with the SED-S, was compared with the child’s chronological age. Interrater reliability was assessed in 25 cases. For this purpose, the information provided by the parents was scored by two psychologists. One psychologist (rater 1) conducted the interview and the second psychologist (rater 2) attended this assessment. Both interviewers scored the provided information independently.
Assessment of the level of ED
The scale was administered as a semi-structured interview by a trained psychologist familiar with the developmental approach for several years with one close carer (120 mothers, 24 fathers, 4 grandmothers, 10 caregivers from the Kindergarden, 2 others). The interview takes about one hour and refers to the child’s behavior within the past six weeks. On domain level, the stage with the highest number of items rated as ‘typical’ was assumed to provide the best estimation of the child’s level of ED. On scale level, a rank-based strategy is proposed, with the highest level of ED within the four lowest-ranking domains determining the overall result. Sappok et al. (2016), Morisse, Sappok, de Neve, and Dosen (2017) and Sappok et al. (2018) reported the administration and scoring of the SED-S in detail [23–25].
Ethics
All parents provided informed consent. The procedures were in accordance with the ethical standards of the institutional ethics committees and with the 1964 Helsinki declaration and its later amendments. The study was approved by the Charité ethics committee, EA2/193/16 (approval granted 02/2017).
Sample characteristics
The sample characteristics are displayed in Table 1. At least 30 children were included in each age group, and both genders were represented equally. Most children lived in their nuclear family and 65 lived with siblings (40.6%). In light of the different ages included, the majority of the children went to kindergarten, while one-third spent the day with a parent at home and a quarter went to school.
Statistical analysis
All analyses were performed using R. There were no missing values on item scores.
- Criterion Validity. At item level (1.1), agreement between the child’s chronological age and the behaviours described in the items of the respective SED-S level of ED was analysed. The proportion of children in a certain age group showing these behaviours was assessed. The items for a respective age group were used as reference and compared with the proportions of ‘yes’ responses in adjacent age groups. For the items constructed to assess SED-S level 1, the proportion of ‘yes’ answers in children from age group 1 was compared to the corresponding proportion for children in age group 2; for SED-S level 2, the results for children from age group 2 were compared to those in age groups 1 and 3, and so forth. These comparisons were based on the hypothesis that children in a certain age group will display the behaviours described in the items for the respective level of ED more frequently than children in adjacent age groups. Comparisons were made using Fisher’s exact test (one-sided). Items were considered inappropriate if a) the response probability was low (< 45%) in the target age group, or b) the response probability was not significantly higher in the reference age group when compared to adjacent age-groups (lack of age group specificity). Due to multiple testing, only differences with p < .01 were considered statistically significant.
At the domain level and overall scale level (1.2), agreement of the child’s chronological age with the SED-S classification was determined in terms of quadratic weighted kappa (κw) with bootstrapped 95% confidence levels (95% CI), as well as the percentage of exact agreement. Sex differences (1.3) were looked at by stratifying this analysis for both boys and girls. Moreover, median values on domain and overall scale level were compared for these two groups.
Within-Profile Homogeneity (1.4) of the SED-S profiles was assessed using Cronbach’s alpha (α) as an overall index for internal consistency. In addition, the difference between the highest and lowest SED-S levels (min-max difference) was computed as an indicator for within-profile homogeneity. Since the sample consisted of typically developing infants who should have even SED-S profiles, internal consistency should be high and high min-max difference should be rare. - Interrater reliability Agreement between raters was determined in terms of percentage agreement and weighted kappa.
Results
Criterion validity at item level
For the majority of items, the response probability within the target age group was significantly higher compared to the adjacent age groups. For each item, Tables 2–9 display the proportions of ‘yes’ answers for every age group and the level of significance when comparing the target age group with the adjacent age groups. Appropriate items (response probability > 45%; age group specificity) are printed in bold.
In the following section, the item validity results are presented domain-wise for items considered as inappropriate: that is, items showing low response probabilities in the target age group or low levels of significance (>.01) compared to the adjacent age groups.
Relating to his/her own body.
In this domain, most items met the criteria for appropriateness. Items b1_5, b2_3, and b3_5 occurred rarely in all age groups. Items b2_5, b4_3, and b5_2 indicated phase non-specificity, as the proportion of individuals in the reference group did not show a specific behaviour significantly more often than did individuals in adjacent age groups. However, in each level of ED, at least 3, but mostly 4 items met the designated criteria.
Relating to significant others.
In phases 2 and 4, only two items each met the inclusion criteria. Items s1_5, s2_1, s2_3, s2_4, s4_1, s4_3, s4_5, s5_1, and s5_3 did not differ significantly from adjacent age groups in terms of the proportion of ‘yes’ answers. Item s3_4 showed low response probabilities in all age groups.
Dealing with change / object permanence.
Only item o3_5 showed low response probabilities in the target age group. Two items in phase 2 (o2_4 and o2_5) and phase 3 (o3_1 and o3_2) were not sufficiently age-specific; neither were four items in phase 4 (o4_1, o4_2, o4_3, and 4_4) and three items in phase 5 (o5_1, o5_2, and o5_5).
Differentiating emotions.
Items e1_3 and e1_4 showed low response probabilities in all age groups. In SED-2, no item met the level of significance for describing age-specific behaviour. For the same reason, six more items across different levels of ED were rated as inappropriate (e3_2, e3_4, e4_1, e4_4, e4_5, and e5_1).
Relating to peers.
In this domain, only one to two items in each SED level met the criteria of appropriateness. Most items were not age-specific: i.e. items p1_2, p1_4, p2_2 to p2_5, p3_2, p3_3, p4_1, p4_4, p4_5 and p5_1 to p5_3, while three items described behaviour rarely seen in all children (p1_3, p1_5, and p3_5).
Engaging with the material world.
In this domain, 18 of the overall 25 items met the inclusion criteria. Three items in phase 5 (m5_1 to m5_3) and one in each of the other levels of ED (m1_4, m2_2, m3_4 and m4_1) described behaviours that were not specific to the respective age group.
Criterion validity at domain level and overall agreement
The agreement of the classifications of the eight SED-S domains with the chronological age group ranged between 65.6% (Relating to Peers, κw = 0.92) and 78.1% (Relating to his/her Own Body, κw = 0.94), Mdn = 70.9%. Kappa was 0.95 for the overall classification, with an exact agreement of 80.6%. Domain-wise rates of agreement, the corresponding kappa values, the proportions of individuals within a certain age group, and the classification in the SED-S phases are summarized at the domain level in Table 10 and at the scale level in Fig 1.
A summary of the valide items will be provided in the supplement.
Sex differences
The analyses of the square weighted kappas for boys and girls separately on domain and overall SED-S level is shown in Table 11. No obvious sex differences could be found.
The comparison of the medians of boys and girls on domain and overall score level can be seen in Table 12.
In the first age group (0 to 6 months), in 4 out of the 8 domains, girls showed significantly higher median values than the boys. In this age group, the girls were also significantly older than the boys (6 vs. 3 months).
Internal consistency and within-profile homogeneity
The internal consistency was high, at α = .99. Overall, the majority of individuals provided homogenous profiles, with 33.8% of individuals showing the same phase across all domains and 53.8% showing a min-max difference of one, 11.3% showing a min-max difference of two and 1.3% showing a min-max difference of three SED levels.
Discussion
The current study normed the SED-S, a scale for the assessment of ED, in a sample of typically developing children, and provides evidence for the criterion validity on item, domain and scale level. The majority of children showed homogenous profiles. Interrater reliability and internal consistency were high.
At item level, the expected pattern emerged for the majority of items: that is, the response probabilities were highest in the target age group and significantly higher compared to the adjacent age groups. Thus, the behaviours described in items which were considered as being ‘appropriate’ are typical for the assigned level of development. However, some items were observed in fewer than half of the children in the determined age group or were not sufficiently age-specific, and therefore should be excluded or rephrased in a revision of the SED-S.
At the domain level, quadratic weighted kappa-values were high and the exact agreement was good. However, the assignment to the assumed level of ED did not fit perfectly to the respective chronological age of the children. This may be caused not only by limitations of the instrument itself, but also by differences in the developmental stage of the various children. Also, in typically developing children, the day-to-day care and the living situation, including the presence and age of siblings, environmental factors and genetic aspects, may lead to more or less advanced levels of ED. Moreover, the age of some children was towards the lower or higher end of the respective age group, which increases the likelihood for assignment to an adjacent stage. The agreement between the children’s chronological age and the SED-S was best for the overall result of the SED-S. Assessment across several domains thus increased the overall accuracy of the scale.
No obvious sex differences could be found for the agreement of the child’s chronological age with the SED-S classification in terms of the quadratic weighted kappa values. Comparing the median values of boys and girls on domain and overall score level resulted in marginal differences in the first age group (0 to 6 months), with 4 out of the 8 domains showing significantly higher median values in girls than the boys. However, the girls were also significantly older than the boys (6 vs. 3 months). If the analysis is corrected for multiple testing (.05/8 = .006), the results are not significant any more. To conclude, we ascribe these marginal differences to the different ages of boys and girls within this age group.
Sex differences in emotional and social functioning are reported in various studies [26, 27]. Functional connectivity of the amygdala with other brain regions involved in stress and emotion processing seems to be sex-specific [28]. However, looking at our data did not reveal any apparent difference between the two groups. Maybe the size of the study sample is not large enough to detect sex specific differences.
Persons with intellectual disability show high prevalence rates of challenging behaviours [18]. So far, assessment of challenging behaviour focusses on environmental aspects: for example, behavioural analysis (NICE 2015). Due to the lack of appropriate concepts and measurements, intrinsic factors that are crucial for adaptive functioning, such as the level of mental functioning, remain under-investigated [7]. Despite the presence of different measures for emotional development, such as the Scheme of Appraisal of Emotional Development (SAED) [4, 29] the Schaal voor het sociaal-emotionele ontwikkelingsniveau (ESSEON-R) [30], the Developmental–Structuralist Approach (Greenspan; 1997a, b), the Levels of Emotional Awareness [31] or the Infant-Toddler Social and Emotional Assessment scale (ITSEA) [32], no study to date has normed the respective scale in a sample of typically developing children. Thus, the described characteristics and behaviours of the various models and measures have never been validated alongside the trajectories of typical development. The current study fills this gap and can be regarded as a proof of concept of the model using corresponding reference ages of typically developing children. Moreover, the above mentioned scales conceptualize ED less comprehensive and more focused on emotional aspects as such, while the SED-S comprises a thorough assessment of various aspects of mental functioning observable alongside the developmental trajectory of the different parts of the limbic system as described in the introduction. Finally, the SED-S is applicable to adults as the items have been phrased accordingly, while the above mentioned measures for ED are designed to be used in children. Each level of development is related to specific emotional needs, motivations and adaptive strategies and leads to certain observable behaviours. This may add a further perspective on the origins of challenging behaviours in persons with developmental delays and support practising psychiatrists in their diagnostic work-up.
In this study, internal consistency was high. In fact, a study by La Malfa et al. [33], which examined the psychometric properties of the SAED, showed similar values (Cronbach’s alpha: .96). Moreover, the proportion of infants showing a min-max-difference greater than 1 was small. This further supports the validity of the SED-S, as typically developing children should show rather homogenous profiles and therefore should be classified in the same or at least adjacent phases across all domains. To conclude, the different items seem to measure different elements of the same aspect, which in this case is ED.
The interrater reliability was very high, as indicated by high exact agreement and quadratic weighted kappa-coefficients above .95. This is comparable to the study conducted by La Malfa et al. (2009; kappa = .75) which assessed the psychometric properties of the SAED [33]. However, the estimate of interrater reliability in the current study may be increased by the fact that both raters participated in the same assessment and scored the same information. Therefore, interrater reliability must be reassessed by two independent evaluations of the raters.
Nonetheless, some limitations of this study have to be considered. The study popula-tion is a selected sample and may not represent the whole population of typically de-veloping infants. The development of the assessed infants was considered ‘typical’ based on the caregivers’ reports and a review of the child’s personal health record; the children were not re-examined by a physician from the study team. The raters were not blinded to the children’s chronological age. Cultural aspects were not addressed systematically. The interview was applied to only one parent, mostly the mothers, despite a multi-informant approach is considered as the best practice for assessment of emotional functioning in children [34]. In this study, Fält et al. (2018) reported a moderate agreement between mother and father ratings (ICC 0.66–0.76) with regard to the child’s problem behavior, while another study points out the unique information of mothers and fathers about their child’s behavioural and emotional problems [35]. Further studies may address the impact of the parents sex by comparing the results of interviews of mothers and fathers.
For the assessment of interrater reliability, both assessors rated the same structured interview. In a next step, the interview should be applied twice by two different raters to additionally assess the reliability of the application of the instrument. Moreover, the current estimates of inter-rater agreement are limited by the fact that only 25 individuals were investigated. Therefore, these results need to be replicated. We suggest using different approaches to inter-rater reliability in further studies, that is, testing comparing the coding of completely independent assessments and using a larger number of interviewers to get a better impression of the raters specificity and generalizability across different interviewers.
Finally, application and validation of the SED-S in an adult IDD population, with and without additional mental and behavioural disorders is pivotal to further validate the scale for its respective usage.
The SED-S showed adequate psychometric properties in terms of criterion-related validity and therefore may be applied in clinical practice to assess the level of ED. The application of the SED-S in a sample of adults with ID and an assessment of its psychometric properties would be the next step to ensure the empirical basis of the scale in this population. We therewith aim to provide a proof of concept for the assessment of the level of ED to add a further perspective on the basic emotional needs, self-regulation strategies and behaviours of persons with developmental delays.
References
- 1. Damasio A. Self comes to mind. Constructing the conscious brain, Vintage, 2012
- 2.
Sroufe LA. Emotional development: The organization of emotional life in the early years. Cambridge University Press, Cambridge (UK), 1995
- 3. LeDoux JE. Emotion circuits in the brain. Annual review of neuroscience 2000; 23: 155–184. pmid:10845062
- 4. Došen A. Applying the developmental perspective in the psychiatric assessment and diagnosis of persons with intellectual disability: part I-assessment. J Intellect Disabil Res 2005; 49: 1–8. pmid:15634306
- 5. Pessoa L. Emergent processes in cognitive-emotional interactions. Dialogues Clin Neurosci 2010; 12: 433–448. pmid:21319489
- 6. Otsuka S, Uono S, Yoshimura S, Zhao S, Toichi M. Emotion Perception Mediates the Predictive Relationship Between Verbal Ability and Functional Outcome in High-Functioning Adults with Autism Spectrum Disorder. J Autism Dev Disord 2017; 47: 1166–1182. pmid:28194554
- 7. Sappok T, Budczies J, Dziobek I, Bölte S, Došen A, Diefenbacher A. The missing link: delayed emotional development predicts challenging behaviour in adults with intellectual disability. J Autism Dev Disord 2014; 44: 786–800. pmid:24002416
- 8. Roth G, Strüber N. Wie das Gehirn die Seele macht. Klett-Cotta, Stuttgart, 2018.
- 9.
Fonagy P, Bateman A (Eds.) Handbook of mentalizing in mental health practice. American Psychiatric Publishing, Inc, Arlington (VA), 2012
- 10. Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 2003; 299: 1898–1902. pmid:12649484
- 11. Schultz W. Behavioural dopamine signals. Trends in Neurosciences 2007; 30: 203–210. pmid:17400301
- 12.
Kernberg O. The inseparable nature of love and aggression. American Psychiatric Publishing, Washington D.C, 2012.
- 13.
World Health Organisation (WHO) International Classification of Diseases for Mortality and Morbidity statistics. Eleventh revision. WHO 2018
- 14. Aldao A, Nolen-Hoeksema S. The influence of context on the implementation of adaptive emotion regulation strategies. Behav Res Ther 2012; 50: 493–501. pmid:22659159
- 15.
Hodapp R, Zigler E. Past, present and future issues in the developmental approach to mental retardation and developmental disabilities. In Developmental psychopathology (eds Cichetti D, Cohen D): 299–331. Wiley, New York, 1995.
- 16.
Greenspan S. Developmentally based psychotherapy, International University Press, Madison, 1997.
- 17. Happe F, Frith U. Annual Research Review: Towards a developmental neuroscience of atypical social cognition. J Child Psychol Psychiatry 2014; 55: 553−577. pmid:24963529
- 18. Cooper SA, Smiley E, Morrison J, Williamson A, Allan L. Mental ill-health in adults with intellectual disabilities: prevalence and associated factors. Br J Psychiatry 2007; 190: 27–35. pmid:17197653
- 19. De Bildt A, Systema S, Kraijer D, Sparrow S, Mineraa R. Adaptive functioning and behaviour problems in relation to level of education in children and adolescents with intellectual disability. J Intellect Disabil Res 2005; 49: 672–681. pmid:16108984
- 20. Felce D, Kerr M, Hastings RP. A general practice-based study of the relationship between indicators of mental illness and challenging behaviour among adults with intellectual disabilities. J Intellect Disabil Res 2009; 53: 243–254. pmid:19017165
- 21. Sappok T, Budczies J, Bölte S, Dziobek I, Došen A, Diefenbacher A. Emotional development in adults with autism and intellectual disabilities: a retrospective, clinical analysis. PloS one 2013; 8: e74036. pmid:24066092
- 22. Sappok T, Zepperitz S. Das Alter der Gefühle: Über die Bedeutung der emotionalen Entwicklung bei geistiger Behinderung, Hogrefe, Bern, 2016.
- 23. Sappok T, Barrett BF, Vanvelde S, Heinrich M, Poppe L, Sterkenburg P et al. Scale of emotional development-Short. Res Dev Disabil 2016; 7: 166–175.
- 24. Sappok T, Zepperitz S, Barrett BF, Došen A. Skala der emotionalen Entwicklung–Diagnostik (SEED)–Diagnostik, Hogrefe, Bern, 2018.
- 25. Morisse F, Sappok T, de Neve L, & Došen A. SEO-V: Schaal voor Emotionele Ontwikkeling van mensen met een verstandelijke beperking-Verkort. Garant, Antwerpen–Apeldoorn, 2017
- 26.
Rutter M. Rutter’s child and adolescent psychiatry, Wiley-Blackwell, Oxford, UK 2010.
- 27. Pavlova MA. Emotion Science in the Twenty-First Century. Time, Sex, and Behavior in Emotion Science: Over and Above. Front Psychol 2017; 8:1211 pmid:28785231
- 28. Kogler L, Müller VI, Seidel EM, Boubela R, Kalcher K, Moser E, et al. Sex differences in the functional connectivity of the amygdalae in association with cortisol. Neuroimage 2016; 134:410–423. pmid:27039701
- 29.
Došen A. Psychische Störungen bei geistig behinderten Menschen, Gustav Fischer Verlag, Stuttgart, 1997.
- 30. Hoekman J, Miedema A, Otten B, Gielen J. De constructie van een schaal voor de bepaling van het sociaal-emotionele ontwikkelingsniveau. Achtergrond, opbouw en betrouwbaarheid van de ESSEON. Nederlands Tijdschrift voor de Zorg aan mensen met verstandelijke beperkingen 2007; 33: 215–232.
- 31. Lane RD, Schwartz GE. Levels of emotional awareness: A cognitive-developmental theory and its application to psychopathology. Am J Psych 1987; 144: 133–143.
- 32. Briggs-Gowan MJ, Carter AS, Skuban EM, Horwitz SM. Prevalence of social-emotional and behavioural problems in a community sample of 1- and 2-year-old children. J Am Acad Child Adolesc Psychiatry 2001; 40: 811–819. pmid:11437020
- 33. La Malfa G, Lassi S, Bertelli M, Albertini G, Došen A. Emotional development and adaptive abilities in adults with intellectual disability. A correlation study between the scheme of appraisal of emotional development (SAED) and Vineland adaptive behaviour scale (VABS). Research in Developmental Disabilities 2009; 30: 1406–14. pmid:19640678
- 34. Fält E, Wallby T, Sarkadi A, Salari R, Fabian H. Agreement between mothers’, fathers’, and teachers’ ratings of behavioural and emotional problems in 3–5-year-old children. PLoS ONE 2018; 13(11): e0206752. pmid:30383861
- 35. Chiorri C, Hall J, Casely-Hayford J, Malmberg LE. Evaluating Measurement Invariance Between Parents Using the Strengths and Difficulties Questionnaire (SDQ). Assessment. 2016 Feb;23(1):63–74. pmid:25604631