Development and psychometric testing of an abridged version of Dundee Ready Educational Environment Measure (DREEM)

Education environment refers to the diverse physical locations in which the students learn. The educational environment of an institution has direct and indirect influence(s) upon the knowledge gained by the students including the motivation to learn, personal safety and their well-being [1]. Dundee Ready Educational Environment Measure (DREEM) is one of the tools developed specifically to assess the educational environment of medical institutions as perceived by the students.

DREEM has 50 items grouped under five domains namely students’ perception of learning (SPoL; n = 12), students’ perception of teachers (SpoT; n = 11), students’ academic self-perception (SASP; n = 8), students’ perception of atmosphere (SPoA; n = 12), and students’ social self-perception (SSP; n = 7). It is a generic and culturally non-specific tool which has been translated and validated in at least eight languages and is used worldwide [2]. In addition to being used to diagnose the deficiencies of the current educational environment, DREEM has also been used to compare different groups, to monitor the same cohort over a period of time and to assess factors influencing the educational environment [3, 4]. DREEM was used among undergraduates [5] including nursing [6, 7] and dental students and various post graduate medical students [8, 9] to explore their perceptions of educational environment.

Though universally accepted and used, investigators followed their own analysis and reporting system of DREEM data [10]. Few studies questioned the five-factor structure and construct validity of the original DREEM tool and also tried to develop abridged versions of DREEM [11‐14]. The other studies have developed an abridged version but the properties of the abridged version have not been tested on another cohort. These abridged versions have not been validated nor have they been widely in use.

Our study developed and validated an abridged version of the DREEM-50 with an aim to have a less resource-intensive (time, manpower), yet valid and reliable, version of DREEM-50 while also avoiding respondent fatigue.

A total of 277 students from group I participated in the development of the abridged version and 141 students from group II in the validation and reliability of the abridged version. The age and gender distribution of the study population is given in Table 1.

The item-total correlation and domain-total correlation of the items selected for inclusion in DREEM-12 are given in Table 2. Correlation between total scores of DREEM-50 and DREEM-12 was 0.88 (p < 0.001). Domain–domain correlation between DREEM-50 and DREEM-12 were 0.75 (SPoL), 0.7 (SPoT), 0.79 (SASP), 0.66 (SPoA) and 0.62 (SSP). Regression analysis revealed that DREEM-12 explained 77.4% of the variance in DREEM-50. The internal consistency of DREEM-12 was 0.83 as compared to 0.86 of DREEM-50.

The item wise and overall scoring system of DREEM-12 was devised in line with the DREEM-50 (Table 3). When calculated for group II, DREEM-50 and DREEM-12 scores yielded the same interpretations on the students’ perceptions about the educational environment: overall and at domain level (Table 4).The test-retest reliability of DREEM-12 was 0.595, p < 0.001.

Confirmatory factor analysis of DREEM-12 construct was statistically significant (LR test of model vs. saturated: χ²(44) = 80.88, Prob> χ² = 0.0006) (Fig. 1).

This study developed and validated DREEM-12, an abridged version of DREEM-50, using two subsets of a population of undergraduate medical students and established good model fit of DREEM-12 by confirmatory factor analysis.

Our study, besides developing an abridged version of DREEM-50, has also confirmed its factor structure and established its validity and reliability. We have employed robust data quality management, through double data entry and validation, and statistical techniques to prove the same. Our study has enrolled students from different years of medical education and thus can be considered representative of the perceptions of students in different stages of medical education.

The study was limited to only students from a single teaching institution in India. However, ours being a study on educational environment, it is also necessary to point out that the quality standards of a medical college with respect to curriculum, teaching methodology, infrastructure, staffing and assessment methods are strictly governed to be uniform by the Medical Council of India. These standards are uniform for all the colleges countrywide. All colleges, government and private, are to abide by these standards, failing which the recognition of the institute is cancelled and the given institute will no more be able to admit candidates. Hence, it will be safe to say that the educational environment offered in colleges across the country are to be uniform. The students are admitted into the undergraduate medical course by a nationwide common competitive entrance examination—National eligibility entrance test. The students do come from different sociodemographic and cultural backgrounds in different parts of the country. But, we do not expect that to significantly affect their perception of their environment. Thus, while the internal validity of our study is commendable, further multi-centric research is required to establish the generalisability of the results outside the country.

The psychometric properties of original DREEM-50 were investigated in few studies. Confirmatory factor analysis found that the five-factor structure of DREEM-50 showed poor model fit in studies conducted in Malaysia and Australia [12, 14]. Both these studies came up with an abridged version of DREEM-50 with a good model fit (Table 5). The Malaysian study derived a 17-item abridged version with good model fit with each domain having three items except the students’ perception of atmosphere which had five items. Similarly, the Australian study derived 12-item abridged version with only four domains after removing student’s social self-perception. The difference in the items chosen in our study, Malaysian and Australian abridged versions might have been due to the following reasons- the methodology of choosing items into the abridged version was different in the three studies. In our study, we had followed the method used in development of WHO QOL BREF. We included items based on their item-total correlations. While the Malaysian version retained the five-domain structure of the original DREEM, the Australian version had done away with the social self-perception domain. Since our study revealed equally good domain-total correlation for all domains, similar to the Malaysian study, we chose to retain all domains in our abridged version. Eight (items 3, 19, 22, 37, 41, 43, 44 and 45) out of the 12 items selected in our study (Table 3) also figure in the Malaysian 17-item version [11, 12]. None of the items in the Australian abridged version are present in the current DREEM-12 except for item 22 [14]. Our abridged version is short like the Australian version and maintained the five-domain structure of the original version like the Malaysian version.

A study from Ireland also mentioned the poor construct validity of the DREEM-50 and found some 17 items having poor fit indices (< 0.7). This study failed to propose an abridged version (33 items) mentioning two reasons, namely, the weakest items may not be same for different settings and fear of affecting the factor structure [13].

Reports on test-retest reliability of DREEM are variable. A study from Brazil reported moderate test-retest reliability of 0.43 [16]. Contrarily, a study validating the Greek translation of DREEM, reported 0.90 [17]. While a generally accepted cut-off of 0.7 is followed, there are no standards laid for test-retest reliability cut off for instruments measuring psychological parameters [18].

DREEM-12, developed as an abridged version of DREEM-50, was found to explain 77.5% of the variance of the scores of the original version and also retained its factor structure, validity and reliability. Future research using DREEM-12 will establish its validity and reliability across different settings.