Unbundling evaluation use

This paper presents an analysis of survey data to examine the association between supervised structured mentoring and students’ intent to pursue a career in science. Data were collected from students in the 10 Building Infrastructure Leading to Diversity (BUILD) research training programs, developed through grants from the National Institutes of Health. Propensity score matching and multinomial logistic regression demonstrated that exposure to BUILD programs—meaning participation in undergraduate research, receipt of mentoring from a primary mentor, and/or participation as a funded scholar and/or associate of each BUILD site’s training program—was associated with increased intent to pursue a science career. These findings have implications for STEM program evaluation and practice in higher education.

We use motivational theories of self-defence to explain how employees of destination management and marketing organisations experience carbon footprint data as a threat. A three-stage study, with a total of 186 employees of destination management and marketing organisations, shows few instances of consonant evaluation of sustainability data that lead to conceptual or instrumental use of indicators. Instead, dissonant cognitive evaluations result in the symbolic engagement and misuse of data to justify previous decisions, promote incremental change and delay sustainability actions. Greater levels of identification with the industry explain cognitive dissonance resulting in moral disengagement, through advantageous comparison, moral justification, and the downplaying, discrediting and disregarding of data.

Past findings concerning whether and how feedback from external evaluations benefit the improvement of schools are inconsistent and sometimes even conflicting, which highlights the contextual nature of such evaluations and underscores the importance of exploring them in diverse contexts. Considering that broad international debate, we investigated the use and impact of feedback from external evaluations in compulsory schools in Iceland, particularly as perceived by principals and teachers in six such schools. A qualitative research design was adopted to examine changes in the schools made during a 4–6-year period following external evaluations by conducting interviews with principals and teachers, along with a document analysis of evaluation reports, improvement plans and progress reports. The findings reveal that feedback from external evaluations has been used for instrumental, conceptual, persuasive and reinforcement-oriented purposes in the schools, albeit to varying degrees. According to the principals and teachers, the improvement actions presented in the schools’ improvement plans were generally implemented or continue to be implemented in some way, and the changes made have mostly been sustained.

Numerous studies have examined the impact of evaluation quality on the use of evaluation, yet the findings are still inconclusive and only derived from examples in countries with a well-established evaluation culture. This article examines the relationship between evaluation quality and the instrumental use of evaluation in administrations with a low maturity of evaluation culture using Poland as a case study. The research method employs quantitative analysis of datasets regarding evaluation quality and recommendation use from the Polish cohesion policy evaluation system.

The use of evaluation findings was measured at two stages: acceptance of recommendations and actual implementation. The results show that the impact of evaluation quality on the use of evaluation is limited and sometimes puzzling. While perceived utility increases the odds of accepting recommendations, the quality of analysis and conclusions reduces it. The actual implementation of recommendations is supported by the robustness of methodology, quality of the analysis and the clarity of the final report. Data reliability and achievement of the study objectives have no significant impact on either the decision to accept or implement recommendations. The problem of symbolic use is discussed as a potential explanation for the observed results.