The conceptual framework of the theory-driven perspective

doi:10.1016/0149-7189(89)90057-8

Evaluation and Program Planning

Volume 12, Issue 4, 1989, Pages 391-396

https://doi.org/10.1016/0149-7189(89)90057-8 Get rights and content

Abstract

This paper attempts to outline the conceptual framework of the theory-driven perspective. Six essential domain theories in a program are identified: (a) treatment, (b) implementation environment, (c) outcome, (d) impact, (e) intervening mechanism, and (f) generalization. Based upon these domain theories, different types of theory-driven evaluations are constructed. This conceptual framework provides a heuristic scheme which enables stakeholders and evaluators to work together to select a theory-driven evaluation that fits the nature of the program and stakeholders' needs within the constraints of resources and timeliness. The theory testing procedures and the role of the theory-driven evaluator are also discussed.

References (14)

H.T. Chen et al.
The theory-driven approach to validity
Evaluation and Program Planning
(1987)
L. Bickman
Using program theory in evaluation
J.S. Brekke
The model-guided method for monitoring program implementation
Evaluation Review
(1987)
M.J. Chandler
Egocentrism and antisocial behavior: The assessment and training of social perspective-talking skills
Development Psychology
(1983)
Chen, H.T. (in press). Theory-driven evaluations: A comprehensive perspective. Newbury Park, CA:...
H.T. Chen et al.
The multi-goal, theory-driven approach to evaluation: A model linking basic and applied social science
Social Forces
(1980)
H.T. Chen et al.
Evaluating with sense: The theory-driven approach
Evaluation Review
(1983)

There are more references available in the full text version of this article.

Cited by (35)

Realist protocol for implementing integrated care pathways in France (waiver 51 LFSS 2018)
2022, Annales Pharmaceutiques Francaises
Le paradoxe des études qui sont prétendument conduites, en vie réelle est qu’elles n’ont pas réussi à se libérer des techniques des essais randomisés dont elles prétendaient s’affranchir. Dans la perspective successioniste qui est celle des méthodes expérimentales ou quasi expérimentales classiques, la cause précède toujours l’effet de manière linéaire, et toute interférence qui est susceptible de menacer la stabilité de cette relation doit être neutralisée en mobilisant les techniques statistiques appropriée. Dans des systèmes complexes où tout bouge en même temps du fait d’interrelations multiples qui rendent impossible la construction d’un contrefactuel, ces éléments ne sont plus appelés à être considérés comme autant de facteurs de confusion qu’il convient de contrôler, mais comme des facteurs décisifs du bon déroulement de l’expérimentation. Le protocole présenté dans cet article propose une technique évaluative alternative mobilisant les enseignements du réalisme critique qui semblent les plus à même d’appréhender ce qui se passe « dans la boîte noire » enregistrant tous les événements qui sont susceptibles de survenir entre la mise en œuvre des expérimentations article 51 et le relevé de leurs résultats. Le rôle de l’évaluateur est de mettre les acteurs au cœur du changement, puisque celui-ci se réalise (ou non) en fonction de leurs réactions et des éléments de contexte. Cette théorie explicative crédible permet de comprendre : comment cela fonctionne ? Pour qui cela fonctionne ? pourquoi les perdants perdent, les gagnants gagnent ? et dans quelles circonstances ?
The paradox of real world researches supposedly conducted in real life is that they did not succeeded in freeing themselves from the techniques of the randomised trials which they pretended to escape. In the successionist perspective of classical experimental or quasi-experimental methods, the cause always precedes the effect in a linear manner, and any interference that is likely to threaten the stability of this relationship must be neutralised by mobilising the appropriate statistical techniques. In complex systems where everything moves at the same time due to multiple interrelationships that make it impossible to construct a counterfactual, these elements are no longer considered as confounding factors that need to be controlled, but as decisive factors in the smooth running of the experiment. The protocol presented in this article proposes an alternative evaluative technique mobilising the teachings of critical realism, which seems to us to be the most appropriate for understanding what happens “in the black box” recording the events that occur between the implementation of the Article 51 experiments and the observed results. The role of the evaluator is to put the actors back at the heart of the change, since it is achieved (or not) according to their reactions and the contextual elements. This credible explanatory theory allows us to understand: how does it work? For whom does it work? Why do losers lose, winners win? and under what circumstances?
Facilitating transition from children's to adult services for young adults with life-limiting conditions (TASYL). Programme theory developed from a mixed methods realist evaluation
2018, International Journal of Nursing Studies
Citation Excerpt :
Mechanisms are located in the reasoning, perceptions, beliefs and resources of those affected by the programme (Higgins et al., 2015; Pawson and Tilley, 2004). The outcomes are the result of mechanisms operating in a particular context (Pawson and Tilley, 1997) with intended outcomes being what programmes strive to achieve (Chen, 1989). Interventions work differently in different circumstances, which means simple replication of interventions in varying contexts may not produce the same outcomes.
Improvements in care and treatment have led to more young adults with life-limiting conditions living beyond childhood, necessitating a transition from children's to adult services. Given the lack of evidence on interventions to promote transition, it is important that those creating and evaluating interventions develop a theoretical understanding of how such complex interventions may work.
To develop theory about the interventions, and organisational and human factors that help or hinder a successful transition from children’s to adult services, drawing on the experience, knowledge, and insights of young adults with life-limiting conditions, their parents/carers, and service providers.
A realist evaluation using mixed methods with four phases of data collection in the island of Ireland. Phase one: a questionnaire survey of statutory and non-statutory organisations providing health, social and educational services to young adults making the transition from children’s to adult services in Northern Ireland and one Health Services Executive area in the Republic of Ireland. Phase two: interviews with eight young adults. Phase three: two focus groups with a total of ten parents/carers. Phase four: interviews with 17 service providers. Data were analysed seeking to explain the impact of services and interventions, and to identify organisational and human factors thought to influence the quality, safety and continuity of care.
Eight interventions were identified as facilitating transition from children’s to adult services. The inter-relationships between these interventions supported two complementary models for successful transition. One focused on fostering a sense of confidence among adult service providers to manage the complex care of the young adult, and empowering providers to make the necessary preparations in terms of facilities and staff training. The other focused on the young adults, with service providers collaborating to develop an autonomous young adult, whilst actively involving parents/carers. These models interact in that a knowledgeable, confident young adult who is growing in decision-making abilities is best placed to take advantage of services – but only if those services are properly resourced and run by staff with appropriate skills.
No single intervention or stakeholder group can guarantee a successful transition. Rather, service providers could work with young adults and their parents/carers to consider desired outcomes, and the range of interventions, in light of the organisational and human resources available in their context. This would allow them to supplement the organisational context where necessary and select interventions that are more likely to deliver outcomes in that context.
A two-step approach to evaluating sustainability-related consumer education interventions
2017, Journal of Cleaner Production
Citation Excerpt :
A well suited approach to shedding light on the mechanisms that lead to the induction of effects–and thus to breaking up the black box–is the theory-driven approach to (impact) evaluation (Astbury and Leeuw, 2010; White, 2009). In this approach, the process of theoretically relating intervention outcomes to different (potential) determinants is called the development of a “causal program theory” (Chen, 1989) or “treatment theory” (Finney and Moos, 1989). Developing such a treatment theory requires assumptions about the potential mechanisms between program inputs, contexts and outcomes.
The characteristics of creative educational interventions and the way they are implemented in the field often make their evaluation a challenging task. This article uses an exemplary intervention from a large-scale consumer education program on climate protection to present the design, method, and results of a two-step evaluation procedure which allows evaluators to cope with such a situation. Step 1 aims to answer the question of whether or not an intervention actually has the intended effects. Step 2 then assesses the factors that contribute to those effects. Thus, such a two-step evaluation yields information, not only on which interventions are effective and should therefore be maintained, but also on how they should be designed to achieve maximum effects.
TEACH: Trial of Education And Compliance in Heart dysfunction chronic disease and heart failure (HF) as an increasing problem
2008, Contemporary Clinical Trials
Citation Excerpt :
The timing of assessments would control for the fact that an assessment post-intervention may over-estimate the treatment effect. This evaluation obviated a number of the previously identified methodological problems [61–73] by following the current gold standard evaluation methodology as proposed by Vingilis et al. [74]. TEACH utilized the following strategies and components for patient education — Educational: oral, visual, audiovisual, written, the audiovisual and written were delivered in hospital to all patients, the CP (PHARM) delivered an oral component using visual aids and reiterate written information; Behavioural: Medication Event Monitor, dosage tailoring, memory aids, calendar.
In this manuscript we outline the development of a randomized control trial.
Heart failure (HF) is one of the leading causes of hospitalization in North America. While great advances have been made in the treatment of this disease, many factors limit the effectiveness of treatment. The objective of this Trial of Education And Compliance in Heart failure patients (TEACH) was to measure in patients with heart failure who were hospitalized and then discharged into the community, the impact of a comprehensive outpatient educational intervention on: 1) a composite endpoint of mortality, readmissions and visits to the emergency room for all causes, 2) the impact of the intervention on health-related quality of life, 3) the impact of the intervention on compliance to medication and 4) the economic impact of providing such an intervention to patients.
This was a prospective cluster randomized controlled clinical trial in patients who were admitted to hospital with a diagnosis of heart failure (HF). Patients were stratified by reason for admission to hospital. Stratum 1 included patients whose primary reason for admission was HF and stratum 2 included patients whose primary reason for hospital admission was not HF.
HF patients who receive an enhanced educational intervention from their community pharmacist will 1) have an absolute risk reduction in the number of events of 20%, 2) be more compliant, 3) will demonstrate less health-related quality of life (HRQoL) impairment as measured by disease-specific and generic HRQoL instruments and 4) will lower costs/Quality Adjusted Life Years (QALY) due to fewer total events per patient.
Outcomes were collected every 3 months from baseline to the end of 1 year. Events were tracked using a comprehensive database that we developed and validated. In order to measure compliance we used electronic measures known as Track Caps (MEMS). We also used pharmacy refill data that was provided to us from all community pharmacies the patient used. This data was analyzed and interpreted using a validated formula developed by Steiner et al. (1996).
The Mechanistic Rewards of Data and Theory Integration for Theory-Based Evaluation
2024, American Journal of Evaluation
Advancing Social Intervention Research Through Program Theory Reconstruction
2023, Research on Social Work Practice

View all citing articles on Scopus

View full text

The conceptual framework of the theory-driven perspective

Abstract

Evaluation and Program Planning

Using program theory in evaluation

The model-guided method for monitoring program implementation

Evaluation Review

Egocentrism and antisocial behavior: The assessment and training of social perspective-talking skills

Development Psychology

The multi-goal, theory-driven approach to evaluation: A model linking basic and applied social science

Social Forces

Evaluating with sense: The theory-driven approach

Evaluation Review