Background
Methods
Participants
Describing our sample and related theoretical expertise
The interview guide and interview
Member checking the hypotheses
Theme generation
Results
Discipline or field | Total |
---|---|
Cognitive psychology | 9 |
Education | 8 |
Medical decision-making | 7 |
Industrial organization or management | 6 |
Social or health psychology | 5 |
Medical education | 5 |
Economics | 3 |
Human factors | 2 |
Expert | Self-described expertise | Other concepts/areas of expertise referred to during interview |
---|---|---|
1. | Brunswikian psychology, diagnostic judgments of physicians, use of vignettes containing clinical cues | Lens modeling, evidence-based medicine, behavior change theory, face validity |
2. | Behavior decision theory, methodological theory of information integration—what cues people pay attention to | Diffusion of responsibility, norm theory, SMART goals, loss aversion theory, scale compatibility, habituation theory, spacing effects |
3. | Human factors, health communication and decision making, gestalt principles, information science | Theories of attention, international design standards, prospect theory, loss aversion, cognitive load, constructivist learning theory, intrinsic/extrinsic motivation |
4. | Self-assessment, behavior change, comparison models | Guided reflection, learner centered agenda, teacher directed agenda |
5. | Diagnostic reasoning of physicians, dual process models, information distortion (gestalt) | Learning theories, theory of planned behavior, extrinsic motivation |
6. | Human factors engineering, iterative design | Theory of planned behavior, theories of operant conditioning, law of effect |
7. | Cognitive psychology, judgment decision making framework, information processing, linguistics | Tufte theory, incentives, Lake Woebegone effect in social psychology |
8. | Applied work in medical decision making, hindsight bias | Fast and frugal heuristics |
9. | Cognitive psychology—how people reason, formulate judgments, and make decisions | |
10. | Personality, social, and health psychology, principles of feedback control | Self-regulation of behavior |
11. | Cognitive psychology, learning, memory, lab research on feedback | |
12. | Behavioral economics, psychology, rational choice | Individual limitations, motivations, ego, mastery, social comparison |
13. | Organizational psychology, feedback research, feedback seeking behavior, feedback environment framework, information processing, achievement goal theory, personality | Self-motives framework, self-enhancement theory, self-determination theory, motivation, dual process models, serial position curve/memory |
14. | Cognitive psychology, measurement, assessment, formative feedback, constructivism, active learning theories | Cognitive load, growth mindset work (Dweck), goal setting theory, display of quantitative information (Tufte), graph design |
15. | Social psychology, attribution/dissonance theory, prospect theory, conflict and dispute resolution, study of influence | Lewinian channel factor identification, self-perception theory, social norms theory, nudge theory (Thaler and Sunstein), motivation, Prospect Theory |
16. | Methodology, resource management principal | |
17. | Bjork’s desirable difficulties | |
18. | Psychology, dual processes, affect and emotion, numeracy and aging | |
19. | Goal setting theory | Cognitive load |
20. | Social psychology, health communication strategies or health decision making and health behavior change, social cognitive theory, theory of planned behavior, adoption process model, social comparison theory, classic theories of attitude and behavior | Study of influence |
21. | Control theory, self-regulation theory, goal setting theory, self-efficacy | Reinforcement theory, partial reinforcement theory |
22. | Industrial organizational psychology, work motivation, team performance, feedback from the standpoint of individual behavior | Self-regulation, control theory, goals and actions, subjective expected utility theory (theory of reasoned action, expectancy theory) |
23. | Educational theory, learning theories, constructivism, socio-cultural learning theories | Reflective learning, motivation, peer learning, communities of practice, social learning, peer scaffolding, role modeling |
24. | Applying psychological principals in clinical practice, sociological learning theory, sociocultural theory, feedback interventions | Discourse theory, activity theory, complexity theory, achievement motivation theory |
25. | Psychology, economics, ethics, patient physician communication, treatment decision making | |
26. | Education research, feedback in education, social cultural theory | |
27. | Industrial organizational psychology, Power’s control theory (self-regulation theory), Carver and Scheier’s social cognitive theory | Gain theory, implementation intentions |
28. | Education, constructivist approach, basic notions of social psychology, multisource feedback or feedback to students from supervisors (Ross and Nesbett), social cultural theory (Vigotsky), humanist theory (Carl Rogers) | Theories of behavior change, self-regulation, feedback intervention theory (Kluger and Denisi), motivation theories, informed self-assessment |
Hypotheses generated
Themes(N = 30) | # of hypotheses (N = 313) | Example hypotheses A&F/A&F interventions will be more effective… |
---|---|---|
Related to the recipient | ||
1. Trust/credibility | 14 | If it is perceived to be without conflict of interest; when recommendations related to the A&F are based on good quality evidence |
2. Motivation/intention | 13 | If it is accompanied with positive reinforcement to those who have improved their performance; when accompanied by incentive |
3. Recipient characteristics | 9 | For those with a mastery goal orientation if it involves comparison to others |
4. Recipient priorities | 9 | When targeted at behaviors that the target feels is important to their professional roles/responsibilities |
5. Attack on self-identity | 7 | When measures are used to prevent a defensive response (providing other “reassuring” messages as well) |
6. Attract/maintain attention | 6 | If they engage the target’s attention |
7. Self-efficacy/control | 5 | If the behavior is under the control of the recipient |
Related to the behavior | ||
8. Remove barriers | 11 | If they address barriers to change in behavior |
9. About aspects of behavior | 7 | For behaviors that are easy compared to those that are harder to do |
10. Decision processes or conceptual model | 4 | If designed with a clear understanding of the decision making process underlying the behavior to be changed |
Related to the content of the A&F | ||
11. Cognitive load | 33 | If as few graphs as possible are presented; without unnecessary depth elements; if the graphical representations are clearly and consistently labeled; when color changes are purposeful and convey meaning; when presenting absolute numbers as opposed to percentages; when graphical clutter is removed; when focused on a few, most important behaviors |
12. Comparisons | 26 | When the benchmark comparison is justified to be a reasonable standard; when a comparator is provided; when multiple individual practice data is presented along with the recipient’s data; if it involves a comparison to the self; if the comparator is specific to the recipient’s own context/practice. |
13. Action plans/coping strategies | 19 | If clear direction on how to change behavior is provided |
14. Feedback specificity | 16 | If individual level provider data is provided; if patient-specific information is provided; if it is as specific as possible |
15. Goal setting | 16 | If it is accompanied by a goal that is specific |
16. Justify need for behavior change | 10 | If accompanied by evidence supporting the behavior change |
17. Cognitive influences | 7 | If emphasis is on what needs to be achieved (loss framing) as opposed to what was achieved (gain framing). |
18. Nature of the data | 6 | If graphical representation displays the variability of data in order to indicate the error or uncertainty |
19. Guide reflection | 6 | If it involves a personal reflection component |
20. Improving memory | 6 | If the reminder messages are presented in real time/point of care; if incorporates an emotional message underlining the desired behavior |
Related to the delivery of the A&F | ||
21. A&F timing | 20 | If individual change data over time is provided; when presented multiple times; when presented at the time of decision making |
22. Social engagement | 17 | If they involve engaging recipients in social discussion about the A&F |
23. Knowledge/learning | 13 | If it creates opportunities to learn |
24. User-guided experience | 6 | When complex information is scaffolded to allow a recipient to get more information if and when they want |
25. In-person A&F | 2 | When provided with human contact |
26. Responding to A&F providers | 2 | If they allow the recipient an opportunity to indicate why a recommended action was not taken. |
Other | ||
27. Opportunity costs | 7 | When there are few costs to change behavior |
28. Environment | 4 | If the environment encourages the desired behavior as the default. |
29. Development process involvement | 2 | When recipients have been involved in the design of the A&F |
30. Single hypotheses | 10 | If they imply some kind of extended commitment; if the recipient generates a response immediately prior to receiving the A&F; if the goal is made public; if it is provided to the intended target for behavior change; it includes multiple modes of information (e.g., pictures and text) |
Related to the recipient (n = 63 hypotheses in seven themes)
Related to the behavior (n = 22 hypotheses in three themes)
Related to the content of the A&F (n = 145 hypotheses in ten themes)
Related to the delivery of the A&F (n = 60 hypotheses in six themes)
Other (n = 23 hypotheses in four themes)
Discussion
-
A&F will be more effective if noun descriptors rather than verbs are used in messaging—‘do not be an over prescriber’ versus ‘please prescribe less’.
-
A&F will be more effective if it incorporates a gaming approach.
-
A&F will be more effective if information about opportunity costs is included; A&F will be more effective when recipients have been involved in the design of the A&F.
-
A&F interventions will be more effective if they involve engaging recipients in social discussion about the A&F.