Background
Methods
Determining concepts for inclusion in the PIC measure
• Consultation purpose | |
• Relevant history, diagnosis and/or management to date | |
• Current management options (independent of study) | |
• Clinical Equipoise regarding trial treatments | |
• Research study purpose or question | |
• Trial arm 1 processes, disadvantages/risks, advantages/benefits | |
• Trial arm 2 processes, disadvantages/risks, advantages/benefits | |
• (3 arm trial only) Trial arm 3 processes, disadvantages/risks, advantages/benefits | |
• Reason for trial or trial purpose | |
• Randomisation | |
◦ Reason for randomisation
| |
◦ Process of randomisation
| |
• Detail on trial treatment options | |
◦ Processes, potential risks and benefit
| |
• Detail on trial procedures | |
◦ Potential risks/costs/burden & benefits of taking part,
| |
◦ Options to refuse or withdraw,
| |
◦ Options for further support in decision making about participation,
| |
◦ Benefits to professional or organisation of P taking part.
| |
◦ Confidentiality of data
| |
◦ Explanation re compensation arrangements
|
Trial | Development of DevPICv1 N = 17 | Phase 1 evaluation N = 6 (DevPICv1) | Phase 2 evaluation N = 6 (DevPICv2) | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CLASS | CHEMORAD | ProtecT | POUT | CSAW | OPTIMA | ACST II | OPTIMA | ProtecT | |||||||
N appointments | 6 | 3 | 8 | 3 | 2 | 1 | 2 | 2 | 2 | ||||||
Blinded category Appt ID:1–6 | Less good 3. | Good 1. 2. | Less good 4. | Good 5. | Less good 6. | Good - | Less good 2. | Good 1. | Less good 4. | Good 3. | Less good 5. | Good 6. | |||
Cancer (Y/N) | N | Y | Y | Y | N | Y | N | Y | Y | ||||||
Surgical arm (Y/N) | Y | Y | Y | N | Y | N | Y | N | Y | ||||||
N of trial arms | 2 | 2 | 3 | 2 | 3 | 2 | 2 | 2 | 3 | ||||||
Recruiter | |||||||||||||||
Surgeon | 5 | 1 | 1 | N/A | 2 | N/A | 2 | N/A | 0 | ||||||
Oncologist | N/A | 2 | 0 | 3 | N/A | 1 | N/A | 2 | 0 | ||||||
Nurse | 1 | 0 | 7 | 0 | 0 | 0 | 0 | 0 | 2 | ||||||
Outcome for trial participation Yes, No or Undecideda
| N = 0 | N = 0 | N = 5 | No = 2 | No = 1 | N = 1 | N = 0 | No = 0 | N = 1 | ||||||
Y = 0 | Y = 0 | Y = 2 | Y = 1 | Y=0 | Y=0 | Yes = 1 | Y = 0 | Y = 1 | |||||||
Undecided = 6 | Undecided = 3 | Undecided = 1 | Undecided =0 | Undecided =1 | Undecided =0 | Undecided = 1 | Undecided =2 | Undecided =0 | |||||||
Length of consultation | Mean:13 m 49 s Range: 8 m 58 s–16 m 22 s | Mean:30 m 46 s Range: 25 m 30s–37 m 55 s | Mean:58 m 24 s Range: 14 m 43 s–1 h 48 m 24 s | 39 m 58 s 26 m 43 s 33 m 19 s | 16 m 50s 24 m 01 s | 31 m 10s | 20 m 22 s 14 m 26 s | 1 h 06 m 27 s 22 m 13 s | 18 m 44 s 43 m 02 s | ||||||
Length of time to rate (min) | N/A | 163 m 169 m 120 m | 75 m 94 m | 81 m | 73 m 55 | 51 m 54 m | 39 m 65 m | ||||||||
Mean time to rate (min) | N/A | 117 m | 56 m |
Formative evaluation of the developmental PIC
Phase 1
-
Feasibility: the time taken to apply the measure to each recruitment appointment was recorded to determine whether it was feasible to continue with it in its existing format. It was assumed that a developmental version should be completed in under an hour.
-
Validity: initial feedback on the face validity of the measure was obtained through free-text comments and feedback from the raters who completed it. Response rates and missing data were identified for individual items to evaluate their acceptability.
-
Reliability: inter-rater reliability was assessed by evaluating differences in item responses made independently by the two raters. Each rater was required to rate each of 20 parameters (two-arm trials) or 23 parameters (three-arm trials – see Additional file 1: Appendix A) four times (evaluating first the quantity then the clarity of recruiter information, the quantity of patient talk and then evidence of understanding shown in patient talk about each parameter) giving a total of 80 ratings per appointment (two-arm trials) or 92 ratings per appointment (three-arm trials) respectively.
-
Stability: the stability (test-retest reliability) of the new measure was assessed by evaluating changes in item responses when the measure was applied to a single appointment by the same researcher, with an interval of at least 14 days. Rating procedure was as described for inter-rater reliability.
-
In interpreting both reliability and stability, a discrepancy of 1 point or less was deemed acceptable on the grounds that this might represent the difference between the presence and absence of information and between ‘mostly clear’ and ‘very clear’ on the scale. Larger discrepancies were noted.
-
Qualitative evaluation: free-text comments recorded by raters on the content and interaction in the recruitment appointment and the application of the measure were collated. Thematic analysis [45] was used to identify emergent themes in relation to the content of the information and patterns of interaction between recruiter and patient (e.g. what and how much each contributed) during discussion.
-
Findings from phase 1 were reviewed by a panel (JW, DE, KNLA, JLD, RB) convened to revise and shorten the measure to produce DevPICv2. In phase 2, it was then applied by the same two researchers (JW and DE) to six new appointments, purposively sampled as before from three trials [38, 39, 43, 46], including two appointments led by nurse recruiters, and again including one appointment from each trial where information provision was comprehensive and clear and another where information provision was less comprehensive and clear (Table 2). As before, raters were blind to these categorisations.
Phase 2
Results
Determining items for inclusion in the DevPIC
Formative evaluation of the DevPIC
Phase 1
Recruiter info–quantity | Recruiter info– clarity | Patient talk–quantity | Patient talk–understanding | |
---|---|---|---|---|
Phase 1: inter-rater reliability | ||||
N of comparisons | 126 | 126 | 126 | 126 |
N of comparisons showing ≤1-point discrepancy (% of total comparisons) | 111 (88.10) | 92 (73.02) | 113 (89.68) | 89 (70.63) |
N of comparisons showing 2-point discrepancy (% of total comparisons) | 13 (10.32) | 5 (3.97) | 10 (7.94) | 7 (5.56) |
N of comparisons showing ≥3-point discrepancy (% of total comparisons) | 2 (1.59) | 29 (23.02)a
| 3 (2.38) | 30 (23.81)a
|
aIncludes those comparisons where N/A (coded 9) was marked for one application of the measure and not the other | ||||
Phase 1: test-retest stability | ||||
N of comparisons | 126 | 126 | 126 | 126 |
N of comparisons showing ≤1-point discrepancy (% of total comparisons) | 124 (98.41) | 116 (92.06) | 124 (98.41) | 114 (90.48) |
N of comparisons showing 2-point discrepancy (% of total comparisons) | 2 (1.59) | 1 (0.79) | 2 (1.59) | 1 (0.79) |
N of comparisons showing ≥3-point discrepancy (% of total comparisons)a
| 0 (0.00) | 9 (7.14)a
| 0 (0.00) | 11 (8.73)a
|
Phase 2
Recruiter information provision | Evidence of patient understanding | |
---|---|---|
Phase 2: inter-rater reliability | ||
N of comparisons | 138 | 138 |
N of comparisons showing ≤1-point discrepancy (% of total comparisons) | 125 (90.58) | 125 (90.58) |
N of comparisons showing 2-point discrepancy (% of total comparisons) | 10 (7.25) | 13 (9.42) |
N of comparisons showing ≥3-point discrepancy (% of total comparisons) | 3 (2.17) | 0 (0.00) |
Phase 2: test-retest stability | ||
N of comparisons | 138 | 138 |
N of comparisons showing ≤1-point discrepancy (% of total comparisons) | 137 (99.28) | 137 (99.28) |
N of comparisons showing 2-point discrepancy (% of total comparisons) | 0 (0.00) | 1 (0.72) |
N of comparisons showing ≥3-point discrepancy (% of total comparisons) | 1 (0.72) | 0 (0.00) |
Appointment | Rater | Does the recruiter consistently convey a position of equipoise and what evidence do you have to suggest this? | Is the P in equipoisea and what evidence do you have to suggest this? | Does the patient accept randomisation as a means to determine treatmenta and what evidence do you have to support this? | Is there evidence that the patient is sufficiently informed by the end of the consultation to make an informed decision and what evidence do you have to support this? |
---|---|---|---|---|---|
1 | 1 | Y | Unclear | Unclear | Unclear |
2 | Y | Y | Y | Y | |
2 | 1 | N | Unclear | Unclear | N |
2 | N | Y | Unclear | N | |
3 | 1 | Y | Y | Unclear | Y |
2 | Y | Y | Y | Y | |
4 | 1 | N | Unclear | N | N |
2 | N | Unclear | N | N | |
5 | 1 | Y | Unclear | N | Unclear |
2 | N | Unclear | N | Unclear | |
6 | 1 | Y | Y | Y | Y |
2 | Y | Y | Y | Y |