Background
Methods
Data collection
Description of patients' condition and the criterion to be optimized
Description of interventions
Data management and analysis
Formulating an MDP model for the treatment of AIS
Formulating a model for MDPs with finite horizon reward criteria
Solutions to the optimality problem
Numerical implementation
No. of cases |
States at t1
|
Actions at t1
| |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
i
1
|
i
2
|
i
3
|
i
4
|
i
5
|
i
6
|
a
1
|
a
2
|
a
3
|
a
4
|
a
5
| |
22 | 2 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 |
0 | 0 | 1 | 0 | 1 | |||||||
0 | 0 | 1 | 1 | 1 | |||||||
1 | 0 | 0 | 0 | 1 | |||||||
1 | 0 | 1 | 0 | 1 | |||||||
1 | 0 | 1 | 1 | 1 | |||||||
16 | 2 | 0 | 0 | 1 | 1 | 2 | 1 | 0 | 0 | 1 | 1 |
1 | 0 | 1 | 0 | 1 | |||||||
1 | 0 | 1 | 1 | 0 | |||||||
1 | 0 | 1 | 1 | 1 | |||||||
1 | 1 | 1 | 1 | 1 | |||||||
...... | ...... | ...... | ...... | ...... | ...... | ...... | ...... | ...... | ...... | ...... | ...... |
No. of cases |
States at t2
|
Actions at t2
| |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
i
1
|
i
2
|
i
3
|
i
4
|
i
5
|
i
6
|
a
1
|
a
2
|
a
3
|
a
4
|
a
5
| |
19 | 2 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 1 |
0 | 0 | 1 | 1 | 1 | |||||||
1 | 0 | 1 | 0 | 1 | |||||||
1 | 0 | 1 | 1 | 1 | |||||||
17 | 2 | 0 | 0 | 1 | 1 | 2 | 1 | 0 | 0 | 0 | 1 |
1 | 0 | 0 | 1 | 1 | |||||||
1 | 0 | 1 | 0 | 1 | |||||||
1 | 0 | 1 | 1 | 1 | |||||||
1 | 1 | 1 | 0 | 1 | |||||||
...... | ...... | ...... | ...... | ...... | ...... | ...... | ...... | ...... | ...... | ...... | ...... |
No. of cases |
States at t1
|
Actions at t1
| |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
i
1
|
i
2
|
i
3
|
i
4
|
i
5
|
i
6
|
a
1
|
a
2
|
a
3
|
a
4
|
a
5
| |
22 | 2 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 |
16 | 2 | 0 | 0 | 1 | 1 | 2 | 1 | 0 | 1 | 0 | 1 |
...... |
No. of cases |
States at t2
|
Actions at t2
| |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
i
1
|
i
2
|
i
3
|
i
4
|
i
5
|
i
6
|
a
1
|
a
2
|
a
3
|
a
4
|
a
5
| |
19 | 2 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 |
17 | 2 | 0 | 0 | 1 | 1 | 2 | 1 | 0 | 0 | 0 | 1 |
...... |
Results
General information
N = 960 | Numbers of cases(n) |
---|---|
Age | |
46-65 years | 308 |
More than 66 years | 652 |
Disease history
| |
None | 252 |
Have at least one | 708 |
Complications
| |
None | 942 |
Have at least one | 18 |
TCM Diagnosis
| |
Apoplexy involving channels or collaterals | 960 |
Syndrome differentiation of TCM (Pattern of TCM)
| |
Yang pettern | 305 |
Yin pettern | 601 |
Composite pattern | 38 |
Other patterns | 16 |
Level of neurological function defect
| |
Level 1: 0-2 scores | 402 |
Level 2: 3-5 scores | 355 |
Level 3: 6-12 scores | 203 |
Level 4: 13-19 scores | 0 |
Level 5: 20-29 scores | 0 |
Optimal combination of treatments for corresponding states
No. of cases |
States at t1
|
Actions at t1
|
Rewards at t2* | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
i
1
|
i
2
|
i
3
|
i
4
|
i
5
|
i
6
|
a
1
|
a
2
|
a3 |
a
4
|
a
5
| ||
130 | 3 | 1 | 0 | 1 | 2 | 2 | 0 | 1 | 0 | 1 | 1 | 4.000 |
122 | 3 | 1 | 0 | 1 | 2 | 1 | 0 | 1 | 0 | 1 | 1 | 1.000 |
57 | 3 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 1 | 1.000 |
51 | 3 | 1 | 0 | 1 | 2 | 3 | 1 | 0 | 0 | 0 | 1 | 6.283 |
50 | 2 | 1 | 0 | 1 | 2 | 1 | 1 | 0 | 1 | 0 | 1 | 2.000 |
43 | 2 | 1 | 0 | 1 | 2 | 2 | 1 | 1 | 0 | 0 | 1 | 3.700 |
41 | 3 | 0 | 0 | 1 | 2 | 1 | 0 | 1 | 0 | 0 | 1 | 0.344 |
40 | 3 | 1 | 0 | 1 | 1 | 2 | 0 | 0 | 0 | 0 | 1 | 5.000 |
38 | 3 | 1 | 0 | 1 | 1 | 3 | 0 | 0 | 1 | 1 | 1 | 7.333 |
35 | 3 | 0 | 0 | 1 | 2 | 2 | 1 | 1 | 0 | 1 | 1 | 1.750 |
33 | 3 | 0 | 0 | 1 | 2 | 3 | 1 | 0 | 0 | 0 | 0 | 12.000 |
31 | 2 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 0.694 |
30 | 2 | 1 | 0 | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 1 | 2.056 |
23 | 2 | 1 | 0 | 1 | 1 | 3 | 0 | 0 | 1 | 1 | 1 | 7.000 |
22 | 2 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0.667 |
22 | 2 | 0 | 0 | 1 | 2 | 1 | 0 | 0 | 0 | 0 | 1 | 1.000 |
21 | 2 | 1 | 0 | 1 | 2 | 3 | 1 | 1 | 1 | 1 | 1 | 5.571 |
20 | 3 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 2.000 |
18 | 2 | 0 | 0 | 1 | 2 | 2 | 0 | 1 | 0 | 0 | 1 | 1.000 |
18 | 3 | 0 | 0 | 1 | 1 | 2 | 0 | 0 | 1 | 1 | 1 | 4.500 |
18 | 3 | 1 | 0 | 1 | 3 | 1 | 0 | 1 | 0 | 0 | 1 | 3.000 |
17 | 2 | 0 | 0 | 1 | 2 | 3 | 0 | 1 | 0 | 1 | 1 | 4.636 |
16 | 2 | 0 | 0 | 1 | 1 | 2 | 1 | 0 | 1 | 0 | 1 | 5.200 |
11 | 3 | 1 | 0 | 1 | 3 | 2 | 0 | 1 | 0 | 0 | 1 | 2.000 |
10 | 3 | 0 | 0 | 1 | 1 | 3 | 1 | 1 | 1 | 1 | 1 | 6.000 |
10 | 3 | 1 | 0 | 1 | 4 | 1 | 0 | 0 | 1 | 1 | 0 | 1.500 |
10 | 3 | 1 | 1 | 1 | 2 | 3 | 1 | 1 | 1 | 1 | 1 | 6.100 |
9 | 2 | 1 | 0 | 1 | 3 | 1 | 0 | 0 | 1 | 0 | 1 | 2.000 |
8 | 3 | 1 | 1 | 1 | 2 | 2 | 0 | 1 | 0 | 0 | 1 | 0.000 |
6 | 2 | 1 | 0 | 1 | 4 | 2 | 0 | 0 | 0 | 0 | 1 | 0.333 |
Cases |
States at t2
|
Actions at t2
|
Rewards at t3* | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
i
1
|
i
2
|
i
3
|
i
4
|
i
5
|
i
6
|
a
1
|
a
2
|
a
3
|
a
4
|
a
5
| ||
127 | 3 | 1 | 0 | 1 | 2 | 1 | 0 | 1 | 0 | 1 | 1 | 1.000 |
119 | 3 | 1 | 0 | 1 | 2 | 2 | 0 | 0 | 0 | 0 | 1 | 4.000 |
60 | 3 | 1 | 0 | 1 | 2 | 3 | 1 | 0 | 0 | 0 | 1 | 4.667 |
53 | 2 | 1 | 0 | 1 | 2 | 1 | 1 | 0 | 0 | 0 | 1 | 1.000 |
51 | 3 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1.000 |
42 | 3 | 0 | 0 | 1 | 2 | 1 | 1 | 1 | 0 | 0 | 1 | 0.167 |
41 | 3 | 1 | 0 | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 0 | 5.000 |
39 | 3 | 1 | 0 | 1 | 1 | 3 | 0 | 1 | 0 | 0 | 1 | 6.000 |
38 | 3 | 0 | 0 | 1 | 2 | 3 | 1 | 0 | 0 | 1 | 1 | 9.000 |
35 | 2 | 1 | 0 | 1 | 2 | 2 | 1 | 0 | 0 | 0 | 1 | 4.000 |
31 | 2 | 1 | 0 | 1 | 1 | 2 | 0 | 0 | 1 | 1 | 1 | 2.000 |
30 | 3 | 0 | 0 | 1 | 2 | 2 | 1 | 1 | 0 | 1 | 1 | 1.333 |
29 | 2 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 1 | 0.500 |
26 | 2 | 1 | 0 | 1 | 2 | 3 | 1 | 1 | 0 | 0 | 1 | 2.571 |
23 | 2 | 1 | 0 | 1 | 1 | 3 | 1 | 0 | 0 | 0 | 1 | 7.000 |
22 | 2 | 0 | 0 | 1 | 2 | 1 | 1 | 1 | 0 | 0 | 1 | 0.667 |
19 | 2 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1.000 |
19 | 2 | 0 | 0 | 1 | 2 | 2 | 1 | 1 | 0 | 1 | 1 | 2.000 |
19 | 3 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 1 | 3.000 |
19 | 3 | 0 | 0 | 1 | 1 | 2 | 1 | 1 | 1 | 0 | 1 | 2.500 |
18 | 2 | 0 | 0 | 1 | 2 | 3 | 1 | 1 | 0 | 0 | 1 | 1.000 |
18 | 3 | 1 | 0 | 1 | 3 | 1 | 1 | 0 | 0 | 0 | 1 | 2.636 |
17 | 2 | 0 | 0 | 1 | 1 | 2 | 1 | 0 | 0 | 0 | 1 | 5.000 |
12 | 3 | 1 | 1 | 1 | 2 | 3 | 0 | 1 | 0 | 0 | 1 | 3.000 |
11 | 3 | 1 | 0 | 1 | 3 | 2 | 1 | 1 | 0 | 0 | 1 | 2.000 |
11 | 3 | 1 | 0 | 1 | 4 | 1 | 1 | 0 | 1 | 1 | 1 | 0.500 |
9 | 2 | 1 | 0 | 1 | 3 | 1 | 1 | 0 | 0 | 1 | 1 | 2.000 |
9 | 3 | 0 | 0 | 1 | 1 | 3 | 1 | 0 | 1 | 1 | 1 | 2.000 |
7 | 2 | 1 | 0 | 1 | 4 | 2 | 1 | 0 | 1 | 1 | 1 | 0.333 |
6 | 3 | 1 | 1 | 1 | 2 | 2 | 1 | 0 | 1 | 0 | 1 | 0.000 |
State
| Stage 1 | Stage 2 | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Code* |
i
1
|
i
2
|
i
3
|
i
4
|
i
5
|
i
6
| Cases |
a
1
|
a
2
|
a
3
|
a
4
|
a
5
|
Rewards
at t2
| Cases |
a
1
|
a
2
|
a
3
|
a
4
|
a
5
|
Rewards
at t3
|
(10036) | 3 | 1 | 0 | 1 | 2 | 1 | 122 | 0 | 1 | 0 | 1 | 1 | 1.00 | 127 | 0 | 1 | 0 | 1 | 1 | 1.00 |
(10037) | 3 | 1 | 0 | 1 | 2 | 2 | 130 | 0 | 1 | 0 | 1 | 1 | 4.00 | 119 | 0 | 0 | 0 | 0 | 1 | 4.00 |
(10038) | 3 | 1 | 0 | 1 | 2 | 3 | 51 | 1 | 0 | 0 | 0 | 1 | 6.28 | 60 | 1 | 0 | 0 | 0 | 1 | 4.67 |
State
| Stage 1 | Stage 2 | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Code |
i
1
|
i
2
|
i
3
|
i
4
|
i
5
|
i
6
| Case |
a
1
|
a
2
|
a
3
|
a
4
|
a
5
|
Rewards at t2
| Case |
a
1
|
a
2
|
a
3
|
a
4
|
a
5
|
Rewards at t3
|
(10031) | 3 | 1 | 0 | 1 | 1 | 1 | 57 | 0 | 0 | 1 | 0 | 1 | 1.00 | 51 | 1 | 0 | 0 | 1 | 1 | 1.00 |
(10032) | 3 | 1 | 0 | 1 | 1 | 2 | 40 | 0 | 0 | 0 | 0 | 1 | 5.00 | 41 | 1 | 0 | 1 | 1 | 0 | 5.00 |
(10033) | 3 | 1 | 0 | 1 | 1 | 3 | 38 | 0 | 0 | 1 | 1 | 1 | 7.33 | 39 | 0 | 1 | 0 | 0 | 1 | 6.00 |