Materials and methods
Dataset
Skill assessment metrics
Construct validity of camera metrics
Camera and conventional metric comparisons
Correlation to conventional performance metrics
Results
New | Intermed. | Exp. |
p value (N vs. I) |
p value (N vs. E) |
p value (I vs. E) | Time | Overall score | |||
---|---|---|---|---|---|---|---|---|---|---|
Corr. coef. |
p value | Corr. coef. |
p value | |||||||
Camera Targeting—Level 1 | 0.25 ± 0.10 | 0.36 ± 0.10 | 0.37 ± 0.11 |
0.0051
|
0.0044
| 0.8392 | 0.6081 |
<0.0001
| 0.4958 |
0.0007
|
Camera Targeting—Level 2 | 0.22 ± 0.07 | 0.32 ± 0.07 | 0.36 ± 0.09 |
0.0009
|
<0.0001
| 0.2132 | 0.7381 |
<0.0001
| 0.6359 |
<0.0001
|
Dots and Needles—Level 1 | 0.02 ± 0.02 | 0.05 ± 0.04 | 0.07 ± 0.05 |
0.0095
|
0.0048
| 0.3155 | 0.6457 |
<0.0001
| 0.5722 |
0.0003
|
Dots and Needles—Level 2 | 0.01 ± 0.02 | 0.03 ± 0.03 | 0.06 ± 0.06 | 0.1296 |
0.0429
| 0.1151 | 0.4483 |
0.0114
| 0.2947 | 0.1075 |
Energy Dissection—Level 1 | 0.03 ± 0.02 | 0.06 ± 0.04 | 0.08 ± 0.05 |
0.0008
|
0.0006
| 0.3565 | 0.5366 |
0.0003
| 0.4945 |
0.0010
|
Energy Dissection—Level 2 | 0.04 ± 0.03 | 0.07 ± 0.04 | 0.08 ± 0.05 |
0.0295
|
0.0114
| 0.4572 | 0.2531 | 0.1151 | 0.2114 | 0.1904 |
Energy Switching—Level 1 | 0.12 ± 0.05 | 0.17 ± 0.07 | 0.20 ± 0.08 |
0.0113
|
0.0039
| 0.4945 | 0.4388 |
0.0036
| 0.4499 |
0.0028
|
Match Board—Level 1 | 0.04 ± 0.03 | 0.11 ± 0.05 | 0.14 ± 0.05 |
<0.0001
|
<0.0001
| 0.1888 | 0.7673 |
<0.0001
| 0.6571 |
<0.0001
|
Match Board—Level 2 | 0.07 ± 0.04 | 0.15 ± 0.07 | 0.18 ± 0.07 |
0.0046
|
0.0002
| 0.2226 | 0.6424 |
<0.0001
| 0.5270 |
0.0011
|
Matchboard—Level 3 | 0.06 ± 0.04 | 0.12 ± 0.05 | 0.14 ± 0.07 |
0.0005
|
0.0003
| 0.4152 | 0.7567 |
<0.0001
| 0.6611 |
<0.0001
|
Needle Targeting | 0.06 ± 0.04 | 0.12 ± 0.08 | 0.13 ± 0.09 |
0.0233
|
0.0061
| 0.6404 | 0.3254 |
0.0355
| 0.3044 |
0.0500
|
Peg Board—Level 1 | 0.04 ± 0.05 | 0.16 ± 0.10 | 0.18 ± 0.08 |
0.0020
|
0.0001
| 0.6706 | 0.3493 | 0.0501 | 0.3624 |
0.0415
|
Peg Board—Level 2 | 0.06 ± 0.03 | 0.19 ± 0.06 | 0.21 ± 0.07 |
<0.0001
|
<0.0001
| 0.4591 | 0.6932 |
<0.0001
| 0.6356 |
<0.0001
|
Pick and Place | 0.04 ± 0.03 | 0.05 ± 0.03 | 0.04 ± 0.02 | 0.3775 | 0.6933 | 0.4804 | 0.1181 | 0.6200 | −0.0394 | 0.8692 |
Ring and Rail—Level 1 | 0.06 ± 0.03 | 0.13 ± 0.08 | 0.16 ± 0.12 |
0.0014
|
0.0019
| 0.5150 | 0.6409 |
<0.0001
| 0.4581 |
0.0018
|
Ring and Rail—Level 2 | 0.05 ± 0.03 | 0.11 ± 0.05 | 0.14 ± 0.08 |
0.0002
|
0.0002
| 0.3582 | 0.6256 |
<0.0001
| 0.5331 |
0.0002
|
Ring Walk—Level 1 | 0.14 ± 0.06 | 0.20 ± 0.08 | 0.20 ± 0.06 |
0.0155
|
0.0065
| 0.9668 | 0.6207 |
<0.0001
| 0.4959 |
0.0007
|
Ring Walk—Level 2 | 0.13 ± 0.05 | 0.19 ± 0.06 | 0.23 ± 0.06 |
0.0099
|
<0.0001
| 0.1230 | 0.7449 |
<0.0001
| 0.4580 |
0.0020
|
Ring Walk—Level 3 | 0.07 ± 0.04 | 0.15 ± 0.06 | 0.19 ± 0.08 |
0.0002
|
<0.0001
| 0.1769 | 0.7885 |
<0.0001
| 0.5099 |
0.0006
|
Scaling | 0.10 ± 0.07 | 0.22 ± 0.08 | 0.24 ± 0.09 |
<0.0001
|
<0.0001
| 0.5702 | 0.1325 | 0.4028 | −0.3042 | 0.0501 |
Suture Sponge—Level 1 | 0.01 ± 0.00 | 0.03 ± 0.04 | 0.03 ± 0.04 | 0.2101 | 0.1332 | 0.8011 | 0.4389 |
0.0362
| 0.3530 | 0.0985 |
Suture Sponge—Level 2 | 0.04 ± 0.02 | 0.08 ± 0.05 | 0.10 ± 0.07 |
0.0015
|
0.0015
| 0.4202 | 0.5657 |
0.0001
| 0.4509 |
0.0031
|
Suture Sponge—Level 3 | 0.03 ± 0.02 | 0.07 ± 0.04 | 0.09 ± 0.05 |
0.0009
|
0.0002
| 0.3024 | 0.4876 |
0.0011
| 0.5585 |
0.0001
|
Thread the Rings | 0.05 ± 0.03 | 0.09 ± 0.06 | 0.11 ± 0.06 |
0.0114
|
0.0020
| 0.6108 | 0.4264 |
0.0049
| 0.4244 |
0.0051
|
Tubes | 0.02 ± 0.01 | 0.05 ± 0.05 | 0.08 ± 0.07 |
0.0035
|
0.0018
| 0.3661 | 0.5165 |
0.0005
| 0.4828 |
0.0014
|
New | Intermed. | Exp. |
p value (N vs. I) |
p value (N vs. E) |
p value (I vs. E) | Time | Overall score | |||
---|---|---|---|---|---|---|---|---|---|---|
Corr. coef. |
p value | Corr. coef. |
p value | |||||||
Camera Targeting—Level 1 | 2.40 ± 1.09 | 1.89 ± 0.75 | 2.09 ± 0.84 | 0.1601 | 0.4062 | 0.5212 | −0.2594 | 0.0931 | −0.1886 | 0.2258 |
Camera Targeting—Level 2 | 1.91 ± 0.65 | 1.40 ± 0.30 | 1.30 ± 0.26 |
0.0143
|
0.0035
| 0.3699 | −0.5880 |
<0.0001
| −0.5925 |
<0.0001
|
Dots and Needles—Level 1 | 1.27 ± 0.58 | 1.05 ± 0.34 | 0.78 ± 0.23 | 0.2607 |
0.0132
|
0.0324
| −0.5928 |
0.0001
| −0.3638 |
0.0292
|
Dots and Needles—Level 2 | 1.94 ± 1.81 | 0.94 ± 0.41 | 0.68 ± 0.21 | 0.0676 |
0.0327
| 0.0633 | −0.3425 | 0.0593 | −0.1825 | 0.3259 |
Energy Dissection—Level 1 | 1.24 ± 0.39 | 0.87 ± 0.25 | 0.71 ± 0.18 |
0.0067
|
0.0002
| 0.0789 | −0.4239 |
0.0057
| −0.2254 | 0.1566 |
Energy Dissection—Level 2 | 1.12 ± 0.33 | 0.92 ± 0.27 | 0.75 ± 0.22 | 0.0882 |
0.0024
| 0.0929 | −0.5038 |
0.0009
| −0.4861 |
0.0015
|
Energy Switching—Level 1 | 1.17 ± 0.27 | 0.97 ± 0.25 | 0.89 ± 0.22 |
0.0490
|
0.0062
| 0.3880 | −0.5365 |
0.0003
| −0.4628 |
0.0023
|
Match Board—Level 1 | 1.42 ± 0.46 | 0.91 ± 0.30 | 0.80 ± 0.23 |
0.0035
|
0.0003
| 0.3118 | −0.6846 |
<0.0001
| −0.5432 |
0.0004
|
Match Board—Level 2 | 1.54 ± 0.67 | 1.00 ± 0.21 | 0.91 ± 0.18 |
0.0175
|
0.0044
| 0.2910 | −0.5752 |
0.0003
| −0.4859 |
0.0031
|
Matchboard—Level 3 | 2.00 ± 0.94 | 1.08 ± 0.32 | 0.90 ± 0.21 |
0.0032
|
0.0003
| 0.1026 | −0.5209 |
0.0004
| −0.3815 |
0.0127
|
Needle Targeting | 1.45 ± 0.58 | 0.98 ± 0.28 | 0.82 ± 0.21 |
0.0112
|
0.0012
| 0.1234 | −0.6561 |
<0.0001
| −0.5669 |
<0.0001
|
Peg Board—Level 1 | 1.84 ± 1.73 | 0.94 ± 0.32 | 0.81 ± 0.20 | 0.1042 | 0.0779 | 0.2971 | −0.4919 |
0.0042
| −0.4002 |
0.0232
|
Peg Board—Level 2 | 2.05 ± 0.99 | 1.05 ± 0.21 | 1.00 ± 0.21 |
0.0020
|
0.0008
| 0.5314 | −0.6704 |
<0.0001
| −0.7447 |
<0.0001
|
Pick and Place | 0.49 ± 0.45 | 0.87 ± 0.23 | 0.93 ± 0.44 | 0.0871 | 0.0882 | 0.7885 | 0.1381 | 0.5615 | 0.1513 | 0.5243 |
Ring and Rail—Level 1 | 2.29 ± 1.33 | 1.03 ± 0.40 | 0.86 ± 0.24 |
0.0027
|
0.0006
| 0.1745 | −0.5503 |
0.0001
| −0.3942 |
0.0081
|
Ring and Rail—Level 2 | 1.70 ± 0.67 | 0.94 ± 0.31 | 0.77 ± 0.20 |
0.0008
|
<0.0001
| 0.0964 | −0.6639 |
<0.0001
| −0.5869 |
<0.0001
|
Ring Walk—Level 1 | 2.35 ± 0.74 | 1.66 ± 0.48 | 1.68 ± 0.42 |
0.0076
|
0.0079
| 0.9028 | −0.4826 |
0.0010
| −0.4300 |
0.0040
|
Ring Walk—Level 2 | 2.20 ± 0.86 | 1.62 ± 0.38 | 1.40 ± 0.28 |
0.0339
|
0.0047
| 0.1102 | −0.5457 |
0.0002
| −0.2502 | 0.1100 |
Ring Walk—Level 3 | 1.77 ± 0.74 | 1.09 ± 0.32 | 0.86 ± 0.21 |
0.0049
|
0.0002
|
0.0449
| −0.6292 |
<0.0001
| −0.4202 |
0.0062
|
Scaling | 1.29 ± 0.51 | 0.99 ± 0.25 | 0.95 ± 0.22 | 0.0652 |
0.0416
| 0.6669 | −0.1298 | 0.4127 | 0.1129 | 0.4766 |
Suture Sponge—Level 1 | 1.31 ± 0.66 | 0.77 ± 0.25 | 0.72 ± 0.36 | 0.0508 |
0.0421
| 0.7679 | −0.5922 |
0.0029
| −0.3949 |
0.0622
|
Suture Sponge—Level 2 | 1.36 ± 0.62 | 1.05 ± 0.42 | 0.73 ± 0.15 | 0.1355 |
0.0019
|
0.0187
| −0.5779 |
<0.0001
| −0.4376 |
0.0042
|
Suture Sponge—Level 3 | 1.36 ± 0.53 | 0.91 ± 0.28 | 0.71 ± 0.18 |
0.0094
|
0.0004
| 0.0512 | −0.6565 |
<0.0001
| −0.6601 |
<0.0001
|
Thread the Rings | 1.36 ± 0.38 | 0.92 ± 0.25 | 0.76 ± 0.22 |
0.0014
|
<0.0001
| 0.1067 | −0.6612 |
<0.0001
| −0.5324 |
0.0003
|
Tubes | 1.30 ± 0.56 | 0.96 ± 0.31 | 0.72 ± 0.17 | 0.0620 |
0.0022
|
0.0321
| −0.5437 |
0.0002
| −0.3995 |
0.0097
|
New | Intermed. | Exp. |
p value (N vs. I) |
p value (N vs. E) |
p value (I vs. E) | Time | Overall score | |||
---|---|---|---|---|---|---|---|---|---|---|
Corr. coef. |
p value | Corr. coef. |
p value | |||||||
Camera Targeting—Level 1 | 4.42 ± 1.64 | 2.98 ± 0.94 | 3.01 ± 1.06 |
0.0089
|
0.0121
| 0.9409 | −0.5254 |
0.0003
| −0.4769 |
0.0012
|
Camera Targeting—Level 2 | 5.01 ± 2.61 | 3.19 ± 0.72 | 2.86 ± 0.62 |
0.0213
|
0.0072
| 0.2261 | −0.5914 |
<0.0001
| −0.6318 |
<0.0001
|
Dots and Needles—Level 1 | 30.59 ± 18.86 | 29.80 ± 28.96 | 20.33 ± 11.50 | 0.9493 | 0.1559 | 0.3015 | −0.3356 | 0.0604 | −0.2810 | 0.1193 |
Dots and Needles—Level 2 | 47.04 ± 35.73 | 31.11 ± 17.89 | 19.42 ± 11.57 | 0.2618 |
0.0497
| 0.1133 | −0.7061 |
0.0001
| −0.5917 |
0.0023
|
Energy Dissection—Level 1 | 58.77 ± 51.49 | 18.49 ± 9.01 | 17.72 ± 10.16 |
0.0102
|
0.0122
| 0.8431 | −0.3421 |
0.0330
| −0.4413 |
0.0049
|
Energy Dissection—Level 2 | 46.46 ± 55.68 | 17.72 ± 10.51 | 15.66 ± 11.16 | 0.0789 | 0.0846 | 0.6470 | −0.2299 | 0.1591 | −0.2325 | 0.1543 |
Energy Switching—Level 1 | 9.60 ± 4.61 | 6.06 ± 2.10 | 5.94 ± 2.90 |
0.0160
|
0.0223
| 0.9068 | −0.3597 |
0.0193
| −0.4618 |
0.0021
|
Match Board—Level 1 | 34.91 ± 41.74 | 9.99 ± 4.83 | 8.22 ± 4.07 | 0.0519 |
0.0382
| 0.3417 | −0.3740 |
0.0246
| −0.3734 |
0.0249
|
Match Board—Level 2 | 14.80 ± 8.63 | 8.24 ± 4.15 | 6.37 ± 2.48 |
0.0343
|
0.0039
| 0.1984 | −0.6241 |
<0.0001
| −0.6240 |
<0.0001
|
Matchboard—Level 3 | 24.81 ± 19.38 | 9.36 ± 4.14 | 10.37 ± 9.24 |
0.0117
|
0.0198
| 0.7301 | −0.5080 |
0.0006
| −0.4461 |
0.0031
|
Needle Targeting | 17.69 ± 14.57 | 12.64 ± 7.75 | 11.07 ± 7.08 | 0.2684 | 0.1582 | 0.6024 | −0.2528 | 0.1063 | −0.4117 |
0.0067
|
Peg Board—Level 1 | 13.72 ± 10.42 | 6.00 ± 2.68 | 5.54 ± 1.79 | 0.0504 |
0.0354
| 0.6753 | −0.5376 |
0.0067
| −0.6100 |
0.0016
|
Peg Board—Level 2 | 17.65 ± 10.06 | 5.74 ± 1.80 | 5.75 ± 3.53 |
0.0004
|
0.0004
| 0.9913 | −0.6202 |
<0.0001
| −0.7285 |
<0.0001
|
Pick and Place | 4.35 ± 1.74 | 11.61 ± 3.14 | 14.40 ± 0.00 | 0.0633 | 0.1329 | 0.5232 | 0.2227 | 0.6714 | 0.2512 | 0.6311 |
Ring and Rail—Level 1 | 12.24 ± 7.30 | 6.14 ± 1.85 | 5.67 ± 2.07 |
0.0092
|
0.0110
| 0.5861 | −0.5208 |
0.0008
| −0.6320 |
<0.0001
|
Ring and Rail—Level 2 | 23.24 ± 16.90 | 10.17 ± 3.99 | 9.03 ± 4.32 |
0.0111
|
0.0064
| 0.4903 | −0.5144 |
0.0004
| −0.5815 |
<0.0001
|
Ring Walk—Level 1 | 8.11 ± 3.50 | 5.17 ± 1.53 | 5.22 ± 1.47 |
0.0087
|
0.0093
| 0.9402 | −0.5911 |
<0.0001
| −0.5098 |
0.0005
|
Ring Walk—Level 2 | 8.36 ± 2.91 | 5.82 ± 1.83 | 4.75 ± 1.30 |
0.0101
|
0.0003
| 0.0976 | −0.6958 |
<0.0001
| −0.4354 |
0.0035
|
Ring Walk—Level 3 | 18.15 ± 9.49 | 7.92 ± 2.94 | 6.10 ± 2.26 |
0.0009
|
0.0001
| 0.0900 | −0.5730 |
<0.0001
| −0.3806 |
0.0129
|
Scaling | 13.88 ± 15.03 | 4.88 ± 1.51 | 4.82 ± 1.62 |
0.0412
|
0.0486
| 0.9287 | −0.0769 | 0.6329 | 0.2046 | 0.1995 |
Suture Sponge—Level 1 | 63.83 ± 87.60 | 45.64 ± 40.59 | 37.95 ± 37.77 | 0.6641 | 0.5034 | 0.7303 | −0.4894 |
0.0462
| −0.6828 |
0.0025
|
Suture Sponge—Level 2 | 29.19 ± 13.24 | 14.98 ± 7.56 | 13.27 ± 9.25 |
0.0021
|
0.0016
| 0.6172 | −0.4445 |
0.0041
| −0.3987 |
0.0108
|
Suture Sponge—Level 3 | 37.81 ± 30.46 | 18.73 ± 10.73 | 20.30 ± 26.50 |
0.0409
| 0.1240 | 0.8456 | −0.3917 |
0.0113
| −0.4650 |
0.0022
|
Thread the Rings | 23.05 ± 11.98 | 13.90 ± 9.06 | 11.90 ± 8.12 |
0.0295
|
0.0094
| 0.5679 | −0.4313 |
0.0044
| −0.2673 | 0.0871 |
Tubes | 41.60 ± 25.24 | 21.47 ± 12.10 | 16.10 ± 8.55 |
0.0169
|
0.0032
| 0.2169 | −0.4978 |
0.0017
| −0.4637 |
0.0038
|