Background
In present time, training outside the patient is widely accepted and several studies have shown that urological skills can be improved by simulation training [
1,
2]. The main advantage of training on simulators is that the patient-related learning curve can be shortened without compromising patient safety. In addition to the classical master-apprentice type of training (see one, do one, teach one), new simulation curricula are required due to the evolution of medical technology, the increasing number of minimally invasive procedures that urologists need to master, the decreasing number of patient-related training hours and patient safety issues [
3‐
7].
Several studies have been conducted on how to develop simulation programs. Ahmed et al. concluded that ‘proficiency-based curricula with well-structured endpoints and objective tools for validating proficiency are crucial in developing a simulation program’ [
8]. Sweet et al. emphasized the value of the backward design principle of Wiggins and McTighe [
9]. According to this principle, the purpose and learning outcome must be determined first, after which learning objectives are established by working backwards from the desired outcomes.
Within surgery, several practical skills training curricula have been implemented and validated, such as the American College of Surgeons/Association of Program Directors in Surgery surgical skills curriculum and the Fundamentals of Laparoscopic Surgery [
10‐
12]. For urology, only a limited number of urological practical skills training curricula have been developed [
13,
14]. Moreover, their structured implementation remains challenging. Possible obstacles in the implementation of a skills curriculum in surgical residency programs are issues like limited personnel, considerable cost and resident working hour restrictions [
15].
In the current Dutch urological curriculum, residents are obliged to attend a number of national practical skills courses. However, reports of the Dutch inspection of health services pointed out that residents’ knowledge of local medical technology is not optimal [
16]. The outline for the Dutch Urology Practical Skills (D-UPS) training program was designed to provide residents and program directors with a structured training program for urological basic skills, including pre-test, procedural steps, simulator training, pitfalls and evaluation. One of the first steps in the development and implementation of a new curriculum is the establishment of acceptability [
9]. To enhance this aspect, our implementation strategy included the early involvement of residents and program directors in the development of the program, prior to its validation and implementation.
This study presents the outline of the newly developed D-UPS program and aimed to answer the questions: ‘How do residents currently and ideally learn their practical urology skills?’ and ‘Which design characteristics may increase the acceptability of urological practical skills training programs such as D-UPS?’
Methods
Development of the Dutch Urology Practical Skills training program
The D-UPS program was designed using the backward design principle of Wiggins and McTighe [
17]. The program combines the acquisition and rehearsal of basic theoretical knowledge, based on theory derived from the national courses and expert input, with practical training of basic urological skills and techniques. The first step in developing each specific training session was a Training Needs Analysis (TNA) [
18,
19]. In the TNA, procedural steps were identified, potential pitfalls analysed and learning objectives defined [
20]. Subsequently, a suitable simulator was selected (Training Media Specification, TMS), with a preference for low fidelity models to limit the costs and simplify logistics.
The D-UPS program was designed by the national project group ‘Training in Urology’ in collaboration with the Dutch Association of Urologists. In the final development of the program, the opinions of residents and program directors were considered.
Study design
In this mixed-method research design, we used a questionnaire to collect quantitative data and semi-structured focus group interviews to collect qualitative data.
Questionnaire
The questionnaire was developed by a multidisciplinary team, consisting of an educationalist (AS) and two experts in urology (BS, AH). The questionnaire contained nine questions or statements rated on a 5-point Likert scale (1 = disagree, 5 = agree), five open-ended questions, one yes/no question and one multiple-choice question. Three questions focused on demographics, three on the participants’ opinions on current practical skills training, and ten on the D-UPS program, e.g. positive endpoints and expected difficulties in future implementation. The full version of the questionnaire is added in Additional file
1, including the definitions of various expected positive endpoints.
Between April 2011 and December 2011, the questionnaire was sent to all 87 Dutch urology residents and 45 program directors in the 25 teaching hospitals, using the online program Survey Monkey (
http://www.surveymonkey.com).
Interviews
For the semi-structured focus group interviews, a topic list was developed by a multidisciplinary team, consisting of an educationalist (AS) and two experts in urology (BS, AH). The topic list consisted of three main themes: 1) current way of learning practical skills, 2) ideal way of learning practical skills, and 3) respondents’ opinions on the design characteristics of the D-UPS program in relation to its acceptability (Additional file
2).
All residents and program directors in the Netherlands (n = 132) were invited by email (BS) to participate in an interview. Those residents and program directors that responded positively to this electronic invitation were divided into groups based on their geographic distribution. Between March and December 2011 the interviews were conducted in five different teaching hospitals across the Netherlands. The interviews were moderated by an independent expert in medical education (SvL). Besides the moderator, one researcher was present to make field notes (BS or AH). Before the interview, participants received one page of information on the content of the D-UPS program. Interviews continued until no new themes emerged. Needed number of interviews was based on saturation of information.
Data analysis
Questionnaire data were graphically displayed using frequency figures. Differences in categorical variables between groups were analysed using the Chi-square test. A p-value <0.05 was considered statistically significant. Analyses were performed using the Statistical Package for Social Sciences version 20.0.
Interviews were audio-recorded and transcribed verbatim by an independent company. Subsequently, transcripts were imported into a software program for qualitative data analysis (Atlas.ti version 7). The transcripts were thematically coded by the principal researcher (BS) using a predefined coding scheme based on the three main themes described earlier. To enhance interobserver reliability, 25 % of transcripts were independently coded by a second researcher (AdV). Discrepancies in initial coding between the two researchers were discussed until consensus was reached, and a final coding scheme was established. Thereafter, all interviews were summarized using the final coding scheme. The responses were categorized into the three themes. Finally, quotes were selected to illustrate findings.
Ethical aspects
Ethical approval was sought from the Catharina hospital’s research and ethics committee. Since patients or patient data were not involved in this study, they ruled that ethical approval was not required according to the Dutch Medical Research (Human Subjects) Act. All included residents and program directors volunteered to participate and anonymity and confidentiality was guaranteed. Informed consent with assurance of anonymity was obtained at the start of each interview.
Discussion
In this study, we aimed to gain insight into current and ideal Dutch urological skills training and presented the outline of the D-UPS program, including the assessment of design characteristics that may increase its acceptability. The results of this study show that Dutch residents in urology currently learn their practical skills ‘by doing’, according to the classic master-apprentice model. Ideally, they would prefer to practice certain procedures on simulation models first, especially in endourology. The acceptability of implementing the newly developed D-UPS program is high. Residents and program directors think this program would provide all residents in urology with a nationwide uniform foundation for training urological techniques. Design characteristics that increase acceptability of the D-UPS/related practical skills training programs are discussed in the next paragraphs.
One of the expected difficulties in implementing the skills training program was ‘materials’. Residents and program directors expressed the belief that practical skills training is only useful if residents practise on realistic, i.e. high fidelity models. This is contradictory to the present outline of the D-UPS program, in which low fidelity models are preferred. In the decision of which simulator to use for skills training it is of paramount importance that the simulator can serve the goal of training. In the development of the D-UPS program, first the learning objectives for training a certain skill were defined and subsequently a suitable simulator was sought. If possible the choice was for a simulator of low fidelity. This was not only to limit the cost, but also to simplify logistics and because for certain basic skills no high fidelity models are available. In the literature it is confirmed that, especially for training basic skills, low fidelity simulators can be of great value. Matsumoto et al. compared the effectiveness of a strictly didactic training in ureteroscopy with training on a low fidelity model and on a high fidelity model [
22]. They showed that training on the low fidelity model had the same degree of benefit as training on the high fidelity model, and both had a significantly higher degree of benefit than the didactic session alone. Since the first eight training sessions focus on basic urological procedures, low fidelity simulators could be suitable. However, when it comes to training more advanced skills sometimes high fidelity simulators, e.g. virtual reality simulation, will be needed. For successful implementation of practical skills training using low fidelity models, it will be of great importance that residents and program directors understand the value of these training models. McDougall et al. designed a 4-year curriculum for urology residency training, with frequent training sessions using mainly low fidelity models [
14]. Although this study included only 8 residents so far and evaluation is ongoing, initial results are encouraging. Most participants stated that this 4-year curriculum provided a better learning experience than the curriculum without structured skills training. Furthermore, while residents and program directors in our study expected one-hour training sessions to be insufficient for some parts of practical skills training, McDougall and colleagues found that acceptance of a weekly hour of training was high [
14]. In their study, the majority of residents indicated that one hour of training was sufficient and provided new clinical information.
Another important expected obstacle, according to residents and program directors in our study, was the logistic integration of practical skills training into the working week. Structured scheduling was suggested as a condition for successful implementation. The importance of scheduling training sessions and making them obligatory was emphasized by Chang and colleagues, who examined the effectiveness of voluntary training in a simulation laboratory as part of the surgical curriculum [
23]. They showed that voluntary use of a surgical simulation laboratory resulted in minimal participation in the curriculum.
Another expected difficulty in implementation was motivation, in particular the motivation of program directors. This concern is in line with the findings of Stefanidis et al., who described the implementation of a proficiency-based laparoscopic skills curriculum in a general surgical residency program and found that this can only be achieved successfully if dedicated faculty and scheduled training time are ensured [
24]. Hence, one of the key success factors for implementation is motivating program directors for their educational role in urology skills training programs.
A remarkable finding was the significant difference in views on the current availability of structured practical skills training in the local teaching hospitals. This was mentioned as current practice by 12 % of residents versus 44 % of program directors. A possible explanation for this difference could be that residents and program directors have different perceptions of the definition of practical skills training, or that some of the residents started their residency only recently, and might not yet have been involved in practical skills training.
To our knowledge, the D-UPS program would be the first curriculum in Europe that provides yearly repetitive practical skills training in the local hospital setting, including the use of the local equipment. The first step in the development and implementation of a new curriculum is the performance of training needs analysis and the establishment of acceptability, which was evaluated in this study. Although the results of this study describe the Dutch situation, which limits generalizability, the outline of the D-UPS program could serve as a blueprint for skills training in other surgical specialties in the Netherlands. Moreover, extrapolation to European countries would be possible, especially those countries with similar residency programs, since up till now there have been limited initiatives for non-patient related skills training curricula.
Where possible, existing validated simulation training is incorporated in the D-UPS program, to avoid duplication and expense. For example, the tasks used in the basic laparoscopy training of the D-UPS program are derived from the validated European Basic Laparoscopic Urological Skills program [
21]. Other possibilities should be further explored.
We acknowledge that validation of the curriculum is of paramount importance in the process of innovating educational programs. However, this is a multi-year process and is considered to be the endpoint of the implementation process. In the process towards this validation it is important to inform colleagues in the field of curriculum development regarding the ongoing developments, since they might profit from the outline of this program an our findings on design characteristics that increase the acceptability of implementing practical skills training in a non-patient-related setting.
The use of a questionnaire and interviews is relatively subjective and might have led to socially desirable answers. To counter this effect, the interviews were moderated by an independent educational expert, and anonymity was guaranteed. Furthermore, residents and program directors were interviewed in separate groups to ensure freedom and safety in expressing opinions. As in any qualitative study, investigator objectivity is a limitation [
25]. This issue was countered by having 25 % of the transcripts coded by two researchers separately.
Competing interests
The authors A.H. de Vries, S.J. van Luijk, A.J.J.A. Scherpbier, A.J.M. Hendrikx, E.L. Koldewijn, C. Wagner and B.M.A. Schout declare that they have no competing interests or financial ties to disclose.
Authors’ contributions
AdV performed statistical and qualitative analysis of the data, contributed to data interpretation and wrote the first draft of the manuscript. SvL made a substantial contribution to the acquisition of data as moderator of the semi-structured focus-group interviews. AS contributed to the conception and design of the study (optimizing questionnaire) and the interpretation of data. AH made a substantial contribution to the conception and design of the study (optimizing questionnaire) and data collection. EK and CW contributed to the interpretation of the data and the second draft of the manuscript. BS designed the study, contributed to data collection, the analysis of qualitative data and interpretation of data. All authors critically revised the manuscript and gave their final approval before submission.