Is there any benefit to adding students to the European council on chiropractic education evaluation teams and general council? An audit of stakeholders

The European Council on Chiropractic Education (ECCE) is an autonomous, international organization whose primary focus is on accreditation (and reaccreditation) of institutions offering chiropractic education and training [1]. Accreditation (and reaccreditation) of institutions is determined by the quality of their chiropractic education and training programmes judged against a set of educational Standards. These Standards, which themselves are evaluated and updated regularly, are intended for use by chiropractic institutions, in both the private and public sectors for self-evaluation of their educational programmes as well as for use by international bodies involved in the recognition and accreditation of chiropractic education worldwide [1]. Currently, the ECCE evaluates 10 programmes in Europe and South Africa.

The ECCE was started by the European Chiropractors’ Union (ECU) in 1981, but has only included two students on General Council and one student on each accreditation evaluation team since 2011 [1‐4]. The ECCE is the only chiropractic specific accreditation body in the world that includes student members [2‐4]. The two students on General Council are nominated by the student bodies of the accredited institutions but serve only 1 or 2 years on Council prior to their graduation [5]. The student members of each accreditation evaluation team are selected by the ECCE’s Executive committee based on recommendations from the management of one of the accredited institutions not involved in the current evaluation. Student members of evaluation teams serve only once due to the fact they are normally final year students at the time of the evaluation event.

It has been generally accepted that students provide unique and positive perspectives to ECCE’s activities and thus are important stakeholders. However, no data have been collected to assess these assumptions based on experiences of ECCE General Council members, non-student accreditation evaluation team members, accredited institutions/programs and the involved students themselves. In particular, it seems important to assess perceptions of students’ usefulness and specific contributions, if any, to ECCE’s activities. There are real and logical benefits for the ECCE to understand the role and effectiveness of student members, not least because there are cost implications. Additionally, the ECCE needs to identify if there are areas of concern regarding the effectiveness of students on council and evaluation teams so that changes in ECCE policies and procedures can occur to remedy deficiencies. Therefore, the aim of this study was to collect feedback from four ECCE stakeholder groups regarding their perceptions and implications of including chiropractic students on the ECCE General Council and accreditation evaluation teams.

This was a mixed-methods audit using questionnaires which included both closed statements requesting a level of agreement, thus providing percentage agreement for each statement option, and open-ended questions/statements requesting written responses. The written responses were particularly important to identify specific issues that may be associated with those statements having a higher percentage of less positive responses.

Four similar questionnaires were designed, one for each of the four stakeholder groups (General Council, Accredited Institutions, Evaluation Team Members who were not students, and Student Evaluation Team Members) (Additional files 1, 2, 3 and 4). The ECCE Quality Assurance Consultant, who had been a member of ECCE for 8 years and who had also participated on several evaluation teams, created the first drafts of each questionnaire. The specific questions selected for inclusion arose from feedback questionnaires used by ECCE for past evaluation visits and modified for use by General Council. The questionnaires were then sent to the ECCE Quality Assurance committee for peer review and revision. Minor revisions of wording were agreed. Finally, the questionnaires were sent to the ECCE executive committee for input, revision and agreement of the final versions. No further recommendations for questionnaire revision arose. Two of the questionnaires (non-student Evaluation Team Member; Accredited Institution) included six closed statements, each requesting the respondents to identify their level of agreement to each statement using the five options: Strongly Agree; Agree; Neither Agree nor Disagree; Disagree; Strongly Disagree (Additional files 2 and 3). The General Council stakeholder group’s questionnaire and the Student Evaluation Team member questionnaire each had five closed statements using the same options (Additional files 1 and 4).

Each questionnaire also included two open-ended statements/questions requesting written responses. There was no limit to the length of any response. These four ‘Additional files’ show the complete questionnaires. Three of the questionnaires asked for ‘Positive comments about your experiences having students on Council or Evaluation Teams’ as well as ‘Areas for improvement needed based on your experiences having students on Council or Evaluation Teams.’ For the questionnaire completed by previous students on evaluation teams, the open-ended questions were: 1) Overall, the strong points of my experiences as a student member of the evaluation team were: 2) What unique contributions did you, as a student, make to the team and the evaluation process?

Questionnaires to General Council members were hard copies, completed individually and collected by the Quality Assurance consultant immediately prior to the start of the November 2018 Council meeting held in London, England. The first author of this study was in attendance and provided the instructions to all in attendance and monitored that no discussions occurred while completing the questionnaires. No identifying personal information was included on any of the questionnaires. Similarly, hard copy questionnaires from the Principals/Department Heads of nine of the ten accredited Institutions were completed independently prior to the start of the November 2018 meetings. One institution was not in attendance so that particular questionnaire was then sent electronically to the Department Head and returned via email. As a result, this particular questionnaire was not anonymous. Questionnaires to the non-student evaluation team members in attendance at the General Council meeting were also completed independently on hard copies and collected not only at the November 2018 Council meeting to those participants who had also been on previous evaluation teams but also sent electronically via email to prior evaluation team members not on ECCE General Council. All questionnaires to students were sent electronically and collected via email. Therefore, the responses collected electronically were not necessarily completely anonymous, but names were not added to the printed and saved questionnaires returned. For those failing to respond to the first email request, a second email reminder was sent. For all stakeholder groups, completion of the questionnaire was considered informed consent to participate in the study.

The response rates for the four different questionnaires were: 100% for both the Institutional feedback (10 responses) and Students on Evaluation Teams feedback (12 responses) questionnaires, 87.0% for the General Council members (20/23 responses) and 89.5% (17/19 responses) for the Evaluation Team members who were not students.

This survey was the first attempt to gather evidence to determine the value of student involvement in key positions on the ECCE Accreditation teams and General Council through evaluating perceptions of all of the key stakeholders. As such, the preponderance of evidence suggests that the stakeholders found value in the students’ participation. The data reveal a niche where the students excel, along with ideas for improvement. From primarily the written answers from non-student members of evaluation teams and responses to the closed questions, it was clear that student members of the evaluation teams are particularly useful in obtaining information from students at the institution being evaluated during the accreditation evaluation events. Not only did the student team members themselves recognize this, but the institutions and non-student evaluation team members pointed this out as well via their written responses to the open questions. Several respondents pointed out that student evaluation team members were more familiar with student issues and thus better able to formulate appropriate questions to obtain relevant information that non-student evaluation team members could not. Related to this, it was also identified that assigning student members of evaluation teams to explore very specific areas (i.e. ‘Standards’) for their questioning and report write-ups during the evaluation visit would be most effective.

Further, the written feedback from the former student members of evaluation teams was extensive and unanimously positive. Not only did they recognize the unique ways that they contributed to the evaluation visit through their rapport with students from the institution, many provided comments concerning how much they learned themselves from the experience. They also felt respected by the institution and other team members. One of their comments deserves particular attention:

“I had the opportunity to see how this kind of evaluation is made and what are the most important points every college needs to have in order to offer an education of quality. After this experience, I got more involved in the academic part of chiropractic and became a teaching assistant, which has been a great decision for my future as a chiropractor on so many levels. I was able to take advantage of this chance I was given and used it as a way to get out of my comfort zone and get to learn to be less introverted, talk in front of many people in stressful situations. I have always seen this experience as a game changer for me in a very helpful way in which I discovered that I was able to do many things I have never thought I could.”

However, the findings regarding the use of students on the ECCE General Council were not as positive as those for the evaluation teams. The most common problem was that these particular students were not prepared for the General Council meetings, were unsure of their role on council and thus unable to contribute as much as they could. This theme of being unprepared arose almost unanimously amongst the written comments. Following on from this, many council members stated that the student members of General Council needed intensive training prior to their meetings. The ECCE acknowledged that, unlike student members of accreditation evaluation teams, no specific training was offered to student members of General Council prior to their first meeting. In spite of this, most members of the ECCE General Council see the potential of student members and the unique contributions that they could make. Whether or not this difference between the effectiveness of students on the ECCE General Council compared to accreditation evaluation teams is due to the way specific students are selected for these roles is unknown and further research is indicated. Students on the evaluation teams are selected from the management of accredited institutions whereas students on General Council are selected by the student bodies of the accredited institutions. Students selected by the institutional managers for evaluation teams are likely to be the top academic students.

The other main problem with the students on the ECCE General Council, is their very short term of service (i.e. 1–2 years). Several written comments identified this issue. Currently, the students on General Council attend at most two meetings prior to their graduation. This is because students are not selected for General Council membership until they have completed at least 2 years of their chiropractic education so that they have some experience as a chiropractic student. They then graduate at the end of their 4th or 5th year, coming off of council at that time. This is reflected in theme #2 from the ECCE General Council feedback questionnaire on ‘areas for improvement needed’ i.e. “Value of experience: Expand the definition of ‘student’ to include those in recognized post-graduate training programmes or degrees so that they can remain on General Council for more than one or two years.”

In recognition of these two important issues of inadequate preparation and a short term of service arising from this study regarding the use of students on General Council, the ECCE made two important changes. The first is that the ECCE expanded the definition of ‘student’ to include those in recognized post-graduate programs or residencies. Thus, students can remain on General Council beyond 1 or 2 years. The second important change implemented by the ECCE is that the Vice President and the Quality Assurance Consultant have been assigned to create training materials and to conduct training workshops for new student members of council prior to their first meeting. This will address the key finding of this study of lack of preparation. Similar workshops are already in place for evaluation team members.

Although the responses to the three surveys focused solely on the use of students on accreditation evaluation teams, were very positive, one issue stands out where the ECCE can improve. This is to assign the student evaluation team member to specific ‘Standards’ during the accreditation visit and report write-up. This would particularly involve those ‘Standards’ most directly related to the student experiences. This does not mean however, that the student members of the evaluation teams could not be involved in questioning other areas during the evaluation visit.

The findings from this study, showing the specific advantages obtained by including students on chiropractic education accreditation evaluation teams and General Councils may be particularly interesting to the other international chiropractic accrediting bodies that do not currently include students as equal members of their evaluation teams or General Councils [2‐4, 7]. They may wish to reconsider their practices based on this evidence. Certainly medical, dental and chiropractic students are integral members on the Swiss Agency of Accreditation and Quality Assurance (AAQ) accreditation evaluation teams for medicine, dentistry and chiropractic in Switzerland, but this does not currently seem to be common practice internationally in medicine [8, 9].

The results from this study suggest both unique and positive contributions that chiropractic students make to ECCE accreditation evaluation teams. Although the results were slightly less positive concerning students on the ECCE General Council, this is likely due to the lack of specific training for their roles and the short time-frame of their current membership. Based on this study, the ECCE has made important changes to processes and procedures in order to maximize the benefits of student involvement. These include 1) the expansion of our definition of ‘student’ to include those in recognized post-graduate programmes or degrees and 2) creating training materials and workshops for new students prior to membership on council.