A framework to improve quality of hospital-based physiotherapy: a design-based research study

Brainwriting is an idea generation technique in which participants write down their ideas about a particular question for a few minutes without talking. Then, each person passes his or her ideas to the next person who uses them as a trigger for adding or refining their own ideas [12]. We used the 6–3-5 brainwriting method. Each panel member was asked to individually write down 3 ideas about a method to improve the quality of hospital-based physiotherapy, based on the previously identified quality themes (Box 1) [6, 7]. After 5 minutes, each panel member was asked to pass their own form to another panellist so that 6 rounds of idea generation could take place. With each new round, participants were asked to involve or to build on previous panellists’ ideas. Because this was an online session, due to Covid restrictions, we used Padlet [16]. Padlet is an online environment to gather opinions or ideas. During the digital brainwriting sessions, the research question was always visible for the participants to ensure that all panellists worked towards the same goal. After the final round, each participant received their original form in return and was asked to individually identify the best ideas on this form in 10 minutes. These ideas were shared with the panel followed by a half-hour panel discussion, aiming for consensus on the ideas that were perceived sufficiently appropriate to proceed to the next part of the meeting, namely the decision matrix. The panel discussion was video recorded for analysis purposes.

To decide which of the remaining ideas from the first part would be the most suitable, we placed each idea in a decision matrix against a set of decision criteria. For this purpose, the panel was first asked to generate ideas for decision criteria, and then to decide by total consensus which of these criteria should be used. After consensus was reached, the matrix form was filled with ideas and criteria, and each panel member was given half an hour to individually test each idea against each criterion. This was done both quantitatively (providing scores on a Likert scale of 1 (very inappropriate) to 5 (very appropriate)) and qualitatively (by writing comments in text boxes). Finally, all panellists sent their form to the moderators and explained their ideas what the design of a framework should look like to the panel. This marked the end of the panel session. All panel discussions were video recorded for analysis purposes.

Quantitative data from the decision matrix were analysed and described using Microsoft Excel. Written qualitative data from the decision matrix were collected and added as comments to the scores. These comments were checked by both moderators against the various video recordings and supplemented if they highlighted new perspectives. This resulted in a final decision matrix. The research team developed a QI framework design by discussing the outcomes of this final decision matrix. The video recordings were also used to check afterwards whether all procedures during the panel session had been conducted correctly.

During the study, we were aware of our positions and maintained a reflexive approach from our perspectives as experienced hospital-based physiotherapist and researcher [RS], as a teacher of physiotherapy and experienced researcher [MM] and as (associate) professors in allied health and medical care and experienced researchers [TH,PB,PW]. None of the authors worked as a hospital physiotherapist in any of the hospitals involved or maintained personal contacts with any of the panellists. We tried to obtain balanced data by having RS conduct the panel, supported by MM. To encourage trustworthiness, a member check of the final decision matrix with all participants was carried out.

The median scores of all the criteria per idea ranged from 3.0 (360-degree feedback) to 4.5 (management information system) (Table 4). The median scores of the other five ideas was 4.0. On three of four criteria, the ‘management information system’ idea received highest scores. Feedback on Patient Reported Experience Measures (PREMs) and Patient Reported Outcome Measures (PROMs), a quality portfolio, and intervision & intercollegiate evaluation scored highest on two of four criteria. Continuing education and peer observation and feedback scored the highest on the criterion of acceptability. The idea of 360-degree feedback was not among the highest scores on any criterion.

Participants discussed potential advantages and disadvantages of the proposed QI methods, which are summarised in Table 3, together with their objectives and construction:

The effects and feasibility of each QI method mentioned by the panel have been described previously in the literature. Overall, these studies suggest positive effects and reasonable feasibility, but also make reservations about each method ranging from the degree of effect, reliability, and validity to efforts with and conditions under which application could be successfull [17‐28]. The results of these studies suggest that, when designing a QI framework for hospital-based physiotherapy, a mixture of these methods may be most appropriate. This allows evening out of advantages and disadvantages of each individual method, because they cover different aspects of professional quality. The result may be a combination of methods that together meet the predefined QI criteria and build a valid and effective framework to improve the quality of hospital-based physiotherapy.

More rigorous research is needed to identify effective and generalizable interventions individually, but also in combination as a multiple method assessment, to improve healthcare quality [29, 30]. This may lead to a more multidimensional approach to quality [31, 32].

The prevailing method of the Individual Quality Register of Physiotherapy of the KNGF in primary care is individually based, where each activity aimed at professional development is rewarded with points [33]. For hospital-based physiotherapy, an integrated approach based on a portfolio of activities would be more appropriate. This is in view of the nature of the work of hospital-based physiotherapists, which can be more short-cycled, more acute, more varied and more multidisciplinary than in primary care. Especially in a healthcare environment that promotes the collaboration of administrators and physicians in ensuring the quality of patient care [34], a multidimensional model also offers advantages in terms of a more flexible applicability to different disciplines pursuing the same quality goal. Also, because hospital-based physiotherapy is bound to other regulations than in primary care, this flexibility of a multidimensional model offers more options for QI. Especially if techniques that are already used in the hospital world, such as the tracer method with peer observation and feedback, are used [25].

The results of this study, summarised in the design of a framework for quality of hospital-based physiotherapy (fig. 1), provides a foundation to develop a quality system for hospital-based physiotherapy. A quality system comprises a management system and a technical system (methods for IQ). Here, the individual professional manages his own quality efforts in a personal portfolio, which is fed by four types of quality improvement methods. These methods each highlight a different aspect of quality so that a total package is created that fits the described nature of work of hospital-based physiotherapy. The management information system concerns all activities in the field of planning, decision-making, organisation, control, evaluation, motivation, training, and involvement of employees to guarantee and improve quality [35]. Within this management information system, quality indicators found in previous research [6, 7] could be implemented.

The composition of a representative panel for hospital-based physiotherapy enables a balanced answer to our research question. Using the principles of design-based research is another strength, as design-based research studies can play an important role in the advancement of theory and practice in designing or redesigning work-based learning environments and assessment programs [14]. Although exact data on its validity and reliability are still lacking, the method of brainwriting has been presented as a novel and efficient alternative to brainstorming that can rapidly inform program implementation at minimal time and cost [36‐38].

We acknowledge the following limitations. Although a design-based research panel can produce collective answers, the achieved consensus is not necessarily accurate; bias can occur in the meeting because one individual’s opinion can be overrepresented. Since the panel meeting was not anonymous, respondents may have felt restrained to speak freely, and may have been subject to social desirability bias, especially considering the high scores that were given to the QI methods. Although the panel represented all key stakeholder groups, there was only one representative for each group in the panel, which may have produced selection bias. Also, gender was not considered in the composition of the panel and thus the formation may not have been sufficiently ‘inclusive’, looking at the diversity of interests and perspectives.

A key limitation is the extent to which the results of this design-based research can be generalised or transferred to other contexts. Seen from the perspective of our design and analysis, we think that extrapolation of our results to the Dutch situation of hospital-based physiotherapy is feasible. From an international perspective, this is more complex because the forces within the health care system differ per country, and the positioning of hospital-based physiotherapy can be quite divergent.

In the search for the right mix of the various QI methods, further studies should investigate what this could look like in terms of impact and feasibility. Within the framework of hospital-based physiotherapy, the QI methods discussed can be further explored, either individually or in certain combinations. If a suitable combination seems to have been achieved, which feeds into a management information system on QI of hospital-based physiotherapy, a follow-up study can be conducted to examine its feasibility and total effect on quality. The main question then is how to measure this quality, and with which quality indicators.