Background
Methods
Analytical framework
Results
Essential factors for successful implementation of an audit system
Drivers and teams
Outreach and supervision
"It is important to have an ' over-see-er ' at facility level and DOH [Department of Health] Outreach person provides support and training" [DS1-S1-GC]
Perinatal review meetings
-
Daily or weekly meetings in the ward, unit or department where deaths were analysed as they occurred.
-
Monthly perinatal review meetings at hospital level were very important and provided a means of encouraging continuous data collection - "Keep on with perinatal meeting → keep PPIP on to collect the data needed" [DS3-S1-G3].
-
Annual meetings held at regional, provincial or national level to report on aggregated death rates and causes of death - "Yearly PPIP meetings - there is an aim in the end" [DS1-S2-GA].
Communication and networking
"There was support from regional office ... It is very important to network in level 1 and level 2 facilities. Working together and communicate and support each other. Networking, communication, buy-in: management - from the top, then downwards." [DS1-S1-GA]
Stages in the implementation and maintenance of the audit programme
Pre-implementation phase
Stage 1 - Create awareness
"In 1999 we just started collecting data. ... I felt there was something wrong. We needed to get a handle on it. Why are babies dying? We didn't get far ... Then we were invited to a provincial PPIP meeting in 2002 or 2003. We got the software ourselves and we were excited about the tool ... Spreadsheets helped. PPIP really helped to give some analysis." [DS1-S1-GA]
"Thirteen years ago, Prof[essor] A came and told us, 'You and Hospital X must start it'. We heard what it was all about and we went with it." [DS1-S1-GC]
"We phoned all the hospitals and said, 'We have to start it [PPIP].' I offered to come to train everyone on the programme and the hospital manager decided in the labour or paeds ward who will carry on with the audits after training." [DS1-S1-GC]
Stage 2 - Commit to implementing audit
"What was very important was the buy-in from everyone, to inform everyone what it [PPIP] was about. We informed our [regional] director and got his buy-in. Then [you] inform management at each place. Also, to inform management in each sub-district and they must identify a responsible person to do PPIP, et cetera." [DS1-S1-GA]
Implementation phase
Stage 3 - Prepare to implement an audit programme
Stage 4 - Implement audit
Institutionalisation phase
Stage 5 - Integrate audit into routine practice
"It's critically important that people don't hide behind a busy schedule to avoid doing audit. It's not an extra; it's as crucial as filling in your clinical notes. If you can get people to run with the idea that it's integral, you don't need to beat them to do it." [DS1-S2-GC]
Stage 6 - Sustain audit
Discussion
STAGE | QUESTIONS TO CONSIDER |
---|---|
Stage 1:
Create awareness
• of problem (e.g. high PNMR) • that something must be done • of audit programmes that could be used | • What is the level of awareness of staff about the problem? • How can we create or improve a general awareness of the importance of audit among staff? • Do we have review meetings (e.g. perinatal review meetings) on a regular basis? ○ If no, how could we organise and institute these meetings? ○ If yes, how could we use the existing meetings to create better awareness of the importance of audit? • Should a specific person at our facility be designated to find out about the programme (e.g. PPIP)? • Could a representative of an institution using the audit programme or the owners of the programme (e.g. PPIP) be invited to come and tell staff about it? • Are other facilities in our area/district using audit programmes? ○ If no, what role can our facility play in reaching out to create awareness of the problem and the importance of audit? |
Stage 2:
Commit to implement audit
• More people commit to implement programme | • Can existing review meetings be used to facilitate commitment from more people? ○ How could we use this? ○ What needs to be done? • Can existing communication channels be used to facilitate commitment from more people? ○ How could we use this? ○ What needs to be done? • Are there specific persons whose commitment is required who need to be approached? ○ People at management level? Who? How? ○ IT officials? Who? How? ○ Clinicians? Who? How? ○ Government officials? Who? How? ○ Other? Who? How? • What steps are needed to get the use of the programme approved in principle? • What kinds of commitment are needed? (E.g. Who? What? Where? When? How?) ○ Written? ○ Verbal? |
Stage 3:
Prepare to implement
• Practical aspects to get the programme up and running | • What do we expect of the driver(s) of the process? • How will we identify the driver(s)? Who should be considered? • What are the financial outlays for getting the programmes and necessary equipment? ○ Must the audit programme be purchased or is it free? ○ What will orientation and training in the use of the programme cost? ○ Do we have a dedicated computer for capturing audit data? - Should more computers be made available? • How do we get budget approval for the purchase of the programme and other equipment (if applicable)? • What kind of support do we have from senior management on the practical aspects of the audit implementation process? ○ In what areas do we need additional support? ○ What else should we do to secure the necessary support? • Has the software for the audit programme been installed? • Who will be responsible for data entry? ○ Has this duty been negotiated with the assigned data capturer(s)? - How well have they bought into the idea? ○ What kind of computer training do they need? ○ What kind of training do they need in the use of the software? - How will that be organised? ○ What kind of support or incentives should be provided to the data capturers to make them aware of the value of their work? • What kind of quality control will be exercised on the captured data? • Who has been trained in interpreting the data and generating reports? ○ If no one, what should be done about it? • Are all the responsible clinicians trained in the use of the computer hardware? • Are all the responsible clinicians trained in the use of the software? • How are facility information officers or IT officials involved in the establishment of the system? ○ How informed are they? ○ What kind of support will they be able to provide? • What kind of support do we need from our outreach clinicians and other health department officials in our preparation for implementation? ○ How could these person(s) assist us in keeping the process on track or speeding it up? |
Stage 4:
Implement audit
• Collection of the first data • Analysis of the results • Dissemination of results | • Have we started collecting data? • How do we keep back-ups of data files? ○ Who is responsible for making back-ups? ○ How often are back-ups made? ○ How do we check that this is done regularly? • Are all persons sharing the computer(s) aware of the programme? (To avoid unintentional deleting of files or programmes) • How can accuracy and regular data entry be improved? ○ Are data always entered immediately? ○ Should refresher sessions on the use of the software be provided? • How is/are the driver(s) of the implementation process coping with the tasks? ○ What kind of additional support do they need? • How is feedback of results given at the review meetings? ○ How can we improve on this feedback? • How good is the attendance of review meetings? ○ How can it be improved? ○ Are attendance registers kept for review meetings? - If yes, how does it help us in what we are doing? - How do we use the information to ensure that all relevant role-players are invited regularly and/or have the opportunity to attend? • Is the usefulness of the audit programme (e.g. PPIP) evident at meetings? ○ If yes, how? ○ How can this be improved? • Are minutes taken at review meetings? ○ If no, how will we get this process going? ○ If yes, how can we improve on what we are doing? - How are the minutes used? - How are recommendations noted in the minutes? - Are timelines and the allocation of specific persons included in the recommendations? • How are the recommendations incorporated into in-service training? • How does the use of the results contribute to accountable behaviour of all relevant role-players? • What kind of support do we need from our outreach clinicians and other health department officials in the implementation process? ○ How could these person(s) assist us in keeping the process on track or speeding it up? • How can communication at different levels be improved? ○ Between providers at the coal face? ○ Between providers and their immediate managers? ○ Between managers and senior management? ○ Between the facility and the other levels of the health system? ○ Other? |
Stage 5:
Integrate audit into routine practice
• Data collection • Analysis of the results • Dissemination of results • Use of findings ↓ Routine practice | • Are audit duties written into the job descriptions of relevant staff members? • Is feedback on audit results regularly provided to all relevant service providers in the institution? ○ How can we improve on this? • Is feedback on audit results regularly provided to senior management in the institution? ○ How can we improve on this? • Is feedback on audit results regularly provided to higher levels in the health system (e.g. district, regional or provincial managers)? ○ How can we improve on this? • How are the audit data interpreted? ○ How can we improve on this? • What kinds of recommendations and improvements are suggested following the interpretation of data? ○ How can we improve this? • How are we faring with the execution of the recommendations? ○ What are we doing well? ○ How can we improve on the things we are not doing so well? • Does our facility need (more) visits from an outreach person or team to assist us with the institutionalisation of the audit programme? |
Stage 6:
Sustain audit
• Data collection • Analysis of the results • Dissemination of results • Use of findings ↓ Sustained over a longer period of time | • How can the regular review meetings be improved and used more effectively? • How often and to whom is feedback given? ○ Monthly? To whom? ○ Quarterly? To whom? ○ Six-monthly? To whom? ○ Annually? To whom? • What are the gaps in our feedback procedures? • How can the feedback to service providers and senior management in the facility be improved? ○ How can engagement in the audit process, the use of the findings and the application of recommendations be improved? • How can the feedback to the higher levels of reporting be improved (e.g. district or provincial levels)? ○ How can involvement from these levels be improved? • Who is responsible for keeping the audit system together? ○ One person? ○ A team? • Who is leading the audit? ○ Who takes responsibility when the leader(s) is/are not there? ○ What kind of succession plan do we have? • How do staffing issues such as rotations and turnovers influence the audit activities? ○ How can staff stability be improved? • What is our facility's responsibility in reaching out to another facility or facilities to introduce and establish an audit programme (e.g. PPIP)? |