Dr Simon Abrams describes how to effectively audit out-of-hours staff using the urgent and emergency care toolkit
In March 2011, RCGP urgent care lead Dr Agnelo Fernandes published the Urgent and Emergency Care Toolkit. He and many colleagues had worked to produce a methodology for developing the effectiveness of patient-facing staff – both clinicians and non-clinicians in urgent and emergency care.
The toolkit enables services to get the best from their staff. As GPs, that means examining delivery of patient care and identifying areas for improvement and is complementary to appraisal and revalidation. This is a summary of how it works in one OOH service in the North West.
The toolkit uses fourteen criteria to assess the performance of clinicians and non-clinicians (Fig. 1). In the Liverpool and Knowsley OOH service auditors review voice recordings from call handlers, observe and interview receptionists and review voice recordings and paper based records from clinicians.
Performance is assessed against each criterion and scored as compliant, partially compliant or non-compliant. Several records per person are audited (four is usually enough). It soon becomes apparent where each person is performing well and where development is needed. Results are fed back to staff and the cycle repeated.
How we used the toolkit
Making it work in an organisation with 60,000 patient contacts a year, over 100 clinicians, 40 call handlers and 25 receptionists requires contributions from a team of part team staff, including a data analyst running a database, a lead clinical auditor, several sessional clinical auditors, a call handler auditor and two receptionist auditors.
But it is not the size of the team that makes the difference. It is the willingness of everyone to take part, to see the process through and pay attention to detail.
As much as it scrutinises the patient facing staff, the auditors must be up to the job. In the OOH service a lot of work goes into benchmarking the auditors. How different auditors’ score each criterion is assessed and reported and auditors are encouraged to discuss amongst themselves what they consider to be compliant, partially compliant or non-compliant. Results are fed back to staff.
Individual and benchmark reports are provided. They enable staff to see their strengths and development needs and to compare themselves against others. Where there are criteria that have scored low across the organisation, work is done to highlight the area and training provided.
For example, Safety Netting may be an issue. This is highlighted in broadcasts and through setting out examples of best practice.
On occasion auditors come across examples of practice that give cause for concern. If the example is low level then that staff member is sent a sheet setting out the details of the encounter and asked to reflect on their performance and feedback. This is a Minor Call to Reflect.
More serious concerns are addressed in a similar way but the clinician is invited to attend a session where the episode is reviewed using expert mentoring and a methodology such as Pendleton’s rules.Clearly, if an example of malpractice is found then further action must be taken.
Overall the system has worked well although it has not been easy to implement. It has enabled the service to evidence some really good practice and to highlight areas where improvement would enable patients to get better care.
Dr Simon Abrams is a GP in Everton and Medical Director of Urgent Health UK
Figure 1 – Toolkit Assessment Criteria Universal Urgent and Emergency Care Clinical Audit Tool
1 Elicits reason for call/visit
2 Identifies EMERGENCY or SERIOUS situations
3 Appropriate HISTORY taking (or algorithms use)
4 Carries out appropriate ASSESSMENT
5 Draws appropriate CONCLUSIONS
6 Makes appropriate MANAGEMENT decisions following assessment
7 Correctly fills in appropriate DOCUMENTATION
8 Appropriate PRESCRIBING behaviour
9 Displays adequate SAFETY–NETTING
10 Did the clinician address any potential SAFEGUARDING issues?
11 Makes appropriate use of IT/Protocols/Algorithms
12 Displays EMPOWERING behaviour
13 Develops RAPPORT
14 Satisfies ACCESS criteria where appropriate