Introduction

 

BrisDoc is committed to providing clinical care and supporting processes to an exemplary standard that ensures patients, their carers and families, and other professionals receive high quality care and support. In order to assure itself that staff are meeting the expected standards, BrisDoc will routinely audit a random sample of all contacts with patients and professionals, across all of its services, whether the contact is over the telephone or in person. Specific audits will also be undertaken to assure BrisDoc of the consistency of standards of care and practice across a service, and that a Service can work effectively and safely within the local healthcare community. Audit will provide evidence that areas of poor practice, and poor systems and processes are recognised and acted upon thereby ensuring BrisDoc continues to provide high quality patient care is accordance with its business model.

BrisDoc is committed to:

• Regularly auditing a random sample of patient and professional contacts,
• Acting appropriately on the results of those audits,
• Ensuring auditors have appropriate experience and support to undertake the role,
• Reporting results to the Urgent Care Service & Quality Board, Practice Services Governance Board, and to the Directors through the corporate dashboard,
• Undertaking an annual programme of clinical and non-clinical audits, as agreed and managed by the above Boards, that assess the outcomes of the BrisDoc services; and inform learning, development, and new/improved processes.

BrisDoc recognises that clinicians outside of surgery based general practice or urgent care settings, and clinicians working a large number of hours outside of normal opening hours are, generally, in need of a higher level of audit than clinicians within surgery based general practice and urgent care settings. In Practice Services it is felt essential to ensure quality and consistency throughout each practice’s service. To ensure targets for funding are met and high standards of patient care achieved, it is essential that consultations are recorded in a manner that records information in a certain way and is reactive to the holistic needs of the patient. With this in mind it is essential that practice EMIS records are audited to ensure consistency and completeness is maintained.

BrisDoc also recognises the need to provide a higher level of monitoring to clinicians who are unknown and therefore untested.

BrisDoc aims that any telephone call is conducted with the upmost professionalism and customer care whilst maintaining the robust process that gains the correct information from or about the patient; provides appropriate, relevant and correct advice, treatment or care; and ensures the caller knows what the next steps should be.

BrisDoc has adopted a strengthened process to provide assurance of robust audit processes and practice. Experience has shown that a small minority of staff generate the great majority of complaints and concerns. More audit resource is therefore dedicated to this small number of staff and to those new to the service

The Process

Severnside Integrated Urgent Care (IUC) Clinical Case Review

Every clinician working in IUC (Nurse, ECP, GP, Pharmacist), whether it is for the Weekday Professional Line (WDPL), Clinical Assessment Service (CAS) or seeing patients face to face (F2F), is regularly audited by a member of the CAS/F2F Clinical Guardian Team (GT) or WDPL auditing team.

All clinical data is extracted from Adastra for patients who had contact with the IUC WDPL, CAS or F2F services and is uploaded onto “Clinical Guardian”, BrisDoc’s secure on-line governance software (appendix 1). The software reconstructs the data into anonymised patient records which can then be randomly selected according to audit criteria for peer review. The Clinical Guardian auditing process is set out in appendix 2. The Royal College of General Practitioners (RCGP) toolkit criteria provide the underpinning standards for audit.

The auditors look specifically for unsafe or worrying patterns of clinical behaviour which may indicate a larger sample size of that clinician’s cases needs to be audited. The Clinical Guardian system, which can be accessed online by each clinician working in the CAS/ F2F and WDPL, provides them with a rolling record of their clinical performance.

The audit process is managed in a slightly different way for the CAS/ F2F services and the WDPL service, reflecting the differing nature of the work and clinical team.

CAS/ F2F Clinical Guardian Team

Individual Auditor Review

CAS/F2F case records are reviewed by the CAS/F2F Clinical Guardian Team. The team incorporates at least two GP auditors, and one of the Nursing/ AHP Team managers with line management and support from the CAS/F2F Lead GP and Urgent Care Deputy Medical Director. The CAS/F2F GT clinicians work regularly in the CAS/F2F setting and have each done so for a minimum of 12 months prior to joining the team. They have an interest and experience in education and leadership, and continue to develop these skills during their work in the team.

The auditors assess each case as “pass without comment”, “pass with comment” (and provide free text positive comments, or logistical feedback), or “refer for group review” (and makes a note of the reason for referral to aid group discussion at the next CAS/F2FGT meeting). When undertaking individual audit, the auditor cannot see the identity of the consulting clinician (unless the clinician has special status, see below). Cases where there are concerns, or learning for the individual or service, are forwarded for ‘group review’ to discuss the case with the wider team and agree feedback/ actions.

CAS/ F2F Clinical Guardian Team Review

The CAS/F2F GT meets fortnightly and requires a quorum of at least two out of three members to participate. The whole team meets approximately once a month to discuss processes, review performance and workload and share views and tips from the preceding weeks of audit. Wider clinical issues and audits may also be discussed.

Cases forwarded at individual audit for “group review” are discussed at the meeting. Again, cases reviewed are assessed as “pass without comment”, “pass with group comment” (a free text comment which may highlight learning or constructive feedback, or particularly positive feedback) or “as discussed by phone or email” (if the case requires the CAS/F2FGT to discuss the case with the clinician, either by phone or email prior to the case score being submitted).

Feedback to individual clinicians and any status change if required is agreed collectively by the Team. All clinicians working in the CAS/F2F services are alerted to feedback by a monthly email from the CG platform, and are encouraged to review their feedback and reply to the team if they wish to.

IUC clinicians interested in the work of the CGT may shadow a meeting having undertaken some individual audits prior to the meeting they attend. For this purpose they will be given auditor access to Clinical Guardian by the Governance Team, and been briefed about confidentiality by the Lead GP and signed a confidentiality agreement when they attend.

CG colour status of CAS/F2F clinicians determines audit requirement (green, purple, yellow, amber or red)

Green clinicians are established clinicians working in the service. 5% of their cases are audited, or a minimum of one case per shift. These cases are anonymised at the individual audit stage. Most clinicians are ‘green’, and remain ‘green’, following the initial audit undertaken when they start working in the CAS/ F2F service.

Purple clinicians are brand new to working in the CAS/F2F services, and have 100% of their first 15-20 case records audited and then one to two telephone recordings listened to (two if there are any concerns during audit or from the first telephone call). Purple clinicians are changed to green when they have had at 15-20 cases audited satisfactorily, and there are no persistent or ongoing concerns/ issues about their work. If the clinician has worked for BrisDoc before but not worked for 6 months or more, their status defaults to purple. If there have not been previous concerns and no new concerns are identified, they may be changed to green after 10-15 cases.

If there are ongoing concerns/issues after 20 cases then clinicians will remain as purple (or perhaps be changed to yellow where 15% of their cases will be put through for audit) until these concerns have been addressed through feedback via email, telephone call or in person by the ‘line manager’ (nurses and AHPs, and employed GPs) or a Clinical Guardian team member (self-employed GPs).

At or around the time of transition of new, purple clinicians to green, the Clinical Guardian team will reach out to speak with the new clinician to share any overall feedback from the initial audit. This is also an important opportunity to seek and hear new clinicians’ feedback about induction, working in the service, Clinical Guardian and any other suggestions they may have.

There are three other clinician categories – yellow, amber and red. Yellow is used to support individual/bespoke audit requirements when more than 25% audit is not required, but the cases need to be identifiable or a higher rate of audit than the 5% green audit is required.

Yellow clinicians stay yellow for a maximum of three months, at which point review and next steps need to be agreed. This would usually include changing to amber, discussion with the Deputy Medical Director and/or Medical Director, and the Head of Nursing/AHPs for a nurse or AHP member of the team. It could also include consideration of review at the BrisDoc clinician Performance Advisory Group (PAG), which is chaired by the Medical Director. Amber cases audited can be selected in accordance with the issues raised on a range of 25% to 100% of cases. The percentage of cases can be changed for amber however, note that all clinicians on amber will have the same percentage of cases put through for audit.

Clinicians who are red on the system have been deemed unsuitable/unsafe to work for BrisDoc and therefore should not have shifts booked. They remain on the Guardian list so that if they were booked in error this would be evident promptly on the audit framework and appropriate steps taken to rectify this. Changing a clinician to red would always require Medical Director/Deputy Medical Director involvement, may require discussion at the PAG or review of the decision at PAG.

Listening to CAS telephone advice calls

The CAS/F2F Clinical Guardian team will routinely listen to one to three randomly selected calls when a new clinician starts work, as part of transitioning from purple to green status and or if there are concerns/ issues about a new clinician. Calls are scored using the modified RCGP telephone triage tool, which has been tailored to reflect the nature of the consultations undertaken in the CAS/F2F setting. It may not be possible to score some of the criteria in any one call, because of the nature of the clinical consultation so more than one call is listened to. Scoring is logged in the spreadsheet and feedback about the telephone conversations can be incorporated into the discussion between a new clinician and the Guardian auditor if needed.

In addition, the CAS/F2F Guardian team have the option to listen to calls on an ad hoc basis if, for example, it is very unclear what has happened from the written record, new concerns/ issues are emerging for a clinician or as part of wider review of an individual clinician triggered by an external issue.

WDPL Clinical Guardian audit

WDPL clinicians’ main workload is related to WDPL cases, but they also provide the GP input into a subset of CAS cases which come through to the team following an NHS Pathways assessment. These CAS cases are audited in line with the CAS/F2F audit processes outlined in section 2.1.1, and this section summarises the audit process for WDPL primary care referral cases only. The CAS/F2F Guardian Team and the WDPL Clinical Lead liaise if needed about clinicians’ audit issues arising for a clinician working across both services.

Individual Auditor Review

The WDPL Team will work collaboratively to peer review each other’s anonymised cases weekly under the supervision of the Clinical Lead GP.

• The content of the case will be specifically considered for whether what is documented is clear and inclusive of all relevant clinical information:
• Presenting complaint and clinician concerns
• Treatment/investigations of presenting problem
• Relevant PMHx/DHx/social Hx
• Have alternative pathways been considered and discussed?
• Has the patient been referred to the appropriate pathway/ward?
• Were potential infection control issues documented?
• Rockwood score/clinical frailty score/NEWS score
• Were any safeguarding issues considered?

Each case is assessed as “pass without comment”, “pass with comment” (and provide free text positive comments, share knowledge, or logistical feedback including what happened to the patient), or “refer for group review” (and makes a note of the reason for referral to aid the Clinical Lead GP review). Cases where there are concerns, or learning for the individual or service, are forwarded for ‘group review’.

Auditors may access ICE/EMIS to find out what happened to the patient in order to inform their feedback to the clinician and enhance the clinician’s learning.

WDPL Clinical Lead GP review

WDPL group review will be by the Clinical Lead GP and/or the Urgent Care Deputy Medical Director (in conjunction with the GT if necessary) where wider discussion will take place and feedback, actions agreed.

Cases/trends/themes of concern will be reported to the Clinical Lead for review and further discussion with the CGT.

CG colour status of WDPL clinicians determines audit requirement (green, purple, yellow, amber or red)

Green clinicians are established clinicians working in the service. 5% of their cases are audited, or a minimum of one case per week. These cases are anonymised at the individual audit stage. Most clinicians are ‘green’, and remain ‘green’, following the initial audit undertaken when they start working in the WDPL service.

Purple clinicians are brand new to working in the WDPL service, and have 100% of their first 15-20 case records audited and then one to two telephone recordings listened to (two if there are any concerns during audit or from the first telephone call). Purple clinicians are changed to green when they have had at 15-20 cases audited satisfactorily, and there are no persistent or ongoing concerns/ issues about their work.

If there are ongoing concerns/issues after 20 cases then the clinician will remain as purple (or perhaps be changed to yellow where 15% of their cases will be put through for audit) until these concerns have been addressed through feedback via email, telephone call or in person by the Clinical Lead GP.

There are three other clinician categories – yellow, amber and red. Yellow is used to support individual/bespoke audit requirements when more than 25% audit is not required, but the cases need to be identifiable or a higher rate of audit than the 5% green audit is required. An example would be a clinician who is returning from a period away from the service (e.g. maternity leave).

Yellow clinicians stay amber for a maximum of three months, at which point review and next steps need to be agreed. This would usually include changing to amber, discussion with the Deputy Medical Director and/or Medical Director, and the Head of Nursing/AHPs for a nurse or AHP member of the team. It could also include consideration of review at the BrisDoc clinician Performance Advisory Group (PAG), which is chaired by the Medical Director. Amber cases audited can be selected in accordance with the issues raised on a range of 25% to 100% of cases. The percentage of cases can be changed for amber however, note that all clinicians on amber will have the same percentage of cases put through for audit.

Clinicians who are red on the system have been deemed unsuitable/unsafe to work for BrisDoc and therefore should not have shifts booked. They remain on the Guardian list so that if they were booked in error this would be evident promptly on the audit framework and appropriate steps taken to rectify this. Changing a clinician to red would always require Medical Director/Deputy Medical Director involvement, may require discussion at the PAG or review of the decision at PAG.

Listening to WDPL telephone advice calls

The WDPL team members will routinely listen to one to three randomly selected calls when a new clinician starts work, as part of transitioning from purple to green status and or if there are concerns/ issues about a new clinician. Calls are scored using the modified RCGP telephone triage tool, which has been tailored to reflect the nature of the consultations undertaken in the WDPL setting. It may not be possible to score some of the criteria in any one call, because of the nature of the clinical consultation so more than one call is listened to. Scoring is logged in the spreadsheet and feedback about the telephone conversations can be incorporated into the discussion between a new clinician and the Guardian auditor if needed.

In addition, the Clinical Lead GP has the option to listen to calls on an ad hoc basis if, for example, it is very unclear what has happened from the written record, new concerns/ issues are emerging for a clinician or as part of wider review of an individual clinician triggered by an external issue. Otherwise calls are only routinely audited as part of PDR (section 2.3.5).

ED Validation Cases

 

Practice Services Clinical Case Review

Audit Tool

Broadmead Medical Centre (BMC), Walk-in-Centre (WIC) and Charlotte Keel Medical Practice (CKMP) cases are audited against a modified version of the RCGP criteria (appendix 3) that include:

• Appropriate history taking
• Carries out appropriate assessment
• Draws appropriate conclusions
• Displays empowering behaviour
• Makes appropriate management decisions
• Appropriate prescribing behaviour
• Displays adequate safety netting
• Addresses potential safeguarding concerns
• Makes appropriate use of IT/templates/READ codes

Practice Nurses and Pharmacists

New nurses and pharmacists will have an anonymised random sample of three consultations reviewed by the Lead Nurse/Deputy/Nurse Team Manager each quarter for one year. Consultations will be audited against the modified RCGP criteria, scored from 0-2 based on evidence and completeness in the case record, and rated green, yellow, amber or red based on percentage scored out of a maximum of 18. The results will be collated in a spreadsheet by the Auditor. Audit results will be fed back to individual nurses as consultations are reviewed/ in 1:1 meetings and the collated results at the time of PDR.

Agency nurses will have a minimum of three consultations audited each month of their assignment which will be managed in accordance with the process in the above paragraph.

After one year of employment routine peer review will be by exception or following adverse findings in topic audits/complaints/incidents/appraisal.

At least two topic audits (in addition to any QOF/DES/PCN/CCG requested audit) will be undertaken each year through which the quality of the consultation will be assessed as per the modified RCGP audit criteria.

Doctors

New GPs will have an anonymised random sample of three consultations reviewed by the Lead GP/delegated deputy each quarter for one year. Consultations will be audited against the RCGP criteria, scored from 0-2 based on evidence and completeness in the case record, and rated green, yellow, amber or red based on percentage scored out of a maximum of 18. The results will be collated in a spreadsheet by the Auditor. Audit results will be fed back to individual GPs as consultations are reviewed/ in 1:1 meetings and the collated results at the time of PDR.

Locum GPs will have a minimum of three consultations audited each month of their assignment which will be managed in accordance with the process in the above paragraph.

After one year of employment routine peer review will be by exception for performance of concern or following adverse findings in topic audits/complaints/incidents/appraisal.

At least two topic audits (in addition to any QOF/DES/PCN/CCG requested audit) will be undertaken each year through which the quality of the consultation will be assessed as per the modified RCGP audit criteria.

Audit reporting

The findings and reports from topic audits will be shared at a regular team meeting. This will provide opportunity to feedback about the general quality of consultations in relation to the RCGP criteria and areas needing improvement.

The practice manager will support the process and provide an impartial view if necessary.

Telephone Call Review

Telephone calls audited will include the following:

Operational

• Those to the IUC Professional Line from GPs, Paramedics, District Nurses, Nursing Home Nurses, Pathology etc. and received by a Call Handler
• Those to and from the WACC line used typically by NHS 111, to offer comfort calls to patients, to make onward referrals to District Nursing Services
• Those to the Patient Line from patients/family who are deteriorating and received by a Call Handler
• BMC/CKMP calls including health navigation, liaison with patients and other organisations.
• Clinician
• Those from any BrisDoc line used by an IUC clinician with a patient or professional colleague to provide clinician advice, triage. A review of these calls may be requested by the Clinical Guardian Team as a consequence of a case record review.
• BMC/CKMP triage calls and telephone consultations.

Operational Staff

Professional Line

The Professional Line enables community based clinicians to access the BrisDoc services 24 hours per day 7 days per week. These calls are answered by operational Call Handlers.

The service operates from Osprey Court during the weekday in hours period (08:00 to 18:30 Monday to Friday) and at Osprey Court/Nicholson House/Knowle West Health Park (Control Rooms) at weekends/bank holidays and between 18:30 – 08:00hrs Monday to Sunday.

It is important that the Professional Line delivers a consistent and accurate service to the clinicians calling to seek advice or to transfer patients to the IUC service.

Clinical Assessment Service

Workflow and Capacity Coordinators (WACCs) and Call Handlers are principally responsible for answering the Professional Line from 18:30 – 08:00hrs Monday to Friday, weekends and Bank holidays with support from the Knowle based Hosts and Shift Managers. They also answer the WACC number used by NHS 111.

Patient Line

The patient line number is provided to patients/family where the clinician has concerns that they may deteriorate/their condition change. This saves the patient having to contact the IUC service back via NHS 111. Typically the number may be given to palliative patients, those waiting a home visit.

BMC/CKMP

All calls in and out of BMC and CKMP are recorded enabling review in relation to complaint and incident investigations, call handler (Receptionists) audits, health navigation advice given.

2.3.2 IUC Clinicians

The Clinical Coordinator and Professional Line GP, based in the CAS, have a dedicated telephone line otherwise clinicians use the telephone in their consulting room/work station.

IUC Call Standards

IUC Call Handlers are expected to collect information accurately and quickly and to ensure that the call is forwarded to the correct clinical queue.

Audit data will include that the correct questions are asked, the case is put to the correct queue / appointment and that the Call Handler is polite and professional and demonstrates confidence and knowledge.

The aim is to answer every call with the upmost professionalism and customer care whilst maintaining the set process that is in place for gaining the correct patient details and the most useful information to be handed on for use by the clinician.

Clinicians are also expected to manage telephone calls with the utmost professionalism and customer care. They need to ensure that the advice, instructions and guidance they give are clearly heard and understood by the patient / carer / colleague they are speaking to. Calls will be audited against RCGP standards relevant for an advice consultation.

All BrisDoc staff are expected to announce their name, profession and service when speaking to a colleague or a patient/family.

Practice Services Call Standards

The aim is to answer every call with the upmost professionalism and customer care. All staff need to ensure that the advice, instructions and guidance they give are clearly heard and understood by the patient / carer / colleague they are speaking to.

Administration staff calls will be audited quarterly see appendix 4 tool.

Clinician calls will be audited against modified RCGP standards relevant for a triage consultation.

All BrisDoc staff are expected to announce their name, profession and service when speaking to a colleague or a patient/family.

Call Selection

IUC Operational Staff

The calls will be selected at random using the Adastra report XXXXX. The number of calls to be reviewed should be 5% of the total Professional and WACC Line calls, with a minimum of two calls per person audited every month. The template in appendix 4 will be used for the audit.

The calls will be downloaded using the Voice Recording Standard Operating Procedure. Once calls have been audited they must be deleted from the relevant shared drive.

Individual Review

Every call handler is routinely audited monthly.  The Line Manager will review a minimum of 2 calls per Call handler as an initial audit, the volume of calls for further review will depend on the results of this initial audit.

IUC Clinicians

Two calls will be selected randomly from each new IUC clinician’s first 25 advice calls to be audited in order for them to “go green”. For the WDPL team calls will be routinely audited as part of the PDR process.

The clinician call audit tool is set out in appendix 5 (note there is a slightly modified version of this tool for the WDPL team to reflect the acute admission calls are not with a patient).

Control Processes

 

Control of the process and a database of the performance of individuals who handle calls is managed by an IUC Team Manager. All incidents and complaints regarding call handlers are brought into this audit.

Control of the process and a database of the performance of individual clinicians is managed by the IUC Deputy Medical Director or Practice Lead GP/Nurse supported by the Governance Team or Practice Manager. In Practice Services a spreadsheet is used to store all data and is used to assess overall performance and highlight particular trends. In IUC data is held in Clinical Guardian and may be exported into an excel spreadsheet.

The consistency and quality of IUC/WDPL Auditors’ “passed” cases will be audited by a Deputy Medical Director or Head of IUC Nursing and AHPs six months after an auditor commences in their role, at 12 months and 8-12 months later before they finish their term as an auditor. The Governance Team will run a clinical guardian report that will randomise audited “passed” cases per auditor for review.

A full review of all incidents, complaints, patient satisfaction and audit occurs monthly within each Service at a Quality Review meeting, and quarterly at an Urgent Care Service & Quality Board/Practice Services Governance Board meeting attended by the Service Managers and/or Clinician Leaders.

Development and Training

 

BrisDoc provides Call Handling shadow shifts for all Call Handlers, this is organised by an experienced Call Handler and will take place in the work setting appropriate to the Call Handler.

Call Handler coaching will be put in place if someone is on an amber status, and will be managed by the employees Line Manager, who will also arrange for mentoring to take place until a significant improvement is shown. Lack of improvement may result in the employee being put into the formal performance review process (see Managing Performance Policy). A call audit criteria form denotes the areas in which the ability and performance of a Call Handler is assessed (see appendix 4).  The Service Manager will use this form to review all calls to ensure continuity of the process and a fair assessment of all staff.

BrisDoc provides regular development activities e.g. clinical forums, shared learning emails. Clinical Guardian auditors will participate in the responsibility for sharing learning via the clinical forums and clinicians’ newsletter. All employed clinicians are provided with a schedule of mandatory training and are provided with other educational opportunities as they arise. Nurses are given additional hours to use for personal development purposes and activities. Non-clinical staff receive customer service training and other learning opportunities.

If a clinician requires review as a result of serious concerns (amber status in Clinical Guardian or red scores in peer review audit), the Deputy Medical Director/Head of Nursing/Lead GP/Nurse will discuss the situation and jointly agree where things are going wrong. The Clinician will reflect on the discussion and take steps to adjust their practice. A supporting Clinician may be instigated to provide one to one observation and training dependent on the situation. If after the follow up review no improvement appears to have been made, the Deputy Medical Director/Head of Nursing/Lead GP/Nurse, supported by other BrisDoc Directors, will arrange a formal meeting with the clinician for further discussion and assessment. The managing performance process may be initiated. A Performance Advisory Group (PAG) meeting should be convened to review the case, provide advice to the Medical Director who will make a decision about the ongoing management of poor performance. This might include:

• Referral to NHS England for GPs
• Referral to a professional regulatory body
• Formal managing performance/disciplinary processes
• Ongoing improvement work with monitoring and supervision.

Staff Status

 

A colour coding system has been developed to denote the governance status of any individual clinician and operational staff member.

In IUC the status determines the percentage of their cases audited. Staff will progress through the scale as their probation period completes and where required, their performance (and therefore their safe practice) is seen to improve.

Status	Description	Audit Requirements
Red	Clinicians deemed clinically unsafe to work in the service.	Nil
Amber	Clinicians under full review as a result of a complaint or clinical concerns raised by the governance/guardian team	25-100% of filtered cases at the discretion of the clinical guardian team
Yellow	Clinicians under standard review as a result of a complaint or clinical concerns raised by the governance/guardian team	15% of filtered cases
Green	Clinicians providing regular good and safe care	5% of filtered cases
Purple	New clinicians in first 1-2 weeks of working for the service	100% of filtered cases
White	Supervised GP Registrars in training	10% of filtered cases

 

Call Handlers

Status	Description	Audit Requirements
Red	Call handler is not currently performing well in the role and needs to be performance managed according to company policy	100% of calls to be monitored by Line Manager as they go through the Performance review process
Amber	Low achievement – call handler manages a basic call but does not adequately follow guidelines or manner, serious improvement actions needed	2 calls to be routinely reviewed each month by Line Manager & a further 10 calls per month to be reviewed by line manager
Yellow	Partial achievement – call handler demonstrates good knowledge and adequately answers call, but has various improvement areas	2 calls to be routinely reviewed each month by Line Manager and a further 5 calls per month to be reviewed by line manager
Green	Full achievement – call handler has demonstrated excellent knowledge of process, patient care and process	2 calls to be routinely reviewed each month by Line Manager
Purple	New call handlers in first month of working for the service	100% of calls to be monitored by Line Manager for 1st month & during induction process

 

Practice Services Clinicians

Clinicians will be placed in one of four different categories and may move category depending on the results of the audit process.

Red	<50% score on average. Consultations are unacceptable and of a low standard. The clinician will need a full audit and review and may have to attend additional training sessions or be supervised. A locum will not be hired again.
Amber	50-69% score on average. Consultations contain more errors or omissions and are considered as needing improvement. Clinician will be informed of errors and will be expected to show an improvement for the next audit review.
Yellow	70-89% score on average. Consultations contain minor errors or omissions but are considered to be of a high standard. Clinician may be informed of error.
Green	90-100% score on average. Consultations are deemed as being excellent. No action is needed.

 

Roles and Responsibilities

 

PERSON	ROLE	RESPONSIBILITY
Dr Kathy Ryan	Medical Director	To chair the Performance Advisory Group To liaise with NHS England and regulatory bodies as required. To act as GMC Liaison Officer for Doctors if/when needed.
Drs Anne Whitehouse/Chris Dykes	IUC Deputy Medical Directors	To line manage the Clinical Guardian Team (CGT) Lead GP. To oversee the performance of Clinical Guardian (CG) processes and management. To review the CG Clinician dashboard data taking appropriate action as required. To chair quarterly Urgent Care Service & Quality Board (UCS&QB) meetings. To lead or assist in investigating clinical complaints and serious incidents. To liaise with clinicians with respect to performance of concern and organise/attend review meetings with clinicians. To provide feedback to Head of Governance.
Dr Caroline Stovell	Practice Services Deputy Medical Director	To monitor the delivery of peer review and topic audits in the practices. To support the Lead Clinicians manage their audit lead responsibilities. To chair the quarterly Practice Services Governance Board (PSGB) meetings. To lead or assist in investigating clinical complaints and serious incidents. To liaise with clinicians with respect to performance of concern and organise/attend review meetings with clinicians. To provide feedback to Head of Governance.
Dr Louise Whyte	IUC Lead GP	To lead the IUC clinical audit process To manage the Clinical Guardian Team of auditors To provide full audit of doctors needing review in Amber & Yellow groups. To promote the use of the CG clinician dashboard with IUC clinicians. To report audit results to DMDs for the UCS&QB. To manage the currency and content of the Clinical Toolkit
Dr Hannah Chapman	Clinical Lead WDPL Clinical Guardian Lead	To oversee the WDPL GP team undertake peer review audit using Clinical Guardian To review cases referred by the GP team from peer review To liaise with the CGT members with respect to daytime CAS cases (audited by the CGT but managed clinically by the WDPL team) To provide support and education for their clinical team that improves/sustains performance.
Drs Catherine Maytum, Caroline Stovell & Shaba Nabi	Lead GPs	To manage the peer review and topic auditing process. To provide support and education for other clinical staff that improves/sustains performance. To report audit results to the PSGB
Frank Burge	Head of IUC Nursing and AHPs	To review cases of individual nurses/AHPs and support individual’s learning and performance improvement.
Liz Turner/Michelle Whittle/??	Lead Nurses	To lead the audit process for Practice and WIC Nurses. To provide support and education for their clinical team that improves/sustains performance. To report audit results to the PSGB
Sarah Pearce	Patient Safety Manager	To provide support to the CGT to ensure their efficient and effective auditing processes.
Clare-Louise Nicholls	Head of Governance	To support the Medical Director and Deputy Medical Directors.
Lucy Grinnell	Head of IUC	To ensure the following processes are undertaken by the Service Delivery Team Managers: ·         initial and routine call audit of all call handling staff ·         maintenance of the database of audit results ·         provision of feedback to members of team and arrange training if necessary. To attend quarterly UCS&QB meetings. To lead or assist in investigating operational complaints and serious incidents.
Dixine Douis/Jane Isaac	Practice Managers	To support Lead GP and others in auditing process by for example: collating results, selecting random sample of cases for audit, arranging audit meetings. To report audit results to the PSGB.
Samantha Hill	Head of Workforce	To support in the formal Performance Management process and attend PAG.
Jenny Parvin Jo Hunt Shelly Joseph Hazel Hayden Stephen Moore (Hannah Chapman re WDPL)	Clinical Guardian Team Auditors	To routinely audit clinicians in all groups. To provide feedback to the Lead GP. To undertake specific audits for information requirements and the BrisDoc audit programme. To liaise with new clinicians to ensure they are settling into their role following induction. To support the investigation of complaints/incidents. To write and review Clinical Toolkit material.
Practice Services salaried clinicians	Auditors	To undertake peer review auditing as part of the practice team. To give and receive honest and open feedback. To take part in review and educational sessions as necessary.

 

Clinical (Topic) Audits

 

The functionality within Clinical Guardian, Adastra reporting and EMIS searches can be used to source data for topic specific audits agreed as part of BrisDoc’s audit programme.

An annual audit programme will be agreed by the Urgent Care Service & Quality Board and Practice Services Governance Board and results reported to this Board. Each audit will have an audit plan written that sets out the objectives, audit methodology, audit criteria, auditors, reporting processes and timescales for the audit. Individual clinicians and operational staff are welcome to participate in audits as part of their continuing professional development.

Commissioner Audits

 

Specific performance audits for the IUC service, set out as Information Quality Requirements in the Contract, include a review of 2% of cases with admission, sent to ED or referred to 999 outcomes for appropriateness; and a review of completeness, accuracy and a clearly documented management plan for the patient’s own GP. These audits are run in Clinical Guardian and an analysis included in the quarterly IUC Quality and Performance Report.

Clinical Guardian Clinician Dashboard

 

Clinical Guardian functionality can report data for each clinician into a personal dashboard that will, when launched, provide them with information about their consultation and prescribing performance in comparison to an IUC average. Clinicians will be encouraged to review and understand their data and reflect on how their performance contributes to the efficient and clinically effective delivery of the IUC service.

Related Policies and Procedures

 

Standard Operating Procedure – Voice Recording

Policy – Managing Performance Policy

 

 

 

 

Appendix 1

Process for Uploading Cases Into Clinical Guardian

The process for retrieving case records from Adastra and uploading them to Clinical Guardian and the audit of those cases is set out below:

 

1. From the main menu select reporting, query builder, BrisDoc Reports, Clinical Guardian NEW FEB 2020. Set the date and time parameters for the report you wish to run (e.g. Monday 08.00hrs to the following Monday 08.00hrs). Select the flower symbol to run the query.

 

 

2. Finalise the report parameters by checking the dates and times, and setting the report filter to IUC All (AGPT for WDPL). Click in the report filter box and select IUC All (double click) from the menu, select okay.

 

3. Allow the query time to run. When the query completes running select results to bring up the data. Select the xml icon and save the report in the relevant folder on the BrisDoc shared drive using the service and dates as the file name. You may have to select permit all access to get to the shared drive.

 

 

 

4. Log into Clinical Guardian to upload the data. From the home page select upload cases. (For WDPL ensure you switch toggle from Severnside IUC to WDPL).

 

 

5. Click on browse to bring up the location on the shared drive of the data to be uploaded. Select the xml file and click on “open”. Then click on “upload import” and the file will be imported into Clinical Guardian. Let the upload process complete without navigating away from this screen.

 

 

 

6. Information about the results will appear on the left of the screen. You may now log out of Clinical Guardian.
7. Delete the spreadsheet of cases from the BrisDoc shared drive.

 

 

 

Appendix 2

Clinical Guardian Audit Process

Introduction

Clinical Guardian is both a methodology used in the routine clinical audit of GP out of hour’s clinicians and the software to support the process. This guide has been written to support clinical auditors in their work.

The Clinical Guardian Methodology

The Clinical Guardian auditing process involves 2 stages.

Stage 1		Stage 2
Individual Audit Trained Clinical Auditors Working Alone Audit from Anywhere Large Numbers of Anonymised Case Records Overview Pass or Refer Positive feedback where appropriate		Group Review Guardian Team Overseen by Lead GP Physical or Virtual meeting Small Number of Case Records Detailed Analysis Identify Trends Positive or Constructive Feedback as Appropriate

 

Individual Audit

The first stage involves reviewing a large number of cases to identify those which might need further evaluation. This process is undertaken by an individual clinical auditor and should take no longer than 30-60 seconds per case.

The objective in this stage is to identify the small numbers of cases which might give rise to concern or learning. IUC clinicians perform consistently to a very high clinical standard. Problems encountered are either “behavioural” or “clinical” or “service”.

	Focus	Solution	Examples
Behavioural issues	·      Individual clinician	·       Feedback, education, support ·       Referral to the appropriate authorities (rare)	·      Poor note keeping ·      Failing to measure basic observations appropriately ·      Concerns about clinical diagnosis or management
Clinical issues	·      Series of different clinicians experiencing difficulty with the same clinical problem	·       Define best practice and ensure that information is available to clinicians ·       Cascade through local education system/ensure relevant information is including in clinical toolkit and clinicians’ newsletter	·      Management of UTIs in under 5s ·      The assessment of potentially suicidal patients
Service issues	·      Problem within the IUC service ·      Problem with another organisation	·       Communication with operational team ·       Communication with other organisations	·      Late reporting of lab results ·      Unable to access EMIS records due to IT issues ·

 

The individual audit stage can result in four outcomes for a case as summarised as follows:

Quadrant 1 Safe Case Passed	Quadrant 2 Unsafe Case Passed
Quadrant 3 Safe Case Referred	Quadrant 4 Unsafe Case Referred

 

Clearly the scenario to be avoided is in quadrant 2. Quadrant 3 will generate more work for the Clinical Guardian Team but does not compromise patient safety. The general advice for auditors undertaking individual audit is “When in doubt, refer!” – leave the decision to the Clinical Guardian Team review and discussion.

The Auditor’s Screen

Clinical data is uploaded to “Clinical Guardian” by administrative staff. Any patient identifiable data (names, addresses etc.) is removed at this stage. The software randomly selects cases for audit in accordance with their colour status.

Data on the auditor’s screen is displayed as follows:

Cases for audit are displayed as either high priority or standard cases.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

A “Standard Case” is presented without the name of the clinician being presented. The patient demographics are presented in red, as are the consultation notes to be audited. A case may have more than one part (e.g. a telephone consultation and a face to face consultation). The part of the consultation in red is the section to be audited. However, if concerns are raised about the part of the case not in red, this should be referred to alert the Clinical Guardian Team who can then audit that part of the case.

 

Comments Box

The “Comments Box” allows the Auditor to record individual comments about the case. If the case is referred to the Guardian Team, then the comments will be seen by the Guardian Team and not by the clinician. If the case is passed as “pass with comment”, any comments will be passed onto the clinician. Comments from an individual Auditor should be positive (everybody appreciates praise) or constructive suggestions. If the individual auditor feels that critical comments should be made, the case is best passed onto the Guardian Team.

Assessment

Each case is graded as either “pass without comment”, “pass with comment” or “review for group review”. The case cannot be finished until one of these options has been selected.

Check-List for Referral

Cases to be referred are at the discretion of the individual auditor. The general rule is “if in doubt, refer”.

The key to the assessment is safety:

• Is this consultation safe for the patient?
• Is this consultation safe for the clinician? – would the records support them in the event of a legal challenge?
• Is this consultation safe for the wider organisation? – is the clinician creating clinical precedents which might be the cause of later problems e.g. excessive use of injected drugs, issuing Patient Line number, or inappropriate use of A+E/999 when safety-netting?
• Does the documented record of the consultation allow for informed and safe continuity of care for the patient’s own GP or for the accepting hospital teams who can view clinician notes in EMIS?

Telephone Consultations

Telephone consultations are difficult. Many clinicians consider them to be by far the most challenging part of IUC work. Problem cases generally fall into one of the following categories:

• Inadequate history
• Inappropriate clinical conclusion
• Inappropriate case closure
• Inadequate safety netting
• Inappropriate assessment of urgency
• Inappropriate referral to 999, ED, other services
• Video Consultations
• Challenges of more widely used video consultation since the Covid-19 pandemic include:
• Confidentiality issues
• Quality and availability of patient audio/visual equipment
• Limitations on quality of assessment
• Recording of video call visual is not possible (audio can be recorded)

Face to Face Consultations

Face to face consultations (either at the IUC Treatment Centre or home visits) generally give rise to fewer clinical concerns. When these occur, they include:

• Inadequate clinical assessment – typically failure to record vital signs e.g. heart rate in children or failure to perform a urinalysis or a pregnancy test.
• Inappropriate prescribing
• Inadequate safety netting

Guardian Team Audit – Group Review

Clinical Guardian Team Meetings

Frequency	Venue	Essential Membership	Optional Membership	Functions
Typically fortnightly with a bimonthly full Team meeting.	Actual or Virtual	2 Clinical auditors (doctors or nurses)	·    Deputy Medical Director ·    Head of IUC Nurses & AHPs ·    Lay member ·    Governance Team member	·  Approve new clinicians ·  Remove failing clinicians (recommendations to DMD) ·  Assess cases for review ·  Give and review feedback ·  Change colour statuses ·  Provide and/or seek specialist advice or best practice guidelines e.g. paediatrics, ophthalmology ·  Induction follow up calls

 

A guardian meeting requires a minimum of two Auditors to be quorate. This is to avoid negative feedback being sent from an individual. There is a group login screen which requires the usernames and passwords of one additional Auditor before the session can begin.

Once signed in, the whole Guardian Team can see the review screen. This is best facilitated by the use of a projector and screen or blank wall.

The review screen lists all the clinicians to be reviewed with their colour status. Clinicians who have been flagged up on the standard “blind” audit are now named. Some clinicians may have more than one case on the review screen. This is particularly true of “high priority” clinicians (amber or purple) where a large number of cases might have been flagged for review but note that they have had a higher proportion of cases audited in the first place which will account for some of the volume.

 

As each case is opened, the Guardian Team can review the clinical contents as well as the comments made by the auditor which give the reason for the referral.

The group assesses the case and may or may not wish to post feedback. Feedback can be free-texted or it is possible to create “snippets” of text to post in the feedback box. This may save time if there are standard comments to make. Where feedback is given to a GP Registrar it is also shared with their Educational Supervisor to ensure discussion and learning takes place.

 

 

The Clinical Guardian Team has a variety of options for the outcome of the review:

	Feedback?	Clinician Informed?
Pass without comment	No	Once per month
Pass with comment	Required	Once per month
Refer for group review	Preferable	No
Pass without comment (from group review)	No	Once per month
Pass with group comment	Required	Once per month
As discussed by phone or email	No	Will have been aware from email or phone before feedback sent via CG.

 

Once the audit is complete, the Clinical Guardian Team may wish to change the colour status of the clinician. An example of this would be from purple to green in the case of a new clinician who the team considers safe to go into the standard audit process.

In the case of doctors causing concern, the opinion about a change in colour status can be facilitated by reviewing the clinician’s record which can be displayed qualitatively and statistically as well as a list of feedback comments. The team may also like to discuss the clinician’s performance on shift with the leadership team, and listen to calls to see if the notes reflect the consultations accurately.

 

 

 

Two additional statuses can be used:

Grey for “inactive” clinicians – a clinician who has not worked for the service for 6 months should be regarded as inactive.

Black for “not a clinician” – sometimes administrative staff appear in “Clinical Guardian” – they do not need to be subject to the audit process.

Further Information

The Clinical Guardian Team may feel further information is required before it can conclude its judgement and therefore feedback on a case. This might include listening to a telephone call associated with the case, or seeking an expert opinion from a colleague. The BrisDoc Governance Team supports this process. CGT members may add comments within CG, viewed only by themselves, to remind them of the actions to date on that case.

Record Keeping

All actions arising from a meeting are recorded in a spreadsheet which tracks responses and completed actions. The spreadsheets are saved in a confidential Clinical Guardian Team folder on the BrisDoc shared drive accessible by the BrisDoc Governance Team, the Clinical Guardian Team, Medical Director and IUC Deputy Medical Directors. The spreadsheet also includes a record of telephone calls to be audited prior to a clinician being given “green” status, follow induction calls to new clinicians, and a blackboard for Team notes/follow up actions.

Feedback

The clinical guardian meeting concludes with a review of any feedback which has been received from clinicians during the previous week. In most cases, this requires a simple acknowledgement and the case can be closed. Occasionally, further action may be required and this should be at the discretion of the Lead GP. Feedback may be “tracked” when the CGT want to ensure the clinician is engaging in the feedback being given. If the clinician has not engaged, or read the comments, then the team may choose to take further action. This would normally start with a nudging email to ask the clinician to log in and review their feedback. If a clinician consistently fails to engage with the feedback process this may be escalated.

Conclusion

The Clinical Guardian Team has the delicate task of balancing the need to ensure safety and quality within the organisation with the need maintain a good working partnership with its clinical workforce. “Clinical Guardian” should therefore be used with care and sensitivity and thought given as to how feedback is phrased and whether a conversation might sometimes be preferable to an impersonal electronic response.

 

 

 

Appendix 3

Practice Services Peer Review Audit Tool

 

Appendix 4

Call Handlers Audit Tool

 

Competency	Criteria	Score	Max score
Effective call process	Greets caller appropriately – giving own name and job role		5
	Takes patient details in correct order		6
	Checks details given (spellings/repeating whilst not giving out any patient information)		5
	Takes correct callers details and records the correct relationship to caller		5
	Gives correct timescales		6
	Explains process to caller of what will happen next		6
	Transfers the call to a clinician in real time (8:00 to 20:00 Monday to Friday )		5
Clinical care	Recognises when to ask further questions to gain more understanding of situation		5
	Recognises when to not ask further questions.		5
	Records appropriate & useful information, which would be helpful to the clinician using good spelling & grammar		6
	Transfers the case to the correct queue / appointment slot		6
Listening skills	Recognises tone of caller (panic/worry in voice)		5
	Allows the caller to speak without cutting over them to speed up the call, but aware of when they have enough information		5
Effective customer service & communication	Demonstrates a polite and professional manner		5
	Adapts approach according to callers needs		5
	Establishes rapport and treats caller with respect and sensitivity and empathy		5
	Conveys confidence in what they are doing		5
	Overall manages call safely and effectively leaving the caller confident in outcome		10
Overall	Score		100

 

Compliance

>90	Full achievement – call handler has demonstrated excellent knowledge of process, patient care and process
<90	Partial achievement – call handler demonstrates good knowledge and adequately answers call, but has various improvement areas
<80	Low achievement – call handler manages a basic call but does not adequately follow guidelines or manner, serious improvement actions needed
<70	Call handler is not currently performing well in the role and needs to be performance managed according to company policy
	New call handlers in first month of working for the service

 

 

Appendix 5

IUC Clinician Call Audit Tool

 

 

 

 

 

Change Register

 

Date	Version	Author	Change Details
22.4.15		CL Nicholls	Update voice recording retrieval process appendix, new Clinical Guardian categories, inclusion of GPST.
19.10.15		CL Nicholls	Updated to include BMC’s revised process for GP audit, revised role titles, inclusion of BMC and clinician calls audit tools in appendices. Update re process for GPSU/T. Inclusion of Kathy Ryan as Medical Director
7.11.18		CL Nicholls	No changes until the PSGB agrees new approach to clinical audit in the practices in spring 2019.
26.10.20	1.4	CL Nicholls	Mapped to new template. Inclusion of CG Clinician Dashboard and PAG. Updated RAG ratings within CG. Updated CG processes mapped into an appendix. Updated roles and responsibilities. General alignment to urgent care and practice services governance structures. Remove reference to SOP for uploading cases to Clinical Guardian.