This site is intended for health professionals only

GPs report ‘biased’ feedback in first revalidation pilots

By Gareth Iacobucci

Exclusive: The first GPs in the country to trial revalidation have faced 'biased' feedback, 'simplistic' questioning and unrepresentative patient surveys as a raft of problems emerge from the initial primary care pilots, Pulse can reveal.

The majority of GPs in pilots commissioned by the RCGP encountered problems with the use of multi-sourced feedback (MSF) and patient questionnaires, two of the central planks of evidence that GPs are currently expected to collect in order to be revalidated.

GPs also expressed concerns at a lack of anonymity and confidentiality in gaining colleague feedback, particularly in small practices, and the excessive workload involved in seeking out colleagues to participate.

The findings - revealed in two RCGP reports obtained exclusively by Pulse - come just days after it emerged that the Department of Health is planning to substantially scale back revalidation, and add weight to the Government's concerns over the excessive cost and bureaucracy of the plans. Pulse revealed earlier this week that health secretary Andrew Lansley has demanded a slimmed-down version of the scheme, with the GMC now planning to halve the amount of MSF GPs are asked to collect.

The pilots also uncovered widespread difficulties for locums and salaried GPs in gathering sufficient evidence for the process.

The lead pilot for GPs in England and Wales, conducted by the University of Warwick, revealed that almost two thirds of GPs backed the portfolio system as an acceptable means of collecting evidence, with more than three quarters supporting the collection of significant event audits, personal development plans and learning credits.

But respondents encountered significant problems with the use of MSF, reporting the potential for biased feedback from colleagues, questioning the ability of colleagues outside of the consultation to judge their perfomance, and reporting that the questions themselves were too simplistic.

The report explained: ‘They [respondents] questioned the ability of colleagues outside the patient consultation, especially those of single-handed GPs, to judge the performance of that practitioner.'

‘Second, GPs considered that choosing close colleagues and subordinate colleagues to complete questionnaires might bias feedback. Third, some of the questions themselves were considered to be either too simplistic or not appropriate.'

There was similar disquiet about the use of patient questionnaires to revalidate GPs, with respondents questioning patients' ability to assess GP clinical skills, and claiming the surveys did ‘not accurately measure a GP's performance'.

As a result of the problems, 57% of GPs deemed MSF an inappropriate means of collecting evidence, with 55% against the use of patient questionnaires.

The majority of respondents (59%) also felt that assessment of GPs' extended roles beyond their main GP role was an inappropriate form of evidence.

Evidence of problems with patient and colleague feedback leaves both aspects increasingly vulnerable to any potential scaling back of the process by the Government, which has already halted the rollout until further pilots take place.

The report concurred that further piloting of both MSFs and patient questionnaires was required, and also suggested that both aspects ‘should be performed out of house' to increase objectivity of survey feedback.

Salaried and locum GPs reported particular problems with evidence gathering, with all respondents to the sessional GP pilot, carried out by the Northern Deanery and Durham University, reporting that they would or did struggle to complete MSF.

Sessional respondents reported that significant event audits would be ‘impossible' to complete for GPs only working in practices for a short period of time, and warned their status as ‘second class citizens' in general practice translated into ‘a lack of engagement and support' in completing appraisals.

Researchers suggested that alternatives to audits should be explored for GPs with ‘no permanent base', that practices should involve sessional doctors more in practice meetings, and that ‘a smaller sample of more meaningful contacts' may be a more appropriate way of conducting MSF for sessional GPs.

Professor Mike Pringle, the RCGP's lead for revalidation, said the problems experienced by GPs across the pilots could partly be explained by the requirement to collect evidence in two months rather than five years, but acknowledged MSF and patient surveys as ‘the two most obvious things to come out' of revalidation.

He said: ‘We may be getting a message that it's more difficult than it actually is. We find a lot of reassurance in these pilots although there are things we need to improve, and lessons we've got to learn.'

Professor Pringle added that recommendations on how to make revalidation more manageable for sessional GPs were already being incorporated into the RCGP's guide to revalidation, which is being updated regularly as the process develops.

Professor Mike Pringle: 'We find a lot of reassurance in these pilots although there are things we need to improve, and lessons we've got to learn' Professor Mike Pringle: 'We find a lot of reassurance in these pilots although there are things we need to improve, and lessons we've got to learn' Download the reports

To read the full reports into the RCGP's first revalidation pilots, please click here.