This site is intended for health professionals only

GPs face ‘systematic bias’ from revalidation feedback as trial finds Asian doctors get lower scores

GPs face a potential minefield of prejudice and ‘systematic bias' from both patients and colleagues, a major road test of the GMC's draft revalidation feedback questionnaires has revealed.

The GMC-commissioned study of nearly 50,000 feedback questionnaires found some evidence that doctors of Asian origin had lower scores than those from white or other ethnic groups, although researchers stressed the association was 'weak' and overall doctors' age, gender and race were not found to be independent predictors of feedback.

Researchers also found GPs received poorer scores from colleagues than hospital doctors, and that locums and those qualifying outside the UK or South Asia had lower scores.

Patients had less favourable impressions of doctors who had not graduated from European universities. Doctors could also expect a lower score when feedback was obtained from a sample of patients that had a lower proportion of white patients and when the doctor in question was not their 'usual' doctor.

Researchers analysed questionnaires for 1,065 doctors from 17,031 colleagues and 30,333 patients. Results are published in the BMJ today.

The findings follow BMA criticism of the use of patient feedback in the revalidation process after research on consultation satisfaction questionnaires found more than 5% of GPs consistently fell below the mean average patient satisfaction level.

Study leader Professor John Campbell, professor of general practice and primary care at the Peninsula Medical School in Exeter, told Pulse: ‘We strongly support multi-source feedback as part of revalidation. But although the GMC's questionnaires are extremely carefully developed, we're conscious their interpretation needs to be carefully managed. There's a potential for systematic bias.'

'Although age, gender and ethnicity weren't independent predictors of patient or colleague feedback, where the doctor trained was.'

‘We also noticed that patient feedback was lower where doctors were working in areas of high ethnicity, so there may be a risk of doctors getting lower scores even when they are working hard. We must be careful how we compare doctors in the inner city with those in the leafy suburbs.'

He said that the reason GPs scored lower than other doctors in colleague feedback might be that GPs were more critical of themselves than other doctors were.

He added: ‘The results show we must be careful how we compare doctors. It may be that those with lower scores are working in more challenging environments.'

Dr Richard Vautrey, GPC deputy chair said: ‘These questionnaires can provide helpful pointers to appraisers and appraisees but no more.'

‘You can't place significant weight on them because they are open to bias and context can vary tremendously. We shouldn't be spending much time and money on doing these too often. We need to treat professionals as professionals otherwise we could end up doing it just because it's a hoop to jump through rather than a useful tool for evaluating performance.'

Dr Una Coales, an RCGP council member and GP in Stockwell, south London, said the results reflected an ‘undercurrent of cultural racial bias' in the UK.

She said: ‘Patients prefer to see a white UK-trained GP over an international medical school graduate as there is a perception that they are getting a "better quality" doctor.'

'Revalidation is also biased against locum doctors. A patient is more likely to give a favourable response in a survey to his regular family GP than a locum doctor he has only met once.'

Niall Dickson, GMC chief executive, said the questionnaires would not be used in isolation when appraising doctors: ‘Being aware and taking account of how patients and colleagues view your practice is important for every doctor but it is only one part of the supporting information that doctors will bring to their appraisals.'

'It will be considered alongside all the other information about a doctor's practice and is not something which you can 'pass' or 'fail'. It assesses an individual doctor's strengths and areas for development to help them improve their practice – it is not a way of comparing doctors with one another.'

RCGP revalidation lead Professor Mike Pringle said: ‘We have recognised that locums were liable to receive lower patient scores, but we need to raise awareness of the other implications from this research.'

‘It is reassuring that the ethnicity, age and gender of the doctor were not key influences on the questionnaire results."

The Department of Health's revalidation support team declined to comment on the findings.


Multisource feedback

Doctors received less-favourable colleague feedback if they were:

* not a UK or South Asian medical graduate

* locum doctors

* a GP or psychiatrist

* employed in a staff grade, associate specialist or equivalent role

Source: BMJ 2011, published online 28 October