Evidence of racial bias in MRCGP exam claims author of independent review, as row erupts over 'contradictory' second version of research
Exclusive The author of a major independent review into the MRCGP exam has launched a blistering attack on both the RCGP and the GMC, accusing them of misrepresenting his report and warning evidence of racial bias must not be swept ‘under the carpet’.
Professor Aneez Esmail – one of the UK’s leading experts on racism in the NHS, who was charged by the GMC to investigate claims of racial bias in the MRCGP’s clinical skills assessment – says he ‘completely refutes’ RCGP claims that his report exonerates the college, after his review found graduates from non-white ethnic groups did worse in the exam.
The RCGP, which is currently facing a judicial review brought by international doctors seeking to declare the exam unlawful, has consistently denied that the exam is in any way discriminatory or biased.
Professor Esmail’s explosive comments come as the final version of his review was published by the GMC and a different version of his research simultaneously appeared as a paper in the British Medical Journal.
Unlike the report published by the GMC – and an earlier leaked draft - the BMJ version of his research concludes that ‘subjective bias due to racial discrimination’ in the clinical skills assessment may be a cause of higher failure rates for UK-born ethnic minority and international candidates taking the exam.
But RCGP chair Professor Clare Gerada attacked the conclusions of the BMJ paper, saying that it ‘contradicts’ the report published by the GMC and ‘misleadingly suggests we may be guilty of bias’.
The GMC declined to comment on the BMJ report, but said that it was clear that the report it had published did not blame the CSA for the differences in pass rates.
But Professor Esmail – speaking exclusively to Pulse - rejected both organisations’ reading of his research and said that he stood by the claims that ‘racial discrimination’ in the CSA might be causing some to fail the exam, as described in the BMJ report.
‘I completely refute that the report exonerates the RCGP,’ he said. ‘Anyone looking at [the CSA] would say it has its flaws and they cannot hide it under the carpet.’
Professor Esmail’s review found a four-fold difference between the performance of white and non-white UK medical graduates taking the CSA and a fifteen-fold difference between white medical graduates and international traineees, even after controlling for age, gender and previous performance on the applied knowledge test.
He said that during the peer review process for the BMJ paper he was asked to clarify his claim that that ‘subjective bias’ could be behind the differential pass rates.
He said: ‘It was put to me: “If you think it is racial bias, then you should say so.”
He accused the GMC of misrepresenting his findings in its response to his report.
‘That is a misreading of the report and they are being very selective,’ he said.
What the BMJ paper says
‘Subjective bias due to racial discrimination in the clinical skills assessment may be a cause of failure for UK trained candidates and international medical graduates’
Source: BMJ 2013, online 26 Sep
The BMJ paper also states that ‘cultural factors cannot explain the differences between white candidates and black and minority ethnic candidates who have trained in the UK’.
But the other version of Professor Esmail’s research– also published today by the GMC – does not include this sentence and does not mention ‘racial discrimination’ as a possible factor in failure rates.
It offers similar conclusions to a draft version leaked to Pulse earlier this month, saying that although there are ‘significant differences’ in failure rates between different ethnic groups in the CSA exam ‘the method of assessment is not a reason for the differential outcomes that we have described.’
In a strongly-worded statement RCGP chair Professor Clare Gerada said that she was ‘shocked and bemused’ that ‘contradictory’ findings had been published in the BMJ.
She claimed that the BMJ paper had been rushed through ‘with undue haste’ and argued that it ‘called into question the findings to which it comes’.
Professor Gerada said: ‘On the very day that Professor Esmail’s official and independent GMC investigation report finds no evidence of discrimination, the same author is publishing a contradictory paper that misleadingly suggests we may be guilty of bias.’
She added: ‘In the official and independent GMC report, Professor Esmail finds that: “the method of [the RCGP] examination is not a reason for the differential outcomes that…have been described. The CSA examination is based on a well established pedagogy which is internationally recognised and used widely in postgraduate examinations’.”’
A GMC spokesperson said: ‘The report by Professor Aneez Esmail found that while there are significant differences in pass rates between different groups of doctors, the way they are assessed in the CSA is not the cause of those differences.’
‘Professor Esmail’s report also makes a number of important observations about the nature of the examinations. He says that “the CSA exam is not a culturally neutral examination and nor is it intended to be. It is not and nor should it be just a clinical exam testing clinical knowledge in a very narrow sense. It is designed to ensure that doctors are safe to practice in UK general practice.”’
‘However, this report should not be a cause for complacency and we need to understand more about why “there are significant differences in failure rates between different groups in the CSA examination”.’
‘Professor Esmail has made a number of important recommendations for the GMC and the RCGP. As he has proposed, we will publish exam results of the GMC’s Professional and Linguistic Assessments Board (PLAB) exams and International English Language Testing (IELT) results, and how they link to college exam results. We are already working with colleges to ensure that examiners have suitable equality and diversity training, and that their performance is monitored and that best practice is followed in setting standards for exams.’
Dr Fiona Godlee, BMJ editor-in-chief, said the paper was fast-tracked, but only as part of ‘standard procedure’, and it underwent full peer review, including full statistical review
She told Pulse: ‘We stand by its findings and look forward to further discussion of the implications for postgraduate training and workforce planning.’