Professor Aneez Esmail is a man used to controversy, but even he was unprepared for the reaction from the RCGP to his independent research into the failure rates of different ethnic groups in the MRCGP exam.
As he prepared to publish a paper in the BMJ concluding that ‘racial discrimination’ could be a factor in lower pass rates among non-white candidates in the clinical skills assessment (CSA) part of the exam, the college took the unprecedented step of instructing its lawyers to write to the journal claiming the article was ‘defamatory’ and seeking changes in the wording.
‘I felt quite threatened’
Professor Aneez Esmail
‘I felt quite threatened,’ says Professor Esmail, professor of general practice at the University of Manchester and a GP in the city. ‘I can’t explain or understand what the RCGP was trying to do.’
Part of the problem was that his official GMC-commissioned report into the differential pass rates of different ethnic groups in the CSA – published the same day as the BMJ paper and based on the same data – failed to make any explicit allegation of discrimination and was widely interpreted as having largely exonerated the exam.
Public relations disaster
The BMJ paper, in contrast, contained the incendiary claim that ‘subjective bias due to racial discrimination’ in the CSA may be a cause of higher failure rates for UK-born ethnic minority and international candidates.
The BMJ defied the college and went ahead with the publication of Professor Esmail’s paper, pitching the RCGP into a public relations disaster that has made national headlines and potentially undermined trust in the MRCGP, which has become an all-or-nothing high-stakes passport to a career in general practice.
The row over Professor Esmail’s research could not have come at a worse time for the college. A judicial review challenging the lawfulness of the MRCGP exam is due to begin shortly, brought by a group of international doctors.
The GMC, meanwhile, is vowing it will implement the recommendations of the review it commissioned from Professor Esmail, including providing better feedback on performance to candidates and publishing the results of its Professional and Linguistic Assessments Board exams and International English Language Testing results and their association with CSA pass rates. But the controversy shows no sign of abating.
Even the BMA, which has trodden carefully in dealing with the controversy, is now seeking legal advice and ‘considering all options’ after BMA chair Dr Mark Porter told a meeting of the British International Doctors’ Association earlier this month that the college had ‘significant questions’ to answer.
He told the audience: ‘Putting out a press release that basically says “there is no case to answer here, move on and don’t look at us and we’ll get on with defending the judicial review” – that is not good enough.’
What is the row all about?
The clinical skills assessment was introduced in its present form into the MRCGP exam in 2010. Since its introduction, official statistics have consistently shown international medical graduates (IMGs) and UK-trained black and minority ethnic (BME) doctors are far more likely to fail the exam than white UK-trained graduates.
The RCGP points to marked differences in rates of IMGs, BME doctors and white UK graduates passing the anonymised AKT as evidence the CSA is not biased. But Professor Esmail pointed out that his research adjusted for AKT attainment.
His review found that even adjusted for AKT performance, a BME IMG is almost 15 times more likely to fail the CSA at the first attempt than a white UK graduate. He also found BME UK graduates are 3.5 times more likely to fail the CSA at their first attempt than their white counterparts.
His BMJ paper concluded that ‘subjective bias due to racial discrimination’ may be a cause of the differing pass rates, although his report to the GMC also stated that ‘the method of assessment is not a reason for the different outcomes… that have been described’ and that it was based on a ‘well-established’ pedagogy’. In any event, both the BMJ paper and the GMC report have also cited other possible causes.
Professor Esmail found ‘preparedness’ of candidates was a key factor, and pointed out that IMGs have more to learn about the UK model of general practice. ‘The clinical skills assessment is not a culturally neutral examination,’ he wrote.
East London GP Dr Sam Everington, a council member and former deputy chair of the BMA who has raised issues of racism in the NHS with Professor Esmail in the past, says the current position of the college is unprecedented.
He says: ‘For an academic body to threaten legal action against an independent journal is extraordinary – completely unacceptable.
‘A number of things worry me, including how the RCGP’s response is perceived externally. The best response to something like this is that, as an organisation, you are taking the matter seriously and that you are looking into it.’
To add to the RCGP’s problems, a Pulse survey of 578 GPs has found 58% feel the RCGP’s handling of the controversy this year has been ‘poor’ or ‘very poor’.
Related articles
International doctors granted judicial review of the MRCGP exam
There has also been a shift in attitudes to the MRCGP over the past few months. According to a separate question in the same survey answered by 644 GPs, the number who believe the exam is unfair to international medical graduates is still very much a minority – 24% – but this compares with just 15% of those asked a similar question last December.
RCGP defence
The RCGP continues to assert that the CSA – in which GP trainees are assessed on their consultation skills in simulated surgery settings – is fair, and that the college will defend it strongly against all allegations of discrimination. It says the differences in the pass rates between white and non-white candidates in the exam are not due to the CSA itself.
It cites lines from Professor Esmail’s independent report to the GMC –published on the same day as the BMJ paper – to justify its claims, highlighting his conclusion that ‘the method of assessment is not a reason for the differential outcomes that… have been described’.
For his part, Professor Esmail claims that is a ‘completely selective quote taken completely out of context’, and he strongly rejects the RCGP’s claim that his GMC report exonerated the exam. Instead, he insists his BMJ paper clearly explains his position, directly referring to the possibility of racial bias in the exam.
Professor Esmail’s work on discrimination in the NHS won him a Harkness Fellowship in 1997 and has resulted in changes to the GMC complaints process and medical school admissions procedures. He says his BMJ paper was more explicit than his official report in referring to the possibility of discrimination as a factor in differing pass rates because he was challenged on this point during the BMJ peer review process. He says: ‘It was put to me: “If you think it is racial bias, then you should say so”.’
Professor Esmail points out that, even after adjustment for performance in the machine-marked applied knowledge test part of the MRCGP, there was a 3.5-fold difference in pass rates between white UK graduates and black and minority ethnic (BME) UK graduates between November 2010 and December 2012.
He says: ‘That is nothing to do with language or culture or anything else. That needs explaining. You have to ask yourself what is happening in the exam.’
‘[The report] misleadingly suggests we may be guilty of bias’
Professor Clare Gerada
But the disparity between the two versions of the research prompted the RCGP to demand the BMJ change the paper before publication and re-issue its press release. Dr Fiona Godlee, editor-in-chief of the BMJ, said: ‘We responded saying that we had no plans to [do so]. This was followed by a lawyer’s letter repeating the demand and adding that they considered it defamatory. We said we felt it would be damaging to our reputation and theirs if we made a change to a research paper in response to their request.’
The RCGP refused to confirm if this information was correct but chair Professor Clare Gerada claims the BMJ paper ‘contradicts’ the report published by the GMC and ‘misleadingly suggests we may be guilty of bias’.
She says: ‘Certain aspects of the MRCGP are currently subject to an ongoing judicial review. We are defending strongly, in these proceedings, all allegations of discrimination.
‘The college has been very open about the differential pass rates for many years, and has commissioned and supported internal and external research to try and identify what the cause or causes may be. Indeed, we were also one of the first medical royal colleges to publish the exam results highlighting the differences.’
Professor Gerada adds: ‘As an organisation committed to equality and diversity, we take multiple steps to ensure our exam is fair, including making sure that all of our examiners and role players receive equality and diversity training.’
But the most pressing issue facing the college is the judicial review that is being brought by the British Association of Physicians of Indian Origin (BAPIO). The date for the full judicial hearing has not yet been confirmed, but BAPIO looks set to press ahead with legal moves that could see judges making the final decision on whether the CSA directly or indirectly discriminates against international medical graduates.
And the process is costing the RCGP dear – the college claims it has already spent £45,000 fighting the allegations, and the bill could approach six figures.
GMC action
The GMC will join the college as co-defendants, following a High Court ruling that there is an ‘arguable’ case that the regulator has failed in its public sector equality duty by not taking action when the disparity in pass rates was revealed.
Professor Esmail had previously criticised the GMC for misrepresenting his findings in its response to his report, but after private discussions he now says he is confident that the regulator will take action. ‘In fairness to the GMC, I wrote [chief executive] Niall Dickson a private letter setting out my concerns about what I found and he wrote back and said “I understand entirely and I am not complacent about what needs to be done”.’
Professor Esmail concludes: ‘I have no doubt that the regulator understands what needs to be done. I have some faith in that process.’
The row is sure to dominate the tenure of incoming RCGP chair Dr Maureen Baker as she steps into the job next month, but Professor Esmail insists there is a simple solution. He says: ‘What I would count as a success is the RCGP saying “we understand that there is an issue here”. That is the first thing.’
