Analysis: Where next for the MRCGP exam?
The troubled GP entrance exam has been ruled lawful, but changes are planned to address the disparity in pass rates for non-white groups, finds Jaimie Kaffash
It was a legal challenge that came close to derailing the RCGP’s entrance exam. But the High Court decision in favour of the college has ended up being something of a curate’s egg.
The judicial review into the MRCGP brought by an international doctors’ group last month ruled that the exam was lawful – a positive result for the college.
But the judge presiding over the case said it was time for the RCGP to ‘eliminate discrimination’ in the MRCGP and address the differences in failure rates between white and non-white medical graduates sitting the clinical skills assessment (CSA) – the role-playing test of GPs’ clinical and communication skills.
Mr Justice Mitting said in his judgment last month that if the RCGP does not act, and its failure to act is the subject of a further challenge, ‘it may well be held to have breached its duty’.
The RCGP welcomed the judge’s ruling and agreed ‘further action is needed’ to support medical graduates who fail the CSA.
But the BMA insists the college must take action ‘immediately’ and the British Association of Physicians of Indian Origin (BAPIO) – the organisation that brought the legal challenge – says it will be calling for major changes and that it has not ruled out an appeal.
Questions have also been raised about the RCGP’s handling of the case, after it spent nearly £300,000 on its legal defence.
A Pulse survey shows that, despite the verdict, almost half of GPs think the RCGP could have handled the case better.
The CSA was introduced as part of the new MRCGP exam in 2008, and was designed to test how well candidates performed treating cases in a mock clinical setting.
The exam was made tougher in 2010, when candidates were given a combined mark over 13 simulated clinical settings – meaning a serious failure in any one of those ‘stations’ would be likely to lead to overall failure.
In the wake of these changes, it soon became apparent that there were huge differences in the number of non-white graduates failing the exam compared with white UK graduates.
International doctors’ groups began lobbying on the issue, leading to a series of 18 reviews into pass rates, carried out by the RCGP and others – but leading to little substantive change to the exam.
All this led to the judicial review held last month. In a courtroom packed with anxious international medical graduates (IMGs) and their relatives, the court heard that the first-time pass rate for UK white graduates in 2012/13 was 97%, but just 40% for foreign trainees. The pass rate for non-white UK graduates was 84%.
Karon Monaghan, QC for BAPIO, told the court: ‘Overall, BAPIO contends that the candidates’ nationality, national or ethnic origin, or colour, has had a significant influence on the outcome of their examinations. The only proper inference, therefore, is that the college has directly discriminated against IMGs and black and minority ethnic candidates.’
She said the college had ‘failed and is continuing to fail to comply with the public sector Equality Duty in repeatedly applying the MRCGP… without modification’.
Peter Oldham, QC for the RCGP, said: ‘On the contrary, all the evidence is of an organisation that is greatly concerned with differential pass rates in the CSA.’
‘The college needs to act to eliminate discrimination’
Mr Justice Mitting
In his final judgment, Mr Justice Mitting found the college had not directly discriminated against any candidates. He also held that any indirect discrimination against IMGs was justifiable because the CSA was a proportionate means of achieving the legitimate aim of protecting patient safety.
In relation to BAPIO’s claim that the RCGP had breached its duties under the Equality Act 2010, the judge concluded it was sufficient that the college had ‘applied its mind’ to the issues by commissioning a number of expert reports, and identifying where and how improvements could be made. The college was held not to be in breach, but the judge said the judge had an ongoing duty to eliminate discrimination and advance equality of opportunity, and must continue to have due regard to those matters.
The judge said: ‘The college needs to act to eliminate discrimination and has identified some of the means by which that need might be addressed and fulfilled.
‘If it does not act, and its failure to act is the subject of a further challenge, it may well be held to have breached its duty.’
He said BAPIO had won a ‘moral victory’ and that: ‘I am… satisfied that the bringing of this claim is likely, in the end, to bring something of value to the medical profession and therefore to the public in general.’
GP trainer’s view
This should never have ended up in court. It did neither side any favours and if the two parties had been able to get together and discuss the problems out of the public eye, it would have been far more beneficial for all involved.
My concern about the whole thing is that it calls into question the validity of the MRCGP full-stop, which would be a mistake. The MRCGP requirement has improved the image of general practice, because it was the only specialty that didn’t have membership of a royal college as an outcome of becoming a senior clinician.
I think international medical graduates are always under the cosh a bit, because of all the problems they face, which have been well documented.
But I would be concerned if the college tried to positively discriminate to address the pass rates. But anyone who feels they have been unfairly discriminated against by the CSA should have a mechanism by which they can be reassessed or re-marked. Whether that is a repeat sitting, or in a different environment, or whether we just video all assessments, there are some simple answers there.
Dr Ian Turner is a GP trainer in Mexborough, South Yorkshire
Robust and fair
The RCGP welcomed the decision. Chair Dr Maureen Baker said after the verdict: ‘We hope today’s judgment means we can now draw a line under the events of the past year and concentrate on delivering a robust and fair exam for future generations of GPs and for the benefit of our patients.’
The college had already taken steps to ensure the exam is fair, she said, including giving examiners and CSA actors ‘diversity training’ – although she recognised there was more work to do.
Dr Baker added: ‘We agree that further action is needed, and we are already working hard to find the best way of supporting the small number of trainees who fail to pass the CSA component of the MRCGP licensing exam, to give them every chance of passing the exam.’
The RCGP says it is developing further learning resources and reviewing the feedback candidates receive after the exam. It says it welcomes any suggestions about how to improve the exam, and that it will work with BAPIO.
But the changes the RCGP’s critics are calling for will prove a major challenge. BAPIO president Dr Ramesh Mehta, a consultant paediatrician, says the association will be looking for more fundamental changes to the exam, such as actors being replaced by real patients.
He says: ‘Although we lost the case, the judge added that the RCGP has to ensure that things are fair and the law of equality is applied.
‘I think there are problems caused by the role-playing where the college brings in actors to play the part of patients and who are trained in a very specific way. Real patients might be better.
Potential changes to MRCGP
• Using two examiners instead of one for the clinical skills assessment (CSA).1
• The use of real patients, instead of actors.2
• Videoing the exams to give candidates the chance to appeal and to help provide improved feedback.3
• Using videos from real consultations, instead of simulated clinical settings.3
• Increasing the number of attempts allowed to pass the CSA to six.3
• Increasing the standard required to pass Professional and Linguistic Assessments Board tests.4,5
1. GMC-commissioned review by Professor Aneez Esmail, September 2014; 2. BAPIO suggestion; 3. Letter from Dr Krishna Kasaraneni, chair of the GPC GP trainees subcommittee, to the RCGP, October 2013; 4. BMJ 2014; 348:g2621; 5. BMJ 2014; 348:g2622
‘There is also evidence from research that having two examiners marking independently at each station is better than one.’
The BMA, which supported the judicial review financially – to the tune of £25,000 – and by gathering evidence, says the RCGP must act on its recommendations, set out in a letter to the college last October.
These included videos of examinations, an increase in the number of attempts allowed to pass the CSA to six and testing communication skills through video assessments and trainer feedback rather than in a mock clinical setting.
Dr Krishna Kasaraneni, chair of the GPC GP trainees subcommittee, says: ‘The best way to resolve these significant variations in pass rates is for all parties to now work together.
‘While the ruling has dismissed claims of discrimination, the judge made it clear that the CSA does put certain groups at a disadvantage. This is what needs to be looked at immediately.’
Dr Kasaraneni’s concerns are echoed by Professor Aneez Esmail, professor of general practice at the University of Manchester and a GP in the city, who was the author of a GMC-commissioned review into the MRCGP exam, published last year.
Professor Esmail says: ‘A lot can be done to help improve it… key priorities are double-marking, better training of the actors and a recognition that there are problems with using this kind of assessment which can be mitigated, better feedback to candidates and looking at the case-mix of the diets and ensuring a greater diversity of examiners.’
However, there is the prospect of a crackdown on IMGs entering GP training. Researchers from University College London and the University of Cambridge recently claimed that the way to reduce the differences between UK graduates’ and IMGs’ pass rates was to raise the pass mark for the Professional and Linguistic Assessments Board tests – the language exams international doctors must take before practising in the UK.
As the dust settles
In the immediate aftermath of the ruling, the MRCGP’s standing, and that of the college, may have suffered. Of the 441 GPs who completed Pulse’s online survey, 46% think the college has handled the dispute badly. Only 16% say nothing about the exam should be changed, while 51% advocate looking at the feedback given to candidates, and 42% think candidates should be given better preparation.
The case has cost the RCGP hundreds of thousands of pounds; it asked the court to make BAPIO pay £288,000 towards its legal costs. In the end, the judge ordered BAPIO to pay £50,000.
As Dr John Ashcroft, deputy chair of Derbyshire LMC, points out, this is GP members’ cash being spent on legal fees rather than improving the exam. He says: ‘Looks like the lawyers won… Good to know the profession’s money is being well spent.’
But Dr Simon Braybook, a GP in Cardiff, vice-chair of the south-east Wales faculty and an associate academic fellow at Cardiff University, says the case has opened up the opportunity to make the MRCGP the ‘fairest postgraduate exam of all the royal colleges’.
He adds: ‘I am certain the work will continue and other royal colleges will follow suit by placing their postgraduate exams under similar scrutiny and transparency. If that happens, it will be money well spent.’