This site is intended for health professionals only

At the heart of general practice since 1960

Read the latest issue online

CAMHS won't see you now

Analysis: Does the row over the CSA have any basis?

As the college faces legal action and criticism from all sides over the MRCGP exam’s clinical skills assessment, Jaimie Kaffash looks at whether the claims have any basis

The opening of the RCGP’s shiny new building in Euston this year should have marked a new start for the college, but old arguments about its entrance exam to the profession keep clouding the horizon.

The clinical skills assessment (CSA), part of the new MRCGP exam, has always been controversial. But now the college faces being dragged through employment tribunals by international medical graduates who claim it is biased against them.

The GMC has launched an investigation into MRCGP pass rates and even the BMA has written to the college urging it to address concerns over the much lower pass rates for ethnic minorities and foreign medical graduates.

The RCGP insists the CSA is fair – chair Professor Clare Gerada even underwent the exam personally to make sure she could understand the concerns of its critics – and a Pulse survey reveals that the majority of GPs also think the exam is reasonable. So why the continuing controversy?

Stark differences

The CSA was introduced in 2007 as part of the revamped MRCGP exam. It tests aspiring GPs’ consultation skills in a mock clinical setting, with actors playing the role of patients and an examiner sitting in to grade the candidates, who must pass 13 different patient scenarios.

The college says the aim of the exam is to ‘test a doctor’s ability to gather information and apply learned understanding of disease processes and person-centred care appropriately
in a standardised context, make evidence-based decisions, and communicate effectively with patients and colleagues’.

But concerns began to emerge soon after its introduction, with international medical graduates – and, to a lesser extent, black and ethnic minority UK graduates – failing the exam at far higher rates than their white, UK-educated colleagues.The latest results available from the college (July 2010 to August 2011) show CSA failure rates of 3.9% for white UK graduates, 15% for UK graduates of south Asian decent and 33% for black UK graduates. The failure rate was even higher for international medical graduates as a whole, at 59%.

As a result of stark figures like this, the RCGP decided to fund an independent assessment of the CSA by King’s College London in November 2010. The college has yet to publish the full findings but, in a letter to all associates in training last year, it said the study of 52,000 cases had found there were ‘no substantial effects of gender or ethnicity on examiner/candidate interactions’.

The college was investigated by the GMC in 2011 after it emerged it had failed to get the required regulatory approval for toughening up the exam and did not warn trainers it expected pass rates to fall.

Legal challenge

Long-suppressed concern over the exam boiled over late last year. At the British Association of Physicians of Indian Origin (BAPIO) conference last November, delegates mandated the organisation to begin collecting funds for a possible judicial review to establish the fairness of the exam.

By the beginning of this year, this fund stood at more than £30,000, with other organisations – most notably the British International Doctors’ Association (BIDA) and the British Pakistani Doctors Forum – joining the call for a rethink of the CSA.

Failure has huge implications for trainees. As well as the costs involved – around £1,525 for each exam taken for associates in training – candidates are only allowed four attempts to pass the CSA and are removed from training after four failures.

A meeting between BAPIO, BIDA and the college in December attempted to resolve the row.

The RCGP offered some concessions: an increase in attempts allowed from four to six; collaborating with BAPIO in running courses; looking into the accountability of trainers; and an invitation to BAPIO to observe CSA examinations.

But it made no concessions on other key demands, such as allowing the CSA to be videoed for use in appeals, and retrospectively allowing failed graduates another try at passing the exam.

Instead of abating, the row has intensified further. BAPIO has given the college notice it will begin legal action this month if the situation is not resolved and the body has vowed to pursue the college through employment tribunals on behalf of those who have failed the CSA four times.

The GMC has waded in, with chief executive Niall Dickson saying the regulator was ‘determined to make progress’ on the issue and would hold talks with the RCGP about the outcomes of recent exams.

Even the BMA GP trainees subcommittee has called on the college to consider videoing all assessments, while an open letter from 18 GP trainers said there was evidence that trainees deemed competent by their trainers were failing the exam multiple times.

Unintentional bias?

One of those trainers, Dr Steve Taylor, a GP in Prestwich, Manchester, says there is no evidence of deliberate bias. But he insists he has trained doctors who were good enough to be GPs, yet failed the CSA.

He says: ‘A lot of people who are failing are not failing work-based assessments, where they are seeing 80 patients a week. Their communication skills week in, week out are actually okay.

‘[The assessments] are not intentionally biased. The college has to pass a certain number of people and they pass the best ones on the day. Because it is a spoken exam, they are likely to be UK graduates.’ 

Dr Taylor says part of the problem is that the CSA is not the best way of measuring whether trainees have the required communication skills.

‘The actors tend to be white, with south-of-England accents, and a lot of the doctors will end up working in inner-city Bradford or Rochdale, where a lot of the patients are from overseas.

‘It is all very well saying we want everyone to speak very good English, which is great, but we have to bear in mind they are going to have to work in areas where patients themselves won’t have English as a first language.’ 

Not all GPs agree with Dr Taylor. A Pulse poll of 229 GPs this month found 50% believe the exam is fair, compared with 15% who think it is unfair and 35% who say they don’t know.

Dr Peter Davies, the RCGP’s Yorkshire faculty representative, says: ‘It is a competency standard. Only those up to the standard will pass it.

‘If international medical graduates need extra or different training, then it needs to be provided, but altering the standard or applying a variable standard is unprofessional.

‘This is about applying a standard fairly and consistently. All examiners are doing that as fairly and consistently as they can.’

Uncertain future

The imminent publication of the 2011/12 exam results and a planned protest march on RCGP headquarters during the next round of exams are likely to keep the issue in the spotlight for a while yet.

The college is adamant that the exam is fair, pointing out the CSA is solely responsible for the failure of only 1% of the 3,000 medical graduates who take the MRCGP every year.

It says the results of the applied knowledge test section of the MRCGP show similar patterns to the CSA, with failure rates at 9% for white UK graduates in 2011, compared with 47% of south Asian international medical graduates. Overall, the failure rate for all UK graduates is 13% compared with 46% of all international medical graduates. Yet the AKT is anonymised, and there have been no claims of bias.

In a set of frequently asked questions on its website, the RCGP says it is ‘confident about the validity’ of the CSA. It says: ‘MRCGP assessment procedures have been well researched and have been approved by the GMC as the regulator.

‘There have been several external reviews over the past four years by leading international assessment experts, which have led to helpful modifications.’

‘Few doctors fail’

Professor Gerada says she is disappointed the college is facing legal action over the CSA: ‘I am very disappointed they have chosen to do this, rather than have a dialogue instead about how we can do the best for the next generation of doctors and our patients.

‘Overall very few doctors who start training fail to obtain the certificate of completion of training and only a very, very small number of GPs – approximately 1% per year of those who sit the MRCGP – are released from training because of a failure to pass the CSA alone.’

A spokesperson for the RCGP says: ‘Discussions are ongoing between the RCGP, BAPIO and BIDA.

‘Another meeting has been arranged for 21 February and the college is keen to work constructively and collaboratively.’

Readers' comments (28)

  • I think it's reasonable for a trainee to be assessed on their ability to think holistically. I also feel that there is not enough evidence at to suggest racial bias. My concerns are as follows:
    1. How do we know that trainers mark consistently & fairly?
    Bearing in mind there is no mark scheme, no video evidence & only ONE EXAMINER present. This facilitates ample room for subjective bias, furthermore this contradicts other more established Royal College exams.

    2. Are the actors consistent?
    No evidence of this at all.

    3. Why in the exams are 90% of actors/patients & examiners white?
    I'm Black British, so I wonder what would happen if we had 90% black or Asian actors or examiners. Since this has never been explored race remains a confounding factor.

    Historically, institutional racism usually starts with denial /apathy (usually by Caucasians) followed by some sort of intervention (usually by non Caucasians). This could be protesting/striking/legal action/doing nothing especially if left feeling disenfranchised. Eventually some sort of ruling takes place for or against. Trust me there have been so many occasions where ruling decides that institutional racism IS a problem. The RCGP are running the risk of serious embarrassment, just look at the Met Police!

    Unsuitable or offensive? Report this comment

  • its the actors who play a duboius role ,record the exam setting and truth will come out

    Unsuitable or offensive? Report this comment

  • BAPIO survey of trainees have revealed desperate situation for some of the trainees. One of them said “'Have good references from 20 hospital Consultants, several GP trainers, 40 excellent MSF,100 PSQ- have been working for NHS for seven years, have seen at least three thousand patients in GP Surgeries, passed AKT with good marks-cannot pass my CSA , about to be kicked out of the programme. If I am not fit to practice why did my trainers not raise concerns, why are the RCGP actors taking centre stage? Are they more genius than my Trainers, Consultants, Patients and colleagues?' The RCGP actors are deciding who is worthy to be a NHS GP!”Another trainee commented ‘This CSA and what they have done to my career has left me with financial trouble. I am in the process of selling the house too as we can't keep up with the mortgage and the credit taken’

    Unsuitable or offensive? Report this comment

  • It doesn't matter a damn where doctors come from so long as they speak English clearly and understand the British culture so that they understand the patient's concern and the patient understands them. Unfortunately in many IMG cases this is simply not the case and under those circumstances they should not be allowed to simply sit as many times as the can so that they finally pass. They should not be allowed to continue working in the NHS. This is not racism but common sense !

    Unsuitable or offensive? Report this comment

  • I'm alarmed by the politicisation of this issue.
    And what is the implication of the comment by Dr Taylor (above)? That if patients don't speak good English then doctors shouldn't have to either? Or that people whose first language isn't English need doctors who've been tested in the patients' language? Where are we going with this?

    IMG doctors should be treated fairly, but the core issue is whether doctors are competent to practice: patient safety should always be our core goal.

    Personally I don't see why videos of the exams shouldn't be made available: surely, they're at least an invaluable re-training aid. If we genuinely want to help people improve, the more specific the feedback the better.

    Unsuitable or offensive? Report this comment

  • Being Asian and British, I must admit that IMGs have poor comprehension. Speaking the language is not the same as understanding mindset, values or belief systems. EU doctors are the same. British Asians often come from closeted backgrounds although I find it surprising they should also have high failure rates. It took me some years to be on a par with White doctors, but I planted myself in a practice in a White heartland, and cut myself from my own Asian network. So I can understand why this problem exists. Medical knowledge is not communication skills and does not define attitudes as a component of the RCGP curriculum.

    Unsuitable or offensive? Report this comment

  • Being Asian and British myself, what does a closeted background mean-(not going to the pub).oh please we all worked in hospitals there is no time to be closeted.
    I think it's wrong to rely on actors to be involved in an end stage exit exam at all no matter how good they are.

    Unsuitable or offensive? Report this comment

  • The very nature and purpose of this examination pretty well ensures the veneer of bias, which is inherent in any professional evaluation. Minimizing this is extremely difficult. What educators have learnt in recent decades is to incorporate techniques of objectivity, eg referring to candidates by numbers rather than looking at names, pictures of their faces and recordings of their actual voices; by recruiting multiple examiners, who score totally independently; and by using multiple-choice questions as adjuncts as much as possible. Ideally, questions and scenarios used need to be published openly after the event, with answers given; and new questions thought up each time the exam is given. Videotaping is also essential, so as to explain confidently to candidates later what it was that caused them to lose marks, and which of the examiners thought so.

    Finally, a good review session lasting a fortnight or so immediately prior to the exam can be invaluable. Perhaps some entrepreneur can organize this.

    Measures such as these are the only way out, although even they still won't ensure that more candidates pass or that there will be fewer lawsuits.

    Unsuitable or offensive? Report this comment

View results 10 results per page20 results per page50 results per page

Have your say