This site is intended for health professionals only

At the heart of general practice since 1960

Analysis: Does the row over the CSA have any basis?

As the college faces legal action and criticism from all sides over the MRCGP exam’s clinical skills assessment, Jaimie Kaffash looks at whether the claims have any basis

The opening of the RCGP’s shiny new building in Euston this year should have marked a new start for the college, but old arguments about its entrance exam to the profession keep clouding the horizon.

The clinical skills assessment (CSA), part of the new MRCGP exam, has always been controversial. But now the college faces being dragged through employment tribunals by international medical graduates who claim it is biased against them.

The GMC has launched an investigation into MRCGP pass rates and even the BMA has written to the college urging it to address concerns over the much lower pass rates for ethnic minorities and foreign medical graduates.

The RCGP insists the CSA is fair – chair Professor Clare Gerada even underwent the exam personally to make sure she could understand the concerns of its critics – and a Pulse survey reveals that the majority of GPs also think the exam is reasonable. So why the continuing controversy?

Stark differences

The CSA was introduced in 2007 as part of the revamped MRCGP exam. It tests aspiring GPs’ consultation skills in a mock clinical setting, with actors playing the role of patients and an examiner sitting in to grade the candidates, who must pass 13 different patient scenarios.

The college says the aim of the exam is to ‘test a doctor’s ability to gather information and apply learned understanding of disease processes and person-centred care appropriately
in a standardised context, make evidence-based decisions, and communicate effectively with patients and colleagues’.

But concerns began to emerge soon after its introduction, with international medical graduates – and, to a lesser extent, black and ethnic minority UK graduates – failing the exam at far higher rates than their white, UK-educated colleagues.The latest results available from the college (July 2010 to August 2011) show CSA failure rates of 3.9% for white UK graduates, 15% for UK graduates of south Asian decent and 33% for black UK graduates. The failure rate was even higher for international medical graduates as a whole, at 59%.

As a result of stark figures like this, the RCGP decided to fund an independent assessment of the CSA by King’s College London in November 2010. The college has yet to publish the full findings but, in a letter to all associates in training last year, it said the study of 52,000 cases had found there were ‘no substantial effects of gender or ethnicity on examiner/candidate interactions’.

The college was investigated by the GMC in 2011 after it emerged it had failed to get the required regulatory approval for toughening up the exam and did not warn trainers it expected pass rates to fall.

Legal challenge

Long-suppressed concern over the exam boiled over late last year. At the British Association of Physicians of Indian Origin (BAPIO) conference last November, delegates mandated the organisation to begin collecting funds for a possible judicial review to establish the fairness of the exam.

By the beginning of this year, this fund stood at more than £30,000, with other organisations – most notably the British International Doctors’ Association (BIDA) and the British Pakistani Doctors Forum – joining the call for a rethink of the CSA.

Failure has huge implications for trainees. As well as the costs involved – around £1,525 for each exam taken for associates in training – candidates are only allowed four attempts to pass the CSA and are removed from training after four failures.

A meeting between BAPIO, BIDA and the college in December attempted to resolve the row.

The RCGP offered some concessions: an increase in attempts allowed from four to six; collaborating with BAPIO in running courses; looking into the accountability of trainers; and an invitation to BAPIO to observe CSA examinations.

But it made no concessions on other key demands, such as allowing the CSA to be videoed for use in appeals, and retrospectively allowing failed graduates another try at passing the exam.

Instead of abating, the row has intensified further. BAPIO has given the college notice it will begin legal action this month if the situation is not resolved and the body has vowed to pursue the college through employment tribunals on behalf of those who have failed the CSA four times.

The GMC has waded in, with chief executive Niall Dickson saying the regulator was ‘determined to make progress’ on the issue and would hold talks with the RCGP about the outcomes of recent exams.

Even the BMA GP trainees subcommittee has called on the college to consider videoing all assessments, while an open letter from 18 GP trainers said there was evidence that trainees deemed competent by their trainers were failing the exam multiple times.

Unintentional bias?

One of those trainers, Dr Steve Taylor, a GP in Prestwich, Manchester, says there is no evidence of deliberate bias. But he insists he has trained doctors who were good enough to be GPs, yet failed the CSA.

He says: ‘A lot of people who are failing are not failing work-based assessments, where they are seeing 80 patients a week. Their communication skills week in, week out are actually okay.

‘[The assessments] are not intentionally biased. The college has to pass a certain number of people and they pass the best ones on the day. Because it is a spoken exam, they are likely to be UK graduates.’ 

Dr Taylor says part of the problem is that the CSA is not the best way of measuring whether trainees have the required communication skills.

‘The actors tend to be white, with south-of-England accents, and a lot of the doctors will end up working in inner-city Bradford or Rochdale, where a lot of the patients are from overseas.

‘It is all very well saying we want everyone to speak very good English, which is great, but we have to bear in mind they are going to have to work in areas where patients themselves won’t have English as a first language.’ 

Not all GPs agree with Dr Taylor. A Pulse poll of 229 GPs this month found 50% believe the exam is fair, compared with 15% who think it is unfair and 35% who say they don’t know.

Dr Peter Davies, the RCGP’s Yorkshire faculty representative, says: ‘It is a competency standard. Only those up to the standard will pass it.

‘If international medical graduates need extra or different training, then it needs to be provided, but altering the standard or applying a variable standard is unprofessional.

‘This is about applying a standard fairly and consistently. All examiners are doing that as fairly and consistently as they can.’

Uncertain future

The imminent publication of the 2011/12 exam results and a planned protest march on RCGP headquarters during the next round of exams are likely to keep the issue in the spotlight for a while yet.

The college is adamant that the exam is fair, pointing out the CSA is solely responsible for the failure of only 1% of the 3,000 medical graduates who take the MRCGP every year.

It says the results of the applied knowledge test section of the MRCGP show similar patterns to the CSA, with failure rates at 9% for white UK graduates in 2011, compared with 47% of south Asian international medical graduates. Overall, the failure rate for all UK graduates is 13% compared with 46% of all international medical graduates. Yet the AKT is anonymised, and there have been no claims of bias.

In a set of frequently asked questions on its website, the RCGP says it is ‘confident about the validity’ of the CSA. It says: ‘MRCGP assessment procedures have been well researched and have been approved by the GMC as the regulator.

‘There have been several external reviews over the past four years by leading international assessment experts, which have led to helpful modifications.’

‘Few doctors fail’

Professor Gerada says she is disappointed the college is facing legal action over the CSA: ‘I am very disappointed they have chosen to do this, rather than have a dialogue instead about how we can do the best for the next generation of doctors and our patients.

‘Overall very few doctors who start training fail to obtain the certificate of completion of training and only a very, very small number of GPs – approximately 1% per year of those who sit the MRCGP – are released from training because of a failure to pass the CSA alone.’

A spokesperson for the RCGP says: ‘Discussions are ongoing between the RCGP, BAPIO and BIDA.

‘Another meeting has been arranged for 21 February and the college is keen to work constructively and collaboratively.’

Readers' comments (28)

  • the indifference everyone has to imgs is appalling

    Unsuitable or offensive? Report this comment

  • Hope things improve.
    It isn't anybody's fault.
    Any "oral exam" will favour the native.
    Problem with CSA is it is just tilting too much.
    Because it is being used only tool for standard setting.
    The other two tools - Akt and WBPA should replace csa for achieving better standard.
    To overcome bias in WBPA GP trainees must be placed in 3 different practices for 6 months which gives 3 trainers to comment on the trainee .

    Unsuitable or offensive? Report this comment

  • Can RCGP explain that in 2007 the failure rate for IMG was 7% below the UK graduate and now the difference is more then 50%.
    2. Why RCGP is so resistant in videoing this exam, before 2007 they used to do evaluate people based on videos submitted by doctors and now they consider video cannot reflect what actually happened in the exam,,, Isnt it Hipocracy.

    Unsuitable or offensive? Report this comment

  • Wonder how much in terms will it cost to video ?
    I am told all rooms used for csa have a camera.
    So I suppose it should not cost very much.
    Seriously some one needs to think.
    For a doctor scoring in 80s in AKT fails CSA , RCGP should think and do what other colleges do.
    They go back to see why this doctor failed because other colleges understand the importance of a knowledgeable doctor.
    I am sure if you survey patients and ask who would they prefer... A) doctor with excellent comm skills but moderately okish medical knowledgeable or B) Doctor with okish comm skills but excellent medical knowledge .

    Unsuitable or offensive? Report this comment

  • Dear ALL
    The question is not all about good or bad // fair or unfair. If there is a perceived notion about IMG 's being indifferent to British culture and British jokes - why are they taken into training ??? Is it only to make use of service provider aspect. Why IMG's are not trained as accordingly with focus to improve their communication and developing understanding of British values. Why at the end of their training they are subjected to racial prejudice. Why only IMG's are expelled and not White British graduates. RCGP // Deanary // Trainer all of them should be made accountable and not poor IMG - we are not refugees and it is only after hardships - IELTS // PLAB // COTS// CbDs // MSF // PSQ // DOPS // Reflective Log entries - You say we are not fit to be a GP. It is absolutely unacceptable - myself being an IMG have no other choice but to accept the insult. I think all the IMG's irrespective of Speciality or GP training should rise above , unite and give a beffiting response about this instituionalised racism.
    Poor IMG

    Unsuitable or offensive? Report this comment

  • Vinci Ho

    The DESIGN of the examination particularly the CSA is questionable. It may not be fair to question the value and aim behind the design . Yes , ensuring those who passed the examination has reached certain standard is undeniable BUT the examination could have a BLIND SPOT and disadvantaged certain section of the takers.
    I am holding on to the simple argument that the discrepancy of percentages is far too big to be ignored.
    Sad to see that this matter has been and will be politicised but it is now inevitable that independent professionals have to be involved.

    Unsuitable or offensive? Report this comment

  • Dear Clare,
    Day after day, you are just causing me shock after another with your answers. And actually it is sad to see that is coming from you. You could be a nice person in real and I have met you before, but the way you are defending the exam has changed the picture I have for you. My post has to be anonymous and that is the least favour I can do for myself. I still have to do the exam and I don’t want to think that my comment today has led the college to make me fail, I promise you that I will come and see you when I get the pass result in my next attempt.
    I carry an advice for you: first don’t be stubborn and look around, have a word with the Home office and ask them what happened when they were sooo stubborn like yourself now. I am sure you will learn that what is right is right and no one has the power to alter it.
    my points now are mainly related to professionalism of the exam: your actors are not trained, they have no idea what reality of acting as a patient is, no one of them has attended acting class in what is related to how they should act as a patient.
    Other royal colleges DONT do videotaping and that is simple to answer, I have passed MRCP, MRCS. In both of them there are two examiners in each station and each examiner has own marking sheet. On the top of that, they have real patients (bear in mind that some stations in MRCS was breaking bad news) so it is not videotaping but the bias level is less than the bias in RCGP. CAN YOU HAVE TWO EXAMINERS IN EACH STATION IN CSA??!! i am sure you would say it is not possible and allow me to tell you why , simple the cost .
    I did ask my trainer who is an examiner about how much they pay them to be csa examiner, it is pretty good pay. On contrast to other examiner of royal college who some of them do it voluntarily, simple they care about their profession and its reputation and mainly their trainees. STOP please comparing RCGP to them.

    When you say that RCGP set the standard and has nothing to do with the training or recruitment!! it is a serious mistake to say that.
    RCGP has broken the rules by not discussing the new settings of CSA with GMC when it was changed and there is an article in PULSE to give you an idea if you dont know.
    Also you have changed the exam settings after i started the training that is by itself a break of contract which i signed knowing that MRCGP is a reflective exam of the real life and the actual thing now that CSA is very silly acting exam. I promise you that I practise with white British doctors in my previous attempt who now passed the exam, from my position as a physician I have no trust to let them see my child if gets ill.
    It is shameful to say that CSA is like a driving test, LORD GOD. That should never be said as an example; CSA should never resemble a driving test. It is a clinical exam, not Sean Connery doing James bond. No doubt that we have to have good interpersonal skills, but this has been tested in 18 months in general practice. You cant assess them in 10 mins in 13 stations. You cant assume that a human will be able to do domestic violence station then learning disability station before going again to breaking bad news station. You are making no sense in that. I bet you as a GP if you have that in a normal day, you won’t feel good after that, so how you would feel when you are in an exam environment!!!!!
    I urge you from your position as a chair to listen, stop defending Sue and others who are really not telling you the truth about how the exam is conducted, please listen and be aware that it is the right time to listen, because IMG is a %45 of the nhs workforce and simply you can’t ignore facts, lastly be a GP for me at least and listen to my clear agenda, JUST videotape the exam please, that is the least sensible thing you can do for the future doctors.

    Unsuitable or offensive? Report this comment

  • This comment has been deleted by the moderator

    Unsuitable or offensive? Report this comment

  • Sudheer Surapaneni

    The only way forward is March and Tribunal. BPDF supports ET action through mediclaw raj chaudhary. Join and be heard.

    Unsuitable or offensive? Report this comment

  • ARY News video - Saturday 9 Feb

    Unsuitable or offensive? Report this comment

View results 10 results per page20 results per page50 results per page

Have your say