This site is intended for health professionals only

At the heart of general practice since 1960

pulse june2020 80x101px
Read the latest issue online

GPs go forth

Babylon claims its in-app clinical advice is ‘on par’ with GPs

Private GP provider Babylon has claimed its app is able to provide clinical advice to patients that is ‘on par’ with doctors, sparking criticism from GP leaders.

Babylon, which also provides its GP at Hand service for NHS patients, said it had used diagnostic questions from trainee GP exams to test the artificial intelligence behind its app, which features a symptom checker and provides patients with medical information and triage advice.

The company said its AI scored 81% when it was tested using RCGP exam questions, whereas the average mark for real-life doctors was 72%, according to data from 2013 to 2018.

However, the RCGP said the claims were ‘dubious’. It highlighted Babylon had used its MRCGP exam preparation questions, which were for ‘revision purposes,’ and that these ‘are not necessarily representative of the full-range of questions and standard used in the actual MRCGP exam’.

The RCGP’s vice chair, Professor Martin Marshall, said: ‘To say that Babylon’s algorithm has performed better than the average MRCGP candidate is dubious.’

Babylon also carried out further tests by collaborating with the Royal College of Physicians, Stanford University’s chief of general primary care, Dr Megan Mahoney, and Yale New Haven Health’s chief population health officer, Dr Arnold DoRosario, which involved using 100 independently devised symptom sets.

According to Babylon, during this further testing, its AI scored 80% for accuracy, while the seven doctors it compared results with achieved an accuracy range of 64 - 94%.

Babylon also said the app’s safety score was found to be 97%, while the GPs were assessed as having an average of 93.1%.

Babylon launched its research on the app’s ability to provide accurate health information at the Royal College of Physicians in London tonight.

Ali Parsa, Babylon’s founder and CEO, said: ‘Babylon’s latest artificial intelligence capabilities show that it is possible for anyone, irrespective of their geography, wealth or circumstances, to have free access to health advice that is on par with top-rated practising clinicians.’

‘Tonight’s results clearly illustrate how AI-augmented health services can reduce the burden on healthcare systems around the world.'

However the RCGP’s Professor Marshall said that while the ‘potential of technology to support doctors to deliver the best possible patient care is fantastic… at the end of the day, computers are computers, and GPs are highly-trained medical professionals’.

‘The two can’t be compared and the former may support, but will never replace, the latter,’ he said.

‘No app or algorithm will be able to do what a GP does…

‘An app might be able to pass an automated clinical knowledge test but the answer to a clinical scenario isn’t always cut and dried, there are many factors to take into account, a great deal of risk to manage, and the emotional impact a diagnosis might have on a patient to consider,’ he added.

Readers' comments (21)

  • Just Your Average Joe

    'Babylon claims its in-app clinical advice is ‘on par’ with doctors'

    My automated phone line with 4 options can do the same job - give paracetamol, antibiotics and diazepam to anyone who chooses option 1, 2 or 3. Option 4 leads to a accredited HCP to issue sick notes.

    Job done.

    Unsuitable or offensive? Report this comment

  • David Banner

    Fine, Babylon’s super computer doc can see all our multi agenda hypochondriacal heartsinks. It will go into meltdown within minutes.
    And who does the patient sue when the inevitable cock ups occur?

    “Look Dave, I can see you’re really upset about this. I honestly think you ought to sit down calmly, take a stress pill, and think things over”

    Unsuitable or offensive? Report this comment

  • AlanAlmond

    Fascinating ..guess they’re hoping for a lot of GP outrage to fuel their hype. Yawn.

    Unsuitable or offensive? Report this comment

  • We should not air this - let them do what they are doing - until they truly replicate a general practice - we still have a while to go

    I think they will be helpful - in the future to deal with the he worried well, I don’t think it will be cheaper.

    Like using Ryan air - the only way to book a ticket is online A maybe we should do this for all patients young or old - the only way to access healthcare is through a chat bot - we can see what the public say then.

    Unsuitable or offensive? Report this comment

  • Vinci Ho

    Medicine is about both science and human touch
    On the 70th birthday of NHS , there are at least two issues worth a serious debate
    (1)How to fund NHS which clearly needs more investments?
    (2)The role of latest technology e.g. AI , robotics ,smartphone apps , social media etc
    Watched the BBC 2 programme celebrating 70th birthday of NHS last night :
    Both Simon Stevens and Lord Darci spoke out. They were addressing these questions .
    As I wrote before , I am yet to hear or see anyone listing the advantages and disadvantages of pushing more innovative (the word chosen by Lord Darci) technologies into NHS ,for the sake of discussion. . There seems to be a narrative that the more new money spent in these technology , the more likely we can solve the crisis we are facing in primary and secondary care (and social care).
    One size cannot fit all . We need to carefully analyse and speculate circumstances where these new technologies will be extremely useful e.g. processing hundreds and thousands of scan images(example in ophthalmology quoted in the programme)but then in less clear-cut but dubious cases , which way AI will turn, positively or negatively , hence , generating false positive or negative as well ? It will be interesting to see more evidences. Presumably, AI is better than radiologists using naked eyes or not? Bear in mind , even for that, how often do we disagree with the radiologist? What will stop us from disagreeing with the machine ? However , it can be a caveat if the whole process , of investigating, interpretation of result and reaching diagnosis , is dominated by AI.
    The biggest question we have been asking about the impact of internet and particularly social media on our young generations, is how are they going to develop their social skills , hence , connecting to each other as human beings. Aren’t we becoming hypocrites to encourage our youngsters to spend even more time on smartphones getting medical help?
    We influence each other by talking and connecting with each other as human beings. GPs know this extremely well because that is practically what we(GPs)are doing everyday and it is also about human touch when we examine our patient.
    Video consultation on smartphone must have its limitations as far as making diagnosis without physical examination is concerned , in at least half of the cases but may be though reliable for certain cases in e.g. dermatology , minor injury.
    It is fine for simple requests like asking a sick note and contraceptive advices etc . But we should also be aware of abusing this device system. GP apps addiction is not totally unreal .
    In the TV programme, the young patient used GP at hand video consultation for symptom of headache because he could not get a GP appointment under 3 weeks ; so are we (GP) feeling comfortable and secure to make a diagnosis and initiate treatment without physical examination for this chap ? May be , may be not ?Otherwise , he will still need a face to face consultation.
    AI must be for facilitating not replacing physicians, at least this current level of technology.Certainly, an AI love affair is not to replace a successful doctor-patient relationship . I cannot see AI completing Pendleton’s seven tasks of consultation in general practice e.g. the sharing of diagnosis and treatment decisions.

    My concern is fundamental: are we simply switching from a human model to a machine model ?Because the latter is potentially more economical in dealing with ever rising demands in NHS , at least in the eyes of economists and politicians. But are ALL these demands ‘justified’?

    Obsession is most dangerous when we are blind in one eye , in face of a crisis .

    Unsuitable or offensive? Report this comment

  • Vinci Ho

    This is what I wrote two days ago:
    As I wrote , I would like to keep an open mind about this . But I also believe everything has its advantages as well as disadvantages. Well , at least that is how I was taught during my vocational training on developing good doctor-patient relationship 24 years ago . We know very well the strengths and weaknesses of the ‘old’ (still current) model of doctor-patient communication skill simply because of long time of practising this model.
    We are yet to learn the same about this new AI-algorithm-smart GP consultation model . As veterans , we are easily labelled as anti-revolution. Equally , the younger colleagues can be criticised as insouciant. I am reminding myself this everyday.
    One size certainly cannot fit all . The current climate of pushing AI by all these tech companies like Google , Amazon , Facebook etc is NOT healthy , at least , politically . We have already seen the goods as well evils of the over-indulgences of Internet and more specifically social media . The way they can influence ‘outcomes’ has arguably crossed the red line of morality in various arenas.
    Until I have seen objective evidence about the real merits of this kind of model of consultation which is to be driven by the tech industry fast and furious , my scepticism always remains.
    Another big issue is confidentiality, the collection of ‘big data’ by these tech giants(hence , further usage )is estranged to our principle of defending patient confidentiality. Clearly , I don’t believe the IT system of NHS will be strong enough to deter the misdemeanour of these tech firms .
    Once again , like I quoted Lincoln in my long comment yesterday :
    One can fool everybody some time and some people all the time . But not everybody all the time.
    The reason I always believe we are Jedi is because we have a code and morality to serve for, that’s why we are different from them.........

    Unsuitable or offensive? Report this comment

  • Knowledge is Porridge

    Bring on the technology! Please help us sift through the stuff and manage the workload. A computer AI can monitor online interactions and learn to collect and analyse the data. The next 10 years will be full of false dawn's, but in maybe 20 years they will be bulletproof.
    Amazon was going to destroy the high street in the 90's. Now 20y later it's doing exactly that...
    I think this is a reason for hope!

    Unsuitable or offensive? Report this comment

  • Babylon feels emboldened to make this claim after the honourable chair of RCGP gave commendations.
    It is clear the powers that be have decided AI is the way to go. Jeremy Hunt once aid doctors (I think he means GPs) will so be obsolete as AI will take over and this is being played out now.

    Unsuitable or offensive? Report this comment

  • Dear All,
    And Babylon claims that i am paid six times more for my 85 year olds than i am for my 15-44 year olds.
    How did the AI deal with "TATT"?
    Paul C

    Unsuitable or offensive? Report this comment

  • How will it deal with Tatt, chuck them of their books and tell them to see a GP I bet(like every other @@@@@ about).We seem to be the backstop for everyone!

    Unsuitable or offensive? Report this comment

View results 10 results per page20 results per page50 results per page

Have your say