The independent evaluation of the Babylon GP at Hand service will not be able to make robust conclusions about its safety and effectiveness.
It will also ‘not include a comprehensive assessment’ of the artificial intelligence (AI) symptom checker the service uses to triage patients.
Ipsos Mori, which is evaluating the rapidly growing new digital GP consultation service, identified the limitations in an interim report which was published today following its initial scoping exercise.
The company will publish its final report on the service, which has signed up over 30,000 patients from GP practices across London over the past year, in March 2019.
Ipsos Mori was awarded the £250,000 piece of work to provide a ‘robust’ analysis of the service’s impacts and outcomes earlier this year.
But today’s interim report said: ‘The evaluation will only be able to provide qualitative evidence as to the safety and effectiveness of the Babylon GP at Hand service, and therefore will be limited in the robustness of the conclusions that can be drawn in this area.’
A report by NHS England was published in November assessing the clinical safety of Babylon GP at Hand services, including the AI symptom checker. It found that each safety case ‘meets the standards required’ by the NHS after a ‘robust assessment methodology’.
NHS England and NHS Hammersmith and Fulham CCG are also continuously evaluating the service.
But Dr Michelle Drage, chief executive of Londonwide LMCs, said the limitations of the Ipsos Mori independent evaluation were ‘disappointing’.
She told Pulse: ‘There is an urgent need for robust and independent evidence on the clinical effectiveness of this technology and the absence of that from the pending evaluation is disappointing.
‘Any proprietary technology which exists to financially benefit a commercial company should have independent, peer reviewed research funded by that company. The NHS should not be left to pick up the bill.’
In March, Pulse reported that the successful bidder of the review would evaluate the outcomes and risks of GP at Hand, including financial risks, capacity issues and the effect on continuity of care.
A Babylon GP at Hand spokesperson declined to comment on the interim report.
Babylon has previously claimed its in-app clinical advice is ‘on par’ with that of GPs.
NHS England is currently pursuing changes to the GP contract to boost more ‘digital-first’ GP consultations, along the lines of what Babylon has to offer.
Meanwhile, health secretary Matt Hancock has been criticised for his open support for the Babylon GP at Hand service, of which he is a patient.
