Claim that AI is 'on-par' with top doctors questioned by leading GPs

Leading doctors have questioned claims that artificial intelligence can provide heath advice which is "on-par" with top-rated medics.

Technology healthcare company Babylon is to present data claiming its artificial intelligence (AI) software achieves equivalent accuracy to human doctors when it comes to giving health advice.

But the Royal College of GPs has said "no app or algorithm will be able to do what a GP does".

Babylon is to present evidence at an event at the Royal College of Physicians headquarters in London which it says demonstrates how its AI can provide health information, rather than a medical diagnosis, in a number of situations.

It said its software can pass relevant sections of the final test for trainee GPs - known as the MRCGP - set by the Royal College of General Practitioners (RCGP).

In test simulations - the RCGP does not publish past papers so Babylon used example questions - Babylon's AI scored 81%.

Babylon said the average pass mark for real life doctors was 72%.

But the College said the claim was "dubious".

To further test the software, the AI was presented with case studies where it was told about a series of symptoms, and its responses were compared to those of seven "highly-experienced" primary care doctors.

In the 100 cases, Babylon's AI scored 80% for accuracy, while the seven doctors achieved an accuracy range of 64-94%, it said.

But it stressed that its AI remains an information service, rather than a medical diagnosis.

Dr Ali Parsa, Babylon's founder and chief executive, said: "Even in the richest nations, primary care is becoming increasingly unaffordable and inconvenient, often with waiting times that make it not readily accessible.

"Babylon's latest artificial intelligence capabilities show that it is possible for anyone, irrespective of their geography, wealth or circumstances, to have free access to health advice that is on-par with top-rated practising clinicians."

But Professor Martin Marshall, vice chairman of the Royal College of GPs, said: "The potential of technology to support doctors to deliver the best possible patient care is fantastic, but at the end of the day computers are computers, and GPs are highly-trained medical professionals - the two can't be compared and the former may support but will never replace the latter.

"No app or algorithm will be able to do what a GP does.

"Much of what GPs do is based on a trusting relationship between a patient and a doctor, and research has shown GPs have a 'gut feeling' when they just know something is wrong with a patient.

"An app might be able to pass an automated clinical knowledge test but the answer to a clinical scenario isn't always cut and dried, there are many factors to take into account, a great deal of risk to manage, and the emotional impact a diagnosis might have on a patient to consider.

"It is also the case that the exam-preparation materials, used by Babylon in this research, will have been compiled for revision purposes and are not necessarily representative of the full-range of questions and standard used in the actual MRCGP exam, so to say that Babylon's algorithm has performed better than the average MRCGP candidate is dubious.

Prof Marshall added: "Babylon's GP at Hand service uses technology in a way that some patients like. But some don't, and the way it is being used risks undermining and damaging traditional general practice services.

"The College has publicly criticised GP at Hand for 'cherry-picking' patients, leaving traditional GP services to deal with the most complex patients, without sufficient resources to do so.

"We stand by this: we do not endorse Babylon, or its GP at Hand service, being used in the way that it is, in the NHS.

"Technology has the potential to transform the NHS, but it must be implemented in an equitable way, that doesn't benefit some patients, and not others, and is not to the detriment of the general practice service as a whole."