Technology behind ChatGPT better with eye problem advice than non-specialist doctors, study test finds | Science & Tech News

The technology behind ChatGPT scored better at assessing eye problems and providing advice than non-specialist doctors, a new study has found.A study led by the University of Cambridge has found that GPT-4, the large language model (LLM) developed by OpenAI, performed nearly as well as specialist eye doctors in a written multiple-choice test.
The AI model, which is known for generating text based on the vast amount of data it is trained on, was tested against doctors at different stages of their careers, including junior doctors without a specialism, as well as trainee and expert eye doctors.Each group was presented with dozens of scenarios where patients have a specific eye problem, and asked to give a diagnosis or advise on treatment by selecting from one of four options.

Image:
Dr Arun Thirunavukarasu – the lead author of the study – at work

The test was based on written questions, taken from a textbook used to test trainee eye doctors, about a range of eye problems – including sensitivity to light, decreased vision, lesions, and itchy eyes.
The textbook on which the questions are based is not publicly available, so researchers believe it is unlikely the large language model has been trained on its contents.GPT-4 scored significantly higher than junior doctors, whose level of specialism is comparable to general practitioners, at the test.The model achieved similar scores to trainee and expert eye doctors, but it was beaten by the top-performing experts.

Follow Sky News on WhatsApp
Keep up with all the latest news from the UK and around the world by following Sky News
Tap here

The research was conducted last year using the latest available large language models.
The study also tested GPT-3.5, an earlier version of OpenAI’s model, Google’s PaLM2, and Meta’s LLaMA on the same set of questions. GPT-4 gave more accurate responses than any of the other models.The researchers have said that large language models will not replace doctors, but they could improve the healthcare system and reduce waiting lists by supporting doctors to deliver care to more patients in the same amount of time.Read more on Sky News:Tourist tax warning in 10 citiesSmacking children should be banned, doctors say

Dr Arun Thirunavukarasu, the lead author of the paper, said: “If we had models that could deliver care of a similar standard to that delivered by humans, that would help overcome the problems of NHS waiting lists.”What that requires is trials to make sure it’s a safe and effective model. But if it is, it could be revolutionary for how care is delivered.”He added: “While the study doesn’t indicate deployment of LLMs in clinical work immediately, it gives a green light to start developing LLM-based clinical tools as the knowledge and reasoning of these models compared well to the expert ophthalmologists.”

Share this
Tags

Must-read

N.W.T. leaders worry wildfires, low water will mean even longer delay for much-needed housing units

Seniors in five N.W.T. communities expecting to move into new homes earlier this spring will now have to wait until at least summer, because of wildfires...

PC candidate says comments on recruiting doctors from India and Pakistan weren’t meant to be derogatory

Lin Paddock, the Progressive Conservative candidate in the upcoming Baie Verte-Green Bay byelection, said Tuesday that comments he made about recruiting doctors in India...

Repairs near completion at Rissers Beach Provincial Park after storm damage

Eight months after post-tropical storm Lee tore through Rissers Beach Provincial Park on Nova Scotia's South Shore, repairs are moving into the final stages.  Rissers...

Recent articles

More like this

LEAVE A REPLY

Please enter your comment!
Please enter your name here